Comparing SBML and LBT Performance by Learning Objective

successful in demonstrating the learning objectives on their first attempt at the test scenarios (drill and emergency scenarios, respectively). The learning objectives were categorized as spatial performance and procedural performance; each will be discussed separately.

2.6.3.1. Spatial Performance

Effective wayfinding in emergencies depends on an individual’s spatial knowledge of the platform layout. Seigel and White (1975) describe a spatial knowledge acquisition model

known as Landmark-Route-Survey. This model provides one explanation for how individuals develop a spatial understanding of an environment. Spatial knowledge usually starts with the recognition of salient landmarks, followed by connecting the landmarks with learned routes. Over time, individuals develop a map-like representation of an environment (e.g. learning how landmarks and routes are interconnected) known as survey knowledge.

The basic muster drill scenarios tested the participants’ recognition of landmarks and their ability to follow designated routes. Table 2.3 shows the percentage of participants who reached competence for each learning objective in the two muster drill scenarios. A clear difference between SBML and LBT groups is seen in the first test scenario (S1) for reaching the correct location and correctly following the egress routes. This difference is less prominent in the second test scenario (S2). The LBT trained groups matched the performance of the SBML group in reaching the correct location and showed improvements in following the designated egress route during the alarm recognition muster drill scenario.

Table 2.3: Percentage of successful participants for drill scenarios Performance Measures

n/a = not applicable. Some performance metrics are not applicable for all test scenarios.

The participants’ route selection and re-routing in the emergency scenarios are good

situations. The emergency scenarios tested the participants’ route and survey knowledge by forcing them to reroute after they encountered an obstructed path. Table 2.4 shows the percentage of participants who were successful at each learning objective in the two emergency scenarios. For these test scenarios, the route selection learning objective was divided into four subcategories, which are listed as items 2 to 5 in Table 2.4. The correct behaviours for the emergency scenarios were to select the safest route or re-route based on the PA information. Individuals who did not re-route until they encountered the hazard and those who did not re-route at all failed the test scenario. This categorization was designed to determine: 1) what information the participants were using to select their egress route, 2) the level of risk the participants were willing to take, and 3) if the participants had sufficient survey knowledge of the platform to re-route if their designated route was compromised by a hazard.

Table 2.4: Percentage of successful participants for emergency scenarios Performance Measures

5. Did not re-route (opened door to hazard

and/or went through the hazard) 7% 47% 37% 2% 12% 32%

The overall percentage of participants who selected the safest route for a given scenario is one indicator of the training efficacy and how well the training helped participants develop egress strategies. As shown in Table 2.4, the SBML trained group was more successful at selecting the safest route at the onset of the emergency situation in scenario S3. The LBT1-trained group was more successful at selecting the safest route in scenario S4. In this case, the SBML trained group did not outperform the LBT1-trained group on their first attempt at the scenario in terms of reaching the correct location and selecting the safest egress route (item 1 and 2 in Table 2.4). This highlights some limitations in the SBML training for developing spatial knowledge. Two possible reasons for the performance variability are: 1) developing survey knowledge takes time and this process is subject to individual variability regardless of training method, and 2) the different training methods (SBML and LBT) resulted in different decision making strategies.

To better understand the limitations of the SBML and LBT training methods, Smith et al. (2017) investigated participants’ decision making strategies in the virtual emergency scenarios using decision tree modeling. This analysis showed that the SBML-trained group tended to employ route selection strategies that involved listening to the PA announcement and taking into consideration information from the announcement. While these route strategies were more successful in general, this does highlight a limitation of the delivery of the training. The SBML training could be improved by focusing more attention to the development of survey knowledge and teaching participants on how to select routes and how to reroute due to blocked routes in the absence of PA announcements. Implementing improved performance assessment and built-in diagnostic tools into the VE, such as

decision tree modeling, could help improve the implementation of the SBML training by addressing each individual’s learning needs.

2.6.3.2. Procedural Performance

The biggest difference between the SBML and LBT training groups was the participants’

compliance (or lack thereof) with safe practices. The SBML group was more compliant with safe practices and showed more risk adverse behaviours than the LBT groups. The majority of SBML participants were able to successfully avoid hazard exposure, close fire doors, and avoid running on the platform. These behaviours were not demonstrated by the LBT training groups.

Overall, the difference in competence and compliance between the SBML and LBT groups is attributed to the delivery of the training. There are two main reasons why the SBML participants outperformed the LBT trained participants: 1) SBML participants received an informative assessment with specific corrective feedback in the training scenarios, and 2) SBML training’s fixed minimum passing standard for each scenario forced the participants to repeat scenarios until competence was demonstrated.

With regards to informative assessment, the SBML training group completed practice scenarios and had in-simulation instructions and feedback. During the SBML training scenarios, participants received immediate in-scenario feedback if they performed the safe practices incorrectly. This allowed the participants to recognize what was correct or incorrect performance. The LBT training groups both received instructions through lecture-style tutorials, followed by after scenario feedback. In the absence of corrective

feedback, the LBT participants repeated the same errors throughout the training and testing scenarios.

Another aspect of SBML training that impacted performance on compliance to safety procedures was the use of a fixed minimum passing score for the training and testing scenarios. The SBML trained participants were only able to proceed to the next training scenario if they reached a minimum passing score in the learning objectives. This restriction in the training ensured all the participants complied with the safety procedures. The SBML participants had more opportunities to learn from their mistakes and adjust their strategy to respond to the emergency situations.