The overall findings indicate that although decision making is generally faster when using the graphical interface (1.60s vs. 2.05s), decisions are more accurate when using the numerical display (87.1% vs. 94.1%). It was found that for scenarios prompting a deny re-task action, with a small distance between the edge of the RAR and the new target, the numeric display maintained 100% decision accuracy compared to the graphical display with 53%. It is worth noting that the small sample used in this trial may explain why the difference in accuracy was not significant across all scenarios and this is an acknowledged limitation.
Overall, the time difference between display groups at making their decisions is around half a second. From a practical perspective, the accuracy gained by using the numerical display should outweigh the benefit of making a decision half a second quicker. The large difference in performance of participants using the different display types seems to occur when a new target appears close to, but not inside, the capability of the weapon. However, this specific scenario could be mitigated by using an under-representation of the capability of the weapon. As discussed in Chapter 3, the basic RAR algorithms used in this real-time study, under-represents the true capability of the GBU. A limitation in this particular study was that the reduced number of participants recruited meant that there was not an opportunity to develop the scenarios presented to participants in such a way to further examine the borderline cases.
Post-task feedback adds further insight into causes of error. Some of the graphical dis- play group participants indicated they were not expecting a new target to appear close to the edge of the RAR despite being fully briefed and demonstrating they understood their task. The open question ‘What thoughts did you have that led you to miss an opportunity to re-task?’ prompted the following responses from group A participants:
• “Not expecting a new target” - Participant A3
• “...it was only inside the green box for a very short time.” - Participant A4
• “...I did not see that situation in previous trials.” - Participant A5
• “The new target appeared when the range of the missile reduced very fast so that I didn’t have time to re-task.” - Participant A6
• “...new target was right on the green boundary and I hesitated” - Participant A9
• “...the green area was rapidly retracting when the new target appeared” - Partici- pant A10
• “...by the time I pressed the key to move the grid lines moved...” - Participant A13
• “...the target was initially in the RAR but was outside by the time I had a chance to react” - Participant A15
Participant A5’s comment highlights the possible learning effect of seeing several sce- narios where a new target appears a long way outside of the weapon’s capability in the run up to a scenario in which a new target appears just inside the RAR boundary. Participants may have performed better if this boundary case scenario had occurred early in the scenario set. The scenario sequence is acknowledged as a limitation of this study with regards to the potential learning effect. Regardless, further improvements could be made by giving clear instructions on the additional procedures when border- line scenarios occur.
It is also worth noting that whilst the significant differences seem small, if 100% decision accuracy is a requirement, only 41% of participants achieved 100% using the graphical display, compared with 59% of participants using the numerical display. Considering the lack of experience the participants had, these results would be improved through training and a stricter set of operating procedures, particularly in relation to new tar- gets that appear close to the border of the weapon’s range capability. No participants were removed from the data set.
Given that this trial was the first study to test the effectiveness of the display, the num- ber of decision aids was reduced to the minimum number necessary to allow participants to make their decisions. Participants’ general remarks with regards to improvements of the display types typically suggest improvements to the cue for a new target appearing and that the system has received a command from the user. The main themes from the qualitative data surround the inclusion of more active prompts to the user and system feedback for actions made by the user. Further suggestions are to do with increasing the manipulation of the colour of the different elements on the screen. Interestingly, although the Numerical Display is sufficient for decision making and participants per- form better when using it, 5 of 17 Numerical Display participants suggested the use of a RAR boundary to aid decision making. Of these five participants, four achieved a score of 100% correct decisions. This supports the notion that improvements need to be made to display the effectiveness of a participant’s decision, for example by integrating a Battle Damage Assessment (BDA) tool into the Remote Operator Terminal so that more detailed feedback is provided that the weapon hit its intended target.