The HOT was developed to provide simulation-based threat reaction training to CC-130 loadmasters and others who work in the CC-130 cargo compartment. The HOT utilizes a simulated crew station to provide students with a simulated view from a CC-130 being engaged by AAA and SAMs. Structured training materials and scenarios enable instructors to teach threat calls to the students in a dynamic, interactive, and controlled simulation. The experiment revealed that the HOT is a valid representation of the threat environment, as shown by expert aircrew initially out-performing novices in the threat reaction task. The HOT demonstrated its ability to train and to lead to expert performance when the performance of novices improved and came to be indistinguishable from that of the experts.
14 DRDC Toronto TR 2009-008
3 Recommendations
• The HOT has demonstrated the ability to improve the threat call performance of novices to that of experts in the course of two 90- minute sessions. The HOT should therefore be considered for use in the training of CC-130 loadmasters. The timing and structure of the training sessions used in this trial should be revised by the CC-130 training community, taking into consideration the current threat scenarios, prevailing administrative constraints, and local knowledge. It should be anticipated that different training regimens will be required for users with different levels of experience, such as novices versus pre-deployment refresher training for operational crews.
• The CC-130 community should consider establishing performance standards for threat calls.
Standards for timeliness, accuracy of the calls, and the syntactical correctness should be considered. The availability of such standards will provide guidance to the development of training scenarios and the amount of HOT training required.
• Several improvements should be made to the HOT to increase the utility of the existing version of the device. The visual imagery should be optimally rendered for the existing projection surface. Also, the time for a student to respond to a threat should be made available to the instructor in a reliable manner. Finally, the appearance of the tracer fire and missile trails should be made more realistic and variable.
• If the HOT is taken into use for operational training, opportunities for confirming the positive transfer of learning from training to operations should be sought. Although it is not expected that enough data could be collected for a statistical analysis to have sufficient power to show a quantitative training effect, interviews could be recorded that would at least document striking qualitative effects.
• Any future generations of the HOT that provide night viewing conditions and night vision goggle simulation (or night vision goggle stimulation) should be subject to a validation study.
DRDC Toronto TR 2009-008 15
4 References
[1] Department of National Defence (2006), Standard Manoeuvre Manual: Tactical Air Transport Procedures Volume 1. SMM-2601(1) Amendment 3. 1 Canadian Air Division.
[2] Department of National Defence (2006), 3500-1 (A3 Tpt Sys) 27 February 2006 Endorsement of the Visual Threat Recognition and Avoidance Training (VTRAT) Statement of Operational Capability Deficiency (SOCD).
[3] Department of National Defence (2006), 2 November 2006 Science & Technology Direct Support to the CF – DRDC Toronto Business Line 2 – Hercules Observer Trainer, DRDC Toronto.
[4] Grant, S.C., Bodner, R., and Lamoureux, T. (2006), Identification and Prioritization of CC−130 Training Device Requirements (U), (DRDC Toronto TR 2006-188) Defence R&D Canada – Toronto.
[5] Hercules Observer Trainer Network Operations 1. Tasking 417 under contract W8475-06BM04. February – March 2008.
[6] Hercules Observer Trainer System Modifications. CAE Professional Services Inc. Call-up 7949-08 under Standing Offer W7711-057949. April 2007 – March 2008.
[7] Hercules Observer Trainer System Development. CAE Professional Services Inc. Call-up 7949-06 under Standing Offer W7711-057949. December 2006 – March 2007.
[8] High Performance Visual System. CAE Professional Services Inc. Call-up 7949-07 under Standing Offer W7711-057949. November 2006 – March 2007.
[9] Lathan, C.E., Tracey, M.R., Sebrechts, M.M., Clawson, D.M., and Higgins, G.A. (2002).
Using virtual environments as training simulators: Measuring transfer. In K. Stanney (Ed.), Handbook of Virtual Environments, pp. 403 – 414, Mahwah NJ, : Lawrence Erlbaum Associates.
[10] Hintze, J. (2008). NCSS (version 7.1.13). NCSS, LLC. Kaysville, UT.
[11] Fitts, P. M. (1966). Cognitive aspects of information processing: III. Set for speed versus accuracy. Journal of Experimental Psychology, 71(6), 849-857
[12] Stewart II, J.E. (1994). Using the backward transfer paradigm to validate the AH-64 Simulator Training Research Advanced Testbed for Aviation. US Army Research Institute Report 1666.
[13] Goettl, B.P., Ashworth, A., McCormick, E. and Anthony, M. (2003). Validation Of Visual Threat Recognition And Avoidance Training Through Analogical Transfer. In Advanced Technologies for Military Training, NATO RTO-MP-HFM-101, NATO.
16 DRDC Toronto TR 2009-008
List of symbols/abbreviations/acronyms/initialisms
[Enter list here, if applicable. If not, delete the page.]
AAA Anti – Aircraft Artillery ANOVA Analysis of Variance
CF Canadian Forces
CFAWC Canadian Forces Aerospace Warfare Centre CGF Computer Generated Forces
DRDC Defence Research & Development Canada
DRDKIM Director Research and Development Knowledge and Information Management
HELO Helicopter
HOT Hercules Observer Trainer
IOS Instructor Operator Station MANOVA Multivariate Analysis of Variance
R&D Research & Development
SAM Surface to Air Missile
SOCD Statement of Operational Capability Deficiency
DRDC Toronto TR 2009-008 17
Distribution list
Document No.: DRDC Toronto TR 2009-008
LIST PART 1: Internal Distribution by Centre 2 DRDC Toronto Library
7 Author
9 TOTAL LIST PART 1
LIST PART 2: External Distribution by DRDKIM 1 Library and Archives Canada
3 Maj Georgie Jones, Canadian Forces Aerospace Warfare Centre - Trenton 3 Capt Chris Rumpel 426 Squadron - CFSI
7 TOTAL LIST PART 2
16 TOTAL COPIES REQUIRED
18 DRDC Toronto TR 2009-008
This page intentionally left blank.
DOCUMENT CONTROL DATA
(Security classification of title, body of abstract and indexing annotation must be entered when the overall document is classified) 1. ORIGINATOR (The name and address of the organization preparing the document.
Organizations for whom the document was prepared, e.g. Centre sponsoring a contractor's report, or tasking agency, are entered in section 8.)
Defence R&D Canada – Toronto
(Overall security classification of the document including special warning terms if applicable.)
UNCLASSIFIED
3. TITLE (The complete document title as indicated on the title page. Its classification should be indicated by the appropriate abbreviation (S, C or U) in parentheses after the title.)
A training study of the Hercules Observer Trainer
4. AUTHORS (last name, followed by initials – ranks, titles, etc. not to be used)
Grant, S.C.
5. DATE OF PUBLICATION
(Month and year of publication of document.)
April 2009
7. DESCRIPTIVE NOTES (The category of the document, e.g. technical report, technical note or memorandum. If appropriate, enter the type of report, e.g. interim, progress, summary, annual or final. Give the inclusive dates when a specific reporting period is covered.)
Technical Report
8. SPONSORING ACTIVITY (The name of the department project office or laboratory sponsoring the research and development – include address.)
Defence R&D Canada – Toronto 1133 Sheppard Avenue West P.O. Box 2000
Toronto, Ontario M3M 3B9
9a. PROJECT OR GRANT NO. (If appropriate, the applicable research and development project or grant number under which the document was written. Please specify whether project or grant.)
20hd11; 13pu
9b. CONTRACT NO. (If appropriate, the applicable number under which the document was written.)
10a. ORIGINATOR'S DOCUMENT NUMBER (The official document number by which the document is identified by the originating activity. This number must be unique to this document.)
DRDC Toronto TR 2009-008
10b. OTHER DOCUMENT NO(s). (Any other numbers which may be assigned this document either by the originator or by the sponsor.)
11. DOCUMENT AVAILABILITY (Any limitations on further dissemination of the document, other than those imposed by security classification.)
Unlimited
12. DOCUMENT ANNOUNCEMENT (Any limitation to the bibliographic announcement of this document. This will normally correspond to the Document Availability (11). However, where further distribution (beyond the audience specified in (11) is possible, a wider announcement audience may be selected.))
Unlimited
13. ABSTRACT (A brief and factual summary of the document. It may also appear elsewhere in the body of the document itself. It is highly desirable that the abstract of classified documents be unclassified. Each paragraph of the abstract shall begin with an indication of the security classification of the information in the paragraph (unless the document itself is unclassified) represented as (S), (C), (R), or (U). It is not necessary to include here abstracts in both official languages unless the text is bilingual.)
The Canadian Forces (CF) operates aircraft in theatres where they are threatened by missiles and gunfire. Timely, coherent and accurate reporting of the threats is vital to defeating them. A training device called the Hercules Observer Trainer (HOT) was developed to support the training of surface-to-air threat reporting. To validate the simulation and determine its training effectiveness, an experiment employing 16 CF air crewmembers was conducted at CFB Trenton. Half of the experimental subjects were qualified tactical aircrew and were deemed experts at making threat calls. The other half consisted of students on a CC-130 Basic Loadmaster course who had not yet qualified as loadmasters on the CC-130 and were deemed novices. Both groups of subjects completed two sessions of making threat calls against 26 surface to air missile and 24 anti-aircraft artillery threats. In the first session, experts made more accurate and more syntactically correct threat calls than novices. By the end of the second session, both groups showed improved accuracy and syntax over their earlier levels of performance, and the performance of the novices was equivalent to that of the experts. It is concluded that the HOT is a valid simulation of the threat call task, that students can use the HOT to learn the task, and that the training is transferrable to the operational environment. It is recommended that HOT be considered for operational training in the CC-130 community. It is further recommended that the training scenarios be adapted and the technology improved to best represent the current operational environment.
Les Forces canadiennes (FC) utilisent des aéronefs qui, dans certains théâtres, sont menacés par des tirs de missiles et de l’artillerie. Des rapports rapides, cohérents et précis sur les menaces sont essentiels pour les contrer. Un appareil appelé observateur d’entraînement du Hercules (HOT) a été mis au point pour donner de la formation sur l’établissement de rapports et les façons d’intervenir en cas de menaces sol-air. Afin de valider l’appareil et d’établir l’efficacité de sa capacité de formation, une expérience a été réalisée à la BFC Trenton, à laquelle seize membres du personnel navigant des FC ont participé. La moitié des sujets de l’expérience faisaient partie de l’équipage aérien tactique et étaient considérés comme des spécialistes de la production de messages signalant une menace. L’autre moitié était composée de stagiaires du cours élémentaire d’arrimeur du CC-130. Ils n’avaient pas encore la qualification d’arrimeur et étaient considérés comme des novices. Les deux groupes de sujets ont participé à deux séances de production de messages signalant la menace de 26 missiles sol-air et de 24 tirs d’artillerie antiaérienne. Lors de la première séance, les spécialistes ont signalé les menaces de manière plus précise et plus exacte au plan syntaxique que les novices. À la fin de la deuxième séance, les deux groupes s’étaient améliorés et avait un rendement équivalent sur les deux aspects. On peut donc conclure que l’appareil HOT permet une simulation valide de la production de messages signalant une menace et que la formation peut être transférée dans un environnement opérationnel. On recommande donc l’appareil HOT pour l’instruction opérationnelle de la collectivité du CC-130. On recommande également l’adaptation des scénarios d’entraînement et l’amélioration de la technologie afin de mieux représenter le contexte opérationnel actuel.
14. KEYWORDS, DESCRIPTORS or IDENTIFIERS (Technically meaningful terms or short phrases that characterize a document and could be helpful in cataloguing the document. They should be selected so that no security classification is required. Identifiers, such as equipment model designation, trade name, military project code name, geographic location may also be included. If possible keywords should be selected from a published thesaurus, e.g. Thesaurus of Engineering and Scientific Terms (TEST) and that thesaurus identified. If it is not possible to select indexing terms which are Unclassified, the classification of each should be indicated as with the title.)
simulator; simulation; Hercules; CC-130; training
www.drdc-rddc.gc.ca
Defence R&D Canada
Canada’s Leader in Defence and National Security Science and Technology
R & D pour la défense Canada
Chef de file au Canada en matière de science et de technologie pour la défense et la sécurité nationale