4.1
Action Research
The research presented in this thesis is based on the development of low cost flight simulation devices, which were tested, evaluated, and modified in an iterative process. This type of research process or design is best described by an action research model. This section defines the principles of action research and the rationale for its use in this study. Action research is a well-established research method used in the social and medical sciences since the mid-twentieth century (McNiff & Whitehead, 2010). Kurt Lewin, the founder of action research, described action research as “a spiral of steps, each of which is composed of a circle of planning, action and fact finding about the result of the action” (Lewin, 1949).
In the 1990s, action research began to be used in scientific investigations of information systems. The method produced relevant research results, because it was grounded in practical action and aimed at solving an immediate problem situation. One of the key characteristics that distinguishes action research from most other research approaches is that action research aims at both improving the subject of the study (the research client), and generating knowledge, achieving both simultaneously at the same time (O'Brien, 2001). The domain of the action research method is characterised by a setting where:
1. the researcher is actively involved, with expected benefit for both researcher and organisation;
2. the knowledge obtained can be immediately applied, there is not the sense of the detached observer, but that of an active participant wishing to utilise any new knowledge based on an explicit, clear conceptual framework;
3. the research is a cyclical process linking theory and practice (Baskerville, 1999; Baskerville & Wood-Harper, 1996).
Some of the main characteristics of action research are that it is (McNiff & Whitehead, 2010):
1. practice based, where practice is both action and research;
2. about improving practice (both action and research) creating knowledge, and developing living theories of practice;
3. focused on improving learning and not just on improving behaviours;
4. about research and knowledge creation and is more than just professional practice;
5. collaborative and focuses on the co-creation of knowledge practices.
The most prevalent action research, details a five-phase cyclical process. The approach first requires the establishment of a client-system infrastructure or research environment (see Fig. 4-1). Then, five identifiable phases are iterated:
1. diagnosing; 2. action planning; 3. action taking; 4. evaluating;
5. specifying learning.
Figure 4-1: The Action Research Cycle (Facsimile) Source: (Baskerville, 1999) Investigating Information Systems with Action Research http://dl.acm.org/citation.cfm?id=374476
The client-system infrastructure is the specification that comprises the research environment. It provides the authority for researchers and practitioners to specify certain actions. The agreement may include the boundaries of the research domain, and the entry and exit of the researchers. It may also allow researchers to disseminate the learning that is gained in the research. The infrastructure should also define the responsibilities of the client and the researchers to one another (Baskerville, 1999). The diagnosing stage, where the cycle begins, involves the identification of an opportunity to improve a process or service or a general problem to be solved at the client organisation. The following stage, action planning, involves the consideration of alternative courses of action to achieve the specified improvement or solve the problem. The action taking stage involves the selection and implementation of one of the courses of action considered previously. The evaluating stage involves the study of possible outcomes of the selected course of action. Finally, the specifying learning stage involves reviewing the outcomes of the evaluating stage and, then utilising this knowledge to construct a model that describes the experimental situation (Koch, 2011).
Technological development has no value without action, and action research encompasses action. Human-computer interaction (HCI) technologies have had a significant impact on modern society and research on HCI has increased worldwide since the 1990s. This has included Internet and Web-based HCI technologies, and personal computer applications. In technology-related research, an action research study could involve the researcher introducing a new technology into an organisation, and simultaneously studying the effects of the technology in that organisation. The expansion of HCI research has coincided with the increasing use of action research in the study of technology-related issues (Koch, 2011).
Information systems prototyping (ISP) has been recently associated with action research despite an absence of theory in its development (Chiasson & Dexter, 2001). The ISP method is defined as an effective information system, and in involving the researcher in a collaborative and facilitative iterative, rigorous, and collaborative/facilitative method. It supports iterative cycles through a precise set of steps in developing process with participants. Depending on the particular ISP method used, these rigorous steps include an
iteration between risk analysis, prototype, software requirements, requirements validation, and further development plan (Chiasson & Dexter, 2001).
4.2
Simulation Design
The Action Research model lends itself to the design of simulation undertaken in this study as the simulation design process follows a similar five-cycle process (Wieringa, 2012). The steps are:
1. Problem investigation; 2. Treatment design; 3. Design validation;
4. Treatment implementation; 5. Implementation evaluation.
In addition, while completing this process of simulation design, the researcher plays three roles:
1. Designer- Designs a system or technique;
2. Helper - Uses the system or technique to help others;
3. Researcher- Draws lessons learned about system or technique.
Simulation is an essential component of aerospace research and design. Its ability to predict complex system behaviour makes it valuable for the analysis and testing of many entities, including vehicles, on-board components such as pilot-interactive systems such as cockpit displays, flight control systems, and flight procedures. Flight simulation can artificially recreate many of these entities combined with the various aspects of the flight environment (Ippolito & Pritchett, 2000). Simulation can fit into all stages of research and design. During basic research and conceptual design, low and medium fidelity simulations can highlight fundamental problems or issues and constraints on the system design. As the design progresses, higher-fidelity models can be added to the system so that its output provides detailed feedback for designers (Ippolito & Pritchett, 2000). Fritz, Gray, and Flanagan (2007) proposed that simulation designers have to consider three levels of
fidelity: environmental, equipment, and psychological. Environmental fidelity ensures that task content is realistic; equipment fidelity ensures that the selected hardware and software is similar to real life; and psychological fidelity ensures that the students have a sense of real immersion when participating in the simulation.
A central characteristic of current flight simulator design is that it incorporates technology standards and COTS hardware and software, which is cost effective (Elite, 2012d; PFC, 2004). Twenty years ago, flight simulators and desktop trainers were proprietary, expensive, and designed in-house with custom components and proprietary databases (Adams, 2008). Utilising COTS technology not only provides potentially high levels of fidelity but also lowers the cost of the system because of the availability of open source or standard software (Meyer, 2010). Today’s systems can replicate very realistic and complex weather patterns, vehicle and aircraft movements, and terrain due to commercially available tools and commercial standards (Mchale, 2009).
Nevertheless, some interface software is not be available as COTS and requires in- house development. The development of flight simulation software can take significant time and resources, to the extent that ‘rapid’ development has been described as that achievable in weeks or even months. In addition, if there is a lack of resources to develop a customised software package then common practice is to re-use already-existing flight simulation software. Existing components can then be modified, and existing flight simulators may have new components added to provide new functionality (Ippolito & Pritchett, 2000).
This thesis reports a survey and five action research cycles. The survey involved the collection of survey data to ascertain the current utilisation and future demand of PCATDs in NZ flight training organisations. The following five cycles comprised the development of five PCATD projects. The projects were characterised by the adoption of the action research philosophy that emphasised close collaboration with the host organisations who were involved with the projects. The development process was an iterative one whereby knowledge gained on each project was utilised in subsequent projects in a process of continuous improvement. Although these projects seem diverse, they had common
characteristics that linked them closely within the action research cycle, these common characteristics included:
1. High risk, as they combined newly released PC-based technologies together with untested software packages (both commercial and open-source) with uncertainty regarding the ultimate success of the project. In all cases, the projects required additional development of hardware and software interfaces that did not exist commercially or as open source. In addition, these interfaces had to be developed with software tools that were mostly beta versions and therefore lacked official support by their authors. This adoption of this difficult approach was driven by the need to constrain or reduce costs and to avoid the necessity of using expensive proprietary hardware and software.
2. All projects used a common software simulator platform, MSFS. Although, the projects did use different versions of the software depending on project requirements. These different versions were regularly upgraded as Microsoft released updates but retained compatibility with earlier versions.
3. Common hardware and software modules for flight controls and avionics were used in several projects.
4. In all projects, there was an emphasis on developing high-resolution visual displays of out- of- the- cockpit- views with a strong intent to improve visual fidelity.
5. All projects involved close collaboration with senior pilots, flight instructors, and pilot trainees. Evaluation and feedback was sought on the IFR/VFR task training effectiveness of the respective PCATDs. In addition, collaboration and assistance was sought on the development of a PCATD training programme, insertion into the training curriculum, and PCATD training documentation. 6. External validation in the form of NZ Civil Aviation Authority IFR/VFR
certification was achieved in two of the projects.
7. Internal validity was sought by an empirical comparative study of one of the PCATDs with a certified FTD to ascertain if the training effectiveness of the PCATD was similar to a certified FTD. In this case, the effect of PCATD training on performance improvement in VFR skills was investigated, due to
The projects included:
1. A Survey of NZ Flight Training Organisations in NZ was conducted to establish how many flight-training organisations in NZ were using Full Flight Simulators (FFs), Flight Training Devices (FTDs); Personal Computer based Aviation Training Devices (PCATDs) and Part Task Trainers (PTTs) in flight training programmes. The aim of the survey was to establish commercial and training opportunities in relation to this research and to establish if there was a demand for low cost PCATDs in NZ based flight training schools. A number of questions were presented to these organisations, to ascertain their current flight simulation inventory, how they were using these devices in their flight training organisation, their level of interest in acquiring an aviation training device, and their future intentions regarding these devices.
2. RNZAF Pilot Training Squadron (PTS) PCATD Project. These PCATDs were the first prototypes to be developed in NZ for ab-initio IFR/VFR skills training of military pilot trainees.
3. Auckland Rescue Helicopter Trust (ARHT) PCATD Project. This PCATD demonstrated the effectiveness of a low cost PCATD for visual flight rules and instrument flight rules helicopter flight training.
a. TracMap GPS Project Extension. This software/hardware interface was the first to be developed in the world as an extension of the ARHT PCATD project. It demonstrated the effectiveness of a low cost PCATD for training visual flight rules search and rescue procedures in conjunction with the TracMap Search and Rescue GPS (Aerial Survey & Search Pattern) unit.
4. Massey University School of Aviation SAV1 PCATD Project. These two PCATDs were the first to be developed in NZ that were designed specifically for VFR skills training, and incorporated multi-display technologies.
a. New Zealand Army Unmanned Aerial Systems (UAV/UAS) Operators Flight Training Project Extension. This project was an extension of the Massey University School of Aviation SAV1 PCATD project as it used an identical flight simulation system. The development of these multi- purpose PCATDs demonstrated the effectiveness of low cost PCATDs for solo rehearsal of VFR procedures towards PPL.
5. Massey University School of Aviation SAV2 PCATD Project. This PCATD demonstrated the effectiveness of low cost PCATD VFR training and used LCD multi-display technologies. Empirical research was conducted on this PCATD to compare its VFR task training effectiveness with a certified FTD.
6. Massey School of Aviation Diamond DA 40 PCATD Project. This PCATD demonstrated the effectiveness of low cost PCATDs for visual flight rules flight training and instrument flight rules training. Also this PCATD was the first device developed in NZ to simulate the Garmin 1000 glass cockpit flight deck combined coupled with a 2 DOF motion platform. This PCATD is also being utilised as a research vehicle for studies on general aviation glass-cockpit automation, scenario based flight training and simulator motion.
The purpose of this research was to evaluate the benefits and cost effectiveness of using a customised PCATD to improve pilot proficiency in performing VFR procedures. Five PCATDs were developed for use in pilot training programmes conducted by flying training organisations. These devices were developed as training aids to assist those organisations in improving the transfer of learning in flight training. This study focuses on the development of these PCATDs and in particular, an empirical study of the transfer of training effectiveness of a PCATD designed specifically for VFR procedural training. The cost of this VFR procedure PCATD represents only a fraction of the financial capital required to purchase a commercially available CAANZ certified FTD. Evidence of the effectiveness of the PCATD in pilot training was determined from comparative studies in two of the research cycles:
x A quantitative analysis on the relative effectiveness of a PCATD compared to a CAANZ certified FTD for improving pilot proficiency in the performance of a standard VFR traffic pattern operation
x An additional analysis was completed to compare the performance of a standard VFR traffic pattern operation by two groups of pilot trainees with different levels of aviation experience on the same PCATD. The level of proficiency required for the execution of these VFR manoeuvres was based on the performance standards outlined in the syllabus of training of the CAA AC61-3 Private Pilot Licenses (CAANZ, 2011e).
4.3
Research Questions
The thesis investigates three critical questions:
1. Can a low cost PCATD be as effective as a CAANZ certified FTD at improving pilot proficiency in the performance of a standard VFR traffic pattern operation?
2. Is there a significant difference in performance of a standard VFR traffic pattern operation on a low cost PCATD between pilots from two different flying training organisations and with different levels of aviation experience? 3. Can low cost PCATDs achieve the fidelity and conformity required for