Initial data for this research effort were requested in 2005 from the U.S. Navy.
Data requests included MCAS results, maintenance statistics, and mishap investigation report results. This information was necessary to define a methodology that the data would support, conduct o f further model development, and refinement from the conceptual phase to a computational model that could be executed and analyzed.
USN and USMC aviation squadrons conduct periodic surveys by maintainers and aircrew to assess safety issues within their command. Since results are not releasable outside o f the command, sanitized data were requested that did not provide identification o f specific squadrons. In order to validate model results, additional data were requested that would provide mishap report summaries that included:
• The summary o f mishap events
• The causal factors identified by the mishap investigation board and approved upon review by the chain o f command.
It was necessary that the mishap summaries could be associated with the respective squadrons' MCAS results from a recent previous survey.
After numerous unsuccessful attempts to obtain requested information, the Naval Safety Center (NSC) in Norfolk, Virginia was approached in 2008 as a repository o f both MCAS data and mishap reports. NSC functions to support the Naval Safety Program by providing guidance and direction, safety data and program services, and the marketing o f safety. N SC 's mission is to provide safety assistance
and advice to senior Navy and Marine Corps leadership in order to enhance the services" warfighting capability, preserve resources, and improve combat readiness by preventing mishaps and saving lives. [Naval Safety Center, 2010]
The NSC Safety Data Manager offered to sanitize both MCAS results and matching mishap reports and coordinated to release the double blind data. This was vetted by the CNAF Judge Advocate General and approved by Safety Officer contingent upon certification by the Safety Data Manager that all the provided
information was sanitized. An updated list o f detailed data requirements was requested from NSC. This is categorized below:
Communication with the Dissertation Faculty Advisor and NSC Safety Data Manager was undertaken to determine an acceptable number o f mishap report summaries to be used to achieve statistical significance. From Fiscal Year 2000 through 2009 there had been 2,300 surveys from 369 commands and the Navy/Marine Corps had experienced 1,696 Class A-C mishaps from 384 commands. A summary o f mishaps sorted by classification (A, B, C) and year is provided in Table 6.
Table 6. Naval Aviation Mishaps Class A-C for Fiscal Years 2000-2009 [Scott, 2009]
Fiscal Year Class A Class B Class C Total
2000 32 21 115 168
2001 21 16 114 151
2002 39 32 106 177
2003 40 29 138 207
2004 34 35 133 202
2005 26 28 131 185
2006 24 37 125 186
2007 19 39 118 176
2008 24 40 102 166
2009 8 21 49 78
Total 267 298 1,131 1,696
The Safety Data Manager requested that this research data inquiry be limited in scope by defining the following characteristics required: timeframe, severity, and/or airframe. In coordination with the Dissertation Faculty Advisor, the decision was made to limit the request for mishap summary reports to squadrons that incurred only Class A mishaps and to receive a minimum number o f 50 mishap summary reports that could be matched against corresponding MCAS results. Additional clarification was provided to limit mishap results to those squadrons that comprised aircraft carrier air wings, since they incurred similar types o f operations and tempos. The
Type/Model/Series included:
• E-2C • EA-6B • F-14 »C-2A
• F/A-18 • S-3B • SH-60 «HH-60
The Safety Data Manager and Customer Support Division Head at the Naval Safety Center took the MCAS data and removed identification containing each squadron's name (i.e., VAW-XXX) and replaced it with a unique 3 digit numeric code
to designate the specific squadron. This 3 digit code was matched against the mishap summary reports for Class A mishap incidents which occurred in corresponding squadrons. The squadron names in the mishap report summaries were replaced with the respective 3 digit code aligned to the squadron. The MCAS data were provided in a tabular spreadsheet and the mishap summary report was furnished in text format.
3.4.1.1. M CA S R E SU L TS DATA
The MCAS results were provided by the Naval Safety Center in tabular spreadsheet format for 2,300 aggregated sets o f survey data. 2,114 o f the unit survey results came from U.S. Navy and Marine Corps squadrons taking the survey, and 186 survey results were derived from organizations representing Aircraft Intermediate Maintenance Departments, Fleet Repair Centers, Naval and Marine Corps Air Stations, Marine Air Logistics Squadrons, Research Developmental Test and Evaluation outfits, and other entities. Each set o f results for a specific unit was assigned a unique 3 digit identification code which served to blind the true identity o f unit. Each set o f aggregated survey data included information contained within Table 7.
Table 7. MCAS Results Raw Data Terminology
Data Description
First Appear Serial Using the 3 digit squadron identification code, this indicated the first time within this total data presentation that a single
particular unit participated by taking the MCAS Squadron Appear
Serial
Using the 3 digit squadron identification code to provide a single set o f MCAS results from a participating unit
Number o f Respondents
Total number o f individuals who participated in the survey. This value ranged from a minimum o f 10 to a maximum o f 742. The mean number o f respondents for the 2,300 sets o f survey data was 118 with a standard deviation o f 73.76
Community Represented the type o f squadron or unit, (e.g., VAQ, VAW, etc.)
Squadron Service Either U.S. Navy or U.S. Marine Corps
First Survey Date The time and date the first respondent submitted a survey Mean Survey Date Represented a computation o f the mean time and date that the
survey was taken in the window between First Survey Date and Last Survey Date
Last Survey Date The time and date that the last respondent submitted a completed survey
Mean Likert Score Response to MCAS Question
Represented the computed mean value for the aggregated Likert Scores from the surveys in the Squadron Appear Serial
The text containing the 43 close-ended MCAS questions and quantitative Likert Scale values per response option are contained in Appendix D.
The mean time o f survey results covered the span from August 23, 2000 through January 6, 2009. The breakdown o f surveys by the communities that compose a Carrier Air Wings is provided in Table 8.
Table 8. Survey Breakdown by Community and Aircraft
Community Primary Mission Aircraft Surveys
HS/HSC Helicopter ASW/Combat SH-60F and HH-60H 159
VAQ Tactical Electronic Warfare EA-6B 138
VAW Airborne Early Warning E-2C 106
VF Fighter F-14B and F-14D 30
VFA Strike Fighter F/A-18A, F/A-18C, F/A-
18D, F/A-18E, and F/A-18F
421
VRC Logistics C-2A 18
VS Sea Control S-3B 57
Total 930
3.4.1.2. A V IA T IO N M ISH A P SU M M A R Y RE SU L TS
The Naval Safety Center provided a written summary o f the Class A mishaps that had occurred in Navy squadrons that comprised Carrier Air Wings for the period from October, 1, 2002 through July 20, 2009. Aviation mishap investigations were conducted in accordance with the Naval Aviation Safety Program Instruction (OPNAVINST 3750.6 series). Per instruction, a mishap investigation panel was convened by the Commanding Officer o f the reporting unit that was the custodian for the aircraft or aircrew involved. (For mishaps involving multiple squadrons, the senior Commanding Officer appointed the board members from involved units.) Upon completion o f the investigation and analysis, the mishap investigation report was reviewed by the Commanding Officer and was routed up through the chain-of- command hierarchy. Each sequential reviewing authority is empowered to make changes to board determinations and recommendations. The final approved version was archived at the Naval Safety Center. The summary provided as data for this research was taken from the final mishap investigation reports and contained the information provided in Table 9.
Table 9. Class A Mishap Summary Report Data Terminology
Data Description
Event Serial Number
5 digit serial number used to uniquely identify the mishap event Aircraft Type / Model / Series used to designate aircraft (e.g., E-2C)
Activity Name Corresponded to Squadron Appear Serial in Table 16. The unique 3-digit number used to indicate a single set o f MCAS results from a participating unit
Event Date Date o f mishap occurrence Event
Summary
Included the following information:
• Description o f incident, equipment damage/loss, and personnel injury/loss
• Narrative summary o f mishap
• List o f causal factors attributed to:
o Aircrew / Personnel o Material
o Supervisory o Facilities o Maintenance
The mishap summary data included 57 separate mishap events that involved 67 aircraft and their aircrew.