T&E S UPPORT TO S OLUTION I MPLEMENTATION DT

T&E support to SI begins with the Development Test (DT) phase of the T&E program. The primary objective of DT is to demonstrate that all specified technical and performance requirements for a system or service have been met. Another objective is to verify that the system or service is fully integrated and stable, and that it has no adverse effect on the rest of the NAS. It may include one or more demonstrations to ensure that the system meets user and operational requirements. DT is performed by the development contractor and witnessed by the ANG-E DT Test Director and DT test team. DT activities may also be monitored by IOA personnel. These test activities can be conducted at the contractor’s facilities, the FAA William J. Hughes Technical Center (WJHTC), and/or FAA field sites.

This section of the handbook describes the processes necessary for DT planning, conduct, and reporting. Additionally, Appendix D identifies the major DT work products along with their associated review and approval cycles.

6.1 DT

O

This section of the handbook describes the processes and best practices to be followed to help ensure a successful DT, which may include:

a) Government or private laboratory testing, or

b) Site Acceptance Testing (SAT), whether conducted by Government personnel or a contractor under Government scrutiny.

DT encompasses all testing necessary to verify contractual requirements and is performed at FAA-approved contractor facilities, FAA laboratories, or FAA field sites. After DT, the FAA will accept the portion of the system demonstrated through a formal Government Acceptance (GA) process.

As shown in Figure 3-1, there are two types of GA: the first type pertains to the acceptance of systems at the WJHTC; the second type involves the acceptance of systems at field sites, as illustrated in Figure 6-2.

6.1.1 DTT

A

The DT process will differ from program to program, depending on size and complexity. It should progress from verifying requirements at the unit level (lowest component) and move incrementally to the fully integrated, fielded system level.

The following items define the standard test activities (which may be modified based on program needs) for DT:

a) DT Software Testing: Verification of the specifications at the B-Level (subsystem). DT Software Testing usually addresses new and modified software broken down to the Computer Software Configuration Item (CSCI), Computer Software Component (CSC), and functional design components.

b) DT Hardware Testing: Verification of the specifications at the B-Level. DT

Hardware Testing usually addresses new and modified hardware broken down to the Hardware Configuration Item (HWCI) and Hardware Design items.

c) Factory Acceptance Testing (FAT): Verification of primarily hardware, firmware, and Commercial Off-the-Shelf (COTS)/Non-Developmental Item (NDI) subsystem components to address B-Level and A-Level (system) specification items.

d) Functional Qualification Testing (FQT): Verification of partially integrated hardware and software subsystem components, including COTS/NDI and modified COTS/NDI subsystem components, to address B-Level and A-Level specification items.

e) DT Integration Testing: Verification of proper installation and functioning of the complete system in the laboratory environment, including the verification of system interfaces with other NAS equipment and Government Furnished Equipment (GFE). f) DT System Testing: Verification of integrated software and hardware components to

address A-Level specification items under conditions that emulate the projected operational conditions.

g) Production Acceptance Testing (PAT): Verification of production line units of developed hardware prior to installations at field sites to address B-Level and some A-Level specification items.

h) Site Acceptance Testing (SAT): Verification of fully-integrated software and

hardware components to address A-Level specification items that could not be tested sufficiently during DT System Testing. SAT also demonstrates that the requirements that were verified under DT System Testing continue to remain in conformance as installed at the operational field site by executing a subset of the DT System Tests. i) DT Regression Testing: Verification of integrated software and hardware

components after changes to either have been made based on anomalies discovered during previous DT test activities. DT Regression Testing also ensures that the changes made did not inadvertently result in a problem elsewhere in the system.

6.1.2 DTT

R

DT verification is based on contractually-required activities that the prime contractor must perform to demonstrate conformance to the FAA-developed system specifications. The prime contractor may develop and maintain separate specifications that are derived from the FAA specifications and approved by the FAA as a CDRL document. The FAA- developed system specifications or the contractor system specification (A-Level) is the “test to” document that drives the conduct of DT. The prime contractor and associated subcontractors may develop subsystem specifications (B-Level) and design documents that form the basis for B-Level verification and system level procedure development.

The DT Test Director and test team review the contractor’s test CDRL documentation and witness all tests during DT. The DT Test Director reviews the team’s comments and recommends Government approval or disapproval to the Contracting Officer (CO) and Contracting Officer’s Representative (COR). The DT test team must be proficient in their particular program domain (e.g., Communications, Navigation, Surveillance, Weather, and Air Traffic Automation) and eventually become experts on the system under test. System expertise is necessary to ensure that the tests performed by the system contractor are valid and

The DT functional flow diagram depicted in Figure 6-1, FAA DT Process Flow, identifies the tasks and functions that support DT from the Screening Information Request (SIR) through delivery of the DT Final Test Report. The tasks and functions that support SAT, from generation of the SAT Plan through the start of Field Familiarization (FF), are identified in Figure 6-2, SAT Process Flow.