System Test Evaluation and Validation Final

System test,

System test,

Evaluation, and

Evaluation, and

Validation

Validation

By By:: Ocampo, Ocampo, Abbygail Abbygail Zabat, Jenilyn Zabat, Jenilyn

System test, Evaluation and

System test, Evaluation and

Validation

Validation

System test, evaluation, and validation activities should be established System test, evaluation, and validation activities should be established

during the conceptual design phase of the life cycle, concurrently ith during the conceptual design phase of the life cycle, concurrently ith the de!nition of

the de!nition of the overall system design the overall system design re"uirements# re"uirements# $$rom thatrom that point on, %he test and evaluation continues by testing of individual point on, %he test and evaluation continues by testing of individual components, the testing of various system elements and ma&or components, the testing of various system elements and ma&or

subsystems and then by the testing of overall system as an integrated subsystems and then by the testing of overall system as an integrated entity#

entity#

 %he ob&ective is to

 %he ob&ective is to adopt a 'progressive( approach that ill leadopt a 'progressive( approach that ill lend it self tond it self to continuous implementation and improvement as the system design continuous implementation and improvement as the system design and development process evolves#

and development process evolves#

 %

 %est and evaluation acest and evaluation activities in this chaptetivities in this chapter can be aligned r can be aligned initially ithinitially ith the design activities describe in chapter)*+ and then etended through the design activities describe in chapter)*+ and then etended through production-construction and system utili.ation#

 %he purpose of this chapter

is to:

• to present an integrated approach for system

test and evaluation

• to facilitate the necessary validation of the

proposed system con!guration and

• to provide assurance that ill indeed meet

System

/e"uirements

and Evaluation

/elationship

2ategories of System test

and Evaluation

• 3#4#5 Analytical and Simulation Evaluation

 %he utili.ation of analytical models, 2A0 methods, and various combinations of these can be e6ective in the

veri!cation of certain design relationships before system is !nali.ed#

for eample, the development of system through

computer graphics can illustrate the relationships among various system elements and accesses for maintenance and support# $rom this, potential interferences, space utili.ation con7icts can be detected#

2ategories of System test

and Evaluation

• %ype 5 %esting

0uring the early phase of detail design, breadboards, bench*test models, engineering models, engineering softare, and service test models are build ith the intent of verifying certain performance and physical design characteristics# %hese models, representing either an entire system or a designated system component, usually operate functionally 8electrically and mechanically9 but do not by any means represent production

e"uipment# n the development of softare, the application of rapid prototyping is sometimes used to verify design ade"uacy#

2ategories of System test

and Evaluation

• %ype 4 %esting

$ormal tests and demonstrations are accomplished during the latter part of detail design hen

preproduction prototype e"uipment, softare, and similar to production e"uipment 8that hich ill be deployed for operational use9, but it is not necessarily fully "uali!ed at this point in time# A test program

may constitute a series of individual tests tailored to the need# Such a program might include the folloing:

 %ype 4 test##

• 5# ;erformance tests are accomplished to verify individual

system performance characteristics# $or instance, tests are designed to determine hether the electric motor ill provide the necessary output, hether the airplane ill perform its intended mission successfully, and so on# Also, it is necessary to verify form, !t, interchangeability, product safety, and other comparable features#

• 4# Environmental "uali!cation < temperature cycling, shoc=

and vibration, humidity, ind, salt, spray, dust and sand,

fungus, acoustic noise, pollution emission, eplosion proo!ng, and electromagnetic interference tests are conducted# %hese factors are oriented to hat the various system elements ill be sub&ected to during operation, maintenance, and during transportation and handling function#

 %ype 4 test##

)# Structural tests are conducted to determine material

characteristics relative to stress, strain, fatigue, bending, torsion and general decomposition#

># /eliability "uali!cation are accomplished on one or more system elements to determine ?%B$# Also, special tests are often designed to measure component life, to

evaluate degradation, and to determine models of failure# +# ?aintainability demonstration < tests are conducted on

one or more system elements to assess the values for mean active maintenance time 8?9, mean corrective maintenance 8?ct9, mean preventive maintenance time 8?pt9, maintenance labor*hours per operating hour

 %ype 4 test##

3# Support e"uipment compatibility tests are often

accomplished to verify compatibility among the prime e"uipment, test and support e"uipment, and

transportation and handling e"uipment

# ;ersonnel test and evaluation are often accomplished to verify the relationships beteen people and

e"uipment, the personnel "uantities and s=ill levels re"uired and training needs, both operator and

maintenance tas=s are evaluated#

# %echnical data veri!cation < the veri!cation of operational and maintenance procedures is accomplished#

C# Softare veri!cation < the veri!cation of operational and maintenance softare is accomplished#

2ategories of System test

and Evaluation

• %ype ) %esting

$ormal tests and demonstrations, started after initial system "uali!cation and prior to the completion of production, are

accomplished at a designated !eld test site by user personnel, operation#

2ategories of System test

and Evaluation

• %ype > %esting

0uring the operational*use phase, formal tests are sometimes conducted to gain further insight in a speci!c area# t may be desirable to vary the

mission pro!le or the system utili.ation rate to

determine the impact on total system e6ectiveness, or it might be feasible to evaluate several alterative support policies to see hether system operational availability can be improved

;lanning for %esting and Evaluation

5#

 %he de!nition and schedule of all tests re"uirements, including anticipated test output for each individual test and

integrated here possible#

4#

 %he de!nition of organi.ation,

administration and control responsibilities#

)#

 %he de!nition of test conditions and maintenance and logistic resource re"uirements#

;lanning for %esting and

Evaluation

># A description of the test preparation phase for each type of testing method, training of testing#

+# A description of the formal test phase#

3# A description of condition and provisions for a retest phase#

;reparation for system test

and evaluation

• %he selection of the items to be tested, • Establishment of test procedures,

• %est site selection,

• Selection and training of test personnel,

• ;reparation of test facilities and resources, • Ac"uisition of support e"uipment,

 %est performance and

reporting

• %his re"uires operating and supporting the

system in a prescribed manner as de!ned in the system test and evaluation plan#

 %est performance and

reporting

• %he folloing "uestions ill be ansered:

5# hat is the true performance and e6ectiveness of the systemD

4# hat is the true e6ectiveness of the

maintenance and logistic support capabilityD )# are the initial speci!ed re"uirements being metD

 %est data re"uirements

5#

t provides ongoing data that are analy.ed to evaluate and assess the performance, e6ectiveness, operation, maintenance, logistic support capability, and so on#

4#

t provides historical data that are

applicable in the design and development of ne systems having a similar function and nature#

0evelopment of a data

subsystem

5#

Success data constitute information

covering system operation and use on a day*to*day basis#

4#

?aintenance data cover each event involving scheduled and unscheduled maintenance#

 %est reporting

5#

 %he !nal e6ort in the evaluation process,

4#

t should reference the initial system evaluation planning document,

)#

t should describe all test conditions,

incorporated system modi!cations during the test, test data, and the results of data analysis#

System

Evaluation

and

2orrective

Action 1oop