ASME 19.1 Test Uncertainty_05.pdf

(1)

A N A M E R I C A N N A T I O N A L S T A N D A R D

Test Uncertainty

ASME PTC 19.1-2005

(Revision of ASME PTC 19.1-1998)

Copyright ASME International

Provided by IHS under license with ASME Licensee=Mott MacDonald Ltd/5956936002 Not for Resale, 08/17/2010 06:03:02 MDT No reproduction or networking permitted without license from IHS

(2)

--`,,``,````````,`,,```,`,,-`-`,,`,,`,`,,`---This is Job # 000608 $$FM Electronic Page # 1 of 10 PL:PTCF Except 9999

(3)

--`,,``,````````,`,,```,`,,-`-`,,`,,`,`,,`---Date of Issuance: October 13, 2006

The 2005 edition of ASME PTC 19.1 will be revised when the Society approves the issuance of the next edition. There will be no Addenda issued to ASME PTC 19.1-2005.

ASME issues written replies to inquiries as code cases and interpretations of technical aspects of this document. Code cases and interpretations are published on the ASME website under the Committee Pages at http://www.asme.org/codes/ as they are issued.

ASME is the registered trademark of The American Society of Mechanical Engineers.

This code or standard was developed under procedures accredited as meeting the criteria for American National Standards. The Standards Committee that approved the code or standard was balanced to assure that individuals from competent and concerned interests have had an opportunity to participate. The proposed code or standard was made available for public review and comment that provides an opportunity for additional public input from industry, academia, regulatory agencies, and the public-at-large.

ASME does not “approve,” “rate,” or “endorse” any item, construction, proprietary device, or activity.

ASME does not take any position with respect to the validity of any patent rights asserted in connection with any items mentioned in this document, and does not undertake to insure anyone utilizing a standard against liability for infringement of any applicable letters patent, nor assumes any such liability. Users of a code or standard are expressly advised that determination of the validity of any such patent rights, and the risk of infringement of such rights, is entirely their own responsibility.

Participation by federal agency representative(s) or person(s) affiliated with industry is not to be interpreted as government or industry endorsement of this code or standard.

ASME accepts responsibility for only those interpretations of this document issued in accordance with the established ASME procedures and policies, which preclude the issuance of interpretations by individuals.

No part of this document may be reproduced in any form, in an electronic retrieval system or otherwise, without the prior written permission of the publisher.

The American Society of Mechanical Engineers Three Park Avenue, New York, NY 10016-5990

Printed in U.S.A.

This is Job # 000608 $$FM Electronic Page # 2 of 10 PL:PTCF 9996

(4)

--`,,``,````````,`,,```,`,,-`-`,,`,,`,`,,`---CONTENTS

Notice . . . vii Foreword . . . viii Committee Roster . . . ix Section 1 Introduction . . . . 1 1-1 General . . . 1

1-2 Harmonization With International Standards . . . 1

1-3 Applications . . . 1

Section 2 Object and Scope . . . . 2

2-1 Object. . . 2

2-2 Scope . . . 2

Section 3 Nomenclature and Glossary . . . . 3

3-1 Nomenclature . . . 3

3-2 Glossary. . . 3

Section 4 Fundamental Concepts . . . . 5

4-1 Assumptions . . . 5

4-2 Measurement Error . . . 5

4-3 Measurement Uncertainty . . . 5

4-4 Pretest and Posttest Uncertainty Analyses. . . 11

Section 5 Defining the Measurement Process . . . . 13

5-1 Overview . . . 13

5-2 Selection of the Appropriate “True Value” . . . 13

5-3 Identification of Error Sources . . . 13

5-4 Categorization of Uncertainties . . . 15

5-5 Comparative Versus Absolute Testing. . . 16

Section 6 Uncertainty of a Measurement. . . . 17

6-1 Random Standard Uncertainty of the Mean . . . 17

6-2 Systematic Standard Uncertainty of a Measurement . . . 18

6-3 Classification of Uncertainty Sources . . . 19

6-4 Combined Standard and Expanded Uncertainty of a Measurement . . . 19

Section 7 Uncertainty of a Result . . . . 22

7-1 Propagation of Measurement Uncertainties Into a Result. . . 22

7-2 Sensitivity . . . 23

7-3 Random Standard Uncertainty of a Result . . . 23

7-4 Systematic Standard Uncertainty of a Result. . . 24

7-5 Combined Standard Uncertainty and Expanded Uncertainty of a Result . . . 24

7-6 Examples of Uncertainty Propagation . . . 24

Section 8 Additional Uncertainty Considerations. . . . 28

8-1 Correlated Systematic Standard Uncertainties. . . 28

8-2 Nonsymmetric Systematic Uncertainty . . . 31

8-3 Fossilization of Calibrations . . . 35

8-4 Spatial Variation . . . 36

iii

This is Job # 000608 $$FM Electronic Page # 3 of 10 PL:PTCF Odd Page

(5)

--`,,``,````````,`,,```,`,,-`-`,,`,,`,`,,`---8-5 Analysis of Redundant Means . . . 36

8-6 Regression Uncertainty . . . 38

Section 9 Step-by-Step Calculation Procedure. . . . 41

9-1 General Considerations. . . 41

9-2 Calculation Procedure. . . 41

Section 10 Examples . . . . 43

10-1 Flow Measurement Using Pitot Tubes . . . 43

10-2 Flow Rate Uncertainty . . . 47

10-3 Flow Rate Uncertainty Including Nonsymmetrical Systematic Standard Uncertainty . . . 50

10-4 Compressor Performance Uncertainty . . . 51

10-5 Periodic Comparative Testing . . . 62

Section 11 References . . . . 69

Section 12 Bibliography. . . . 71

Figures 4-2-1 Illustration of Measurement Errors . . . 6

4-2-2 Measurement Error Components . . . 7

4-3.1 Distribution of Measured Values (Normal Distribution) . . . 8

4-3.3 Uncertainty Interval . . . 11

5-3.1 Generic Measurement Calibration Hierarchy . . . 14

5-4.3 Difference Between “Within” and “Between” Sources of Data Scatter . . . 16

7-6.2 Pareto Chart of Systematic and Random Uncertainty Component Contributions to Combined Standard Uncertainty . . . 27

8-2.1 Schematic Relation Between Parameters Characterizing Nonsymmetric Uncertainty. . . 32

8-2.2 Relation Between Parameters Characterizing Nonsymmetric Uncertainty . . . 34

8-5.1 Three Posttest Cases . . . 37

10-1.1 Traverse Points (Example 10-1) . . . 44

10-2.1 Schematic of a 6 in. ⴛ 4 in. Venturi. . . 48

10-4.1 Typical Pressure and Temperature Locations for Compressor Efficiency Determination. . . 57

10-4.7 The h-s Diagram of the Actual and Isentropic Processes of an Adiabatic Compressor. . . 61

10-5.1-1 Installed Arrangement . . . 63

10-5.1-2 Pump Design Curve With Factory and Field Test Data Shown . . . 64

10-5.1-3 Comparison of Test Results With Independent Control Conditions . 64 10-5.2 Comparison of Test Results Using the Initial Field Test as the Control . . . 67

Tables 6-4-1 Circulating Water Bath Temperature Measurements (Example 6-4.1) . . . 20

6-4-2 Systematic Uncertainty of Average Circulating Water Bath Temperature Measurements (Example 6-4.1) . . . 21

7-6.1-1 Table of Data (Example 7-6.1) . . . 25

7-6.1-2 Summary of Data (Example 7-6.1) . . . 26

7-6.2-1 Table of Data (Example 7-6.2) . . . 26

7-6.2-2 Summary of Data (Example 7-6.2) . . . 27

8-1 Burst Pressures (Example 8-1-1) . . . 29

iv

This is Job # 000608 $$FM Electronic Page # 4 of 10 PL:PTCF Even Page

(6)

--`,,``,````````,`,,```,`,,-`-`,,`,,`,`,,`---8-6.4.5 Systematic Standard Uncertainty Components for Yˆ Determined

from Regression Equation. . . 40

9-2-1 Table of Data. . . 42

9-2-2 Summary of Data . . . 42

10-1.2 Average Values (Example 10-1) . . . 44

10-1.3-1 Standard Deviations (Example 10-1) . . . 45

10-1.3-2 Summary of Average Velocity Calculation (Example 10-1) . . . 45

10-1.6 Standard Deviation of Average Velocity (Example 10-1) . . . 46

10-1.9 Uncertainty of Result (Example 10-1) . . . 48

10-2.1-1 Uncalibrated Case (Example 10-2) . . . 48

10-2.1-2 Absolute Sensitivity Coefficients in Example 10-2 (Calculated Numerically) . . . 50

10-2.1-3 Absolute Sensitivity Coefficients in Example 10-2 (Calculated Analytically) . . . 51

10-2.1-4 Absolute Contributions of Uncertainties of Independent Parameters (Example 10-2: Uncalibrated Case) . . . 52

10-2.1-5 Summary: Uncertainties in Absolute Terms (Example 10-2: Uncalibrated Case) . . . 52

10-2.1.1-1 Relative Uncertainty of Measurement (Example 10-2: Uncalibrated Case) . . . 52

10-2.1.1-2 Relative Contributions of Uncertainties of Independent Parameters (Example 10-2: Uncalibrated Case) . . . 53

10-2.1.1-3 Summary: Uncertainties in Relative Terms for the Uncalibrated Case . . . 53

10-2.1.1-4 Relative Uncertainties of Independent Parameters (Example 10-2: Calibrated Case) . . . 53

10-2.1.1-5 Relative Contributions of Uncertainties of Independent Parameters (Example 10-2: Calibrated Case) . . . 54

10-2.1.1-6 Summary: Uncertainties in Relative Terms for the Calibrated Case . . . 54

10-2.1.1-7 Summary: Comparison Between Calibrated and Uncalibrated Cases . . . 54

10-3-1 Absolute Contributions of Uncertainties of Independent Parameters (Example 10-3: Uncalibrated, Nonsymmetrical Systematic Uncertainty Case) . . . 55

10-3-2 Summary: Uncertainties in Absolute Terms (Example 10-3: Uncalibrated, Nonsymmetrical Systematic Uncertainty Case) . . . . 56

10-4.1-1 Elemental Random Standard Uncertainties Associated With Error Sources Identified in Para. 10-4.2 . . . 56

10-4.1-2 Independent Parameters . . . 57

10-4.1-3 Calculated Result . . . 57

10-4.3.2-1 Inlet and Exit Pressure Elemental Systematic Standard Uncertainties . . . 58

10-4.3.2-2 Inlet and Exit Temperature Elemental Systematic Standard Uncertainties . . . 59

10-4.7 Evaluation of Analysis Error . . . 62

10-5.1-1 Pump Design Data (Tcp20°C) . . . 63

10-5.1-2 Summary of Test Results . . . 63

10-5.2-1 Uncertainty Propagation for Comparison With Independent Control . . . 66

10-5.2-2 Summary: Uncertainties in Absolute Terms . . . 66

10-5.2-3 Summary of Results for Each Test . . . 66

v

(7)

--`,,``,````````,`,,```,`,,-`-`,,`,,`,`,,`---10-5.3-1 Uncertainty Propagation for Comparative Uncertainty . . . 68

10-5.3-2 Sensitivity Coefficient Estimates for Comparative Analysis . . . 68

Nonmandatory Appendices A Statistical Considerations . . . 73

B Uncertainty Analysis Models . . . 84

C Propagation of Uncertainty Through Taylor Series . . . 87

D The Central Limit Theorem . . . 92

vi

(8)

--`,,``,````````,`,,```,`,,-`-`,,`,,`,`,,`---NOTICE

All Performance Test Codes must adhere to the requirements of ASME PTC 1, General Instructions. The following information is based on that document and is included here for emphasis and for the convenience of the user of the Supplement. It is expected that the Code user is fully cognizant of Sections 1 and 3 of ASME PTC 1 and has read them prior to applying this Supplement.

ASME Performance Test Codes provide test procedures which yield results of the highest level of accuracy consistent with the best engineering knowledge and practice currently available. They were developed by balanced committees representing all concerned interests and specify procedures, instrumentation, equipment-operating requirements, calculation methods, and uncertainty analysis.

When tests are run in accordance with a Code, the test results themselves, without adjustment for uncertainty, yield the best available indication of the actual performance of the tested equipment. ASME Performance Test Codes do not specify means to compare those results to contractual guarantees. Therefore, it is recommended that the parties to a commercial test agree before starting the test and preferably before signing the contract on the method to be used for comparing the test results to the contractual guarantees. It is beyond the scope of any Code to determine or interpret how such comparisons shall be made.

vii

(9)

--`,,``,````````,`,,```,`,,-`-`,,`,,`,`,,`---FOREWORD

In March 1979 the Performance Test Codes Supervisory Committee activated the PTC 19.1 Committee to revise a 1969 draft of a document entitled PTC 19.1 “General Considerations.” The PTC 19.1 Committee proceeded to develop a Performance Test Code Instruments and Apparatus Supplement which was published in 1985 as PTC 19.1-1985, “Measurement Uncer-tainty,” and which was intended—along with its subsequent editions—to provide a means of eventual standardization of nomenclature, symbols, and methodology of measurement uncertainty in ASME Performance Test Codes.

Work on the revision of the original 1985 edition began in 1991. The two-fold objective was to improve the usefulness to the reader regarding clarity, conciseness, and technical treatment of the evolving subject matter, as well as harmonization with the ISO “Guide to the Expression of Uncertainty in Measurement.” That revision was published as PTC 19.1-1998, “Test Uncertainty,” the new title reflecting the appropriate orientation of the document. The effort to update the 1998 revision began immediately upon completion of that docu-ment. This 2005 revision is notable for the following significant departures from the 1998 text:

(a) Nomenclature adopted for this revision is more consistent with the ISO Guide.

Uncer-tainties remain conceptualized as “systematic” (estimate of the effects of fixed error not observed in the data), and “random” (estimate of the limits of the error observed from the scatter of the test data). The new aspect is that both types of uncertainty are defined at the standard-deviation level as “standard uncertainties.” The determination of an uncertainty at some level of confidence is based on the root-sum-square of the systematic and random standard uncertainties multiplied times the appropriate expansion factor for the desired level of confidence (usually “2” for 95%). This same approach was used in the 1998 revision but the characterization of uncertainties at the standard-uncertainty level (“standard devia-tion”) was not as explicitly stated. The new nomenclature is expected to render PTC 19.1-2005 more acceptable at the international level.

(b) There is greater discussion of the determination of systematic uncertainties.

(c) There is new text on a simplified approach to determine the uncertainty of

straight-line regression.

ASME PTC 19.1-2005 was approved by the PTC Standards Committee on September 13, 2005, and was approved as an American National Standard by the ANSI Board of Standards Review on November 3, 2005.

viii

(10)

--`,,``,````````,`,,```,`,,-`-`,,`,,`,`,,`---PERFORMANCE TEST CODE COMMITTEE

19.1 ON TEST UNCERTAINTY

(The following is the roster of the Committee at the time of the approval of this Supplement.)

OFFICERS

R. H. Dieck, Chair W. G. Steele, Vice Chair

G. Osolsobe, Secretary

COMMITTEE PERSONNEL J. F. Bernardin, Pratt & Whitney

D. A. Coutts, WSMS

R. H. Dieck, Ron Dieck Associates, Inc. R. S. Figliola, Clemson University H. K. Iyer, Colorado State University J. Maveety, Intel Corp.

J. A. Rabensteine, Environmental Systems Corp. M. Soltani, Bechtel National Corp.

W. G. Steele, Mississippi State University

ix

(11)

--`,,``,````````,`,,```,`,,-`-`,,`,,`,`,,`---PERFORMANCE TEST CODES STANDARDS COMMITTEE

OFFICERS

J. G. Yost, Chair J. R. Friedman, Vice Chair S. D. Weinman, Secretary COMMITTEE PERSONNEL P. G. Albert P. M. McHale R. P. Allen M. P. McHale J. M. Burns J. W. Milton W. C. Campbell S. P. Nuspl M. J. Dooley A. L. Plumley A. J. Egli R. R. Priestley J. R. Friedman J. A. Rabensteine G. J. Gerber J. W. Siegmund P. M. Gerhart J. A. Silvaggio T. C. Heil W. G. Steele R. A. Johnson J. C. Westcott D. R. Keyser W. C. Wood S. J. Korellis J. G. Yost HONORARY MEMBERS W. O. Hays F. H. Light MEMBERS EMERITI R. L. Bannister G. H. Mittendorf R. Jorgensen R. E. Sommerlad x

(12)

--`,,``,````````,`,,```,`,,-`-`,,`,,`,`,,`---TEST UNCERTAINTY ASME PTC 19.1-2005

Section 1 Introduction

1-1 GENERAL

This Supplement has significant additions and Sections that have been rewritten to both add to the available technology for uncertainty analysis and to make it easier for the practicing engineer. Throughout, the intent is to provide a Supplement that can be utilized easily by engineers and scien-tists whose interest is the objective assessment of data quality, using test uncertainty analysis.

1-2 HARMONIZATION WITH INTERNATIONAL STANDARDS

It is recognized that this Supplement and prom-ulgated international uncertainty standards and/ or guides must be in harmony. In rewriting this Supplement, great care was taken to assure contin-ued harmony with the International Organization for Standardization (ISO) Guide to the Expression

of Uncertainty in Measurement (GUM) [1]. For the

practicing engineer, this harmonization means the elimination of such ambiguous terms as bias, preci-sion, bias limit, and precision index. In addition, careful attention was paid to discriminating be-tween errors, the effects of errors, and the estima-tion of their limits, which is the uncertainty.

The term “bias” is not used in this Supplement. Instead, the combined terms of “systematic error” and “systematic uncertainty” are used. The former describes an error source whose effect is systematic or constant for the duration of a test. The latter describes the limits to which a systematic error may be expected to go with some confidence.

The term “precision” also is not used in this Supplement. Instead the combined terms of “ran-dom error” and “ran“ran-dom uncertainty” are used. The former describes an error source that causes scatter in test data. The latter describes the limits to which a random error may be expected to reach with some confidence.

Throughout the Supplement, the term “stan-dard” uncertainty has been introduced to improve

1

This is Job # 000608 $$U5 Electronic Page # 1 of 1 PL: PTCT Start Odd

harmony with international guidelines and stan-dards. In this Supplement, “standard” uncertainties are always equivalent to a single standard devia-tion of the average.

The most common confidence level used in this Supplement is 95% although methods for employing alternate confidences are also given. The confidence level of 95% is applied to “expanded” uncertainty. This term, too, was included in this Supplement for improved harmony with interna-tional guidelines and standards.

While this Supplement is in harmony with the ISO GUM, this Supplement emphasizes the effects of errors rather than the basis of the information utilized in the estimation of their limits. The ISO GUM utilizes two major classifications for errors and uncertainties. They are “Type A” and “Type B.” Type A uncertainties have data with which to calculate a standard deviation. Type B uncertain-ties do not have data to calculate a standard deviation and must be estimated by other means. This Supplement utilizes two major classifica-tions for errors and uncertainties. They are “sys-tematic” and “random.” Random errors (whose effects are estimated with “Random Standard Un-certainties”) cause scatter in test data. Systematic errors (whose effects are estimated with “System-atic Standard Uncertainties”) do not.

Harmonization of this Supplement with the ISO GUM is achieved by encouraging subscripts with each uncertainty estimate to denote the ISO Type, i.e., using subscripts of either “A” or “B.”

1-3 APPLICATIONS

This Supplement is intended to serve as a refer-ence to the various other ASME Instruments and Apparatus Supplements (PTC 19 Series) and to ASME Performance Test Codes and Standards in general. In addition, it is applicable for all known measurement and test uncertainty analyses.

The paramater values and uncertainty levels used throughout the examples are for illustrative purposes only and are not intended to be typical of standard tests.

(13)

--`,,``,````````,`,,```,`,,-`-`,,`,,`,`,,`---ASME PTC 19.1-2005 TEST UNCERTAINTY

Section 2 Object and Scope

2-1 OBJECT

The object of this Supplement is to define, de-scribe, and illustrate the various terms and meth-ods used to provide meaningful estimates of the uncertainty in test parameters and methods, and the effects of those uncertainties on derived test results.

Analysis of test measurement and result uncer-tainty is useful because it

(a) facilitates communication regarding

measure-ment and test results;

(b) fosters an understanding of potential error

sources in a measurement system and the effects of those potential error sources on test results;

(c) guides the decision-making process for

select-ing appropriate and cost-effective measurement sys-tems and methodologies;

(d) reduces the risk of making erroneous

deci-sions; and

2

This is Job # 000608 $U10 Electronic Page # 2 of 1 PL: PTCT Start Even

(e) documents uncertainty for assessing

compli-ance with agreements.

2-2 SCOPE

The scope of this Supplement is to specify proce-dures for evaluation of uncertainties in test parame-ters and methods, and for propagation of those uncertainties into the uncertainty of a test result. Depending on the application, uncertainty sources may be classified either by the presumed effect (systematic or random) on the measurement or test result, or by the process in which they may be quantified (Type A or Type B). The various statistical terms involved are defined in the No-menclature (subsection 3-1) or Glossary (subsection 3-2).

The end result of an uncertainty analysis is a numerical estimate of the test uncertainty with an appropriate confidence level.

(14)

Section 3 Nomenclature and Glossary

3-1 NOMENCLATURE

bXpsystematic standard uncertainty

com-ponent of a parameter

bX_kpsystematic standard uncertainty

asso-ciated with the kth elemental error

source

bRpsystematic standard uncertainty

com-ponent of a result

bXYpcovariance of the systematic errors

in X and Y

b+, b−pupper and lower values of

nonsym-metrical systematic standard uncer-tainty

Np number of measurements or sample

points or observations available

(sample size)

Rp result

sRprandom standard uncertainty of a

result

sXpstandard deviation of a data sample;

estimate of the standard deviation of

the population ␴x

sXprandom standard uncertainty of the

mean of N measurements

SEEp standard error of estimate of a least-squares regression or curve fit

tp Student’s t value at a specified

confi-dence level with ␯ degrees of

free-dom, i.e., t95,␯

up combined standard uncertainty Up expanded uncertainty

U+_{, U}−

pupper and lower values of the

non-symmetrical expanded uncertainty

Xp individual observation in a data

sam-ple of a parameter

Xp sample mean; average of a set of N

individual observations of a pa-rameter

␤p(unknown) true systematic error;

fixed or constant component of ␦

3

This is Job # 000608 $U15 Electronic Page # 3 of 2 PL: PTCT Start Odd

␦p(unknown) total error; difference

be-tween the assigned value of a param-eter or a test result and the true value

⑀p(unknown) true random error;

ran-dom component of ␦

␪pabsolute sensitivity

␪′p relative sensitivity

␮p(unknown) true average of a

popu-lation

␯pnumber of degrees of freedom

␴p(unknown) true standard deviation

of a population

␴2

p(unknown) true variance of a

popu-lation

Indices

Ip total number of variables ip counter for variables

jp counter for individual measurements Kp total number of sources of elemental

errors and uncertainties

kp counter for sources of elemental

er-rors and uncertainties

Lp total number of correlated sources

of systematic error

lp counter for correlated sources of

sys-tematic error

Mp total number of multiple results mp counter for multiple results Np total number of measurements

3-2 GLOSSARY

calibration hierarchy: the chain of calibrations that

links or traces a measuring instrument to a primary standard.

calibration: the process of comparing the response

of an instrument to a standard instrument over some measurement range.

confidence level: the probability that the true value

falls within the specified limits.

(15)

degrees of freedom (␯): the number of independent

observations used to calculate a standard deviation.

elemental random error source: an identifiable source

of random error that is a subcomponent of total random error.

elemental random standard uncertainty (sX_k): an

esti-mate of the standard deviation of the mean of an elemental random error source.

elemental systematic error source: an identifiable

source of systematic error that is a subcomponent of the total systematic error.

elemental systematic standard uncertainty (bXk): an

estimate of standard deviation of an elemental systematic error source.

expanded uncertainty (UXor UR): an estimate of the

plus-or-minus limits of total error, with a defined level of confidence, (usually 95%).

influence coefficient: see sensitivity.

mean (X): the arithmetic average of N readings. parameter: quantity that could be measured or taken from best available information, such as temperature, pressure, stress, or specific heat, used in determining a result. The value used is called the assigned value.

population mean (␮): average of the set of all population values of a parameter.

population standard deviation (␴): a value that

quanti-fies the dispersion of a population.

population: the set of all possible values of a

pa-rameter.

random error (⑀): the portion of total error that

varies randomly in repeated measurements of the true value throughout a test process.

random standard uncertainty of the sample mean (sX): a value that quantifies the dispersion of a

sample mean as given by eq. (4-3.3).

4

This is Job # 000608 $U15 Electronic Page # 4 of 2 PL: PTCT Even Page

result (R): a value calculated from a number of

parameters.

sample size (N): the number of individual values

in a sample.

sample standard deviation (sx): a value that quantifies

the dispersion of a sample of measurements as given by eq. (4-3.2).

sensitivity: the instantaneous rate of the change in a result due to a change in a parameter.

standard error of estimate (SEE): the measure of

dispersion of the dependent variable about a least squares regression or curve.

statistic: any numerical quantity derived from the

sample data. X and sX are statistics.

Student’s t: a value used to estimate the uncertainty

for a given confidence level.

systematic error (␤): the portion of total error that

remains constant in repeated measurements of the true value throughout a test process.

systematic standard uncertainty (bX): a value that

quantifies the dispersion of a systematic error associated with the mean.

total error (␦): the true, unknown difference between

the assigned value of a parameter or test result and the true value.

traceability: see calibration hierarchy.

true value: the error-free value of a parameter or

test result.

Type A uncertainty: uncertainties are classified as

Type A when data is used to calculate a standard deviation for use in estimating the uncertainty.

Type B uncertainty: uncertainties are classified as

Type B when data is not used to calculate a standard deviation, requiring the uncertainty to be estimated by other methods.

uncertainty interval: an interval expressed about a

parameter or test result that is expected to contain the true value with a prescribed level of confidence.

(16)

Section 4 Fundamental Concepts

4-1 ASSUMPTIONS

The assumptions inherent in test uncertainty analysis include the following:

(a) The test objectives are specified.

(b) The test process, including the measurement

process and the data reduction process, is defined.

(c) The test process, with respect to the conditions

of the item under test and the measurement system employed for the test, is controlled for the duration of the test.

(d) The measurement system is calibrated and all

appropriate calibration corrections are applied to the resulting test data.

(e) All appropriate engineering corrections are

applied to the test data as part of the data reduction and/or results analysis process.

For expanded uncertainty, 95% confidence levels have been used throughout this document in accor-dance with accepted practice. Other confidence levels may be used, if required. (See Nonmanda-tory Appendix B.)

4-2 MEASUREMENT ERROR

Every measurement has error, which results in a difference between the measured value, X, and the true value. The difference between the mea-sured value and the true value is the total error,

␦. Since the true value is unknown, total error

cannot be known and therefore only its expected limits can be estimated. Total error consists of two components: random error and systematic error (see Fig. 4-2-1). Accurate measurement requires minimizing both random and systematic errors (see Fig. 4-2-2).

4-2.1 Random Error

Random error,⑀, is the portion of the total error

that varies randomly in repeated measurements throughout the conduct of a test. The total random error in a measurement is usually the sum of the

5

contributions of several elemental random error sources. Elemental random errors may arise from uncontrolled test conditions and nonrepeatabilities in the measurement system, measurement meth-ods, environmental conditions, data reduction tech-niques, etc.

4-2.2 Systematic Error

Systematic error, ␤, is the portion of the total

error that remains constant in repeated measure-ments throughout the conduct of a test. The total systematic error in a measurement is usually the sum of the contributions of several elemental sys-tematic errors. Elemental syssys-tematic errors may arise from imperfect calibration corrections, mea-surement methods, data reduction techniques, etc.

4-3 MEASUREMENT UNCERTAINTY

There is an inherent uncertainty in the use of measurements to represent the true value. The total uncertainty in a measurement is the combination of uncertainty due to random error and uncertainty due to systematic error.

4-3.1 Random Standard Uncertainty

Any single measurement of a parameter is influ-enced by several different elemental random error sources. In successive measurements of the param-eter, the values of these elemental random error sources change resulting in the random scatter evident in the successive measurements. If an infinite number of measurements of a parameter were to be taken following the defined test process, the resulting population of measurements could be described statistically in terms of the population

mean, ␮, the population standard deviation, ␴,

and the frequency distribution of the population. These terms are illustrated in Fig. 4-3.1 for a population of measurements that is normally dis-tributed. For measurements with zero systematic

(17)

Fig. 4-2-1 Illustration of Measurement Errors

error (refer to para. 4-2.2), the population mean is equal to the true value of the parameter being measured and the population standard deviation is a measure of the scatter of the individual mea-surements about the population mean. For a

nor-mal distribution, the interval ␮ ± ␴ will include

approximately 68% of the population and the

inter-val ␮ ± 2␴ will include approximately 95% of the

population.

Since only a finite number of measurements are acquired during a test, the true population mean and population standard deviation are unknown but can be estimated from sample statistics. The sample mean, X, is given by

X p

兺

N

jp1Xj

N (4-3.1)

where Xj represents the value of each individual

measurement in the sample and N is the number of measurements in the sample. The sample standard

deviation, sX, is given by

6

sXp

冪

兺

N

jp1

(Xj− X)2

N − 1 (4-3.2)

Since the sample mean is only an estimate of the population mean, there is an inherent error in the use of the sample mean to estimate the population mean. For a defined frequency distribu-tion, the random standard uncertainty of the

sam-ple mean, sX, can be used to define the probable

interval about the sample mean that is expected to contain the population mean with a defined level of confidence. The random standard uncertainty of the sample mean is related to the sample standard deviation as follows:

sXp sX

冪

N (4-3.3)

For a normally distributed population and a

large sample size (N > 30), the interval X ± sX is

expected to contain the true population mean with

68% confidence and the interval X ± 2sXis expected

to contain the true population mean with 95% confidence [where the value 2 represents the Stu-dent’s t value for 95% confidence and degrees of

(18)

Fig. 4-2-2 Measurement Error Components

freedom of greater than or equal to 30 where the degrees freedom for the random standard uncertainty is N−1 (see subsection 6-1)].

In general, increasing the number of measure-ments collected during a test and used in the preceding formulas is beneficial as

(a) it improves the sample mean as an estimator

of the true population mean;

(b) it improves the sample standard deviation as

an estimator of the true population standard devia-tion; and

(c) it typically reduces the value of the random

standard uncertainty of the sample mean.

4-3.2 Systematic Standard Uncertainty

Every measurement of a parameter is influenced by several different elemental systematic error sources. Each of these elemental systematic error

7

This is Job # 000608 $U20 Electronic Page # 7 of 8 PL: PTCT Odd Page

sources contributes a constant, but unknown,

er-ror, ␤Xk, to the successive measurements of a

parameter for the duration of the test (the subscript

k is used to denote a specific elemental error

source). As ␤Xk is constant for the test, the error

imparted to the average value of successive mea-surements, X [as given by eq. 4-3.1], is equivalent to the error imparted to each individual

measure-ment. While␤Xkis unknown, it may be postulated

to come from a population of possible error values from which a single sample (error value) is drawn and imparted to the average measurement for the test. Knowledge of the frequency distribution and standard deviation of this population permits de-scribing the uncertainty in X due to this single sample elemental systematic error in terms of a confidence interval. The elemental systematic

standard uncertainty, bXk, is defined as a value

(19)

Fig. 4-3.1 Distribution of Measured Values (Normal Distribution)

that quantifies the dispersion of the population of

possible␤X_kvalues at the standard deviation level.

All of the elemental systematic errors associated with a measurement combine to yield the total

systematic error in the measurement, ␤X. As with

elemental systematic error, total systematic error is constant, unknown, and may be postulated to come from a population of possible error values from which a single sample (error value) is drawn and imparted to the average measurement for the

test. Total systematic standard uncertainty, bX, is

defined as a value that quantifies the dispersion

of the population of possible ␤X values at the

standard deviation level. Typically, total systematic standard uncertainty is quantified by

(a) identifying all elemental sources of systematic

error for the measurement;

(b) evaluating elemental systematic standard

un-certainties as the standard deviations of the possible systematic error distributions; and

(c) combining the elemental systematic standard

uncertainties into an estimate of the total systematic standard uncertainty for the average measurement.

4-3.2.1 Identifying Elemental Sources of System-atic Error. Attempting to identify all of the

elemen-tal sources of systematic error for a measurement is an important step of an uncertainty analysis, as failure to identify any significant source of systematic error will lead to an underestimation

8

of test uncertainty. Attempting to identify all ele-mental sources of systematic error requires a thor-ough understanding of the test objectives and test process. For further discussion refer to subsection 5-4.

4-3.2.2 Evaluating Elemental Systematic Stan-dard Uncertainties. Once all elemental sources of

systematic error are identified, elemental system-atic standard uncertainties for each source are evaluated. By definition, an elemental systematic standard uncertainty is a value that quantifies the

dispersion of the population of possible␤Xkvalues

at the standard deviation level. As ␤Xk is both

constant and unknown during a test, successive measurements of a parameter do not provide suffi-cient data for direct computation of a standard deviation as described in para. 4-3.1. Therefore, the evaluation of an elemental systematic standard uncertainty requires that a standard deviation be evaluated from engineering judgment, published information, or special data.

4-3.2.2.1 Engineering Judgment. When

nei-ther published information or special data is avail-able, it is often necessary to rely upon engineering judgment to quantify the dispersion of errors asso-ciated with an elemental error source. In these situations, it is customary to use engineering analy-ses and experience to estimate the limits of the

(20)

elemental systematic error at 95% confidence. In other words, an interval is estimated which is expected to contain 95% of the population of

possible␤Xkvalues. Not withstanding information

to the contrary, the analyst typically assumes that

the population of possible ␤Xk values is normally

distributed, that the estimation of the limits of the error is based upon large degrees of freedom, and that the limits of error are symmetric (equally spread in both the positive and negative direc-tions). Based upon these assumptions, the elemen-tal systematic standard uncertainty is estimated as follows:

bXkp

BXk

2 (4-3.4)

The variable BXkin the preceding equation

repre-sents the 95% confidence level estimate of the

symmetric limits of error associated with the kth

elemental error source. In certain situations, knowl-edge of the physics of the measurement system will lead the analyst to believe that the limits of error are nonsymmetric (likely to be larger in either the positive or negative direction). For treatment of nonsymmetric systematic uncertainty see subsec-tion 8-2. The value of 2 in the equasubsec-tion is based on the assumption that the population of possible systematic errors is normally distributed. If the analyst thinks that the error distribution might be other than normal, such as uniform (rectangular), then a different factor would be used to convert the 95% confidence level estimate of the systematic error limits to an elemental systematic standard uncertainty (see Nonmandatory Appendix B). Also, there is some level of uncertainty associated with

the estimate of BXk. This uncertainty in the estimate

can be converted into a degrees of freedom for the systematic standard uncertainty as shown in

Nonmandatory Appendix B. Usually, the BX_k

esti-mates are made such that this degrees of freedom will be large (≥30). Using the recommendations in Nonmandatory Appendix B, it can be shown that this large degrees of freedom (≥30) corresponds to

an uncertainty in the estimate of BXkof 13% or less.

4-3.2.2.2 Published Information. For some

el-emental systematic error sources, published infor-mation from calibration reports, instrument specifi-cations, and other technical references may provide quantitative information regarding the dispersion of errors for an elemental systematic error source in terms of a confidence interval, an ISO expanded

9

uncertainty statement, or a multiple of a standard deviation. If the published information is presented as a confidence interval (limits of error at a defined level of confidence), then the elemental systematic standard uncertainty is estimated as the confidence interval divided by a statistic that is appropriate for the frequency distribution of the error popula-tion. The specific value of this statistic must be selected on the basis of the defined confidence level and degrees of freedom associated with the confidence interval. For a normal distribution, the Student’s t statistic is used. For a 95% confidence level and large degrees of freedom, the value of the Student’s t statistic is approximated as 2 and eq. (4-3.4) would apply (refer to Nonmandatory Appendix B for values of the Student’s t statistic at other confidence levels and degrees of freedom). For situations in which the frequency distribution and degrees of freedom are unspecified, a normal distribution and large degrees of freedom are often assumed. For situations involving other frequency distributions, refer to an appropriate statistics text-book. If the published information is presented as an ISO expanded uncertainty at a defined coverage factor (sometimes referred to as a “k factor”), then the elemental systematic standard uncertainty is estimated as the expanded uncertainty divided by the coverage factor. If the published information is presented as a multiple of a standard deviation, then the elemental systematic standard uncertainty is estimated as the multiple of the standard devia-tion divided by the multiplier.

4-3.2.2.3 Special Data. For some elemental

systematic error sources, special data may be ob-tained that manifests the dispersion of the

popula-tion of possible, unknown ␤X_k values. Possible

sources of this special data include

(a) interlaboratory or interfacility tests; and (b) comparisons of independent measurements

that depend on different principles or that have been made by independently calibrated instruments; for example, in a gas turbine test, airflow can be mea-sured with an orifice or a bell mouth nozzle, or computed from compressor speed-flow rig data, turbine flow parameters, or jet nozzle calibrations. For these cases, the elemental systematic stan-dard uncertainty may be evaluated as follows:

bX_kp

冪

₁ N␤X k

兺

N_X k j p 1 (Xk_j− Xk)2 NXk− 1 (4-3.5)

(21)

where

NX_kpthe number of special data values used

in the computation of bX_k

N␤X k

pthe number of independent samples from

the population of possible␤X_kvalues that

are averaged together in the computation of the average measurement for the test (X)

Xkpthe average of the set of special data

Xkjpthe j

th _{data point of the set of special}

data that manifests the dispersion of the

population of possible ␤Xk values

associ-ated with the kth elemental error source

For most measurements (especially those made using a single instrument calibrated at a single laboratory and installed in a single location), only a single sample from the population of possible

␤Xk values is included in the computation of

the average measurement for the test (X) and

hence N␤X

k

p 1. The following illustrate some

possible cases where N_␤

X_kmay be greater than one.

(a) Several independent measurement

meth-ods that depend on different principles are used to measure the same parameter. The results from each of the measurement methods (each determined as an average value over the duration of the test) are used as input to eq. (4-3.5) to evaluate the elemental systematic standard uncertainty associated with the error inherent to the various measurement methods. If the average measurement reported for the test is the average of the results from all of the

measure-ment methods, then the value for N␤X

k

used in eq. (4-3.5) is equal to the number of independent mea-surement methods employed.

(b) An instrument is sent to multiple

labora-tories to obtain calibration data for the instrument prior to using the instrument in a test. The results from each of the independent laboratories (each de-termined as an offset to be applied to the instrument when measuring a specific input level) are used as input to eq. (4-3.5) to evaluate the elemental system-atic standard uncertainty associated with the error inherent to the various laboratories. If the average measurement from the instrument reported for the test is based upon application of the average offset from all of the laboratories, then the value for

N␤_X k

used in eq. (4-3.5) is equal to the number of independent laboratories employed.

4-3.2.3 Combining Elemental Systematic Stan-dard Uncertainties. Once evaluated, all of the

ele-mental systematic standard uncertainties influenc-ing a measurement are combined into an estimate

10

of the total systematic standard uncertainty for

the measurement, bX. Provided all elemental

sys-tematic standard uncertainties are evaluated in terms of their influence on the parameter being measured and in the units of the parameter being measured, these elemental systematic standard un-certainties are combined per subsection 6-2. Other-wise, these elemental systematic standard uncer-tainties are combined per subsection 7-4. In some cases, elemental systematic standard uncertainties may arise from the same elemental error source and are therefore correlated. See subsection 8-1 for a detailed discussion.

4-3.3 Combined Standard Uncertainty and Expanded Uncertainty

As mentioned previously, the total uncertainty in a measurement is the combination of uncertainty due to random error and uncertainty due to sys-tematic error. The combined standard uncertainty of the measurement mean, which is the total uncer-tainty at the standard deviation level, is calculated as follows:

uXp

冪

(b_X)2+ (s_X)2 (4-3.6)

where

bXpthe systematic standard uncertainty

sXpthe random standard uncertainty of the

mean

The expanded uncertainty of the measurement mean is the total uncertainty at a defined level of confidence. For applications in which a 95% confidence level is appropriate, the expanded un-certainty is calculated as follows:

UXp 2u_X (4-3.7)

where the assumptions required for this simple equation are presented in subsection 6-4. Expanded uncertainty is used to establish a confidence inter-val about the measurement mean which is expected to contain the true value. Thus, the interval

X ± UX is expected to contain the true value with

95% confidence (see Fig. 4-3.3).

(22)

Fig. 4-3.3 Uncertainty Interval

4-4 PRETEST AND POSTTEST UNCERTAINTY ANALYSES

(a) The objective of a pretest analysis is to

estab-lish the expected uncertainty interval for a test re-sult, prior to the conduct of a test. A pretest uncer-tainty analysis is based on data and information that exist before the test, such as calibration histories, previous tests with similar instrumentation, prior measurement uncertainty analyses, expert opinions, and, if necessary, special tests.

A pretest uncertainty analysis allows corrective action to be taken, prior to expending resources to conduct a test, either to decrease the expected uncertainty to a level consistent with the overall objectives of the test or to reduce the cost of the test while still attaining the objectives. Possible corrective actions include

(1) selecting alternative testing methods that

rely upon different analysis procedures, testing un-der different conditions, and/or measurement of different parameters;

11

(2) selecting alternative measurement methods

by varying test instrumentation, calibration tech-niques, installation methods, and/or measurement locations; and

(3) increasing sample sizes by increasing

sam-pling frequencies, increasing test duration, and/or conducting repeated testing.

Additionally, a pretest uncertainty analysis facili-tates communication between all parties to the test about the expected quality of the test. This can be essential to establishing agreement on any devi-ations from applicable test code requirements and can help reduce the risk that disagreements regard-ing the testregard-ing method will surface after conductregard-ing the test.

(b) The objective of a posttest analysis is to

estab-lish the uncertainty interval for a test result, after conducting a test. In addition to the data and infor-mation used to conduct the pretest uncertainty anal-ysis, a posttest uncertainty analysis is based upon the additional data and information gathered for the test including all test measurements, pretest and

(23)

posttest instrument calibration data, etc. A posttest uncertainty analysis serves to

(1) validate the quality of the test result by

demonstrating compliance with test requirements;

12

(2) facilitate communication of the quality of

the test result to all parties to the test; and

(3) facilitate interpretation of the quality of the

test by those using the test result.

(24)

Section 5 Defining the Measurement Process

5-1 OVERVIEW

The first step in a measurement uncertainty analysis is to clearly define the basic measurement process. This simple step, often overlooked, is essential to successfully develop and apply the uncertainty information. Consideration must be given to the selection of the appropriate “true value” of the measurement and the time interval for classifying errors as systematic or random. This section provides an overview of how the measurement process should be defined.

5-2 SELECTION OF THE APPROPRIATE “TRUE VALUE”

Depending on the user’s perspective, several measurement objectives or goals and hence corres-ponding “true values” (measurements with ideal zero error) may exist simultaneously in a measure-ment process. For example, when analyzing a thermocouple measurement in a gas stream, sev-eral starting points or “true values” can be selected. The starting point for the analysis could begin with the “true value” defined as the metal tempera-ture of the thermocouple junction, the gas stagna-tion temperature or juncstagna-tion temperature corrected for probe effects, or the mass flow weighted aver-age of the gas temperature at the plane of the instrumentation. Any of the aforementioned “true values” may be appropriate. The selection of the “true value” for the uncertainty analysis must be consistent with the goal of the measurement [3].

5-3 IDENTIFICATION OF ERROR SOURCES

Once the true value has been defined, the errors associated with measuring the true value must be identified. Examples of error sources include imperfect calibration corrections, uncontrolled test conditions, measurement methods, environmental conditions, and data reduction techniques. Esti-mates to reflect the extent of these errors are

13

represented as uncertainties. These uncertainties in the measurement process can be grouped by source

(a) calibration uncertainty

(b) uncertainty due to test article and/or

instru-mentation installation

(c) data acquisition uncertainty (d) data reduction uncertainty

(e) uncertainty due to methods and other effects

5-3.1 Calibration Uncertainty

Each measurement instrument may introduce random and systematic uncertainties. The main purpose of the calibration process is to eliminate large, known systematic errors and thus reduce the measurement uncertainty to some “acceptable” level. Having decided on the “acceptable” level, the calibration process achieves that goal by ex-changing the large systematic uncertainty of an uncalibrated or poorly calibrated instrument for the smaller combination of systematic uncertainties of the standard instrument and the random uncer-tainties of the comparison. Calibrations are also used to provide traceability to known reference standards or physical constants, or both. Require-ments of military and commercial contracts have led to the establishment of extensive hierarchies of standards laboratories. In some countries, a national standards laboratory is at the apex of these hierarchies, providing the ultimate reference for every standards laboratory. Each additional level in the calibration hierarchy adds uncertainty in the measurement process (see Fig. 5-3.1).

5-3.2 Uncertainty Due to Test Article and/or Instrumentation Installation

Measurement uncertainty can also exist from interactions between (a) the test instrumentation and the test media or (b) between the test article and test facility. Examples of these types of uncer-tainty are

(25)

Fig. 5-3.1 Generic Measurement Calibration Hierarchy

(a) Interactions Between the Test Instrumentation and Test Media:

(1) Installation of sensors in the test media may

cause intrusive disturbance effects. An example could be the measurement of airflow in an air condi-tioning duct. Depending on the design of the pitot static probe, it may affect the measured total and static pressure and thus the calculated airflow.

(2) Environmental effects on

sensors/instru-mentation may exist when the sensors experience environmental effects that are different from those observed during calibration. These may be such things as conduction, convection, and radiation on a sensor when installed in a gas turbine.

(b) Interactions Between the Test Article and Test Facility:

(1) Test-facility limitations for certification

test-ing affects product measurement uncertainty. An example may be an air conditioner that was bench tested in a laboratory but used in an automotive mechanics shop. The effect of the oily air can influ-ence the quoted rating of the unit. A second example is the testing of a gas turbine engine in an altitude facility. The facility simulates altitude by lowering the ambient pressure at the test article exhaust and raising the inlet pressure at the engine inlet. In

appli-14

cation the inlet pressure is elevated due to the ram drag effects of the aircraft. A correction factor must be applied that corrects between uninstalled to in-stalled aircraft engine performance.

(2) Facility limitations for testing may require

extrapolations to other conditions. An example is the testing of an automotive engine. The fuel con-sumption of an automotive engine changes with altitude and speed. An automotive test facility may only be able to test at specified altitudes and speeds, and the effects at other altitude conditions may need to be extrapolated.

5-3.3 Data Acquisition Uncertainty

Uncertainty in data acquisition systems can arise from errors in the signal conditioning, the sensors, the recording devices, etc. The best method to minimize the effects of many of these uncertainty sources is to perform overall system calibrations. By comparing known input values with their mea-sured results, estimates of the data acquisition system uncertainty can be obtained. However, it is not always possible to do this. In these cases, it is necessary to evaluate each of the elemental

(26)

uncertainties and to combine them to predict the overall uncertainty.

5-3.4 Data Reduction Uncertainty

Computations on raw data are done to produce output (data) in engineering units. Typical uncer-tainty sources in this category stem from curve fits and computational resolution. With the recent advances in computer systems, the computational resolution uncertainty sources are often negligible; however, curve-fit error uncertainty can be signifi-cant. Other examples of data reduction uncertainty include

(a) the assumptions or constants contained in the

calculation routines;

(b) using approximating engineering

relation-ships or violating their assumptions; and

(c) using an empirically derived correlation such

as empirical fluid properties.

These additional uncertainties may be of either a systematic or random nature depending on their effect on the measurement.

5-3.5 Uncertainty Due to Methods and Other Effects

Uncertainties due to methods are defined as those additional uncertainty sources that originate from the techniques or methods inherent in the measurement process. These uncertainty sources, beyond those contained in calibration, installation sources, data acquisition, and data reduction, may significantly affect the uncertainty of the final results.

5-4 CATEGORIZATION OF UNCERTAINTIES

This Standard delineates uncertainties by the effect of the error (i.e., systematic and random). This categorization approach supports the identifi-cation, understanding, and managing of test uncer-tainties. If the nature of an elemental error is fixed over the duration of the defined measurement process, then the error contributes to the systematic uncertainty. If the error source tends to cause scatter in repeated observations of the defined measurement process, then the source contributes to the random uncertainty.

Because measurement uncertainties are catego-rized by the effect of the error, the time interval and duration of the measurement process can be important considerations and so must be clearly

15

stated. The significance of this is discussed in para. 5-4.2. In addition, the objective of the test may affect the categorization as discussed in para. 5-4.3.

5-4.1 Alternate Categorization Approach

An alternate approach, which is used in the ISO GUM, categorizes the uncertainties based on the method used to estimate uncertainty. Those evalu-ated with statistical methods are classified as Type A, while those, which are evaluated by other means, are classified as Type B. Depending on the selection of the defined measurement process, there may be no simple correspondence between random or systematic and Type A or Type B.

5-4.2 Time Interval Effects

Errors that may be fixed over a short time period may be variable over a longer time period. For example, calibration corrections, which are as-sumed fixed over the life of the calibration interval, can be considered variable if the process consists of a time interval encompassing several different calibrations. The time interval must be clearly specified to classify an error, and it may not always be the same interval as the test duration. For example, when comparing results among various laboratories, it may be appropriate to classify an error as random rather than as systematic even though that error may have been constant for the duration of any single test.

The effects of a time interval may also be impor-tant when considering the stability and control of a test process. The stability of a measurement method is a generic concept related to the closeness of agreement between test results. Process stability is estimated from observations of scatter within a data set and is treated as a random error. Variabil-ity in independent test results obtained under different test conditions, varying experimental set-ups, or configuration changes allow for additional between-test random errors.

5-4.3 Test Objective

The classification and number of error sources are often affected by the test objective. For example, if the test objective is to measure the average gas mileage of model “XYZ” cars, the variability among or between cars of the same model must be considered. Random error obtained in a test from a given car would not include car-to-car

(27)

Fig. 5-4.3 Difference Between “Within” and “Between” Sources of Data Scatter

variations and thus would not represent all random error sources. To observe the random error associ-ated with car-to-car variability, the experiment would need to be run again using a random selection of different cars within the same model (see Fig. 5-4.3). The total variation in the test result is greater than that observed from a test of a single given car. This variation would be more representative of the total random error associated with determining gas mileage for the fleet of model “XYZ” cars. Of course, if the data of interest is gas mileage of a given single car, then the estimated variation with testing the representative given car is an appropriate estimate for the random error. The same short-term and long-term effects must be applied for other variables affecting gas mileage (temperature, altitude, humidity, road conditions, driver variations, etc.).

16

5-5 COMPARATIVE VERSUS ABSOLUTE TESTING

The objective of a comparative test (also known as a back-to-back test) is to determine, with the smallest measurement uncertainty possible, the net effect of a design change. The first test is run with the standard or baseline configuration. The second test is then run in the same facility with the design change and hopefully with instruments, setups, and calibrations identical to those used in the first test. The difference between the results of these tests is an indication of the effect of the design change. Depending on whether common instru-mentation, setups, and calibrations are used be-tween comparative tests, the effects of correlated uncertainties (see Section 8) may cause the total uncertainty of the difference between the test re-sults to be less than the uncertainty of each separate test result. An example of back-to-back uncertainty analysis is shown in Example 8.1 in subsection 8-1.