D.2 Basic Aerodynamic Research Tunnel
6.2 Impedance Uncertainty, Quantification, and Quality Control
Quantification, and Quality Control
This section addresses the metrics used to quantify imped- ance uncertainty and the relevance of such metrics to the task of applying statistical process control technology to achieving target liner impedance spectra in an operational aircraft engine to within a specified range of uncertainty.6.2.1
Statistical Process Control: A Tool for
Impedance Quality Control
A single realization of a liner design in an operational engine is subject to random and systematic errors that result in frequency-dependent deviation of the installed impedance from the targeted values. The limiting of such errors to an interval centered on the target impedance is traditionally the task of manufacturing quality control. Traditional quality control is based on a statistical sampling of the final product and thus does not detect an out-of-compliance condition until after it occurs. Statistical process control (SPC) monitors the inherent variability of a manufacturing process at critical points (“voice of the process”) to provide a warning of an impending out-of-compliance condition (Refs. 8 and 9). The power of SPC is to provide a basis to intervene in the manu- facturing processes at critical points to prevent production of out-of-compliance units. This approach to quality control would achieve an installed liner impedance under operational conditions to within some specified total error. The user- specified variation of the end product from the target (“voice of the customer”) is what makes SPC an attractive approach for impedance quality control. It fits well with the SPC concept because the statistical sampling of the installed liner impedance is impractical on a routine basis. It is the judicious sampling of the manufacturing process that is the critical aspect of SPC, and that makes it an attractive conceptual framework for specifying installed impedance uncertainty in operational aircraft engines. As with SPC applied to widget production, the task is to anticipate an impending out-of- compliance condition at selected stages in the design and fabrication process. To some degree, manufacturing process monitoring is already being implemented in current acoustic liner production. What may be missing, however, is a robust statistical database for the various subprocesses which constitute the voice of the process and that may be used in conjunction with a user-defined uncertainty limit such as a confidence interval centered on the desired impedance. For a liner impedance spectrum, such confidence intervals are frequency dependent. The most critical (and potentially useful) aspect of applying the SPC paradigm to assure performance compliance is the creation of a laboratory- validated, semi-empirical impedance prediction model that serves as a kind of proxy for impedance measurements on the installed product (assumed inaccessible on a routine basis). Thus, in what follows, we apply uncertainty analyses to
impedance predictions and measurements for the POHC single-layer liner.
6.2.2
Specification Limits, Uncertainty Limits, and
Confidence Intervals
The goal of SPC, for liner implementation on an operational engine, is to estimate the confidence with which an installed impedance complies with an end-user-defined uncertainty range centered on the target impedance. When quantified in terms of a probabilistic statement this uncertainty range is generally called a confidence interval because it estimates the level of confidence that the implemented impedance will fall within the stated uncertainty interval. Because it is not practical to “measure” the impedance of the finished product (i.e., installed impedance), a semi-empirical impedance prediction model provides the basis for estimating not only impedance mean values but also the standard deviation about the mean. These two statistical quantities are crucial to estimating compliance of the process with a user-specified uncertainty limit. Thus SPC is a statistical tool and for its successful application relies on a sufficiently robust statistical database. Both the statistical database and the selected semi- empirical liner impedance prediction model are crucial for the successful application of SPC to estimating installed imped- ance uncertainty.
The success of a quality control program, in the SPC con- text, depends upon a validated impedance prediction model and an informed customer specification limit. Model valida- tion is done in a laboratory environment and is the basis for employing the model as a kind of proxy for the voice of the process. Given a creditable voice of the process, an informed customer specification limit on a target impedance spectrum translates into an optimal choice (both technically and economically) for the entity ultimately responsible for compliance of an aircraft engine noise certification require- ment. Without realistic specification limits, presumably determined from allowable uncertainty contributed by the turbofan noise to the aircraft footprint, the liner producer simply does not know how “tight” the process input parame- ters need to be! Thus, since there are no a priori specification limits provided for the POHC liner studied in this assessment, the focus is on the credibility of the voice of the process. This takes the form of comparisons of semi-empirical impedance prediction models and laboratory measurements. These comparisons are done in terms of an uncertainty analysis on statistical datasets of measured impedance spectra and Monte Carlo simulations applied to impedance prediction models.
6.2.3
Uncertainty Metrics
An unbiased measurement repeated N times on a hypothet- ical, statistically stable, parent population exhibits random fluctuations about the mean of the N sample values. For unimodal (e.g., Gaussian) statistical distributions, the sample mean approaches that of the parent population mean
(sometimes denoted “true value”), as N increases without limit. In practice, the sample mean, as well as other statistical parameters of interest, is always an estimate of the parent population counterpart. Real measurement processes (and predictive models) may impose unknown bias error (also called systematic error) on the sample statistics to cause a nonrandom shift in sample statistical parameters relative to their true values. This fact constitutes the major challenge for uncertainty analysis as systematic error is not known a priori and is thus not easily separated out from random error. Conceptually, at least, the total error in a single measurement (or prediction) comprises an unknown combination of systematic and random errors (Ref. 10). Again, because the true value is unknown, the total error is also unknown. However difficult the distinction between systematic and random errors may be in practice, these distinctions are nevertheless helpful. Hence, systematic errors are said to be due to assignable causes, and random errors are said to be due to unassignable causes.
Systematic and random errors are sometimes differentiated by the terms accuracy and precision that, in conventional usage, are synonymous. In the technical context here, accuracy denotes closeness to the true value and precision denotes the degree of clustering around a mean value (not necessarily the true value). In this document, we find it convenient to use all
of these descriptors. There are four combinations of precision and accuracy as illustrated in Figure 6.3. Generally, improving measurement methodology increases accuracy, whereas improving measurement technique increases precision. The reader will note that increasing accuracy or precision corre- sponds to an improved measurement, but not in the same manner. Correspondingly, a decrease in systematic error or random error corresponds to an improved measurement. Thus, these descriptors bear an inverse relationship to each other. While the above definitions have been discussed in terms of measurements, we also intend to apply them to impedance model predictions.
As will be discussed in Section 6.3.5, “Relevance of Labor- atory Impedance Measurement,” a propagation model must be employed to measure (educe) the impedance of a test liner. This indirect measurement process affords ample opportunity for accuracy and precision errors to arise. To help mitigate these errors, at least for the impedance measurement technolo- gies employed at the NASA Langley Research Center, studies have been conducted via well-understood absorbing structures (validation liners) consisting of parallel, capillary-like channels embedded in a rigid matrix (Ref. 11). The impedance behavior of such structures is nearly linear (impedance independent of excitation level and mean flow speed), making the impedance predictable from first principles. This
Figure 6.3.—Depiction of accuracy versus precision for parameter x, as used in this chapter. (a) High accuracy, high precision; (b) High accuracy, low precision; (c) Low accuracy, high precision; (d) Low accuracy, low precision.
prediction capability allows very accurate and precise impedance spectra to be determined. This capability allows these liners to be used to establish a baseline uncertainty for validating the impedance measurement methodologies and processes at Langley (Ref. 12), which have been used to validate both normal incidence and grazing incidence impedance measurement processes (Refs. 13 and 14).
For the purposes of this study, the metric for uncertainty is chosen to be the 95% confidence interval (Ref. 10). These intervals are calculated from the statistics generated by repeated impedance measurements and predictions. Calcula- tion of 95% confidence intervals, as used here, assumes input parameter statistics to be Gaussian distributed. The precision part (random error) of the uncertainty analysis consists of simply comparing 95% confidence intervals for impedance measurements and predictions. The accuracy part (systematic error) of the uncertainty analysis consists of comparing the measured and predicted mean values. These confidence intervals are deemed to have a 95-percent probability of containing the “truth.” For the purposes of this study, the truth is assumed to be the mean values that would be attained with a large number (parent population) of measurements or predictions. Correspondingly, there is a 95-percent confidence that the next measurement (or simulation, in the case of prediction models) will fall within the 95% confidence interval. Any difference between the measured and predicted mean values is deemed to be systematic error (due to assigna- ble causes). The confidence interval widths are deemed to arise from unassignable causes (random variability). These definitions are the bases for all the commentary on the many data charts to follow in Section 6.6, “Results and Discussion.”
6.2.4
Coverage Factor for Small Sample Sizes
When estimating a confidence interval for a small sample size taken from a normally distributed parent population, a coverage factor is introduced to account for the fact that the sample probability density function follows a Student’s
t-distribution for small sample sizes (Ref. 10). Because
sufficiently large sample sizes (number of tests or simulations) are generally impractical, especially in experimental work, the underlying assumption of parent population normality cannot be confirmed. In the present task, this situation arises both in the use of the nonlinear data reduction equation for the impedance measurement methodologies and in the prediction models. The impedance prediction models presented in this assessment involve nonlinear equations and are thus almost guaranteed to generate non-Gaussian output distributions for sufficiently large excursions of the inputs from their respective means. This is true even when the input distributions (proba- bility distributions of the input parameters used in the prediction model) are Gaussian and large sample sizes are employed. To allay the concern for the usage of a coverage factor under these circumstances, a Monte Carlo process was used to acquire a large number of simulations (impedance predictions). Excellent agreement between the coverage
factor-based confidence intervals and those calculated directly from the Monte Carlo simulations is achieved.