Simple paired t-test calculations

Paired t-test calculations may be performed, pair-wise, on all possible combinations of the seven individual means to ascertain statistical agreement according to the following equation for paired t-tests:

t = X1 - X2 / s (1/n1 + 1/n2)1/2 (2)

s = [(n1 - 1)s12 + (n2 - 1)s22]/(n1 + n2 - 2) (3)

where

X1and s1 are the mean concentration and standard deviation, respectively, found by method/laboratory specified in the first member of the pair,

X2and s2 are the mean concentration and standard deviation, respectively, found by method/laboratory specified in the second member of the pair,

and where the assumption is that s1 = s2 (i.e. variances are assumed to be homogeneous).

By way of example, calculated paired t-test results are given here for selected data from the four methods: AAS (AAS01 lab 33), AES (AES03 lab 59), NAA (NAA01 lab 15) and XRE (XRE01 lab 47), giving six possible pair-wise comparisons. Comparison of data within the following pairs, 33/59, 33/15, 33/47, 59/15, 59/47 and 15/47 gave respective calculated t's of 7.295 (S), 3.075 (S), 3.856 (S), 1.157 (NS), 6.090 (S) and 2.642 (S). Comparison with predicted t's at the respective degrees of freedom at the usual test level of p = 0.05 suggests non-significant (NS) and significant (S) differences as indicated.

Similarly, F-tests using variances (V) for the identical six possible pairs of data gave the following F ratios: V59/V33 = 19.38 (S); V15/V33 = 59.62 (S); V33/V47 = 1.48 (NS); V15/V59 =

3.08 (NS); V59/V47 = 28.77 (S); and V15/V47 = 88.51 (S). Comparison with predicted F ratios at the respective degrees of freedom for p = 0.05 suggests non-significant (NS) and significant (S) differences as indicated. Thus, for these six methods/laboratories, statistical calculations indicate some differences among means and variances.

Although actual statistical calculations do indicate some differences among means and inhomogeneity of variances, in the RM project from which the K/Wheat Gluten example is taken, such data were combined and utilized for calculation of reference concentration values and associated uncertainties. It is up to the initiating analyst to decide on pooling criteria and whether strict adherence to non-significant t-tests and F-tests need be maintained. The variances discussed so far are simply those generated within laboratories, i.e. within-laboratory variances, including within-unit and among-unit variances; it is surmised that consideration and addition of real errors (e.g. based on between-laboratory deviations) to those indicated in Fig. 2(c) would indicate more universal agreement among means and uncertainties. Plots in Fig. 2(a), 2(b) and 2(d) may be consulted for additional examples of visual comparisons and indications of the kind of data accepted for generation of reference concentration values in that RM characterization project.

In that project, such individual t-test and F-test calculations were not performed but the data tests were subjected to ANOVA calculations to extract the three variance components. Some statistical ANOVA calculations for the accepted set of analytical results are presented in Table VI. The following results were generated by detailed ANOVA calculations: overall mean (reference value): 472.2 mg/kg; within-unit variance (ıw2): 106.03033; among-unit variance (ıu2): 205.915144; among-laboratory/method variance (ıL 2): 634.060559. The reference value was computed as the arithmetic average of equally weighed individual laboratory means. The associated SD was calculated from the three variance components according to the equation:

SD = (ıw2 + ıu2 + ıL2 )1/2 (4)

SD = (106.03033 + 205.915144 + 634.060559)1/2 SD = 30.76

where each ı indicates the estimates of the associated variance component obtained from a type I

(hierarchal) variance component analysis.

Thus the recommended value and associated uncertainty (rounded to appropriate numbers of significant figures) is 472 ± 61 mg/kg, (mean ± 95% confidence limits) where the latter was calculated from the SD using the appropriate t value (usually 2) based on the degrees of freedom. It is worth pointing out that the confidence limits in this work, as in that of other RM producers, is based onstandard deviation and not standard error, as by IRMM (BCR). Using standard deviation leads to more realistic estimates of uncertainties and is related to standard deviations estimated for a single

future determination on the RM. Use of standard errors provides an unrealistically tight, narrow uncertainty estimate. Table IV is an example of a Table from a Report of Investigation [50] listing reference concentrations of constituent elements in RM Wheat Gluten WG 184 (NIST RM 8418). The following statement used by the author in reports of analysis for co-operatively produced AAFC/NIST RMs [52] may be taken as a guide for the reporting target values for in-house developed QCMs:

Reference values, weight percentage or mg/kg (ppm), presented in Reports of Investigation provided with these RMs are based on the dry material, and are equally weighed means of results from generally at least two, but typically several, different analytical methods applied by analysts in different laboratories. Uncertainties are estimates expressed either as 95% confidence intervals or occasionally as intervals based on ranges of accepted results for a single future determination based on a sample weight of at least 0.5 g. These uncertainties, based on among-method and laboratory/among-unit and within-unit estimates of variances, include measures of analytical method and laboratory imprecisions as well as biases and material inhomogeneity.

TABLE VII. REFERENCE CONCENTRATIONS OF CONSTITUENT ELEMENTS IN WHEAT GLUTEN REFERENCE MATERIAL WG 184 (NIST RM 8418)

Major Constituents (weight %)

Element Content and Methods(b) uncertainty (a)

Nitrogen 14.68 ± 0.26 I01 I02 J01 J02

Sulphur 0.845 ± 0.085 B02 B03 F04 J02 M02

Chlorine 0.362 ± 0.022 D01 F02 K01 K02 Phosphorus 0.219 ± 0.015 B02 B03 F01 F02 M01

Sodium 0.142 ± 0.011 A01 B01 B02 D01

Minor and Trace Constituents (mg/kg)

Magnesium 510 ± 47 A01 B02 B03 D01

Potassium 472 ± 61 A01 B02 B03 D01 E01

Calcium 369 ± 35 A01 B02 B03 D01 E02

Iron 54.3 ± 6.8 A01 B02 B03 D01 D03 E01 E02

Zinc 53.8 ± 3.7 A01 B02 B03 D03 E01

Manganese 14.3 ± 0.8 A01 B02 B04 D01 E01 E02

Aluminium 10.8 ± 3.0 A05 B02 B03 D01

Copper 5.94 ± 0.72 A01 A05 B02 C03 C06 E01 H01

Selenium 2.58 ± 0.19 B02 C01 C04 D01 D03 G01

Strontium 1.71 ± 0.26 B02 B03 C03 E01

Barium 1.53 ± 0.26 B02 B03 C03

Molybdenum 0.76 ± 0.09 B02 C03 C06 D01 D03 F01 H06

Nickel 0.13 ± 0.04 A16 H01

Lead 0.10 ± 0.05 A05 A16 C03 H01

Cadmium 0.064 ± 0.022 A04 A05 A16 C03 D03 H01

Iodine 0.060 ± 0.013 D03 D05 D06 F02 H03

Chromium 0.053 ± 0.013 A12 C05 D03

Cobalt 0.010 ± 0.006 A16 D01 H01

Mercury 0.0019 ± 0.0006 A10 D03

(a) Best estimate values, weight percent or mg/kg (ppm), are based on the dry material, dried according to instructions in this report and are equally-weighted means of results from generally at least two, but typically several, different analytical methods applied by analysts in different laboratories. Uncertainties are estimates expressed either as a 95% confidence interval or occasionally (Co, S, Se) as an interval based on the range of accepted results for a single future determination based on a sample weight of at least 0.5g. These uncertainties, based on among-method and laboratory, among-unit and within-unit estimates of variances, include measures of analytical method and laboratory imprecisions and biases and material inhomogeneity.

(b) Analytical method codes and descriptions are provided above.