Tests for Comparing Several Populations 1 Tests for Comparing Several Means

This section describes procedures to test the differences between several sample means from different populations either against a control population or among themselves. For example, the test described in this section could be used identify whether or not there are differences

between several drinking water wells or could be used to identify if several downgradient wells differ from an upgradient well.

In this situation, it would be possible to apply the tests described in Section 3.3.1 multiple times. However, applying a test multiple times underestimates the true false rejection decision error rate. Therefore, the test described in this section controls the overall false rejection decision error rate by making the multiple comparisons simultaneously.

3.4.1.1 Dunnett’s Test

PURPOSE

Dunnett’s test is used to test the difference between sample means from different populations against a control population. A typical application would involve different cleaned areas of a hazardous waste site being compared to a reference sample; this reference sample having been obtained from a uncontaminated part of the hazardous waste site.

ASSUMPTIONS AND THEIR VERIFICATION

Multiple application of any statistical test is inappropriate because the continued use of the same reference sample violates the assumption that the two samples were obtained independently for each statistical test. The tests are strongly correlated between themselves with the degree of correlation depending on the degree of similarity in number of samples used for the control group and investigated groups. The test is really best suited for approximately equal sample sizes in both the control group and the groups under investigation.

LIMITATIONS AND ROBUSTNESS

Dunnett’s method is the same in operation as the standard two-sample t-test of Section 3.3.1 except for the use of a larger pooled estimate of variance and the need for special t-type tables (Table A-14 of Appendix A). These tables are for the case of equal number of samples in the control and each of the investigated groups, but remain valid provided the number of samples from the investigated group are approximately more than half but less than double the size of the control group. In this guidance, only the null hypothesis that the mean of the sample populations is the same as the mean of the control population will be considered.

SEQUENCE OF STEPS

Directions for the use of Dunnett’s method for a simple random sample or a systematic random sample are given in Box 3-25 and an example is contained in Box 3-26.

EPA QA/G-9 Final

QA00 Version 3 - 39 July 2000

Box 3-25: Directions for Dunnett’s Test for Simple Random and Systematic Samples

Let k represent the total number of populations to be compared so there are (k-1) sample populations and a single control population. Let n1, n2, ... nk-1 represent the sample sizes of each of the (k-1) sample populations and let m represent the sample size of the control population. The null hypothesis is H0: µi - µC# 0 (i.e., no difference between the sample means and the control mean) and the alternative hypothesis is HA: µi - µC > 0 for 1 = 1, 2, ..., k-1 where µi represents the mean of the ith sample population and µC represents the mean of the control population. Let " represent the chosen significance level for the test.

STEP 1: For each sample population, make sure that approximately 0.5 < m/ni < 2. If not, Dunnett’s Test should not be used.

STEP 2: Calculate the sample mean, X¯i (Section 2.2.2), and the variance, si

2 _{(Section 2.2.3) for each of the k} populations (i.e., i = 1, 2, ... k).

STEP 3: Calculated the pooled standard:

s m s n s n s m n n D C n k k k = − + − + + − − + − + + − − − − ( ) ( ) ... ( ) ( ) ( ) ... ( ) 1 1 1 1 1 1 2 1 2 1 1 2 1 1 1

STEP 4: For each of the k-1 sample populations, compute

t X X s n n i i C D i C = − + 1 1

STEP 5: Use Table A-14 of Appendix A to determine the critical value TD(1-") where the degrees of freedom is (m - 1) + (n1 - 1) + . . . + (nk-1 - 1).

STEP 6: Compare ti to TD(1-") for each of the k-1 sample populations. If ti >TD(1-") for any of the sample populations, then reject the null hypothesis and conclude that there are differences between the means of the sample populations and the mean of the control populations. Otherwise, conclude that there is no difference between the sample and control population means.

Box 3-26: An Example of Dunnett’s Test for Simple Random and Systematic Samples

At a hazardous work site, 6 designated areas previously identified as ‘hotspots’ have been cleaned. In order for this site to be a potential candidate for the local Brownfields program, it must be demonstrated that these areas are not longer contaminated. Therefore, the means of these areas will be compared to mean of a reference area also located on the site using Dunnett’s test. The null hypothesis will be that there is no difference between the means of the ‘hotspot’ areas and the mean of the reference area. A summary of the data from the areas follows.

Reference Area IAK3 ZBF6 3BG5 4GH2 5FF3 6GW4 Number of Samples: 7 6 5 6 7 8 7 Mean: 10.3 11.4 12.2 10.2 11.4 11.9 12.1 Variance: 2.5 2.6 3.3 3.0 3.2 2.6 2.8 Ratio: m/n 7/6 = 1.16 7/5 = 1.4 7/6 = 1.16 7/7 = 1 7/8 = 0.875 7/7 = 1 ti: 1.18 1.93 0.11 1.22 1.84 2.00

STEP 1: Calculate the ratio m/n for each investigated area. These are shown in the 4th _{row of the table above.} Since all of these ration fall within the range of 0.5 to 2.0, Dunnett’s test may be used.

STEP 2: The sample means, X¯i and the variance, si

2 _{were calculated using Sections 2.2.2 and 2.2.3 of} Chapter 2. These are shown in the 2nd _{and 3}rd _{row of the table above.}

STEP 3: The pooled standard deviation for all 7 areas is:

s_D = − + − + + − − + − + + − = = = ( ) . ( ) . ... ( ) . ( ) ( ) ... ( ) . . . 7 1 2 5 6 1 2 6 7 1 2 8 7 1 6 1 7 1 110 4 39 2 831 168 1

STEP 4: For each ‘hotspot’ area, ti was computed. For example, t1

11 4 10 3 1 68 1 6 1 7 118 = − + = . . . .

These are shown in the 5th _{row of the table above.}

STEP 5: The degrees of freedom is (7 - 1) + (6 - 1) + . . . + (7 - 1) = 39. So using Table A-14 of Appendix A with 39 for the degrees of freedom, the critical value TD(0.95) = 2.37 and TD(0.90) = 2.03.

STEP 6: Since none of the values in row 5 of the table are greater than either 2.37 or 2.03, it appears that none of the ‘hotspot’ areas have contamination levels that are significantly different than the reference area. Therefore, this site may be a potential candidate to be a Brownsfield site.

NOTE: If an ordinary 2-sample t-test (see Section 3.3.1.1) had been used to compare each ‘hotspot’ area with the reference area at the 5% level of significance, areas 2BF6, 5FF3, and 69W4 would have erroneously been declared different from the reference area, which would probably alter the final conclusion to include the site as a Brownfields candidate.

EPA QA/G-9 Final

QA00 Version 4 - 1 July 2000

CHAPTER 4