Objectives:
How does WQ compare between 2+ sites Types of designs (paired vs. unpaired) Case study - French Creek
Types of designs Paired/blocked
Paired (2 sites) or Blocked (2+ sites) designs are similar Synoptic (same day) or near synoptic (same week?) readings are taken
Interested in the average “difference”.
Not necessary to have a random sample to times. It may be preferable to select times to enhance contrast (e.g. sample at low and peak flows).
Paired t-test; Single Factor Randomized Blocked ANOVA
Independent samples (not part of this course).
2+ sites to be compared.
Separate, random samples taken from each site. Interested in compare the MEANS across the sites. Independent sample t-test; Single Factor Completely Randomized Design ANOVA (a.k.a. One-way ANOVA)
French Creek - data set available Five locations along French Creek
Monthly + two sets of 5-in-30 samples starting late-July and late-October
(Near) Synoptic data
How do the readings compare across the sites?
Compare readings in 2 sites. Align the data by date
Find the difference in readings or log(ratio) of readings
Use the difference if the range of values is small so that the differences between sites are relatively similar (e.g. a consistent difference of about 2 units).
Use the log(ratio) if the range of values is large so that differences between sites are NOT relatively similar (e.g. range from 2 to 200 units), but RATIO of readings (e.g. one site’s readings is about twice the other site).
Compute the mean difference, se of mean difference, 95% confidence interval for mean difference and see if the 95% confidence interval includes 0.
If want a p-value, test the hypothesis that mean difference in population is zero.
Turbidity Readings
Sampling BARCLAY GRAFTON NEW WINCH
Week BRG COOMBS ROAD HWY -ESTER
2010.0427 1.9 1.1 0.9 1.9 1.4 2010.0525 1 0.7 0.6 0.7 1 2010.0622 1.8 0.6 0.3 0.7 1 2010.0726 0.7 0.3 0.8 0.5 0.8 2010.0816 0.7 0.3 1.5 0.6 0.9 2010.0824 0.8 0.5 0.4 0.4 1.4 2010.0831 0.9 0.3 1.1 0.6 1.6 2010.0907 0.7 0.3 1.6 0.5 2.5 2010.0914 0.8 0.3 0.2 0.4 0.7 2010.102 0.5 0.4 0.2 0.3 0.6 2010.1026 4.3 2 1.2 2.3 1.1 2010.1102 7.4 4 2.1 4.2 1.9 2010.1109 2.5 1.3 1.3 2 1.3 2010.1116 1.9 1.1 0.6 0.8 2010.1229 5.3 3.8 4.3 6.2 3.1 74 / 118
Compute the difference or log(ratio) between two sites (e.g. Barclay vs. Coombs).
Drop cases where missing values are present.
Compute using Excel functions
Number of differences/log-ratios - count() Mean difference/log-ratio - average() Std dev of diff/log-ration - stdev.s () 95% CI half width - confidence.t() 95% CI mean ± 95% CI half width
t-test for testing equality of means - t.test() Also take anti-logs of mean and 95% CI for log-ratio
Barclay averages .947 NTU higher than Coombs with a 95% c.i. of between 0.45 and 1.44 NTU higher than Coombs. Barclay is, on average, approx 1.98x (95% CI 1.70x to 2.30x) larger than Coombs.
CAUTION: 95% c.i. say nothing about individual differences or individual ratios, i.e. NOT CORRECT TO SAY that 95% of differences lie between 0.45 and 1.44 NTU.
CAUTIONS: Excel does not deal with missing data very nicely. Look what happens if last reading for Coombs is missing.
Similar output from JMP with additional graphs to show that log(ratio) is likely “better” choice than difference:
A formal p-value is un-necessary but can be obtained as well.
Because the p-values are very small, there is strong evidence of a difference (on average) between the two sites.
Pairing is induced by synoptic readings
Delete any pairs with missing values. More advanced software (e.g. R/SAS/JMP) can also incorporate missing data but this is beyond scope of course.
CAUTION: EXCEL does NOT handle missing data well.
Compute differences and/or log(ratio)
Use differences if readings are similar over time
Use log(ratio) if large variation in readings and RATIO is consistent over time
Compute mean difference or log-ratio and 95% confidence interval for population mean difference or log-ratio.
Is 0 included in the 95% confidence interval? If so, then there is no evidence of a difference (on average).
CAUTION: 95% confidence interval say NOTHING about individual differences or log-ratios
Use anti-log on mean and 95% c.i. to convert log-ratio back to ratios.
It is possible to extend the analysis to 3+ more sites with synoptic readings.
Make an array of week by site Record actual values or ln(values)
CAUTION: EXCEL does not ALLOW for ANY missing data. You must exclude entire sampling week if any data is missing. This can lead to a LARGE loss of data.
JMP/SAS/R can gracefully deal with missing data.
Notice that readings on 2010.1116 are dropped FOR all sites because missing data at New Highway
Known as a Randomized Block Design
Blocks = Synoptic Times = device for “pairing up” observations that are affected in similar way.
Assume that differences among sites is relatively consistent across the blocks == NO INTERACTION between blocks and sites.
(OR) Assume that ratio among sites is relatively consistent across the blocks so that differences of log(values) is relatively consistent (e.g. site A might always be about 2x larger than site B).
Again start with plot of values as seen earlier
Here, column effect = effects of SITES
P-value = 0.02, so there is some evidence of a consistent difference in the MEANS among sites
Does NOT indicate which sites could have the same or different means. Need to follow-up with a Tukey Multiple Comparison Procedure to identify which pairs of sites could have different means.
Look at estimates of MARGINAL means:
Critical range is CR = Qa
q
MSE
# blocks where Qa is value from
Studentized range with df1 = #sites and df2 = dfMSE. In this case
we look for (5, 52) df (see previous slide) at http://www.stat. duke.edu/courses/Spring98/sta110c/qtable.html and find the Qa= 4.20 and CR = 4.20
q
0.65
14 = 0.91.
This is VERY tedious and error prone in EXCEL – use a proper package such as JMP/ R/ SAS etc.
The output is “automatic” and more-informative and can handle missing data.
P-value is small, so there is some evidence of a difference in means among the sites. It does not indicate where the difference may lie.
This indicates which Sites could have the same mean.
Think of ’paint-chips’ to understand overlap in ranges of sites that could be the same.
Provides estimates of effects and confidence intervals for each pairwise difference.
Typically used for synoptic data to see if sites are comparable Exactly 2 sites:
Find difference or log(ratio) of readings from both sites. Drop any sites with missing data.
Find mean and 95% confidence interval for difference in MEAN See if 0 is included in the confidence interval.
Find p-value using Paired t-test.
3+ Site:
Analyze either raw data or log(data).
Use Randomized Block Design analysis (Excel: Two-factor with no replication).
Look at ANOVA table at either Rows/Columns effects that correspond to SITES.
Program Tukey Multiple Comparison procedure by hand (groan)!
CAUTION: EXCEL does not deal with missing values correctly - GIVES WRONG ANSWERS.
CAUTION: EXCEL is very clumsy in finding where the differences lie. You must program Tukey procedure. You will make mistakes in doing this!
CAUTION: EXCEL does not provide other output to check assumptions of the models.