Measurement Errors
Introduction to Study Skills & Research Methods (HL10040)
Professor James Betts
Lecture Outline:
•Measurement Errors Continued
•Types of Errors
•Assessment of Error
•Introduction to Inferential Statistics
•Chi-Squared tests
•Assessment Details.
Measurement Errors
• Virtually all measurements have errors
– i.e.
Measured Score = ‘True’ Score Error
Therefore inherently linked to SD
• Reliability and Measurement Error are not the same, rather Reliability infers an acceptable degree of Measurement Error.
Energy Intake (calories per day)
1500 2500 3500 4500 5500
Number of People
0 20 40 60 80 100 120 140
160 This variability
between methods is caused by both
systematic and error factors
Direct Record
Retrospective Recall
SD
Total Variance
(SD2)
This total variance can then be
‘partitioned’
Systematic Variance
Error Variance
Types of Errors
• Systematic Error
– Any variable causing a consistent shift in the mean in a given direction
e.g. Retrospective diet records tend to omit the snacks between meals
• Random Error
– The fluctuation of scores due to chance
e.g. Innaccurate descriptions of the food consumed
Systematic Error
Skin-Fold Callipers
Hydrostatic Weighing
% Body-fat
Subject 1 Subject 2 Subject 3 Subject 4
10 12 8 11
17 22 14 12
Random Error
Skin-Fold Callipers
Hydrostatic Weighing
% Body-fat
Subject 1 Subject 2 Subject 3 Subject 4
14 18 10 9
11 15 21 17
Assessment of Error
• Systematic Error
Descriptive Statistics
4 12.00 22.00 16.2500 4.34933
4 8.00 12.00 10.2500 1.70783
4 Hydrostat
Callipers
Valid N (listwise)
N Minimum Maximum Mean Std. Deviation
Assessment of Error
• Random Error
8. 009. 0010. 0011. 0012. 00Callipers
Correlations
1 .527
. .473
4 4
.527 1
.473 .
4 4
Pearson Correlation Sig. (2-tailed)
N
Pearson Correlation Sig. (2-tailed)
N Callipers
Hydrostat
Callipers Hydrostat r2 = 0.278
r = 0 infers lots of error r = 1 infers no error
Assessment of Error
• Systematic &
Random Error
Callipers HydroStat. Difference Mean
10.00 17.00 7.00 13.50
12.00 22.00 10.00 17.00
8.00 14.00 6.00 11.00
11.00 12.00 1.00 11.50
14.00 11.00 -3.00 12.50
18.00 15.00 -3.00 16.50
10.00 21.00 11.00 15.50
9.00 17.00 8.00 13.00
Assessment of Error
• Systematic &
Random Error
12.00 14.00 16.00
Mean
0.00 5.00 10.00
differences
Mean = 4.63
The “Bland-Altman” Plot 3 points of visual assessment:
-Systematic Error: are points evenly distributed about the zero line?
-Random Error: do points deviate greatly from the mean line?
-Nature of error: is the error consistent left-right?
Examples of Bland-Altman Plots
12.00 13.00 14.00 15.00 16.00
Mean
0.00 5.00 10.00
Mean difference Zero
Examples of Bland-Altman Plots
12.00 13.00 14.00 15.00 16.00
Mean
0.00 5.00 10.00
Mean difference Zero
Examples of Bland-Altman Plots
12.00 13.00 14.00 15.00 16.00
Mean
0.00 5.00 10.00
Mean difference Zero
Examples of Bland-Altman Plots
12.00 13.00 14.00 15.00 16.00
Mean
0.00 5.00 10.00
Mean difference
Zero
Examples of Bland-Altman Plots
12.00 13.00 14.00 15.00 16.00
Mean
0.00 5.00 10.00
Zero
Why is Error Important
• Measurement Error is clearly of importance when
evaluating the agreement between two measurement tools
• A consideration of error is also relevant when attempting to establish intervention effects/treatment differences
i.e. where some of the variance between trials is due to the independent variable...
Systematic Variance Total Variance
between trial 1
& trial 2
Systematic Variance
Error Variance Dependent Variable
Extraneous/
Confounding (Error) Variables
Independent
Variable Primary
Variance
So researchers strive to increase the proportion of variance due to IV.
Smallest Worthwhile Effect
It would appear that even a small amount of primary
variance from an ergogenic aid would guarantee victory to either competitor…
…however, the error variance is such that a re-run could produce entirely different results…
Total Variance between trial 1
& trial 2
Systematic Variance
Error Variance Dependent Variable
Extraneous/
Confounding (Error) Variables
So researchers strive to increase the proportion of variance due to IV.
Scientific Reasoning (Logic)
General Theory
Specific Observation
Inductive Reasoning
Formation of a theory grounded in your own observations
Deductive Reasoning
Confirmation of a theory from your own observations
p-values give the probability of seeing this evidence assuming this general rule is true
Introduction to Inferential Statistics
• Before our next lecture you will be conducting some inferential statistics in your lab classes…
• All you need to be aware of at this stage is that the
‘p-value’ represents the probability of the observed variance occurring if the null hypothesis is true
i.e. p = 0.05 infers a 5 % probability of making your observation if in fact the IV has no effect
n.b. this DOES NOT mean that you will find this result in 95/100 test-retests or that your false positive rate is 5 %
Quantitative Analysis of Nominal Data
• Recall that nominal data infers that variables are dichotomous, i.e. belong to distinct categories
e.g. Athlete/Non-Athlete, Male/Female, etc.
• We know that such qualitative data can be coded quantitatively to allow a more objective analysis
• Nominal data does not require any consideration of normality and is analysed used a Chi2 test.
The Chi-Squared Test
• Goodness of fit χ2 test
– A comparison of your observed frequency counts against what would be expected according to the null hypothesis
i.e. null hypothesis infers equal dispersion (50:50)
• Contingency χ2 test
– A comparison of two observed frequency counts
Goodness of fit χ
2test
• Is a leisure centre used more by males than by females?
– n =150
Observed Frequency
Expected Frequency
Male 62 75
Female 88 75
Gender
62 75.0 -13.0
88 75.0 13.0
150 Male
Female Total
Observed N Expected N Residual
Goodness of fit χ
2test
SPSS Output
Test Statistics
4.507 1 .034 Chi-Squarea
df
Asymp. Sig.
Gender
0 cells (.0%) have expected frequencies less than 5. The minimum expected cell frequency is 75.0.
p-value AKA a.
significance level
Contingency χ
2test
• Are elite athletes more likely to take nutritional supplements than non-athletes
– n =60
Do take supplements
Do not take supplements
Athletes 18 12
Non-athletes 11 19
Chi-Square Tests
3.270b 1 .071
2.403 1 .121
3.301 1 .069
.120 .060
3.216 1 .073
60 Pearson Chi-Square
Continuity Correctiona Likelihood Ratio Fisher's Exact Test Linear-by-Linear Association N of Valid Cases
Value df
Asymp. Sig.
(2-sided)
Exact Sig.
(2-sided)
Exact Sig.
(1-sided)
Computed only for a 2x2 table a.
0 cells (.0%) have expected count less than 5. The minimum expected count is 14.
50.
b.
Group * Response Crosstabulation Count
18 12 30
11 19 30
29 31 60
Athletes Non-Athletes Group
Total
Do take supplements
Dont take supplements Response
Total
Contingency χ
2test
SPSS Output
Assumptions for Chi-Squared
• Although ND not required…
• Cells in the table should all be independent
i.e. one person could have visited the leisure centre twice
• 80 % of the cells must have expected frequencies greater than 5 and all must be above 1
i.e. the more categories available, the more subjects needed
• Cannot use percentages
i.e. a 15:45 split cannot be expressed as 25%:75%
Selected Reading
• I know error and variance can be confusing topics, try these:
• Atkinson, G. and A. M. Nevill. Statistical methods for assessing
measurement error (Reliability) in variables relevant to sports medicine.
Sports Medicine. 26:217-238, 1998.
• Hopkins, W. G. et al. Design and analysis of research on sport performance enhancement. Med. Sci. Sport and Exerc. 31:472-485, 1999.
• Hopkins, W. G. et al. Reliability of power in physical performance tests.
Sports Medicine. 31:211-234, 2001.
• Atkinson, G., ''What is this thing called measurement error?'' , in
Kinanthropometry VIII: Proceedings of the 8th International Conference of the International Society for the Advancement of Kinanthropometry (ISAK) , Reilly, T. and Marfell-Jones, M. (Eds.), Taylor and Francis, London , 2003.
Coursework (60% overall grade)
• Your coursework will require you to address
ONE
of the following research scenarios:– 1) Effect of Plyometric Training on Vertical Jump – 2) Effect of Ice Baths on Recovery of Strength
– 3) Effect of Diet on the Incidence of Muscle Injury – 4) Effect of Footwear on Sprint Acceleration
– 5) Effect of PMR on Competitive Anxiety.
Coursework Outline
• For the selected scenario you will need to:
– Perform a literature search in order to provide a comprehensive introduction to the research area – Identify the variables of interest and evaluate the
research design which was adopted
– Formulate and state appropriate hypotheses
– Summarise descriptive statistics in an appropriate and well presented manner…
Coursework Outline
• Cont’d…
– Select the most appropriate statistical test with justification for your decision
– Transfer the output of your inferential statistics into your word document
– Interpret your results and discuss the validity and reliability of the study
– Draw a meaningful conclusion (state whether hypotheses are accepted or rejected).
Coursework Details (see unit outline)
1000 words (2000 absolute maximum)
• Any supporting SPSS data/outputs to be appended
• To be submitted on Thursday 12th December Assessment Weighting
Evaluation & Analysis (30 %) Reading & Research (20 %) Communication & Presentation (20 %)
Knowledge (30 %)
Coursework Details
• All information relating to your coursework
(including the relevant data files) are accessible via the unit web page:
www.bath.ac.uk/~jb335/Y1%20Research%20Skills
%20(FH10040).html
Web address also referenced on shared area
Mid-Term Test (40% overall grade)
• NEXT WEEK
• This test will involve short answer questions covering all the information covered so far
• Mostly knowledge recall but will require
understanding and possibly some calculations
• Duration = 50 min
So…
Mid-Term Test (40% overall grade)
• Surnames: A-K
– Arrive promptly at 11.10 am for start of test at 11.15 am – Exit in silence afterwards
• Surnames: L-Z
– Arrive promptly at 12.10 am for start of test at 12.15 am – Exit however you like!
