Transcript Document

Measurement Errors
Introduction to Study Skills & Research Methods (HL10040)
Dr James Betts
Lecture Outline:
•Measurement Errors Continued
•Types of Errors
•Assessment of Error
•Introduction to Inferential Statistics
•Chi-Squared tests
•Assessment Details.
Measurement Errors
• Virtually all measurements have errors
– i.e.
Measured Score = ‘True’ Score  Error
Therefore inherently linked to SD
• Reliability and Measurement Error are not the
same, rather Reliability infers an acceptable
degree of Measurement Error.
160
This variability
between methods is
caused by both
systematic and
error factors
Direct Record
140
Number of People
120
Retrospective
Recall
100
80
60
40
SD
20
0
1500
2500
3500
4500
Energy Intake (calories per day)
5500
(SD2)
Caused by
systematic error
Systematic
Variance
Total
Variance
This total variance can
then be ‘partitioned’
Caused by
random error
Error
Variance
Types of Errors
• Systematic Error
– Any variable causing a consistent shift in the
mean in a given direction
e.g. Retrospective diet records tend to omit the snacks
between meals
• Random Error
– The fluctuation of scores due to chance
e.g. Innaccurate descriptions of the food consumed
Systematic Error
% Body-fat
Subject 1 Subject 2
Subject 3
Subject 4
Skin-Fold
Callipers
10
12
8
11
Hydrostatic
Weighing
17
22
14
12
Random Error
% Body-fat
Subject 1 Subject 2
Subject 3
Subject 4
Skin-Fold
Callipers
14
18
10
9
Hydrostatic
Weighing
11
15
21
17
Assessment of Error
Body-Fat
20
15
%
• Systematic
Error
25
10
Evidence of bias
between means
5
0
Condition
Descriptive Statistics
N
Hydrostat
Callipers
Valid N (listwise)
4
4
4
Minimum
12.00
8.00
Maximum
22.00
12.00
Mean
16.2500
10.2500
Std. Deviation
4.34933
1.70783
Assessment of Error
• Random
Error
12.00
11.00
r = 1 infers no error
In general, good agreement
requires r > 0.7
Callipers
r = 0 infers lots of error
10.00
Correlations
Callipers
Hydrostat
Pearson Correlation
Sig . (2-tailed)
N
Pearson Correlation
Sig . (2-tailed)
N
Callipers
1
.
4
.527
.473
4
Hydrostat
.527
.473
4
1
.
4
r2 = 0.278
9.00
8.00
12.00
14.00
16.00
18.00
Hydrostat
20.00
22.00
Assessment of Error
• Systematic &
Random Error
Callipers
10.00
12.00
8.00
11.00
14.00
18.00
10.00
9.00
HydroStat.
17.00
22.00
14.00
12.00
11.00
15.00
21.00
17.00
Assessment of Error
• Systematic &
Random Error
Callipers
10.00
12.00
8.00
11.00
14.00
18.00
10.00
9.00
HydroStat.
17.00
22.00
14.00
12.00
11.00
15.00
21.00
17.00
Difference
7.00
10.00
6.00
1.00
-3.00
-3.00
11.00
8.00
Mean
13.50
17.00
11.00
11.50
12.50
16.50
15.50
13.00
Assessment of Error
• Systematic &
Random Error


10.00

-Systematic Error: are points evenly
distributed about the zero line?

diffe rences
The “Bland-Altman” Plot
3 points of visual assessment:

5.00
Mean = 4.63

0.00
-Random Error: do points deviate
greatly from the mean line?

12.00
-Nature of error: is the error
consistent left-right?

14.00
Mean
16.00
Examples of Bland-Altman Plots
differences
10.00
5.00
Zero
Mean difference
0.00
12.00
13.00
14.00
M ean
15.00
16.00
Examples of Bland-Altman Plots
differences
10.00
5.00
Mean difference
Zero
0.00
12.00
13.00
14.00
M ean
15.00
16.00
Examples of Bland-Altman Plots
differences
10.00
5.00
Zero
Mean difference
0.00
12.00
13.00
14.00
M ean
15.00
16.00
Examples of Bland-Altman Plots
differences
10.00
5.00
Mean difference
Zero
0.00
12.00
13.00
14.00
M ean
15.00
16.00
Examples of Bland-Altman Plots
differences
10.00
5.00
Zero
0.00
12.00
13.00
14.00
M ean
15.00
16.00
Why is Error Important
• Measurement Error is clearly of importance
when evaluating the agreement between two
measurement tools
• A consideration of error is also relevant when
attempting to establish intervention
effects/treatment differences
i.e. where some of the variance between trials is
due to the independent variable...
Dependent Variable
Independent
Variable
Total Variance
between trial 1
& trial 2
Systematic
Variance
Extraneous/
Confounding
(Error) Variables
Error
Variance
Dependent Variable
Independent
Variable
Total Variance
between trial 1
& trial 2
Extraneous/
Confounding
(Error) Variables
Primary
Variance
Systematic
Variance
Systematic
Variance
Error
Variance
So researchers strive to increase the proportion of variance due to IV.
Dependent Variable
Maximise
effect
(20 pints?)
Independent
Variable
Total Variance
between trial 1
& trial 2
Primary
Variance
Extraneous/
Increase
Systematic
control
Confounding
Variance
(Error) Variables
Error Variance
So researchers strive to increase the proportion of variance due to IV.
Introduction to Inferential Statistics
• Before our next lecture you will be conducting
some inferential statistics in your lab classes
• All you need to be aware of at this stage is that
the ‘P-value’ represents the probability that
total variance is not due to primary variance
i.e. P = 0.01 infers a 99 % probability variance in the
DV is not due to pure chance
(i.e. 1 % likelihood of your result occurring if there is in fact no effect)
Introduction to Inferential Statistics
• Before our next lecture you will be conducting
some inferential statistics in your lab classes
• All you need to be aware of at this stage is that
the ‘P-value’ represents the probability that
total variance is not due to primary variance
i.e. P = 0.10 infers a 90 % probability variance in the
DV is not due to pure chance
(i.e. 1 % likelihood of your result occurring if there is in fact no effect)
Introduction to Inferential Statistics
• Before our next lecture you will be conducting
some inferential statistics in your lab classes
• All you need to be aware of at this stage is that
the ‘P-value’ represents the probability that
total variance is not due to primary variance
In exercise science, we must be at least 95 % sure that
our effect is due more than pure chance before
concluding a ‘significant’ difference.
i.e. P  0.05
n.b. this DOES NOT mean that you will find this result in
95/100 test-retests or that your false positive rate is 5 %
n.b. this DOES NOT mean that you will find this result in
95/100 test-retests or that your false positive rate is 5 %
Quantitative Analysis of Nominal Data
• Recall that nominal data infers that variables are
dichotomous, i.e. belong to distinct categories
e.g. Athlete/Non-Athlete, Male/Female, etc.
• We know that such qualitative data can be coded
quantitatively to allow a more objective analysis
• Nominal data does not require any consideration
of normality and is analysed used a Chi2 test.
The Chi-Squared Test
• Goodness of fit χ2 test
– A comparison of your observed frequency counts
against what would be expected according to the
null hypothesis
i.e. null hypothesis infers equal dispersion (50:50)
• Contingency χ2 test
– A comparison of two observed frequency counts
Goodness of fit χ2 test
• Is a leisure centre used more by males
than by females?
– n =150
Observed
Frequency
Expected
Frequency
Male
62
75
Female
88
75
Goodness of fit χ2 test
SPSS Output
Male
Female
Total
Gender
Observed N
62
88
150
Expected N
75.0
75.0
Residual
-13.0
13.0
Test Statistics
Chi-Squarea
df
Asymp. Sig.
P-value AKA
significance level
Gender
4.507
1
.034
i.e. significant difference in
the proportion of users
according to gender
a. 0 cells (.0%) have expected frequencies less than
5. The minimum expected cell frequency is 75.0.
Contingency χ2 test
• Are elite athletes more likely to take
nutritional supplements than non-athletes
– n =60
Do take
supplements
Do not take
supplements
Athletes
18
12
Non-athletes
11
19
Contingency χ2 test
SPSS Output
This is the test
of interest
Group * Response Crosstabulation
Count
Group
Response
Do take
Dont take
supplements
supplements
18
12
11
19
29
31
Athletes
Non-Athletes
Total
Total
30
30
60
Chi-Square Tests
Pearson Chi-Square
Continuity Correction a
Likelihood Ratio
Fisher's Exact Test
Linear-by-Linear
Association
N of Valid Cases
Value
3.270b
2.403
3.301
df
1
1
1
Asymp. Sig.
(2-sided)
.071
.121
.069
Exact Sig.
(2-sided)
Exact Sig.
(1-sided)
.120
3.216
1
.073
60
a. Computed only for a 2x2 table
b. 0 cells (.0%) have expected count less than 5. The minimum expected count is 14.
50.
.060
i.e. no significant difference
in the proportion of users
according to group
Assumptions for Chi-Squared
• Although ND not required…
• Cells in the table should all be independent
i.e. one person could have visited the leisure centre twice
• 80 % of the cells must have expected frequencies
greater than 5 and all must be above 1
i.e. the more categories available, the more subjects needed
• Cannot use percentages
i.e. a 15:45 split cannot be expressed as 25%:75%
Selected Reading
• I know error and variance can be confusing topics, try these:
• Atkinson, G. and A. M. Nevill. Statistical methods for assessing
measurement error (Reliability) in variables relevant to sports medicine.
Sports Medicine. 26:217-238, 1998.
• Hopkins, W. G. et al. Design and analysis of research on sport performance
enhancement. Med. Sci. Sport and Exerc. 31:472-485, 1999.
• Hopkins, W. G. et al. Reliability of power in physical performance tests.
Sports Medicine. 31:211-234, 2001.
• Atkinson, G., ''What is this thing called measurement error?'' , in
Kinanthropometry VIII: Proceedings of the 8th International Conference of
the International Society for the Advancement of Kinanthropometry
(ISAK) , Reilly, T. and Marfell-Jones, M. (Eds.), Taylor and Francis,
London , 2003.
Coursework (60% overall grade)
• Your coursework will require you to address
2 of the following research scenarios:
– 1) Effect of Plyometric Training on Vertical Jump
– 2) Effect of Ice Baths on Recovery of Strength
– 3) Effect of Diet on the Incidence of Muscle Injury
– 4) Effect of Footwear on Sprint Acceleration
– 5) Effect of PMR on Competitive Anxiety.
Coursework Outline
• For each of the 2 scenarios you will need to:
– Perform a literature search in order to provide a
comprehensive introduction to the research area
– Identify the variables of interest and evaluate the
research design which was adopted
– Formulate and state appropriate hypotheses
– Summarise descriptive statistics in an appropriate
and well presented manner…
Coursework Outline
• Cont’d…
– Select the most appropriate statistical test with
justification for your decision
– Transfer the output of your inferential statistics
into your word document
– Interpret your results and discuss the validity and
reliability of the study
– Draw a meaningful conclusion (state whether
hypotheses are accepted or rejected).
Coursework Details (see unit outline)
• 2000 words maximum (i.e. 1000 for each)
• Any supporting SPSS data/outputs to be appended
• To be submitted on Thursday 11th December
Assessment Weighting
Evaluation & Analysis
(30 %)
Reading & Research
(20 %)
Communication & Presentation (20 %)
Knowledge
(30 %)
Coursework Details
• All information relating to your coursework
(including the relevant data files) are accessible
via the unit web page:
www.bath.ac.uk/~jb335/Y1%20Research%20Skills%20(FH10040).
html
Web address also referenced on shared area
Mid-Term Test (40% overall grade)
• NEXT WEEK
• This test will involve short answer questions
covering all the information covered so far
• Mostly knowledge recall but will require
understanding and possibly some calculations
• Duration = 50 min
So…
Mid-Term Test (40% overall grade)
• Surnames: A-H
– Arrive promptly at 11.10 am for start of test at 11.15 am
– Exit in silence afterwards
• Surnames: I-Z
– Arrive promptly at 12.10 am for start of test at 12.15 am
– Exit however you like!
[email protected]