Transcript Correlation & Causal Comparative Research

Correlation & Causal
Comparative Research
Class 6
This Week’s Schedule
• Today – Review and continue w/ statistical analysis
• Tuesday
– 3 individual meetings 9-10am
– Full class (Stats & Method) 10-11am
– 3 individual meeting 11am-12noon
• Wednesday
– 9am-10am – Music ed history (Skype w/ Eastman Class)
Show and tell!!
– 10:00am-11:00am – Qualitative Research-Full class
– 11:00am-12noon-3 individual meetings
This Week’s Schedule
• Thursday
– 4 Project Presentations (20 minutes)
– 2 qualitative/historical 5ish minute presentations
(in pairs & a trio)
– Disseminating research
• Friday
– 5 Project Presentations
– 2 qualitative/historical 5ish minute presentations
(in pairs & a trio)
Assignments
• Tuesday – Work on presentations, projects, etc.
• Wednesday
– Read Queen Bees and Wanna Bees chapt. 1 OR 6
– Read one historical article from the Journal of Historical
Research in Music Education. Be prepared to write or discuss
– Chapter 3 – Method
• Thursday & Friday
– Project Presentations (20 minute w/ 5 minutes for questions &
discussion)
– Informal presentation in pairs of a qualitative or an historical
article
• Monday, July 22 by 5pm – Final Project Proposal
Who & When
• Tuesday Meetings
– 9-10 (3)
– 11-noon (3)
• Weds Meetings
– 11-noon (3)
• Thursday
– Project Presentations
(4)
– Qual./Hist.
presentations (2
pairs)
• Friday
– 5 presentations
– Qual./Hist. trio
presentation
APA Format
• Headers
– Chapter title = Level 1
– Others = Level 2 (Flush left)
• Remember title page, page #s
• Running Head – < 50 characters total. Goes in the
header flush left
• Research Question after purpose statement & before
need for study (Header?)
• Commas = …apples, oranges, and grapes.
Types of Data – Revised
simple to complex; lowest to highest
• Nominal/Categorical = numbers as labels
– Male/female (1 or 2)
– Sop/Alto/tenor/bass (1, 2, 3, 4)
• Ordinal = ranks
– Contest ratings
• Interval = Scale (equal distance b/w each number)
– Contest scores (1-100)
– Lack of meaningful zero (0 on test = no knowledge?, 0 temperature =
arbitrary) or meaningful ratios (2x as smart?)
• Ratio =
– Equal interval data
– True zero possible (0 decibels, 0 money)
– Ratios can be calculated in a meaningful way [2x as loud, ½ money,
height, weight, depth (a lake can dry up) (?), etc.]
Terms
• Inferential statistics
• Parametric vs. non-parametric
• Assumptions/Parameters
– Variances?
– Randomization
• Mean vs. variance
• Used to compare 2 groups and no more?
• Independent vs. dependent (paired or correlated)
• One tail vs. Two tail tests?
Terms
• What is I have more than 2 groups? I need
a…?
• If there is a significant difference in the test
above, then what do I need to do?
• Why do we test the significance of the
difference in variances?
• What if the variances are sig. different?
Statistical Significance
• Probability that result happened by chance and not
due to treatment
–
–
–
–
–
Expressed as p
p < .1 = less than 10% (1/10) probability…
p < .05 = less than 5% (1/20) probability…
p < .01 – less than 1% (1/100) probability…
p < .001 – less than .1% (1/1000) probability…
• Computer software reports actual p
• alpha level = probability level to be accepted as
significant set b/f study begins
• Statistical significance does not equal practical
significance
Statistical Power
• Likelihood that a particular test of statistical
significance will lead to the rejection of null
hypothesis
– Parametric tests more powerful than nonparametric. (Par.
more likely to discover differences b/w groups. Choice
depend on type of data)
• The larger the sample size, the more likely you will be
to find statistically significant effects.
• The less stringent your criteria (e.g., .05 vs. 01 vs.
001), the easier it is to find statistical significance
Statistical Tests
http://pspp.awardspace.com/ (Windows)
http://bmi.cchmc.org/resources/software/pspp (Mac)
http://vassarstats.net/
See Handout from Friday
•
•
•
•
•
Awareness of non-parametric tests
3 groups, ordinal data?
2 groups, interval data?
2 groups, nominal/categorical data?
Relationship b/w two groups, ordinal data?
Independent Samples t-test
• Used to determine whether differences between two
independent group means are statistically significant
• n = < 30 for each group. Though many researchers
have used the t test with larger groups.
• Groups do not have to be even. Only concerned with
overall group differences w/o considering pairs
– [A robust statistical technique is one that performs well even if its assumptions
are somewhat violated by the true model from which the data were
generated. Unequal variances = alternative t test or better Mann-Whitney U]
• Application: Explore Data
– Compare science tests of inst & non-inst. students
Correlated (paired, dependent) Samples ttest
• Used to determine differences between two means
taken from the same group, or from two groups with
matched pairs are statistically significant
– e.g., pre-test achievement scores for the whole song group
vs. post-test achievement scores for the whole song group
• Group size must be even (paired)
• N = < 30 for each group
• Application: Compare Reading & Math test scores of
Instrumental Students
Compare 2 means
• Need sample of at least 10
• Work like Independent and dependent t tests
• Independent
– Mann Whitney U
• Application: Data set #3. Is there a sig. diff. b/w Final ratings at
Site 1 vs. site 2?
• Pairs or dependent samples
– Wilcoxon signed ranks
• Application: Data set #2. Is there a sig. difference b/w rating of
judges 1 & 2?
ANOVA
•
•
•
•
Analyze means of 2+ groups
Homogeneity of variance
Independent or correlated (paired) groups
More rigorous than t-test (b/w group & w/i group
variance). Often used today instead of T test.
• F statistic
• One-Way = 1 independent variable
• Two-Way/Three-Way = 2-3 independent variables
(one active & one or two an attribute)
One-Way ANOVA
• Calculate a One-Way ANOVA for data-set 1 – All noninstrumental tests
• Post Hoc tests
– Used to find differences b/w groups using one test. You
could compare all pairs w/ individual t tests or ANOVA, but
leads to problems w/ multiple comparisons on same data
– Tukey – Equal Sample Sizes (though can be used for unequal sample
sizes as well)
– Sheffe – Unequal Sample Sizes (though can be used for equal sample
sizes as well)
ANCOVA – Analysis of Covariance
• Statistical control for unequal groups
• Adjusts posttest means based on pretest
means.
• [example]
http://faculty.vassar.edu/lowry/VassarStats.ht
ml
•
[The homogeneity of regression assumption is met if within each of the groups there is an linear
correlation between the dependent variable and the covariate and the correlations are similar b/w
groups]
Effect Size (Cohen’s d)
http://www.uccs.edu/~faculty/lbecker/es.htm
http://www.uccs.edu/~lbecker/
• [Mean of Experimental group – Mean of Control group/average SD]
• The average percentile standing of the average treated (or experimental)
participant relative to the average untreated (or control) participant.
• Use table to find where someone ranked in the 50th percentile in the
experimental group would be in the control group
• Good for showing practical significance
– When test in non-significant
– When both groups got significantly better (really effective vs. really
really effective!
• Calculate effect size:
– Treatment group: M=24.6; SD=10.7
– Control Group: M=10.8; SD=7.77
Cohen's Standard
LARGE
MEDIUM
SMALL
Effect Size
Percentile Standing
Percent of Nonoverlap
2.0
97.7
81.1%
1.9
97.1
79.4%
1.8
96.4
77.4%
1.7
95.5
75.4%
1.6
94.5
73.1%
1.5
93.3
70.7%
1.4
91.9
68.1%
1.3
90
65.3%
1.2
88
62.2%
1.1
86
58.9%
1.0
84
55.4%
0.9
82
51.6%
0.8
79
47.4%
0.7
76
43.0%
0.6
73
38.2%
0.5
69
33.0%
0.4
66
27.4%
0.3
62
21.3%
0.2
58
14.7%
0.1
54
7.7%
0.0
50
0%
Chi-Squared
• Measure statistical significance b/w frequency
counts (nominal/categorical data)
• http://www.quantpsy.org/chisq/chisq.htm
• Test for independence: Compare 2 or more
proportions
• Goodness of Fit: compare w/ you have with what is
expected
– Proportions of contest ratings (I, II, III or I & non Is)
– Agree vs. Disagree
• Weak statistical test
Correlation
Pearson
Spearman
Cronbach’s alpha (α)
Correlational Research Basics
• Relationships among two or more variables
are investigated
• The researcher does not manipulate the
variables
• Direction (positive [+] or negative [-]) and
degree (how strong) in which two or more
variables are related
Uses of Correlational Research
• Clarifying and understanding important
phenomena (relationship b/w variables—
e.g., height and voice range in MS boys)
• Explaining human behaviors (class
periods per weeks correlated to practice
time)
• Predicting likely outcomes (one test
predicts another)
Uses of Correlation Research
• Particularly beneficial when experimental studies are difficult
or impossible to design
• Allows for examinations of relationships among variables
measured in different units (decibels, pitch; retention
numbers and test scores, etc.)
• DOES NOT indicate causation
– Reciprocal effect (a change in weight may affect body image, but body
image does not cause a change in weight)
– Third (other) variable actually responsible for difference (Tendency of
smart kids to persist in music is cause of higher SATs among HS music
students rather than music study itself)
Interpreting Correlations
– r
• Correlation coefficient (Pearson, Spearman)
• Can range from -1.00 to +1.00
– Direction
• Positive
– As X increases, so does Y and vice versa
• Negative
– As X decreases, Y increases and vice versa
– Degree or Strength (rough indicators)
•
•
•
•
< + .30; small
< + .65; moderate
> + .65; strong
> + .85; very strong
– r2 (% of shared variance)
• % of overlap b/w two variables
• percent of the variation in one variable that is related to the variation
in the other.
• Example: Correlation b/w musical achievement and minutes of
instruction is r = .86. What is the % of shared variance (r2)?
– Easy to obtain significant results w/ correlation. Strength is most
important
Application
• Rate your principal & school quality on a scale of 1-7
• Principal: (1=highly ineffective; 2=ineffective;
3=somewhat ineffective; 4=neither effective nor
ineffective; 5=somewhat effective; 6=effective;
7=highly effective
• School cleanliness: (1=very dirty; 2=dirty;
3=somewhat dirty; 4=neither dirty or clean;
5=somewhat clean; 6=clean; 7=very clean)
• Type of data? Calculation (Pearson or Spearman?)
• Reliability (Cronbach’s alpha)
www.gifted.uconn.edu/siegle/research/.../reliabilitycalculator2.xls
Interpreting Correlations (cont.)
• Words typically used to describe correlations
– Direct (Large values w/ large values or small values w/ small
values. Moving parallel. 0 to +1
– Indirect or inverse (Large values w/small values. Moving in
opposite directions. 0 to -1
– Perfect (exactly 1 or -1)
50 75 9
– Strong, weak
40 62 14
– High, moderate, low
35 53 20
– Positive, Negative
24 35 45
• Correlations vs. Mean Differences 15
21 58
– Groups of scores that are correlated will not necessarily have
similar means (e.g., pretest/posttest). Correlation also works w/
different units of measurement.
Statistical Assumptions
• The mathematical equations used to determine various correlation
coefficients carry with them certain assumptions about the nature of the
data used…
– Level of data (types of correlation for different levels)
– Normal curve (Pearson, if not-Spearman)
– Linearity (relationships move parallel or inverse)
• Non linear relationship of # of performances & anxiety scores =
Young students initially have a low level of performance anxiety,
but it rises with each performance as they realize the pressure and
potential rewards that come with performance. However, once
they have several performances under their belts, the anxiety
subsides. (
– Presence of outliers (all)
– Ho/mo/sce/da/sci/ty – relationship consistent throughout
• Performance anxiety levels off after several performances and
remains static (relationship lacks Homoscedascity)
– Subjects have only one score for each variable
Correlational Approaches for Assessing
Measurement Reliability
• Consistency over time
– test-retest (Pearson, Spearman)
• Consistency within the measure
– internal consistency (split-half, KR-20,
Cronbach’s alpha)
– Spearman Brown Prophecy formula
• 2r/(1 + r)
• Among judges
– Interjudge (Cronbach’s Alpha)
• Consistency b/w one measure and another
– (Pearson, Spearman)
Reliability of Survey
• What broad single dimension is being studied?
– e.g. = attitudes towards elementary music
– Preference for Western art music
– “People who answered a on #3 answered c on #5”
• Use Cronbach’s alpha
– Measure of internal consistency
– Extent to which responses on individual items
correspond to each other
Spearman Brown
Prophesy Formula
• Reliability = ___n x r___
1+(n-1)r
• n=number of times we multiply items to get
new test length (10 item to 20 item – n=2)
• For a test of 10 items w/ reliability (α) of .60
–
–
–
–
(15 items) 1.5 x .60/1+(1.5 - 1).60 = reliability for test 1.5x size
(20 items) 2 x .60/1+(2-1).60 = reliability for a test 2x size
(25 items) 2.5 x .60/1+(2.5 – 1).60 = reliability for test 2.5x size
(5 items) .5 x .60/1+(.5 – 1).60 = reliability for test .5 size