Transcript There is no significant difference in the mean overall test

EDU5950
SEM1 2015-16
RELIABILITY ANALYSIS
-CRONBACH ALPHA
TEST FOR NORMALITY
STEPS IN CONDUCTING RESEARCH
MEASUREMENT



Consists of assigning numbers or labels to units of
analysis to represent the variables/phenomenon
under study.
The key variables need to be translated into numbers
so the researcher can analyze the data using
statistics.
The process of measurement consists of moving from
the theoretical definition of a variable (typically called
the construct definition) to the concrete mode of
measuring that variable in the research study.
MEASUREMENT
(REFER TO TEACHER EFFICACY DATA)





When designing an instrument, keep in mind the
following:
The conclusions drawn in a research study are only as
good as the data that is collected.
The data that is collected is only as good as the
instrument that collects the data.
A poorly designed instrument will lead to bad data,
which will lead to bad conclusions.
Therefore, developing a good instrument is the most
important part of conducting a high quality research
study.
VALIDITY AND RELIABILITY
Validity is the most important consideration
in developing and evaluating measuring
instruments.
 Validity is the degree to which evidence and
theory support the interpretations on the
meaning of the scores derived from the
instrument
 Content Validity: “based on expert ratings of
the items” in the test or measurement
 Construct Validity: “based on the extent of
scores derived from the instrument truly
reflect the theory behind the psychological
construct being measured.

5
RELIABILITY


Reliability refers to “how well we are measuring
whatever it is that is being measured (regardless of
whether or not it is the right quantity to measure).”
In statistics or measurement theory, a measurement
or test is considered reliable if it produces consistent
results over repeated testings.
-D. Rindskopf, Reliability: Measurement. In: Neil J. Smelser and
Paul B. Baltes, Editor(s)-in-Chief, International Encyclopedia of
the Social & Behavioral Sciences, Pergamon, Oxford, 2001,
Pages 13023-13028.
(http://www.sciencedirect.com/science/article/B7MRM-4MT09VJ2XN/1/083e3cc0b8b9d4e027b0ba214dcd9fa3)
6
HOW CAN RELIABILITY BE ESTABLISHED?



Test-Retest reliability – administer a test or instrument
to the same group of individuals on two occasions and
correlate the two sets of scores
Equivalent-Forms reliability – administer two
equivalent forms of test or instrument to the same
individuals
Internal-Consistency measures of reliability – test
whether all the items in the test or instrument are
measuring the same thing. It is a measure of
homogeneity of the items.



Split-Half reliability
Kuder-Richardson procedures
Coefficient Alpha (Cronbach Alpha after Lee Cronbach)
7
•RELIABILITY ANALYSIS
REFER TO MATHEMATICS TEACHERS’
EFFICACY DATA
1. RECODE ALL NEGATIVE ITEMS
2. CLICK ANALYZE => SCALE => RELIABILITY
ANALYSIS
(YOU WILL GET THE RELIABILITY DIALOG BOX)
3. TRANSFER THE MEASURED ITEMS TO THE RIGHT
BOX – 14 ITEMS FOR TEACHERS’ EFFICACY SCALE
4. SELECT THE FOLLOWING ICONS => CONTINUE
=> OK
Item Statistics
Mean
Std. Deviation
N
My teacher wants us to enjoy learning maths
3.7419
1.55696
62
My teacher understand our problems in learning maths
3.9355
1.48071
62
My teacher try to make mathematics lessons interesting
3.9839
1.53101
62
4.2097
1.48365
62
My teacher show us step by step and how to solve maths problems
4.2258
1.53023
62
My teacher listen carefully to what we say
4.1290
1.18022
62
My teacher is friendly to us
3.5323
1.50101
62
My teacher gives us time to explore new maths problems
3.7581
1.19679
62
My teacher wants us to understand the content of this maths class
4.4032
1.38445
62
My teacher explains why mathematics is important
3.9677
1.49280
62
We do a lot of group work in mathematics class
3.1129
1.31952
62
3.9677
1.51460
62
A_TF13 RECODE
4.0161
1.41990
62
A_TF14 RECODE
4.5161
1.25112
62
My teacher appreciates it when we try hard, even when our results are not so
good
My teacher thinks mistakes are okey as long as we are learning from them
Reliability Statistics
Cronbach's
Alpha
.891
Cronbach's
N of Items
Alpha Based on
Standardized
Items
.890
14
Item-Total Statistics
Scale Mean if
Item Deleted
My teacher wants us to enjoy
learning maths
My teacher understand our
problems in learning maths
My teacher try to make
mathematics lessons
interesting
Scale Variance if Corrected Item- Squared Multiple
Item Deleted Total Correlation
Correlation
Cronbach's
Alpha if Item
Deleted
51.7581
134.285
.753
.731
.874
51.5645
146.479
.425
.421
.890
51.5161
134.287
.768
.784
.874
51.2903
136.242
.735
.890
.875
51.2742
134.399
.765
.882
.874
51.3710
143.024
.690
.652
.879
My teacher appreciates it
when we try hard, even when
our results are not so good
My teacher show us step by
step and how to solve maths
problems
My teacher listen carefully to
what we say
51.9677
138.097
.667
.643
.879
51.7419
142.752
.689
.643
.879
51.0968
140.056
.669
.845
.879
51.5323
137.696
.684
.623
.878
52.3871
160.536
.048
.446
.904
51.5323
143.761
.491
.615
.887
A_TF13 RECODE
51.4839
155.467
.181
.574
.900
A_TF14 RECODE
50.9839
147.983
.471
.534
.887
My teacher is friendly to us
My teacher gives us time to
explore new maths problems
My teacher wants us to
understand the content of
this maths class
My teacher explains why
mathematics is important
We do a lot of group work
in mathematics class
My teacher thinks mistakes
are okey as long as we are
learning from them
REPORTING
The pilot study was administered to 31
graduate students undertaking Statistics for
Educational Research. The internal
consistency estimates, based on Cronbach’s
alpha, were satisfactory thus acceptable.
 Based on 14 items, the Cronbach’s alpha
obtained is .891. However further analysis
using 12 items (by omitting item 11 AND 13)
the internal consistency increased to .904.

Inter-Item Correlation Matrix - - a 4-Item Summated Scale for Measuring ORG. LOYALTY (n = 518)
1. If I were completely 2. I feel a
4. I don't have a strong
free to choose, I would strong sense of 3. I often think personal desire to
continue working for my loyalty to the of leaving the continue working for my
current employer
org I work for org. I work for
current employer
1. If I were completely free to choose, I would continue
working for my current employer
1.000
.690
.646
.743
2. I feel a strong sense of loyalty to the org I work for
.690
.573
.674
3. I often think of leaving the org. I work for
.646
1.000
.573
1.000
.711
4. I don't have a strong personal desire to continue
working for my current employer
.743
.674
.711
1.000
Response Options: 7-Point Scales (1= Strongly Disagree, 7 = Strongly Agree)
Reliability Statistics
Cronbach's
Alpha
.891
Cronbach's Alpha
Based on
Standardized Items
N of Items
.892
4
Item-Total Statistics
Scale Mean if
Item Deleted
Scale
Variance if
Item Deleted
Corrected ItemTotal
Correlation
Squared
Multiple
Correlation
Cronbach's
Alpha if Item
Deleted
9.81
9.81
.789
.633
.849
9.71
12.803
.720
.539
.874
10.06
11.857
.721
.541
.876
9.76
11.447
.816
.668
.838
If I were completely free to choose, I would
continue working for my current employer
I feel a strong sense of loyalty to the org I work
for
I often think of leaving the org. I work for
I don't have a strong personal desire to continue
working for my current employer
Use
these for
item
analysis;
i.e.,
determining
quality of
individual
items.
•Test for normality of scores
TEST FOR NORMALITY
TEST FOR NORMALITY
TEST FOR NORMALITY
Descriptives
Statistic
Mean
95% Confidence Interval for Mean
5% Trimmed Mean
Median
Variance
TEACHER_E
FFICACY
Std. Deviation
Minimum
Maximum
Range
Interquartile Range
Skewness
Kurtosis
Lower
Bound
Upper
Bound
3.9643
3.7321
Std. Error
.11613
4.1965
3.9507
3.7857
.836
.91443
2.21
5.71
3.50
1.52
.388
-.852
.304
.599
TEST FOR NORMALITY
Tests of Normality
Kolmogorov-Smirnova
Statistic
.110
TEACHER_EFFICACY
a. Lilliefors Significance Correction
df
Shapiro-Wilk
Sig.
62
.061
Statistic
.951
df
Sig.
62
.016
TEST FOR NORMALITY
Extreme Values
Case Number
Value
23
5.71
1
33
5.71
2
5
5.64
Highest
3
27
5.64
4
11
5.57
5
TEACHER_EFFICACY
12
2.21
1
24
2.50
2
51
2.71
Lowest
3
45
2.71
4
59
2.93a
5
a. Only a partial list of cases with the value 2.93 are shown in the table of lower
extremes.
TEST FOR NORMALITY
TEST FOR NORMALITY
TEACHER_EFFICACY Stem-and-Leaf Plot
Frequency
Stem & Leaf
1.00
2 . 2
6.00
2 . 577999
15.00
3 . 000012222222224
14.00
3 . 55556677778899
6.00
4 . 001222
9.00
4 . 556677779
6.00
5 . 001333
5.00
5 . 56677
Stem width:
1.00
Each leaf:
1 case(s)
TEST FOR NORMALITY
TEST FOR NORMALITY
TEST FOR NORMALITY
EXPLORING DATA
•
Normality can be assessed in SPSS using the Explore option
of the Descriptive Statistics menu.
•
In this example we will assess the normality distribution of
Scores of Test I. We wish to assess for the sample as a whole.
If we want to do separately for subgroup within our sample,
moves the variables that define these subgroups into the
factor list.
TO ASSESSING NORMALITY

Click Analyze => Descriptive Statistics =>
Explore.

Click and move variables you require statistics,
graphs and test for into Dependent List.

In Display section, select Both. In the Explore
Statistics dialogue, Descreptive are selected by
default.Click Continue.

Click on the Plot button. Select Histogram and
Normality plots with test. Click on Continue.

Click on Option. In the Missing value section, click
Exclude cases pairwise. Click continue. Then OK.
OUTPUTS
OUTPUTS




The descriptive statistics is
shown in the tables.
To obtain the 5% Trimmed
Mean, SPSS removes the top
and bottom 5% of your cases
and recalculate a new mean
value.
Compare the original mean
with new trimmed mean to
see whether some of the
more extreme scores are
having a strong influence on
the mean.
If these two mean values are
very different, you may need
to investigate these data
point further.
OUTPUTS




The Kolmogorov-Smirnov
statistic assess the
normality of the
distribution of scores.
A non-significant result (
Sig value more than .05 )
indicates normality.
The actual shape of the
distribution can be seen in
the Histograms.
For this Histograms, scores
appear to be reasonably
normally distributed.
OUTPUTS




The Normal Q-Q Plots shows
the observed value for each
score is plotted against the
expected value from the
normal distribution.
A reasonably straight line
suggests a normal
distribution.
The Detrended Normal Q-Q
Plots displayed in the output
are obtained by plotting the
actual deviation of the
scores from the straight
line.
There should be no real
clustering of points, with
most collecting around the
zero line.
EXAMPLE OF WRITE-UP
Ho1: There is no significant difference in the mean overall test
performance in the learning of the Straight Lines topic between the
graphing calculator (GC) strategy group and the conventional
instruction (CI) strategy group.
The means and standard deviations of the overall test performance for
both the GC and the CI strategy groups are provided in Table 4.4. The
overall performance test ranged between 0 and 40. Mean overall test
performance of the GC strategy group was 16.81 (SD=4.76) while mean
overall test performance of the CI strategy group was 12.53 (SD=4.99). An
independent t-test analysis showed that the difference in the means were
significant, t(38)=2.78, p<.05.
The results indicated that there was a significant difference in the mean
overall test performance in the learning of the Straight Lines topic between
the GC strategy group and the CI strategy group. The magnitude of the
differences in the means was considered large based on Cohen (1988)
with eta squared =.17. The guidelines proposed by Cohen (1988) for
interpreting this value are: .01 = small effect, .06 = moderate effect, .14 =
large effect. This finding indicated that the GC strategy group had
performed significantly better than the CI strategy group.
EXAMPLE WRITE-UP
Table 4.3: Independent samples t-test to compare means
monthly test performance before experiment in Phase I
Group
GC strategy
n
21
M
59.00
SD
10.25
MD
t
df
p
CI strategy
19
59.26
21.19
-.263
-.049
25.41
.961
Table 4.3 shows the results of the analysis. The total monthly test performance was
100. The mean performance for the GC strategy group and the CI strategy group were
59.00 and 59.26 respectively. Levene’s test indicated that the assumption for equal
variance has been violated, F=9.95, p<.05. Therefore the reading for the output for the
independent t-test was based on the reading for equal variance not assumed. The
results of the t-test showed that there was statistically no significant difference
between the mean monthly test performance for the GC strategy group and the CI
strategy group, t(25.41)=−.049, p>.05. This suggested that the students’ mathematics
performance for both groups in the group did not differ significantly.
H01: There is no significant difference in the mean overall performance in the
learning of statistic between the PBL-Tr, PBL-Web and Conv groups.
The means and standard deviations of the overall performance for the PBL-Tr, PBLWeb and the Conv strategy groups and also results of the ANOVA test. The overall
performance score ranged between 0 and 100. Mean overall performance of the
PBL-Tr group was 70.08 (SD=12.08) while mean overall performance of the PBLWeb group was 80.18 and (SD=17.2) and the mean of overall performance of the
Conv strategy group was 56.6 (SD=20.38). An ANOVA test analysis showed that the
difference in the means were significant, F(2,61)=6.35, p<.05.
The results indicated that there was significant difference in the mean overall
performance in the learning of statistics between the three groups. The magnitude of
the differences in the means was considered large based on Cohen (1988) with eta
squared (ES) = 0.172. The guidelines proposed by Cohen (1988) for interpreting
this value are: .01 = small effect, .06 = moderate effect, .14 = large effect. Based on
Post-Hoc test the mean of overall performance for Conv group was significantly
lower than PBL-Tr and PBL-Web groups. However, PBL-Tr did differ significantly
from PBL-Web group at 5% level of significance.