Transcript day5

Sampling (cont.)
Instrumentation
Measurement Plan Due 3/7
Sampling Demonstration
In Small Groups
• Check each others literature review and
hypothesis of choice.
– Do literature review, rationale, and hypothesis go
together?
– Offer constructive criticism-suggestions.
• Then,
– Identify your
• Target population
• Accessible population
– Sampling strategy
• Strengths
• Weaknesses
Instrumentation
Discuss Jared Diamond
Soft Sciences are Often Harder
than Hard Sciences
Instrumentation
Instructions: Circle the choice that indicates your opinion.
1. Teachers’ unions should be abolished.
Strongly
agree
(5)
Agree
(4)
Undecided
(3)
Disagree
(2)
Strongly
disagree
(1)
2. School administrators should be required by law to teach at least one class in a
public school classroom every year.
Strongly
agree
(5)
Agree
(4)
Undecided
(3)
Disagree
(2)
Strongly
disagree
(1)
3. Classroom teachers should be able to choose the administrators in their schools.
Strongly
agree
(5)
Agree
(4)
Undecided
(3)
Disagree
(2)
Strongly
disagree
(1)
What are Data?
• Data refers to the information researchers obtain
on the subjects of their research.
• Demographic information or scores from a test
are examples of data collected.
• The researcher has to determine what kind of
data they need to collect.
• The device the researcher uses to collect data is
called an instrument.
Key Questions
• The instruments and procedures used in collecting data is
called instrumentation.
• Questions arise regarding the procedures and conditions
under which the instruments will be administered:
–
–
–
–
Where will the data be collected?
When will the data be collected?
How often are the data to be collected?
Who is to collect the data?
• The most highly regarded types of instruments can provide
useless data if administered incorrectly, by someone disliked
by respondents, under noisy, inhospitable conditions, or when
subjects are exhausted.
Validity, Reliability, and Objectivity
• Validity is an important consideration in the choice of an
instrument to be used in a research investigation
– It should measure what it is supposed to measure
– Researchers want instruments that will allow them to make
warranted conclusions about the characteristics of the subjects
they study
• Reliability is another important consideration, since
researchers want consistent results from instrumentation
– Consistency gives researchers confidence that the results
actually represent the achievement of the individuals involved
• Objectivity refers to the absence of subjective judgments
Usability
• An important consideration for any researcher in choosing or
designing an instrument is how easy the instrument will actually be
to use.
• Some of the questions asked which assess usability are:
•
•
•
•
•
How long will it take to administer?
Are the directions clear?
How easy is it to score?
Do equivalent forms exist?
Have any problems been reported by others who used it?
• Getting satisfactory answers can save a researcher a lot of time
and energy.
Ways to Classify Instruments
• Who Provides the Information?
– Themselves: Self-report data
– Directly or indirectly: from the subjects of the study
– From informants (people who are knowledgeable
about the subjects and provide this information)
Types of Researcher-completed
Instruments
•
•
•
•
Rating scales
Interview schedules
Tally sheets
Flowcharts
• Performance
checklists
• Observation forms
Types of Subject-completed
Instruments
•
•
•
•
Questionnaires
Self-checklists
Attitude scales
Personality
inventories
• Achievement/aptitude
tests
• Performance tests
• Projective devices
• Sociometric devices
Scientific America
Handwriting Analysis
Item Formats
• Questions used in a subject-completed instrument can
take many forms but are classified as either selection or
supply items.
• Examples of selection items are:
•
•
•
•
True-false items
Matching items
Multiple choice items
Interpretive exercises
• Examples of supply items are:
• Short answer items
• Essay questions
Unobtrusive Measures
• Many instruments require the cooperation of the respondent in one
way or another.
• An intrusion into an ongoing activity could be involved which causes
a form of negativity within the respondent.
• To eliminate this, researchers use unobtrusive measures, data
collection procedure that involve no intrusion into the naturally
occurring course of events.
• In most cases, no instrument is used, however, good record keeping
is necessary.
• They are valuable as supplements to the use of interviews and
questionnaires, often providing a useful way to corroborate what
more traditional data sources reveal.
Types of Scores
• Quantitative data is reported in the form of scores
• Scores are reported as either raw or derived scores
– Raw score is the initial score obtained
• Taken by itself, a raw score is difficult to interpret, since it has little meaning
– Derived score are scores that have been taken from raw scores and
standardized
• They enable researchers to say how well the individual performed compared to
others taking the same test
• Examples include:
– Age and Grade-level Equivalents
– Percentile Ranks
– Standard scores are mathematically derived scores having comparable
meaning on different instruments
Examples of Raw Scores and
Percentile Ranks
Raw
Score
95
93
88
85
79
75
70
65
62
58
54
50
Cumulative
Frequency
1
1
2
3
1
4
6
2
1
1
2
1
N = 25
Percentile
Frequency
25
24
23
21
18
17
13
7
5
4
3
1
Rank
100
96
92
84
72
68
52
28
20
16
12
4
Norm-Referenced vs. CriterionReferenced Instruments
• All derived scores give meaning to individual scores by
comparing them to the scores of a group.
• The group used to determine derived scores is called the
norm group and the instruments that provide such
scores are referred to as norm-referenced instruments.
• An alternative to the use of achievement or performance
instruments is to use a criterion-referenced test.
• This is based on a specific goal or target (criterion) for
each learner to achieve.
• The difference between the two tests is that the criterion
referenced tests focus more directly on instruction.
Descriptive Statistics
Statistics vs. Parameters
• A parameter is a characteristic of a population.
– It is a numerical or graphic way to summarize data
obtained from the population
• A statistic is a characteristic of a sample.
– It is a numerical or graphic way to summarize data
obtained from a sample
Types of Numerical Data
•
There are two fundamental types of
numerical data:
1)
2)
Categorical data: obtained by determining the
frequency of occurrences in each of several
categories
Quantitative data: obtained by determining
placement on a scale that indicates amount or
degree
Techniques for Summarizing and
Presenting Quantitative Data
• Visual
–
–
–
–
Frequency Distributions
Histograms
Stem and Leaf Plots
Distribution curves
• Numerical
– Central Tendency
– Variability
Summary Measures
Summary Measures
Variation
Central Tendency
Arithmetic
Mean
Median Mode
Range
Variance
Standard Deviation
Measures of Central Tendency
Central Tendency
Average (Mean)
Median
n
X 
X
i 1
n
N

X
i 1
N
i
i
Mode
Mean
• The most common measure of central
tendency
• Affected by extreme values (outliers)
0 1 2 3 4 5 6 7 8 9 10
Mean = 5
0 1 2 3 4 5 6 7 8 9 10 12 14
Mean = 6
Median
• Robust measure of central tendency
• Not affected by extreme values
0 1 2 3 4 5 6 7 8 9 10
Median = 5
0 1 2 3 4 5 6 7 8 9 10 12 14
Median = 5
• In an Ordered array, median is the
“middle” number
– If n or N is odd, median is the middle number
– If n or N is even, median is the average of the
two middle numbers
Mode
•
•
•
•
•
•
A measure of central tendency
Value that occurs most often
Not affected by extreme values
Used for either numerical or categorical data
There may may be no mode
There may be several modes
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Mode = 9
0 1 2 3 4 5 6
No Mode
Variability
• Refers to the extent to which the scores on a quantitative
variable in a distribution are spread out.
• The range represents the difference between the highest
and lowest scores in a distribution.
• A five number summary reports the lowest, the first
quartile, the median, the third quartile, and highest score.
– Five number summaries are often portrayed graphically by the
use of box plots.
Variance
• The Variance, s2, represents the amount of variability of the
data relative to their mean
• As shown below, the variance is the “average” of the
squared deviations of the observations about their mean
s
2
( x  x)


i
n 1
2
Standard Deviation
• Considered the most useful index of variability.
• It is a single number that represents the spread of a
distribution.
• If a distribution is normal, then the mean plus or minus 3
SD will encompass about 99% of all scores in the
distribution.
Calculation of the Variance and Standard
Deviation of a Distribution (Definitional formula)
Raw
Score
85
80
70
60
55
50
45
40
30
25
Mean
54
54
54
54
54
54
54
54
54
54
X–X
31
26
16
6
1
-4
-9
-14
-24
-29
2
(X – X)
961
676
256
36
1
16
81
196
576
841
2
Σ(X – X)
Variance (SD ) =
N-1
2
Standard deviation (SD) =
=
3640
=404.44
9
2
√
Σ(X – X)
N-1
Comparing Standard Deviations
Data A
11 12 13 14 15 16 17 18 19 20 21
Mean = 15.5
S = 3.338
Data B
11 12 13 14 15 16 17 18 19 20 21
Mean = 15.5
S = .9258
Data C
11 12 13 14 15 16 17 18 19 20 21
Mean = 15.5
S = 4.57
Facts about the Normal Distribution
• 50% of all the observations fall on each side of the
mean.
• 68% of scores fall within 1 SD of the mean in a
normal distribution.
• 27% of the observations fall between 1 and 2 SD
from the mean.
• 99.7% of all scores fall within 3 SD of the mean.
• This is often referred to as the 68-95-99.7 rule
The Normal Curve
Different Distributions Compared
Fifty Percent of All Scores in a Normal
Curve Fall on Each Side of the Mean
Probabilities Under the Normal Curve
Standard Scores
• Standard scores use a common scale to indicate how an
individual compares to other individuals in a group.
• The simplest form of a standard score is a Z score.
• A Z score expresses how far a raw score is from the
mean in standard deviation units.
• Standard scores provide a better basis for comparing
performance on different measures than do raw scores.
• A Probability is a percent stated in decimal form and
refers to the likelihood of an event occurring.
• T scores are z scores expressed in a different form (z
score x 10 + 50).