Transcript Survey Lecture

Survey Designs
Basic Elements of
Questionnaires and
Survey Research
Questions in the Survey
Variable
Variable Type
Data Type
Dependent/
Independent
Nominal/ Ordinal/
Interval/ Ratio
Age
Independent
Ratio
Undergraduate
Independent
Nominal
Independent
Nominal
Independent
Nominal
(Where?)
First degree
(What field?)
Current degree
(What field?)
Questions:
Demographic & Descriptive

Typically the first few questions in a survey help to
describe the respondent in some way that establishes
what subgroup of the population he/she represents

Population variables may include things like
age, income, years of education completed
these are ratio data—they can be averaged
place of birth, highest degree, ethnicity
these are nominal data—they name categories

Population variables are typically independent variables
5. How much to you agree or
disagree with this statement:
“Most days I am enthusiastic about being a student.”





Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
Opinion questions may be independent or
dependent, depending on how they are used
in the survey
5. How much to you agree or
disagree with this statement:
“Most days I am enthusiastic about being a student.”





Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
The scale used here is a “5-point Likert scale”
Likert Scales:








Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
Usually 5 point, but some prefer 7 point or 9 point
Typically a Likert scale provides ordinal data
May be reduced to nominal by combining all
“Agree” categories and all “Disagree” categories
Summary Idea
When using surveys, it is important to consider
-variable type (dependent or independent)
-data type (nominal, ordinal, interval, ratio)
These features will affect the analysis plan,
which is the procedures that can be used to
analyze the data.
What research questions?
1. Age
6. Groups you belong to
(gender, student, ethnicity)
2. Place of first degree
3. Field of first degree
7. Academic activities
8. Opinion about Malaysia
Government—English
4. Field of current degree 9. How the use of English
has affected you
5. Enthusiastic student
10. Hours in a week spent
in activities
Survey Designs

Longitudinal—changes within a population
over time

Cohort Studies—a longitudinal study of a
subgroup in a population (over time)

Panel Study—a cohort study of the same
group of people (over time)
Survey Designs
Cross Sectional—data are collected at one
point in time




Can measure current attitudes, beliefs opinions
Can measure needs
Can be used to evaluate a program
May be used to study one group, or compare groups
Study Designs
Time of Data Collection
Study Over Time
Study at a Point in Time
Longitudinal
Cross-sectional
Population
Trends
Subgroup
Changes
Same
Group
Changes
Attitudes Community Program
and
Needs
Evaluation
Practices
Trend
Cohort
Panel
Single group
or Comparisons between groups
Who Did I Survey?

UKM Students
 Graduate Students
 Students Taking Research Class
 Students in Prof. Subahan’s Class
 Students who have e-mail access
 Students who have Internet access
 Students willing to do the survey
Levels of selection




Population (Who do I want to talk about?)
Group of individuals that have one characteristic
(independent variable) in common
Target Population (Who can be chosen?)
List of individuals that qualify for study (they share
one or more an independent variables of importance)
(also known as Sampling Frame)
Sample (Who is actually chosen?)
The set of individuals selected to be surveyed
Respondents (Who actually takes part?)
The group of individuals who actually take part
Levels of Selection

Population—UKM Graduate students

Target Population—Graduate students in
Prof. Subahan’s class


Sample—Members of Prof. Subahan’s class
who have e-mail and Internet access.
Respondents—Member’s of Prof. Subahan’s
class in the sample who responded (n=30+)
Inferences From Data
For n=30
 Masters Student 83.3%
 Doctoral Student 20.0%
 Age = 35.9 years
For n=10 (first 10 respondents)
 Masters Student 67.2%
 Doctoral Student 35.0%
 Age = 34.9 years
Limitations of Information
Where did you do your undergraduate degree?
 Malaysia. 83.9%
 Asian country other than Malaysia. 3.2%
 Europe. 3.2%
 North America. 0.0%
 Africa 0.0%
 Other 9.7%
Inaccuracies in data
Tell which of the following groups you consider
yourself to belong to. If more than one applies to you,
check all that apply:
Male
53.3%
Female 60.0%
In this case, all respondents fit one category or the other.
The total number of responses should be 100%.
Forms of Data Collection





Mailed questionnaires
Electronic questionnaires
One-on-one interviews
Telephone interviews
Focus group interviews
Focus group—a small group of respondents
assembled to respond to survey topics
Open-ended Responses
Describe how the teaching of mathematics and
science in English has affected you.

Thematic analysis
researcher reads responses and looks for themes that
help to describe the situation, using an external
theoretical framework to inform analysis and
conclusions

Targeted analysis (Grounded Theory)
researcher draws categories from data and looks for
statements that relate—building theory that informs
further analysis and conclusions
Thematic Analysis
Describe how the teaching of mathematics and
science in English has affected you.
I am more confident and dare to talk to people using
English. The quality of my teaching seems like it has
decreased! I cannot explain some points very clearly
and sometimes I cannot give examples spontaneously.
However, I like to teach science in English. From this
system, I get to know the weaknesses of my English
and I have the chance to force myself to improve and
practice my English.
Thematic Analysis
Effect on the person (outside of classroom)
 I am more confident and dare to talk to people using English.
 I get to know the weaknesses of my English and I have the
chance to force myself to improve and practice my English.
Effect on teaching
 The quality of my teaching seems like it has decreased! I
cannot explain some points very clearly and sometimes I
cannot give examples spontaneously.
Affective (attitude) response
 However, I like to teach science in English.
 I am more confident and dare to talk to people using English.
Designing A Good Survey



Clear questions
What would you like to do with research?
Single questions
Have you seen people pushing other on
trains and not waiting until people have
disembarked?
Positive wording
Do you think that people should not get on
trains until others disembark?
Designing A Good Survey



Keep language simple and common
How can a concerns-based adoption model
explain responses to teaching in English?
Watch out for overlapping responses
Would you rather have rice for lunch or fish?
Make sure questions include all in sample, or
use branching to clarify.
Do you consider riding the Rapid KL to be
pleasant?
Pilot Testing
Describe how the teaching of mathematics and
science in English has affected you.
“None. I am not a teacher.”
“I find the lesson more interesting and I think it is one
way to improve on my language proficiency as well.”
“At first I found it hard to teach in English. By now I am
more comfortable with it.”
How might I reword the survey item?
Response Rate
137 students in the Sample
30 Respondents
21.89% Response Rate (What % actually respond?)
Ways to improve response rate
 Direct contact
 Use of incentives (but not coercion)
 Follow up procedures
 Avoid sensitive questions
Sensitive Questions
What is your age?
Of 30 overall respondents, 30 responded
Describe your opinion of the decision by the
Malaysian government to require that English
be used to teach mathematics and science.
Of 30 overall respondents, 24 responded.
Standards of Practice
Ethical practices in research






Human subjects review
Anonymity vs. Confidentiality
Fairness in analysis
Knowledge of data uses
Opportunity to refuse
Member checking of results (when possible)
Standards of Practice



Avoid bias within questions, especially in
face-to-face interviews.
Train interviewers, especially if there are to
be follow-up questions that are unscripted.
Be honest regarding the purposes of the
questionnaire, the extent to which the results
will be shared, and the level of
anonymity/confidentiality.
An Abstract: KBSM Study
The focus of the study is to determine teachers’
attitudes towards the implementation of the
KBSM geography curriculum, and its
relationship with age, academic qualification,
teaching experience and level taught.
This study was carried out on a sample of 125
trained geography teachers. The same group of
teachers completed a questionnaire three
times; at the beginning, middle and end of the
two-year implementation period.
Discuss the Study Attributes
What is the population?
What is the target population?
What are the variables?
Which variables are dependent and independent?
Is the study longitudinal or cross-sectional?
What is a Likert scale question that could be used?
What is one possible open-ended question?
What is a variable that could provide ratio data?
What is a variable that could be sensitive?
What issues might the research face?
An Abstract: MARA Study
The objective of this study was to identify the
differences in personality factors, family
environments, locus of control, socioeconomic status
and prior achievement between high and low
achievers, and between male and female students.
The subjects of the study were 356 high and low
achieving fifth formers from eight MARA Junior
Science Colleges. The personality traits were
measured using the Cattell’s Sixteen Personality
Trait Questionnaire, and the locus of control was
measured using the Rotter’s Internal-External scale.
Discuss the Study Attributes
What is the population?
What is the target population?
What are the variables?
Which variables are dependent and independent?
Is the study longitudinal or cross-sectional?
What is a Likert scale question that could be used?
What is one possible open-ended question?
What is a variable that could provide ratio data?
What is a variable that could be sensitive?
What issues might the research face?
An Abstract: Islamic Education
The aim of this study is to explore the effectiveness of
Islamic Education among the Muslim Students of UTM. In
the questionnaire, 52 questions were asked. There were 50
questions with predetermined variables and two open-ended
questions. The following predetermined variables were used.
(i)
(ii)
(iii)
(iv)
Improvement of knowledge about Islam
Attitude of student towards Islamic Education
Moral values in the student’s daily life
The obedience to Islamic teaching among students
The variables were viewed with regard to sex difference,
duration of exposure to Islamic Education, and achievement.
Discuss the Study Attributes
What is the population?
What is the target population?
What are the variables?
Which variables are dependent and independent?
Is the study longitudinal or cross-sectional?
What is a Likert scale question that could be used?
What is one possible open-ended question?
What is a variable that could provide ratio data?
What is a variable that could be sensitive?
What issues might the research face?
Let’s Summarize
What are five terms you should remember?
What are three things that can affect a survey?
What are three ways that surveys can be used
in research?
What are three things that can be problems?