Transcript QUESTIONNAIRE DESIGNS

Review of literature
Research idea
Research problem
Conclusions/
recommendations
Research objectives
Discussion of results
Hypotheses
Data analysis
Frameworks/
definition of variables
Data processing
Study design
Data-collection
Instrumentation
Study population/sampling
INSTRUMENTATION
Development and validation of data-collection
tool
•
•
•
•
•
Questionnaires
Interview schedules
FGD guides
Lab form sheets
Checklists
QUESTIONNAIRE
DESIGNS
Outline
1. Concepts and definitions
2. Steps in questionnaire construction
3. Other issues in questionnaire development and
administration
DEFINITION
• A questionnaire is a data-collection tool used to gather information
about an individual, group, things, phenomena or events.
• It is an instrument that can measure attitudes, cognitions, behaviors
or other attributes.
• It is a written document which…
1) respondent fills in by himself (self-administered) or with
assistance from the researcher (interviewer-assisted)
2) collected by field interviewers (interviewer-administered)
3) may be sent through standard mail or cybermail.
Developing a questionnaire is not easy. It is a
tedious and time-consuming process.
• Use existing relevant questionnaires
• Modify existing questionnaires to suit the research
requirements.
• Items from several questionnaires can be combined.
It is far more difficult to produce a valid and
reliable questionnaire than you might think
• Maximize response rates
• Increase validity and reliability of responses
• Careful choice on questions to be included
• Careful design (wording, formatting, sequencing) of
individual questions
• Lucid explanation of the purpose of the questionnaire
• Compliance to ethical standards
• Pilot testing and eventually finalizing the instrument
There are two common mistakes made by
inexperienced investigators:
1)asking too many questions, the result of which
may never be analyzed; and
2)asking too few questions that may be
inadequate in satisfying the research
objectives.
Before you decide in using a
questionnaire…
Be certain that a questionnaire is the best way to
obtain your information
What information do you require?
Who are your respondents? What are their characteristics and
abilities?
What are your restraints in terms of time, money and
manpower?
Have you considered other methods such as professional
records, key informant interviews, focus group discussions,
laboratory tests and secondary documents?
Questionnaires are more suitable…
• when distance is a problem
• when the respondents are busy people
• when accurate information is required rather
than opinions, and
• when time is short.
STEPS IN QUESTIONNAIRE
CONSTRUCTION
1. An effective questionnaire depends on the clarity
of the research problem.
• Questionnaire data must be able to provide an
answer to the research problem.
• Make sure that you have a clear, solvable research
problem.
2. The path to solve the research problem is
through a clear statement of the research
objectives
– The specific research objectives must be able to
indicate what information is needed to solve the
research problem
– Only information that are relevant to the research
objectives should be obtained.
– Keep the research objectives handy when you are
working on the questionnaire.
EXAMPLE
1. Research problem: Why do persons with leprosy delay their
medical consultation?
2. General objective: To assess the reasons why persons with
leprosy delay their medical consultation.
Specific objectives:
1. To know the prevalence of people with leprosy in the sample
community.
2. To know the ratio of patients seeking medical help vis-a-vis those who
have not sought any medical attention
3. To determine their socio-demographic characteristics.
4. To assess their perceptions and attitudes about their health conditions
5. To identify the reasons why patients seek or do not seek medical help.
3. The research objectives will help us decide
what data need to be collected
Health-seeking
Socio-demographics
Prevalence
Ratio
with/with
out
leprosy
questions
Ratio healthseeker/no
health seeker
questions
age, sex, ses, cs,
educ
questions
Perceptions/attitudes
Concept of
disease
questions
Feelings about
having the
disease
questions
Factors in
health
seeking
Socio-cultural,
psychological.
Economic, health
service, clinical
questions
4. The research objectives will also help us
decide whether the questionnaire is the best
tool to collect that data that we need
What type of information is needed to be captured?
(facts, attitudes, behavior, beliefs, attributes)
How deep should the information be (description,
dynamics, how, why)
5. What type of questionnaire do you need?
1. Descriptive
2. Measurement scales
3. Combination of descriptive and scales
6. What type of questions would you like to
include?
• Open-ended: What is your opinion about the present state of
skin care in the Philippines?
•
Closed-ended: Are you in favor or not in favor of the current
skin care system?
[ ] Yes, I am in favor
[ ] No, I am not in favor
[ ] Could not say/don’t know
•
Partially closed-ended: Which of the following do you think
is most responsible for the popularity of cosmetic
dermatology in the country?
[
[
[
[
[
] endorsement from celebrities
] testimonies of those who have undergone cosmetic dermatology
] increasing concerns of people about good physical appearance
] influx of new technologies
] others _________ (please specify)
Open vs. Closed
Open-ended question
•
•
•
•
Greater freedom for respondent to answer
Could better capture the depth of response
Processing and analysis of data is more complicated
Entails more work from respondents
Closed-ended question
• Requires less effort and less facility with words
• Easier to analyze
• Less time-consuming
• Difficult to develop good closed-ended questions
7. How would you style the questions?
• List - The respondent is given a list of choices, any of which
may be selected
• Category- Only one response can be selected
• Rank -Respondent is asked to place something in order
• Rating scales -A rating device is used to record response
• Quantity- Response is a number giving the amount
List questionsThe respondent is given a list of choices, any of which may be
selected
Please choose the types of skin diseases that are common in this
community.
Scabies
Leprosy
Tinea Versicolor
Eczema
Acne
Melasma
Psoriasis
E
R
Others (please specify______)
Category question
Each respondent’s answer can only fit one
category
How often do you visit your dermatologist?
 Once a month
 2-3 times a month
 At least once every 6 months
 At least once a year
 Others ____
What’s wrong with these responses?
Do not forget to use ‘Don’t Know’, ‘Other’, ‘Not
Applicable’.
That is, do not force people to answer questions that
they might not be able to answer
Ranking question
Respondent is asked to place something in order
• Should not ask respondent to rank more than 15
items
• In telephone questionnaire, should not ask
respondent to rank more than four responses
Ranking question
• Rank-order the three most important reason why
you specialized in dermatology (where 1 indicates
the most important reason):
Interesting
Profitable
More job opportunities
Most in demand
Less academically demanding
Others __
Rating scales questiona rating device is used to record the response
1. Likert type
“Sun exposure is bad for my skin”
/---------------/-----------------/----------------/----------------/
Strongly
Agree
Agree
Neither Agree
nor Disagree
Disagree
Strongly
Disagree
My experience of depression during the past week was:
[
[
[
[
]
]
]
]
none
a bit
moderate
a lot
Rating scales question
• Semantic differential (SD) or adjectival rating scale. In the SD,
a pair of bipolar objectives are used to express the subject’s
feelings about the concept.
Our present skin care system……….
Is efficient
1 2 3 4 5 Is not efficient
Is accessible
1 2 3 4 5 Is not accessible
Of quality
1 2 3 4 5 Not of quality
Sensationalized
1 2 3 4 5 not sensationalized
Guttman scale – where the respondents rate a variable
on a sliding scale.
“How confident are you with your facial appearance?
Are you…. (Check one)
 Unable to go out of your house at all?
 Able to go out but with make up?
 Able to go out even without make up?
Quantity question
• When did you start seeing a dermatologist?
________
• How many times a month do you go to a skin care
clinic? _____
• Keep words as simple as possible (Grade 6 level).
• Keep the question clear and specific.
• Avoid double-barreled questions. (e.g., Do you feel dizzy
and feverish after undergoing liposuction?)
• Avoid using double negative statements (e.g., The lack of
laws to control exorbitant professional fees for skin care is
not a problem in the Philippines)
• Avoid too demanding questions (e.g., ranking a large
number of items).
• Avoid leading or biased questions
• Make sure that questions are applicable to all
respondents.
• Avoid threatening or embarrassing questions.
• Avoid using jargons and other foreign phrases
•
9. You need to code the responses
• Coding consists of assigning code numbers to the response
categories
1. Pre-coding
• In pre-coding, code numbers are assigned to each response
category while the questionnaire is being constructed
although this is limited to questions whose answer categories
are known in advance.
I believe in trying out different doctors to find out which one I think will give
me the best care." (Circle One)
(1) Strongly Agree
(2) Agree
(3) Disagree
(4) Strongly Disagree
2. Post-coding refers to coding conducted after questionnaires
have been completed by respondents. It is used for complex
open-ended questions where pre-coding cannot easily be
done.
1. List down all responses to an open-ended question
2. Make broad categories based on responses.Code each
category
3. Put each response in a certain category with its corresponding
code.
10. The questionnaire must have a good format
1. The questionnaire must look and read simply and clearly.
POOR FORMAT: "What is your approximate monthly income?"
____ Under P 5,000 ____ P 5,000 - P 9,999
____ P10,000 - P 19,999 ____ P20,000 or more
BETTER FORMAT: "What is your approximate monthly income?”
[ 1 ] Less than P 5,000
[ 2 ] P 5,000 - P 9,999
[ 3 ]P10,000 - P 19,999
[ 4 ]P20,000 or more
2. Questions should be grouped in ways which will seem sensible
and understandable to respondents, and should not appear
to jump randomly from one topic to another.
Use a transition statement, i.e., an introductory phrase to explain
why the next questions are asked so that respondents retain
a sense of the logical flow of the questionnaire.
“The past questions asked about some of your socio-demographic
characteristics, now, the next questions will inquire about your opinions
about the skin care system in the country…”
3. If some questions are not relevant to all respondents, filter
questions can be used which direct respondents to skip
certain questions.
1. Are you currently practicing as a skin specialist?
Yes/No
If your answer is “No”, proceed to question 3.
2. How long have you been practicing as a skin specialist?____
3. What factors have hindered your practice?
Filter questions should not require extensive page flipping by the
respondents or memory of answers to earlier questions.
11. Sequencing of questions
• Straightforward and easy to answer questions at the
beginning of questionnaire.
• Complex and difficult questions placed towards the
middle of the questionnaire.
• Personal and sensitive questions towards the end of the
questionnaire
Never start a questionnaire with an open question that
requires a lot of writing.
• Since some demographic questions may be
perceived as threatening (for example, level of
income), put these questions at the end of the
questionnaire.
• If a questionnaire deals with more than one topic,
complete questions on a single topic before moving
on to a new topic.
• Arrange types of questions to increase variety and decrease
response set (i.e., the tendency to respond in a certain
pattern, without proper regard to the particular questions).
• Keep the questionnaire as short as possible by removing
questions that are redundant, or unlikely to be analyzed.
• Consider the salience of the questions to respondents when
deciding how long the questionnaire should be. Respondents
will be more willing to complete a questionnaire if the topic is
relevant and interesting.
12. Do a cover letter explaining the purpose and
mechanics of the questionnaire, and ethical issues
1. Introduce self as interviewer or investigator
2. Tell what the research is all about, its purpose and why
respondent’s response is important,
3. Say how information will be asked and how long it will take to
complete
4. Assure confidentiality of responses or anonymity, how results
will be used
5. State whom to contact if have any queries, who to return filled
questionnaires to and date by which should be returned
6. Thank recipient for their help
7. Signature, name and title.
Include an informed consent form
The objectives and mechanics of the survey was explained to me and I
understood it well. I I also understand that my participation is voluntary
and I can withdraw from the survey after having agreed to participate. I
am free to refuse to answer any question that is asked in the
questionnaire.
I hereby provide INFORMED CONSENT to take part in this study entitled….
Participant signature:
Witness:
Date:
Sign:
13. Translate the questionnaire into some native
dialect, if necessary
2 forward translator
1 backward translator
Pilot testing
Pre-testing or piloting of the questions is not optional
— it is crucial. If time or money does not permit
pilot-testing of the questionnaire, then the study
should not be conducted at all.
14. Pilot test the questionnaire
Pilot-testing can be used to sharpen the focus of questions,
structure certain responses, eliminate irrelevant, vague
questions, and can be of great help in revising the wording of
questions.
Other aspects can be piloted such as:
• paper color
• Font
• timing of each section of the questionnaire (to avoid
overburdening respondents)
Pilot testing
1. Ask expert or group of experts to comment on the
representativeness and suitability of the questions
2. It is a good idea to obtain peer evaluation of the draft
questionnaire. Test the revised draft on yourself, friends,
relatives or co-workers.
3. Administer the questionnaire to a group as similar as possible
to the final population in your sample
The following questions can be addressed through piloting:
• Do the respondents understand what the questionnaire is
about?
• Do they feel comfortable answering the questions?
• Is the wording of items clear?
• Do the respondents draw the same meaning as you intended?
• Are the answers adequate for the purposes of the study? Is
there sufficient detail? In the case of open-ended questions, is
there so much variety among answers that the data will be
difficult to analyze?
• Are there regional differences in the interpretation of
questions? Are there local expressions that should be
incorporated into the items to avoid ambiguity?
• Which items or sections are most difficult for the respondent
to answer?
• Which seem to produce irritation,embarrassment or
confusion?
• Are there any items that the respondent considers irrelevant?
• Is the questionnaire too long?
• Do the filter questions and skips work properly?
• Are the instructions clear?
• Are the transitions from question to question and section to
section smooth?
Cognitive debriefing form
Did you have
difficulty
understanding this
question?
What does this
question mean to
you?
Is the question
relevant to your
condition?
How would you
have worded this
question?
Did you have
difficulty
understanding this
response choice?
What does this
response option
mean to you?
How would you
have worded this
response option?
Is it consistent with
the question?
Did you have
difficulty
understanding this
instruction?
What does this
instruction mean to
you?
How would you
have worded this
instruction?
Question 1
Question 2…..
Response choice
1a
1b……
Instructions
15. Administer the questionnaire
• Once the questionnaire has been piloted and modified where
necessary, it can be printed in bulk and distributed to the
sample population.
• Before distribution, you must have decided on your sample
population, the sample size, and how the questionnaire will
be distributed.
The questionnaire can be:
1. self-administered
2. interviewer-assisted
3. interviewer-administered
4. mailed or cybermailed
5. asked through telephone interviews