Transcript Measurement

Descriptive Research &
Questionnaire Design
Descriptive Research

Survey versus Observation

Survey


Primary data collection method based on communication with
a representative sample of individuals
Observation

Systematic process of recording behavioral patterns of people,
objects, and occurrences without questioning or
communicating with them



Involves recording -- someone (something) has to do the recording
Deals with behaviors -- can’t observe attitudes, emotions, etc.
Broad-based -- may observe, people, objects, occurrences
Survey Research

Methods of Administration




Personal Interview -- face-to-face communication
between an interviewer and respondent
Telephone Interview -- information is gathered
over the telephone
Mail Questionnaire -- self-administered survey
sent via the mail
Electronic Questionnaire -- survey distributed and
gathered via e-mail or the Internet
Survey Research

Personal interviews

Advantages





Probing ability
Longer survey instruments
Completeness
Use visual aids
Disadvantages




Interviewer bias
Not anonymous
Expense
Ability to set up specific times for call backs
Survey Research

Telephone interviews

Advantages






Efficiency
Probing ability
Ability to have “pure random samples”
Ability to set up specific times for call backs
Direct data entry (CATI)
Disadvantages

Limited reach (not all have telephones)




Cell phone issues
Annoyance
Limited duration
No visuals
Survey Research

Mail questionnaires

Advantages





Low in cost
Respondent convenience & anonymity
No interviewer bias
Longer surveys
Disadvantages




Lack of speed
Low response rates (usually below 50%; often much lower)
No interviewer
Cost can get high
Survey Research

Electronic Questionnaires

E-mail surveys




Convenient
Limited reach
Best used in conjunction with another method (i.e., mail or
telephone)
Web-based surveys




Convenient
Limited reach
Some visual aids may be used
Best application:


In conjunction with another method
Panels
Primary Data

When to use communication versus
observation?

When data collection needs to be versatile


Have a wide range of data needs
Want to know needs, ideas, descriptions of customers,
for example


Able to gather by “communicating” with a respondent
Some data cannot be observed (e.g., attitudes)
Primary Data

When to use observation versus
communication?



When a respondent’s willingness to provide
information is low
When the potential for interviewer bias is high
Certain types of data points can only be observed
(e.g., behaviors)
Survey Research

Cross-Sectional versus Longitudinal Designs


Cross-sectional -- snapshot of population at one
point in time
Longitudinal -- fixed sample of population
elements repeatedly measured over time

Allows for tracking of certain changes over time (e.g.,
brand switching, brand loyalty)
Observation-Based Research

Nature of observation

No interaction



No non-response do to unwillingness to participate
Little social desirability
Characteristics of observation


Objectivity
Accuracy
Questionnaire Design

What is a questionnaire?

Formalized schedule for collecting data from
respondents


Outlines information to be gathered
Key criteria of a good questionnaire


Relevance to the problem at hand
Accuracy in terms of its measures
Questionnaire Design

Components of a Questionnaire





ID data
Request for cooperation
Instructions
Information sought
Classification data
Questionnaire Design

Procedure for developing a questionnaire

More of an art than a science (We will cover the
following steps)





Specify the information to be sought
Determine the type of questionnaire and method of
administration
Determine content of individual questions
Determine response form to each question
Determine specific wording of each question
Questionnaire Design

Procedure for developing a questionnaire
(cont’d)




Determine question sequence
Determine physical characteristics of the
questionnaire
Re-examine the above steps and revise (if
necessary)
Pretest questionnaire and revise
Questionnaire Design

Information sought


Guided by the problem definition and the
hypotheses
Be careful to focus on the problem at hand and
the necessary hypotheses

“Interesting” information will only make the questionnaire
longer which will likely contribute to non-sampling error
Questionnaire Design

Type of questionnaire and method of
administration

Depends on the problem at hand

Structured versus unstructured depends on the problem
definition
Questionnaire Design

Determine content of individual questions



Is a question necessary?
Are several questions necessary, or just one?
Do typical respondents have the necessary
information to answer the question?

Example -- Opinion Metallic Metals Act of 1947

If they don’t they will still answer it
Questionnaire Design

Determine content of individual questions
(cont’d)

Will respondents provide the information?

Options for sensitive questions





Hide the question
State the question using terms that suggest the behavior or
attitude in question is not unusual
Phase the questions in terms of others
State the question with response categories (easy to check
a response box)
Randomized Response Model
Questionnaire Design

Form of the response


Open-ended
Fixed alternative

Dichotomous


Yes/No
Multi-chotomous




Strongly Agree
Agree
Disagree
Strongly Disagree
Questionnaire Design

Question wording






Simple words
Avoid ambiguous words
Avoid leading questions
Avoid implicit alternatives
Avoid estimates
Avoid double-barreled questions
Questionnaire Design

Question Sequence


Simple & interesting opening questions
Funnel approach



Start broad and get narrow
Difficult or sensitive questions late
Classification information last
Questionnaire Design

Determine physical characteristics




Securing response – this is most important
Facilitate handling and control
Re-examination and revision
Pretests