Transcript CCT 333: Imagining the Audience in a Wired World

CCT 333: Imagining the
Audience in a Wired World
Class 7: Quantitative
Research Methods
Quantitative Methods
• Unlike qualitative, involves metrics
reduced to numbers
• Why helpful?
• Why problematic?
Surveys/Questionnaires
• Common method of obtaining
information from broad cross-section of
people
• Quality of information directly depends
on quality and purpose of questions and
who is surveyed
• Online tools help - e.g.,
http://www.surveymonkey.com
Sampling
•
•
•
•
Who is included in sample?
How are they reached?
Response rate issues
Responders vs. non-responders - are they
qualitatively different groups?
• Directly impacts quality of results - e.g., 1936
presidential poll
General Questionnaire
Guidelines
•
•
•
•
•
Understandable
Unambiguous
Collects data that is actually valuable
Can be easily analyzed
I’d add - limited in scope - take
respondent’s attention span and
willingness to help into account!
Question Guidelines
• Specific better than general
• Open/closed-ended questions - benefits
and challenges
• Opening and closing preamble and
instructions important - especially if
you’re not there to supervise
• Test it before you use it
Likert Scale Q
• 1-5, 1-7, 1-9 scales
• Midpoint - what does it mean? If no
opinion, give that option
• Take care in too many consecutive
items with same polarity of options leads to patterned responses
• Semantic differential can be effective
Scales
• Set of related questions measuring
attitudes, beliefs, orientations etc.
• Ex: multiple intelligences (others?)
• Scales must actually measure what they
claim, not be redundant
• Verify authenticity (esp. in web
searches) - many scales are
meaningless
Experiments
• Controlled specific measurement of
phenomenon
• Often used to determine causation - not just X
related to Y but X causes Y
• Inferential vs. descriptive statistics - not
simply 68% do X, but that this leads to
something else
Benefits and Limitations
• Benefits: Controlled environment, measured
responses, quantitative data that can lead to
causal links
• Limitations: Must simplify environment to
minimize other potential explanatory
variables, creating rather fake environment
and tasks
Data Mining
• Observation without being there quantitative artifacts - e.g., Web access
logs, click regions, eye tracking
• PeopleMeter example
• Records consequences of actual action
- but potential privacy and collection
issues (e.g., social networking helmet)
Imagining the Audience
in a Wired World
• Hey, the course title means something!
• Hierarchical task analysis and GOMS descriptions of cause and effect at functional
level
• Definitely important for planning computing
systems (and often used - e.g. flowcharts,
UML)
• Why? Computers are not all that bright.
GOMS
•
•
•
•
•
Goals
Operators
Methods
Selection Rules
Very possible that human and system
GOMS differ - which causes problems
Choosing Tools
• Methods are like tools in a toolbox - all
are useful for something - but you don’t
hammer a nail with a screwdriver
• Goals of research should primarily
influence choice of tools
• What else does?
Other factors influencing
method choice
• Type of data needed - qual vs. quant,
descriptive vs. inferential
• Cost and time to collect data
• Cost and time to analyze data
• Triangulation needs
• Contextual requirements
Physical research
example
• 2002 racecar seat
• Partially materials selection, stress
calculations etc. - but mostly ergonomic
• Quantitative measures of 5-95%
percentile team members, and
everyone in between
• Lots of individual testing though too
Org. research example
• Social network questionnaire - who trusted
whom in six domains
• Correlated with three scales interdependence, independence and
proactivity
• Correlational study - what relations existed
between scales and position in network?
• Relations verified by respondent reflection
and personal experience
• Redesign implications
Next week
• A look at how simple user interaction
gets complex when you add a few more
humans or technologies
• Think ahead Q: What technologies get
more complicated to use when more
people are involved?