Transcript Document

Experiments and statistics
Classic “lab” study
 Studying cause and effect
– “novel navigation for faster task times”
 But it might be something else…
 Ensure causal chain
QNT, Paul Cairns, University of York
2
Big problem
 People vary
– Unlike say gravity or water
 Context influences people
 Need to isolate causal chain
 Need to account for natural variation
QNT, Paul Cairns, University of York
3
Experimental design
 Independent variable (IV)
– Causes change in
 Dependent variable (DV)
– Which is numeric
 Confounding variables
– Are eliminated
QNT, Paul Cairns, University of York
4
Statistics





Accounts for natural variation
Assume IV has no effect (null)
Prediction is crucial
Calculate probability of DV data
Unlikely data suggests real effects
QNT, Paul Cairns, University of York
5
Challenges
 Hard concepts: involvement, aesthetics
– Construct validity
 Context
– External/ecological validity
 Very small questions
QNT, Paul Cairns, University of York
6
Surveys and interviews
How to ask questions
 Questionnaires
 Interviews
 Focus groups
QNT, Paul Cairns, University of York
8
Outcomes




Demographics
Reveal thoughts/opinions/experiences
Formulate new hypotheses
Measurement
– Questionnaire as instrument
QNT, Paul Cairns, University of York
9
Asking questions
 Closed
– Likert scales
– Options
 Open
 Wording
QNT, Paul Cairns, University of York
10
Bad questions
 “Why don’t you like this interface?”
 “Isn’t this good?”
 “We made this. What do you think of
it?”
 “I don’t like the colours and would
prefer a simpler interaction: Agree?”
 “How old are you?”
QNT, Paul Cairns, University of York
11
Analysis
 Closed questions: counting and stats
 Open questions: coding
– Patterns, themes
– Dimensions and attributes
– Theoretical sampling
QNT, Paul Cairns, University of York
12
Challenges
 Learning to ask questions
 Qualitative analysis
 Informing design
QNT, Paul Cairns, University of York
13
Case studies
Misconceptions
 Sample of one
 “Go in and learn”
 Purely descriptive
QNT, Paul Cairns, University of York
15
Characterising Case Studies




Current
Context and object blur
Multiple sources of evidence
Theoretically informed
QNT, Paul Cairns, University of York
16
Outline of process
 Theoretical prediction
 Gather data
– Obs, interviews, documents, resources
 Analyse
– Maintain causal chain
 Theoretical development
QNT, Paul Cairns, University of York
17
Methodological development




Devise method to address problems
Use method in a project
Analyse impact of method
Develop method
QNT, Paul Cairns, University of York
18
Challenges
 No formal analysis methods
 Often need intervention
 Action Research for HCI?
– Notion of participation
– Joint solutions
QNT, Paul Cairns, University of York
19