Transcript Where does user and task analysis come from?

Ethnography
• Useful method studying people’s behavior and
understandings
• Can learn from anthropologists, sociologists,
others who have extensive experience with
this method
• Also in IS214, Usability
• IS272 – Qualitative Methods – addresses in
more detail
Ethnography
• “Properly done, provides detailed insight into the
concepts and and premises that underlie what people
do – but that they are often unaware of.” Forsythe
• Methods
– Participant observation
– Observation
• Audiotaped, videotaped
– Interviews
• Formal and informal
– Analysis of records
• Analytical perspective
– Not beginning with theory but with observation
– “grounded theory”
– “Make the familiar strange”
Examples from our readings
• Ling: observing people using cell phones in
public spaces
– “experimenting,” e.g., crowding them
• Taylor & Harper: “ethnographically-informed”
study of young people’s use of mobile phones
– 18 hours of observation of phones and phonerelated use in college cafeteria, hallway, and
surrounding area >> field notes
– 8 video-taped group interviews with 6 students,
over 10 weeks and various locations
Presuppositions
• Commitment to studying activities in natural settings
• concern with understanding relation of particular
activities to the constellation of activities and
resources that characterize a setting
• Detailed descriptions of lived experience
– how people actually behave, not (just) their accounts
– withhold judgment, recommendations, design
• Members’ point of view
– Use their categories, language
• Your point of view affects what you see and
understand
Ethnography vs. Ethnomethodology
• Ethnography: method(s) and orientation
• Ethnomethodology: analytic perspective
– Study of a group’s/area’s common sense
knowledge, procedures, considerations by which
people make sense of and act on circumstances in
which they find themselves
– Parallels ethnobotany, ethnopsychology…
Impetus for Ethnography in HCI
• To understand human-computer interaction,
need to understand social and material
contexts of interactions
• Awareness that human intelligence socially
created/achieved; can’t replicate in devices
• Desire to support cooperative human activity
(e.g., CSCW)
Central Premises
• It is difficult for people to articulate tacit
knowledge and understandings of familiar
activities
– So we observe them as well as talk to them
• Participants act (toward technology) based on
their own understandings and meanings
– So we listen to them as well as observe them
Ethnographic Tools
• Field notes
– Make detailed notes on what is observed
• Need to be done as soon as possible during/after
observation
– Separate interpretation from observation
• Photos, video/audiotapes, & transcripts
– Reusable record of exactly how people act, what
they say
– Repeated observation reveals unseen details
– Precise wording used by participants may be
revealing (e.g., Taylor & Harper)
Time
• Collect data over a long enough period of time to
question own assumptions, viewpoint
– Each observer brings own experience, understandings,
values, which may differ from the participants’
• Work with participants long enough to gain their trust
– E.g., a series of personal failures may indicate systemic
failure
– Underlying issues, problems, practices, habits, assumptions
may shape what you observe
• Stay in a setting to see situation unfold over time
• See situation through multiple viewpoints
– Each participant has limited view, also
Ethnography and HCI
• studies of work
– where new technology might be intro’d but w/o
explicit design agenda
• studies of technology in use
– situated use of specific technologies, classes of
technology
• participatory/work-oriented design
– people who use/are affected involved in design –
based on their understandings of their work
Ethnography and Systems Building
• Gathering customer requirements:
Understand their work, context, interactions –
on site
• Prototype evaluation: e.g., PARC workoriented design project, put a working
prototype in the workplace
• Field evaluation: study use and integration of
product/service on site
Ethnography and Studying Use/Behavior
• Advantages:
–
–
–
–
Real world – observing technology-in-use
Settings, uses, users, conditions of use…
Can see things people wouldn’t think to report
Can ask questions at moment of interesting activity
• Disadvantages:
– Can only study a limited number of people, who are willing
to have you around
– Can you follow people around without affecting their
behavior?
• Teens and cell phones?
– Do you have the time required to observe range of activity?
Misconceptions & Responses
Forsythe, “It’s Just a Matter of Common Sense”
1) Anyone can do ethnography – it’s just a matter of common
sense.
2) Being insiders qualifies people to do ethnography in their own
work setting.
3) Since ethnography does not involve preformulated study
designs, it involves no systematic method at all – “anything
goes.”
4) Doing fieldwork is just chatting with people and reporting what
they say.
5) To find out what people do, just ask them!
6) Behavioral and organizational patterns exist “out there” in the
world; observational research is just a matter of looking and
listening to detect these patterns.
Responses
1) Anyone can do ethnography: Actually, ethnography runs
counter to common sense, since it requires one to identify and
problematize things that insiders take for granted. It takes a
good deal of training and experience to learn to do this. It may
also take courage on occasion, since insiders tend to
experience their own assumptions as obvious truths.
2) Being insiders qualifies people to do ethnography: Ethnography
usually works best when conducted by an outsider with
considerable inside experience. The ethnographer’s job is not
to replicate the insiders’ perspective but rather to elicit and
analyze it through systematic comparison between inside and
outside views - includes detecting tacit knowledge, something
that by definition is generally invisible to insiders.
Responses (cont)
3) Ethnography involves no systematic method: anthropologists
see ethnographic work as technical in nature and take seriously
issues of methodological appropriateness, procedure, and
validity. Proper ethnography involves systematicmethod and
epistemological discipline.
4) Doing fieldwork is just chatting with people: Doing fieldwork
certainly involves talking to people, but this is no more the
entire task than system buildingis “just typing” or medical
diagnosis is “just talking to patients.” The important point is
what one is doing when typing or talking. Competent
fieldworkers do not take what people say at face value; they
treat people’s views as data, not results. The job of the social
scientist is to understand and analyze what people say.
Responses (cont)
5) Just ask them: Ethnography does entail eliciting people’s
understandings of their own and others’ behavior, but only the
most naive of fieldworkers would treat such understandings as
reliable data about systematic behavioral patterns.
Anthropologists know from our observational tradition hat
people’s verbal representations of their own behavior are often
partial and sometimes incorrect. In other words, it is imperative
to watch people engaged in activity as well as to ask them
about it.
6) Behavioral and organizational patterns exist “out there” in the
world: Observational research is sometimes perceived by others
as just a matter of looking to see what is “out there”; one need
only look and listen…This common misconception fails to grasp
is the selectivity and interpretation that go into the process of
gathering careful ethnographic data, writing useful fieldnotes
and analyzing the data in an appropriate and systematic way.
Difficulties with Ethnography
• Harder to do well than it appears
• High resources demands
– Human resources – time and expertise
– Calendar time
• Lots of information to analyze
• Difficult to translate observations and
understandings for others
• How to link to design?
• How to use to develop designs for more
general use, other than this setting?
But:
• Useful as an orientation, set of principles
• Important reminder to stay grounded in the
users’ actual experience and understandings
• Useful reminder to observe and listen
Rapid Ethnography – in general
• Team of researchers – divide up, share
observations, interact with one another
• Triangulation: multiple data collection
methods
– But should be doing this anyway
• Iterative data collection and analysis
– Ditto
“Quick and dirty” Ethnography
•
•
•
•
Focused interviews
Unattended video
“Interactive feature conceptualization”
Most often: short time, focused, fast in and
out
Key Elements of Rapid Ethnography
(Millen)
• Narrow the focus of field research before
entering field.
– Important activities
– Key informants
• Use multiple interactive observation
techniques, looking for exceptional and useful
user behavior
• Use collaborative and computerized iterative
data analysis methods
How rapid ethnography (Millen) diverges
from traditional ethnography
•
•
•
•
•
“Objectivity”
Speed
Focus
“Exceptional” occurrences
Interpretation
Downside of Rapid Ethnography
• Too focused
– Too narrow a view
– Find what you expect to find
• Too few informants to get a broad view, find the people you
really need
• Not enough understanding of the situation to know
–
–
–
–
What’s important
When to collect data
Whom to talk with
How participants understand situation
• Not enough time
–
–
–
–
for exceptions to surface or patterns to appear
to build trust
to shift own thinking to understand nuances of situation
to understand informants’ relationships to situation
• Not skeptical enough – or too skeptical
Other Ways to Adapt Ethnography to
Rapid Development Cycles
• Uncouple research schedule from product
development cycle
– On-going data collection to feed into design when
and as needed
• Pair ethnography with short-term methods
(like focus groups)
• Look ahead several iterations of product
design
Some Issues in Ethnography
• Your relationship with site, participants
– Jeopardizing your future relationship, access?
– Identification with the people you talk with
• Reconciling multiple points of view
• Confidentiality and trust
– What happens when you know something they don’t know?
• When to stop? How much is enough?
• How much control to give participants over your
report?
• What to do when the greatest problems/needs are
supposedly outside the scope of your study?