Transcript Multi-Mediating Groups (MMG) Sponsored by the ESRC and BT

The effects of relevance of on-screen
information on gaze behaviour and
communication in 3-party groups
Emma L Clayes
University of Glasgow
Supervisor: Prof. Anne Anderson
Co-Supervisor: Jim Mullin
BT Supervisor: Dr. David Hands
Sponsored by UK ESRC and BT
Outline
Background & Hypotheses
 Design and Method
 Results
 Summary
 Future work
 Conclusion

Representations: form and location

Many studies concerning mediated
communication (Finn et al, 1997) with
different technologies, measures and
tasks

Little research on the impact of
different representations (e.g Sellen,
1995; Parise et al, 1996)
Representations and relevance

How does the form and location of
representations impact on gaze and
communication?

How does this interact with the relevance
of on screen information ( e.g. task related
information and representations in terms
of the task role)
Evaluation techniques in CMC

Many different measures used to examine
computer mediated communication (e.g.
dialogue analysis, task performance,
questionnaires)

Additional factors (e.g. task, no of
participants, technology) and different
methods have led to conflicting results
Methodological issues
Three-party groups
 Problem solving and social tasks
 Communication analysis,
questionnaire responses and task
performance
 Eye-tracking as an evaluation
technique

Eye-tracking in Psychology and HCI

Well established measure of human
information processing (Rayner, 1998)

Eye-gaze computer interfaces (Jacob,
1991)

Recent studies examining gaze and CMCVelichkovsky et al (1997), Mullin et al
(2001)
Communication and Attention

Eye-tracking as an evaluation technique:
What do users attend to when using
remote communication systems?

Does the relevance of on screen
information impact on gaze behaviour?

Are patterns of gaze related to patterns of
interaction within the group?
Eye-tracking in CMC


Exploratory questions:
Is it possible to obtain significant amounts
of eye data during a non-restricted
interaction?
 Are participants consistent in their
patterns of gaze across a screen?
 Are these patterns meaningful?
Communication and Relevance

Relevance of on screen information (e.g.
video data and video links)

Shared visual data more useful than video
conference links (Daly-Jones et al 1998)
Communication and Relevance

Relevance of other collaborators (e.g. task
role and status)

Status effects in mediated communication
(Dubrovsky et al 1991, France et al, 2000)
Study 1

What on-screen features attract most
gaze?
 Does the position of representations on
screen impact on mediated
communication?
 Does this interact with the type of task and
relevance of information presented on
screen?
Task 1-Problem-solving task

Map task-collaborative problem-solving
task
 Two Instruction Givers (IG1 and IG2) have
to instruct one other person (Instruction
follower-IF) on how to draw a route on their
map
 Therefore, relationship between IG and IF
more relevant to task success than IG1
and IG2
Task 1-Display Screen
Task 1-Hypothesis I

Hypothesis I- Users will gaze more
often at shared visual data ( i.e. the
map) than video links of remote
collaborators
Task 1-Hypothesis II

Do participants look equally often at
the IF and IG video images?

Does the location of the video image
affect gaze behaviour?
Task 1-Hypothesis III

Do patterns of gaze reflect patterns of
interaction within the group?

I.e. Do participants talk more often to
the person they look most often at?
Design
2 factor mixed design

Relevance (map, video-IF, video-IG)
within subjects

Location (video-IF top, video-IF
bottom) between subjects
Method

Subjects- 10 groups of 3, 20 eyetracked
 2 maps, order of maps and location
of IF video balanced
 3 participants in different rooms-high
quality audio and video links Eye tracked participant always an IG
Results

Screen divided into areas of interest for
eye gaze analysis

Percentage analysis on different areas of
interest and different fixations
Eye Data-Pictorial analysis
Results-Hypothesis I

Participants spent significantly more time
looking at the map (72.4%) than video
image of IF (12%) and IG (10%)

Relevance of on-screen information
impacts on gaze behaviour
F(2,36)=258.15, p<.001
Results-Hypothesis II
Do participants look more often at IF
compared to IG and is this affected
by location?
 Main effect of relevance (IF 12%, IG
10%) p<.05
 No effect of location p>.05.

Results-Hypothesis II

Significant interaction between relevance
of video image and location of video
image F(1,18)=5.73,p<.05

Participants looked more often at the
instruction follower - only significant when
the IF video was located in the top left of
the screen (F=10.48, p<.05)
Results-Hypothesis III

Do patterns of gaze reflect patterns of
interaction within the group?
 Does the eye tracked participant direct
more turns of speech to the IF or IG?
 Turn combination analysis: IG1-IG2, IG1-IF
 No effects of relevance (task role-IG2/IF),
location of IF or an interaction between
the two.
Results-Hypothesis III
Turn Combination results
IG1-IF
IG1-IG2
IF top left
20.7
22.6
IF bottom left
19.5
23.2
20.1
22.9
Average no of turns by IG1-43
Task 1-Summary

Users look more often at shared visual
data (map 72.4%) than video images (22%)

Users gaze more often at IF than IG-only
significant when IF video is located in top
left of screen
Task 1-Summary

Communication patterns reflect that
IG directs almost equal number of
turns to IG and IF- does not reflect
patterns of gaze

Positional effect for salient video
images?
Task 2-Relevance and Status
Realistic task-mediated business
meeting
 Two confederates- one high status
(boss) and one low status (marketing
assistant)
 Video data presented not essential to
complete task

Task 2-Relevance and Status

Hypothesis I- Do participants gaze more
often at shared data compared to video
images when the information is relevant,
but not essential to complete the task.
Task 2-Relevance and Status

Hypothesis II- Do participants look more
often at a high-status collaborator?

Does the location of the high status video
image impact on gaze behaviour?
Task 2-Relevance and Status

Hypothesis III- do patterns of gaze reflect
patterns of interaction within the group?

I.e do participants talk more often to the
person they look most often at?
Task 2-Design
2 factor mixed design

Relevance (visual graphic, video-high
status, video-low status) within subjects

Location (video-high status top, videohigh status bottom) between subjects
Task 2-Method

2 confederates, 20 eye-tracked
participants

3 participants in different rooms-high
quality audio and video links
Task 2-Display Screen
Results-Hypothesis I

Participants spent significantly less time
looking at the low status video image
(18%) than both the task feature (24.5%)
and the high status video image (35%)

Relevance of on-screen information
impacts on gaze behaviour (F(2, 36)=8.19,
p<.05), no effect of location or an
interaction.
Results-Hypothesis II


Do participants gaze more often at the high
status confederate compared to the low
status confederate and does this interact
with the location of video images?
Only sig main effect of status
F(1,18)=22.19,p<.05, no effect of location or
an interaction
 Participants gazed more often at the high
status confederate (35%) compared to the
low status confederate (24.5%)
Results-Hypothesis III

Do patterns of gaze reflect patterns
of interaction within the group?

Do participants direct more turns of
speech to the high status
confederate compared to the low
status confederate?
Results-Hypothesis III

Turn combination analysis: sig effect
of status, no effect of location or an
interaction

Participant directs more turns of
speech to high status confederate
and this reflects patterns of gaze
Results-Hypothesis III
Turn Combination results
Participant-LS Participant-HS
10.5
2.9
HS top left
9.3
2.7
HS bottom left
9.9
2.8
Average no of turns by Participant-13
Task 2-Summary

Participants gaze less often at low status
video compared to high status video and
task feature
 Participants gaze more often at high
status video regardless of location
 Interaction in Task 1 not replicated- video
images attract more gaze and main effect
of status found
 Patterns of gaze reflect patterns of
communication
Study 1-Summary

Participants gaze more often at shared
data in task 1, not in task 2
 Significant interaction between location
and relevance of video images in task 1,
not in task 2
 Overwhelming impact of status on gaze
and communication in task 2
 Communication patterns reflect patterns
of gaze in Task 2, not Task 1
Study 1-Summary
Eye tracking provides valuable
information about mediated
interaction
 Distribution of attention related to
experimental task manipulations
 Positional effect for salient video
images?

Study 1-Eye-tracking in CMC

Exploratory questions about eye tracking
answered
 Data capture rate high –managed to track
on average 70% of participants who took
part
 Average of 80% of fixations directed on
screen during task
 Patterns of gaze consistent and
meaningful
Future Work




Same tasks, different design
Video images placed in 4 corners of the screen,
therefore 4 conditions for each task
Task 1:Provisional results suggest interaction not
replicated- always gaze more often at the map
Task 2:Difference in gaze distribution to videos
smaller when video images are placed on same
side of the screen than when they are placed on
opposite sides of the screen
Conclusion

Eye tracking can be used successfully
as an evaluation technique
 Task differences and status effects
robust- reflected in both patterns of
gaze and communication
 May be positional effect for salient
video images?
Conclusion
Implications for the design of remote
communication systems

Context in which system is to be
applied very important e.g. social or
problem solving
 Further research required on positional
effects-may be used to enhance or
reduce amount of gaze directed to
representations