Transcript Gameplay Analysis through State Projection

Gameplay Analysis through
State Projection
Erik Andersen1, Yun-En Liu1, Ethan Apter1,
François Boucher-Genesse2, Zoran Popović1
1 Center
for Game Science
Department of Computer Science
University of Washington
FDG 2010
2 Department
of Education
Université du Québec à Montréal
June 21st, 2010
We want to know how people play
We want to know how people play
We want to know how people play
?
We want to find…
We want to find…
• Player confusion
We want to find…
• Player confusion
• Player strategies
We want to find…
• Player confusion
• Player strategies
• Design flaws
Patterns in data
SELECT * FROM replays WHERE location=x AND
time>y AND attempt>3 AND
death=“grenade”…
Patterns in data
SELECT * FROM replays WHERE location=x AND
time>y AND attempt>3 AND
death=“grenade”…
Confusion? Strategies?
Traditional Playtesting
Statistical Methods
• Surveys
• In-game statistics
Statistical Methods
• Surveys
• In-game statistics
Visual Data Mining
Lets people see patterns in data
Bungie (Halo 3)
Visual Data Mining
Lets people see patterns in data
• Dynamic information?
Bungie (Halo 3)
Visual Data Mining
Lets people see patterns in data
• Dynamic information?
• Games with no map?
Bungie (Halo 3)
But what about?
But what about?
But what about?
But what about?
“Playtraces”
Start
Goal
“Playtraces”
Start
Goal
“Playtraces”
Start
Goal
“Playtraces”
Start
Confusion?
Distance to goal
Goal
Refraction
Refraction
• Massive educational data mining
Classic Multidimensional Scaling
• 2-D projection of points
in high-dimensional
space
• Clusters game states
based on some distance
function
State Distance
State Distance
State Distance
State Distance
Action Distance
da (s1, s2)
State Distance
Start
Confusion?
Distance to goal
Goal
Distance to Goal
dg (s1, s2) = abs(dg (s1, sg) - dg (s2, sg))
Distance Functions
Action distance
Distance to goal
Combined
Refraction Distance Function
d (s1, s2) = (da (s1, s2) + dg (s1, s2)) / 2
Playtracer Framework
Easy level
Difficult level
Failure
Chance To Win
Chance To Win
Evaluation
Evaluation
• 35 children from K12 Virtual Academies
Evaluation
• 35 children from K12 Virtual Academies
• Mostly third and fourth-graders
Evaluation
• 35 children from K12 Virtual Academies
• Mostly third and fourth-graders
• About 15 levels
Evaluation
• 35 children from K12 Virtual Academies
• Mostly third and fourth-graders
• About 15 levels
• The game logged all player actions
Analysis
Analysis
• Player confusion
Analysis
• Player confusion
• Player hypotheses
Analysis
• Player confusion
• Player hypotheses
• Design flaws
Analysis
• Player confusion
• Player hypotheses
• Design flaws
Level 2
Level 2 Solution
Level 2 Solution
Level 2 Visualization
Level 2 Visualization
Confusion: Hitting target from wrong
side
Refinement
Refinement
Confusion: Using pieces incorrectly
Confusion: Using pieces incorrectly
Confusion: Using pieces incorrectly
Confusion: Using pieces incorrectly
Analysis
• Player confusion
• Player hypotheses
• Design flaw
Level 4
Level 4 Solution
Level 4 Visualization
Level 4 Visualization
Level 4 Visualization
Hypothesis: Satisfy bottom target
Hypothesis: Get laser near targets
Hypothesis: Overload bottom target
Analysis
• Player confusion
• Player hypotheses
• Design flaws
Level 4 Visualization
Level 4 Visualization
Design flaw: Deadly state
Refinement
Limitations
Difficult to find good distance function
Limitations
Difficult to find good distance function
Limitations
Difficult to find good distance function
Limitations
Large game spaces
Conclusions
• Useful for game analysis
Conclusions
• Useful for game analysis
• We are expanding and refining Playtracer
Big Open Problems
How to
Big Open Problems
How to
– specify distances between game states
Big Open Problems
How to
– specify distances between game states
– differentiate types of confusion
Big Open Problems
How to
– specify distances between game states
– differentiate types of confusion
– classify player strategies
Acknowledgements
Marianne Lee
Emma Lynch
Justin Irwen
Happy Dong
Brian Britigan
Dennis Doan
François Boucher-Genesse
Seth Cooper
Taylor Martin
John Bransford
David Niemi
Ellen Clark
Funding:
NSF Graduate Fellowship, NSF,
DARPA, Adobe, Intel, Microsoft
Cycles
Acyclic Paths
Player Tracking