Transcript document

A Qualitative Study of
Animation Programming in the Wild
Aniket Dahotre, Yan Zhang, Christopher Scaffidi
ESEM 2010
Roles for animation programming
•
•
•
•
A vehicle for getting kids excited about programming
An environment for teaching programming skills
A medium for communicating and entertaining
A platform for research aimed at raising usability of
programming tools
• Examples of animation programming tools:
○
2
Logo, KidSim, AgentSheets, Alice, Hands, Scratch
Intro  Technical  Social  Remixing  Closing
Scratch as a particular programming tool
• Turing-complete
language
• Events, loops,
conditionals,
sprites, sound…
• Drag-and-drop
programming
+
Online community
for sharing, trying,
discussing &
remixing
animation projects
3
Intro  Technical  Social  Remixing  Closing
Scratch online repository
4
Intro  Technical  Social  Remixing  Closing
Statements and anecdotes
from related work
• Scratch site is “the YouTube of interactive media”.
• Users are “a new generation of creative, systematic thinkers
comfortable using programming to express ... ideas”
• Supports “creative appropriation… the utilization of someone
else’s creative work in the making of a new one”
• “The site’s collection of projects is wildly diverse, including
video games, interactive newsletters, science simulations, virtual
tours, birthday cards, animated dance contests, and interactive
tutorials, all programmed in Scratch”
• One project obtained over 100 user comments.
• Several programmers formed collaborative partnerships.
• One particular Tetris game was remixed dozens of times.
5
Intro  Technical  Social  Remixing  Closing
Questions for our study
• To what extent is Scratch succeeding as a basis for
developing programming skills in the wild?
○ Technical programming skills
○ Social skills
○ Remixing/reuse skills
6
Intro  Technical  Social  Remixing  Closing
Research methods
• Screen-scraped 100 randomly-selected animations
○
Including their code, usage statistics, and user comments
• Developed coding schemes for several research questions
○
○
○
○
○
○
One student examined subset of animations, proposed coding scheme
This student and another independently applied scheme to half
Negotiated modifications to coding scheme
Then checked each other’s work on the other half of the animations
< 10% disagreement; negotiated to resolve all disagreements
Coalesce codes to increase clarity of presentation as needed
• In a few cases, related work provided relevant coding schemes
○
Which we applied without modification
• A few research questions could be answered directly with
quantitative (non-coded) data
7
Intro  Technical  Social  Remixing  Closing
Functional roles of animations
Static trial 6%
Education 8%
To teach skill
or knowledge
Animated
trial 33%
Game 25%
To entertainingly
challenge
Story 11%
Ani-inter trial
17%
To communicate
fictional plot
56%: No clear functional role
8
Intro  Technical  Social  Remixing  Closing
Use of programming constructs:
repository vs. afterschool “Clubhouse”
Clubhouse
Repository
≥ 1 variable
≥ 1 conditional
≥ 1 loop
≥ 1 input event handler
≥ 2 scripts
≥ 1 script
0%
9
20%
40%
60%
80%
100%
Intro  Technical  Social  Remixing  Closing
Apparent design patterns
Controller
methodWithLoop()
Sprite
Sprite
Sprite
onMessage()
10
AbstractHandler
onHit(type1)
onHit(type2)
onHit(type3)
CollisionHandler
onHit(type2)
CollisionHandler
onHit(type1)
onHit(type2)
Sprite of type 1
Sprite of type 2
Intro  Technical  Social  Remixing  Closing
Conclusion regarding
technical skill development
• Relatively successful platform for development of
technical programming skills
○
○
○
11
Comparable primitive use relative to Clubhouse users
Some demonstration of patterns (perhaps subconscious)
Comparable in complexity to spreadsheets and other programs
created by end-user programmers (see paper for details)
Intro  Technical  Social  Remixing  Closing
Kinds of user comments exchanged
Approval, no suggestion
Compliment
27%
Chit-chat
26%
Other comments
from users, other
than project
creator
Feature
suggestion
21%
Approval + idea
Author
replies 20%
Criticism 6%
Comments from
project creator
Disapproval
12
Intro  Technical  Social  Remixing  Closing
Active collaboration on projects
• By “collaboration,” we mean
○
○
○
joint design or implementation
by a team of multiple people
consciously working on a common intellectual goal
• Reviewed user comments
○
○
○
○
E.g., to find comments like “I used your suggestion – thanks!”
E.g., or like “Thanks, my friend helped me with that.”
Also examined source code to look for any action on suggestions
Looked for comments revealing code edits by multiple people
• We found no such indications of any collaboration at all.
13
Intro  Technical  Social  Remixing  Closing
Conclusion regarding
social skill development
• Uneven success as platform for development of
social programming skills
○
○
○
14
Half of projects received comments
Most comments were complimentary and offered suggestions
But unable to find evidence of actual collaboration
Intro  Technical  Social  Remixing  Closing
Frequency of downloading and remixing
• Related work: 15% of projects were created by remixing
• But what fraction of projects are used to create remixes?
• Of our 100 projects…
○
○
○
15
50% were downloaded at least once
10% were remixed at least once
5% were remixed at least once by other users
Intro  Technical  Social  Remixing  Closing
Changes made to programs during remixing
• 20 of our 100 projects were created by remixing. Of these…
Number of remixes
14
12
10
8
6
4
2
0
multimedia
changes
16
script
feature
bug
changes add/change removed
bug added
Intro  Technical  Social  Remixing  Closing
Conclusion regarding
remixing skill development
• Uneven success as platform for development of
remixing skills
○
○
○
○
○
17
Most remixes simply involved multimedia tweaks
Few attempts at script modification during remixing
Nearly half of script modifications led to major bugs
Even the biggest modifications were still fairly small (see paper)
Frequency of remixing unimpressive vs other systems (see paper)
Intro  Technical  Social  Remixing  Closing
Results and
opportunities for future work
• Technical skill development: relatively successful
○
Perhaps still a need for helping animators to do higher-level design
• Social skill development: uneven success
○
Definitely a need for helping animators to collaborate
• Remixing skill development: uneven success
○
18
Definitely a need for helping animators to remix code
Intro  Technical  Social  Remixing  Closing
Additional empirical questions for future work
• Do animators consciously understand design patterns?
• What usually happens after code downloads?
• Is some hard-to-detect collaboration somehow occurring?
• What kinds of interaction happen in the online forums
(outside the context of particular projects)?
• How well do these programming skills transfer to other
programming languages and tools?
19
Intro  Technical  Social  Remixing  Closing
Thank You…
• For the opportunity to present.
• For your questions, thoughts, and constructive feedback.
20
Intro  Technical  Social  Remixing  Closing
% of projects with ≥ this many events
Frequency of interactions between users
100
90
views
80
comments
downloads
70
remixes
60
50
40
30
20
10
0
1
2
5
7
10
20
50
70
100
Number of events
21
Intro  Technical  Social  Remixing  Closing
Size of repository animations
by animations’ functional role
Project size (# primitives)
250
animation-specific
200
150
144
95
68
100
50
0
22
general programming
73
88
82
87
56
12
13