Transcript Document
Episodes
a shared approach for timing web pages
Steve Souders
Google
http://stevesouders.com/episodes/
Disclaimer: This content does not necessarily reflect the opinions of my employer.
measuring to onload
many current timing frameworks stop at the onload event
• Keynote, Gomez
• Fasterfox, YSlow
• IE, FF, etc. ("Done" in the statusbar)
onload doesn't capture "page ready" in Web 2.0
• some apps render content above the fold before onload
and that is the "page ready" point
• some apps do critical JS and DOM after onload
• Ajax actions often don't have onload
measuring Web 2.0 apps: 2 choices
manual scripting - identify timing points using scripting tools
(Watir, Selenium, KITE, etc.)
• accuracy (does script author know key points?)
• high switching costs (eg, moving from Watir to KITE)
• high maintenance costs
• redundant work (think # of Google Search scripts created
worldwide)
• synthetic (no real user measurements)
programmatic scripting - timing points are marked by JavaScript
in the page itself (Jiffy, Gomez, etc.)
• observer effect
• reinvent the wheel
• JS instrumentation in pages has to be changed across
frameworks
programmatic scripting is preferred
embedding timing in the actual web app reduces maintenance
costs and improves accuracy (app developer more likely to
know the appropriate timing points and keep them up-to-date)
measures real user traffic
creating a shared approach resolves many drawbacks
• no need to reinvent
• could be used by all companies, frameworks, and tools
a shared approach: scenario 1
web company wants to measure real traffic
current approach:
• write a JS library
• instrument page
• beacon back measurements
• write code to generate reports
shared approach:
• use Episodes
• instrument page
• beacon back measurements
• use existing code to generate reports
a shared approach: scenario 2
web company wants to measure their pages using a
service (Keynote, Gomez, WebMetrics, etc.)
current approach:
• create manual scripts (hard to maintain, different from other
scripting tools) or use service provider's JS library (different
from other JS libraries)
• run synthetic tests
shared approach:
• pages already instrumented using Episodes; Episodes
understood by service's test agent
• run synthetic tests
a shared approach: scenario 3
web company wants to measure competitors' pages
using a service (Keynote, Gomez, WebMetrics, etc.)
current approach:
• create manual scripts (hard to maintain, different from other
scripting tools)
• run synthetic tests
shared approach:
• pages already instrumented using Episodes; Episodes
understood by service's test agent
• run synthetic tests
a shared approach: scenario 4
client apps (browser, dev tool) want to measure pages
current approach:
• measure to the onload event or end of HTTP traffic
shared approach:
• measure Episodes
Episodes – a shared approach
ideal vision of what Episodes looks like:
• uses events to signal episodes
window.postMessage("EPISODES:measure:pageready", "*");
• supports multiple listeners
window.addEventListener("message", callback, false);
• inserting markers is separate from gathering episodes
• most of the implementation is built into browsers (postMessage,
addEventListener, aggregating episode times)
• browsers record when the current page's request began
window.navigator.starttime
• many (most) web sites instrument their pages with Episodes
• industry-wide agreement on 5-10 Episode names: totaltime,
pageready, timetofirstbyte, timefromfirstbyte, etc.
• measurement service providers support Episodes
• browsers and dev tools report Episodes
Episodes – code sample
<html>
<head>
<script>
window.postMessage("EPISODES:mark:firstbyte", "http://stevesouders.com");
window.postMessage("EPISODES:measure:backend:starttime:firstbyte", "http://stevesouders.com");
window.postMessage("EPISODES:mark:jquery.min.js", "http://stevesouders.com");
</script>
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.2.6/jquery.min.js"></script>
<script>
window.postMessage("EPISODES:measure:jquery.min.js", "http://stevesouders.com");
window.onload = function() {
window.postMessage("EPISODES:measure:frontend:firstbyte", "http://stevesouders.com");
window.postMessage("EPISODES:measure:totaltime:starttime", "http://stevesouders.com");
window.postMessage("EPISODES:done", "http://stevesouders.com");
}
</script>
</head>
<body>
<div class=siteheader style="color: #444; font-size:32pt;">
...
</body>
</html>
Episodes - next steps
• solicit feedback, get agreement
• implement JS library using custom events (e.g., YUI) for
browsers that don't support postMessage
• .js hosted from a shared URL for greater caching
• browser plug-ins (Gears, Firebug, etc.) support
window.navigator.starttime
• measurement frameworks and dev tools start supporting
Episodes
• greater visibility of performance
FAQ
Q: Will this weigh down the page making it slower?
A: Yes, but less than current approaches that are already getting
adoption. As the bulk of implementation moves into the browser, impact
will be even less.
Q: Won't people game the standardized episode names?
A: There are a few dynamics that encourage sites to accurately
instrument their pages. They need this so that their own metrics are
accurate. And if they game the system bloggers will call them on it.
Q: Why would companies want to cooperate?
A: Cooperating drives down costs. You can't hide the truth. Competitors
already measure each other, it's just inefficient. Raising the cost to
measure doesn't hinder the biggest competitors.
Q: Where'd you get the name "Episodes"?
A: From Doug Crockford. Thanks, Doug!