| impact on page - | One of "small", "med", and "big" based on the content's size, number of HTTP requests, and performance issues. |
| Page Speed - | Performance score from Page Speed. |
| YSlow - | Performance score from YSlow. |
| doc.write - | Does the snippet use document.write? |
| total reqs - | Total number of HTTP requests. |
| JS ungzip - | Number of bytes for scripts after any compressed scripts are uncompressed. |
| DOM elems - | Number of DOM elements created. This does not include DOM elements inside iframes. |
| The median number of milliseconds loading the snippet adds to this page. This is gathered from real users by taking the delta of loading the page with and without the snippet. |
The end of the load time depends on what kind of snippet is being measured. For "Ads" snippets, the load time ends when the HTTP waterfall image is done loading. For all other snippets, the load time ends with the window load event. The reason for this difference is for Ads I want to measure the load time of the ad framework itself, but not the actual ad. I want to encourage the ad networks to develop better snippets for their frameworks. It's important for the actual ads themselves to be faster, but that'll have to be a subsequent project.
The load time is measured the first time the user visits this page in a single browser session in a single day. This is done to increase the likelihood that the relevant resources will not be in the user's cache. The page is loaded again without the snippet (if the user chooses to do so). Any resources that are not part of the snippet (i.e., resources that are mine such as the image for the waterfall chart) are guaranteed to be loaded again by tweaking the querystring.
One possible source for inaccuracy is DNS lookups. These would only be done on the first page view and not the second. But since the time measurement starts when the HEAD of the document is received, the DNS lookup for "stevesouders.com" is not included. And any DNS lookups for the snippet's domains are rightly included in the time measurement.
Another possible source for inaccuracy is the cache. It is possible that the user could visit this page and have the snippet's resources in the cache. However, restricting any single user to only one measurement per day mitigates this impact. Most users won't visit this page multiple times across multiple days. And even if they do, many of the snippets set resource expiration times at less than a day. Still, it's possible that the browser may send a conditional GET request that would make the snippet load faster than if the cache was empty. However, in the real world some users would also experience a cached scenario. So this is a variable that I can't track, but I feel that the resulting data is still representative of the real world experience.