The way I look at this is the best way to over come such a complex task as determining when a site is ready to use, as far as real humans are concerned, is to use those users and their brains to do all the heavy lifting.

A metric that determines when the users begin interacting with content in relation to the start of the browsers work (HTTP GET, AJAX fired etc), would be the most useful, and probably fairly easy to implement – JS at top of the page (yes I know that is a sin) that watches the mouse/keyboard for user behaviour in the region of the browser display (or more specific areas of the DOM for AJAX events) (clearly you’d want to ignore users reading their emails whilst waiting for a page to load).

Clearly this is only of use in systems where RUM is possible (not dev, test, CI etc).

Paul

I just performed some testing on Chrome 26.0.1410.64 (for 64-bit Vista), and I found the following:

No preload, or ‘metadata’ => about 10 sec of buffering.

For example, after the page first loads, the “Played time ranges” is empty, and the “Buffered time ranges” is 0-10.89 sec.

After I play the first 19.8 secs, , the “Played time ranges” is 0-19.8 secs (as expected!), and the and the “Buffered time ranges” is 0-28.76 sec.

I’m reading those values off the myVideo.buffered.length and myVideo.played.length JS variables

My test video is about 8 mins (MP4)

Romain: The best alternative is preload=auto AFAIK.

Kudos for taking initiative on this. Something worth mentioning for the masses is that you already have page load screenshots on HTTP Archive for those who want to start going down the image analysis route. I’m not sure how easy it is to consume those images in bulk, but they are there nonetheless and give us the ability to identify some of the differences you mentioned (i.e. gmail vs amazon).

All the best,
Jared

Nothing is perfect (therefore everything comes with some type of imperfection). Asking which metric is like asking which statistical calculation (Arithmetic Mean versus Geometric Mean versus etc). If the answer is contextual, then suggest ReadyState or OnLoad are still two of the least imperfect metrics available.

Leo

I’ve haven’t looked at it in any detail but I believe that it is no longer exclusive to Internet Explorer.

This is the problem with virtually every metric. We want “making our websites faster for users”. But it’s hard to define, and hard to measure. So we settle for something that is easier to measure, but isn’t quite what we really want. And then we cross or fingers and hope there is some correlation between what we measure and what we actually want. Sadly, many metrics degrade into well-intentioned folklore, best-practices, and a cargo cult mentality.

I applaud your efforts! I look forward to some progress toward anything which can accurately indicate “faster for users”, yet works across browsers and is external to the site under test.

To test a page, Selenium has to click on elements. If those elements don’t exist yet, you get a test failure.

If you have enough tests, you’re guaranteed that at least one of them will click ‘too soon’ and then every test run results in failures that have to be evaluated by hand. I have found this to be exhausting for everyone involved so we try to avoid this.

What we’ve done is put a DOM element in the page that gets modified when the page has ‘settled’. We still occasionally forget to do that and run into test failures, but mostly we can fix them.

Onload fails at this sometimes, so do the rendering metrics. Therefore if there is initiative to better measure “rendering time”, we also need to take a look at “time to interact”.