domInteractive: is it? really?

August 7, 2015 10:50 am | 7 Comments

Netflix published a great article, Making Netflix.com Faster, about their move to Node.js. The article mentions how they use performance.timing.domInteractive to measure improvements:

In order to test and understand the impact of our choices, we monitor a metric we call time to interactive (tti).

Amount of time spent between first known startup of the application platform and when the UI is interactive regardless of view. Note that this does not require that the UI is done loading, but is the first point at which the customer can interact with the UI using an input device.

For applications running inside a web browser, this data is easily retrievable from the Navigation Timing API (where supported).

I keep my eye out for successful tech companies that use something other than window.onload to measure performance. We all know that window.onload is no longer a good way to measure website speed from the user’s perspective. While domInteractive is a much better metric, it also suffers from not being a good proxy for the user experience in some (fairly common) situations.

The focus of Netflix’s redesign is to get interactive content in front of the user as quickly as possible. This is definitely the right goal. The problem is, domInteractive does not necessarily reflect when that happens. The discrepancies are due to scripts, stylesheets, and fonts that block rendering. Scripts can make domInteractive measurements too large when the content is actually visible much sooner. Stylesheets and custom fonts make domInteractive measurements too small suggesting that content was interactive sooner than it really was.

To demonstrate this I’ve created three test pages. Each page has some mockup “interactive content” – links to click on and a form to submit. The point of these test pages is to determine when this critical content becomes interactive and compare that to the domInteractive time.

Scripts: Loading the domInteractive JS – too conservative test page, we see that the critical content is visible almost immediately. Below this content is a script that takes 8 seconds to load. The domInteractive time is 8+ seconds, even though the critical content was interactive in about half a second. If you have blocking scripts on the page that occur below the critical content, domInteractive produces measurements that are too conservative – they’re larger than when the content is actually interactive. (WebPageTest results for Chrome, Firefox, IE & Safari.)

Stylesheets: On the other hand, loading the domInteractive CSS – too optimistic test page, the critical content isn’t visible for 8+ seconds. That’s because this content is blocked from rendering until the stylesheet finishes downloading, which takes 8 seconds. However, the domInteractive time is about half a second. If you use stylesheets, domInteractive may produce measurements that are too optimistic – they’re smaller than when the content is actually interactive. (WebPageTest results for Chrome, Firefox, IE & Safari.)

Fonts: The results for the domInteractive Font – too optimistic have a domInteractive time of about half a second in all browsers, but the time at which the critical content is visible varies. In Safari, the critical content isn’t visible for 8+ seconds while waiting for the font file to load. In Chrome and Firefox, the critical content is blocked for 3 seconds at which time it’s rendered in a default font. That’s because Chrome and Firefox use a three second timeout when waiting for fonts; if the font doesn’t arrive within three seconds then a default font is used. IE11 displays the critical content immediately using a default font. In Chrome, Firefox and IE11, the content is re-rendered when the font file finishes downloading after 8+ seconds. If you use custom fonts for any of your critical content, domInteractive may produce measurements that are too optimistic in Chrome, Firefox, and Safari – they’re smaller than when the content was actually interactive. (Unless you load your fonts asynchronously like Typekit does.) (WebPageTest results for Chrome, Firefox, IE & Safari.)

One solution is to avoid these situations: always load scripts, stylesheets, and fonts asynchronously. While this is great, it’s not always possible (I’m looking at you third party snippets!) and it’s hard to ensure that everything is async as code changes.

The best solution is to use custom metrics. Today’s web pages are so complex and unique, the days of a built-in standard metric that applies to all websites are over. Navigation Timing gives us metrics that are better than window.onload, but teams that are focused on performance need to take the next step and implement custom metrics to accurately measure what their users are experiencing.

NOTE: I hope no one, especially folks at Netflix, think this article is a criticism of their work. Quite the opposite, their article is what gave me the idea to do this investigation. Netflix is a great tech company, especially when it comes to performance and sharing with the tech community. It’s also likely that they don’t suffer from these custom font and blocking script issues, and thus domInteractive is producing accurate interactive times for their site.

7 Responses to domInteractive: is it? really?

  1. Nice find! Like you said there’s no magic bullet to determine how fast your pages are. But I think for most cases (and for Netflix’s as well) what you care about is how a given metric changes over time versus what the absolute value is.

    So even in your example where domInteractive was too conservative because of the blocking script, as long as that remains constant, you can tell if something new you’ve down lowers or raises the overall number and act accordingly.

    I think domInteractive is a good enough stake in the ground on which to set up a performance budget and then you can evaluate changes relative to that stake.

    Thanks again for sharing!

  2. The “too optimistic” test page isn’t correct, I think. In addition to the delayed web font resource, there is also a style sheet LINK which loads a type=js resource with a 8-second delay. This is what causes the rendering delay, not the web font. Note that most browsers have a 3-second web font timeout after which the text is rendered using a fallback font.

  3. Å ime: Great catch! I made two mistakes that clouded the results: 1) When testing the stylesheet, the content-type was incorrectly JS so there was no blocking. In fact, a stylesheet will ALSO be too optimistic. 2) I left the misconfigured stylesheet in the font test and thus didn’t see that the font delay is shorter.

    I’ve rewritten the post with these corrections. Thanks!

  4. Hi Steve,

    Thanks a lot for this article, we had the same thoughts about the Netflix article, asking ourselves how Netflix team could be able to measure TTI with Performance Timing API.

    Nevertheless, I’ve a question about your article, because you seem to consider an element is interactive as soon as it has been displayed. IMO, it’s too optimistic!
    Javascript execution could freeze the page interactivity, or a script could be required for an element to be really interactive.

    TTI should be specific to a given action, no?
    As pointed out at the end, I don’t see other reliable way than using custom metrics.

  5. I think your right Steve about custom metrics. Every website is different and will have different business goals, what might work with one site might not necessarily work with another. A product based website may want to focus on TTI for their search bar, whilst a news site may want to concentrate on rendering their feature news component.

  6. Can this article be applied for getting Google Speed insight score 100%. While optimizing customer site we found that the greatest number of bugs was dealt with domInteractive CSS?

  7. Can we use a more general approach to mark the TTI?
    In every page, the dom in first viewport is the most important, so we can put a script just below the element (nearly the last element of first viewport), the script can mark the time. It enforce the stylesheets before it to be loaded then rendered the page and show it, as browsers worked.