JavaScript Performance
Last night I spoke at the San Francisco JavaScript Meetup. I gave a brand new talk called JavaScript Performance that focuses on script loading and async snippets. The snippet example I chose was the Google Analytics async snippet. The script-loading part of that snippet is only six lines, but a lot of thought and testing went into it. It’s a great prototype to use if you’re creating your own async snippet. I’ll tweet if/when the video of my talk comes out, but in the meantime the slides (Slideshare, pptx) do a good job of relaying the information.
There are two new data points from the presentation that I want to call out in this blog post.
Impact of JavaScript
The presentation starts by suggesting that JavaScript is typically the #1 place to look for making a website faster. My anecdotal experience supports this hypothesis, but I wanted to try to do some quantitative verification. As often happens, I turned to WebPagetest.
I wanted to test the Alexa Top 100 URLs with and without JavaScript. To load these sites withOUT JavaScript I used WebPagetest’s “block” feature. I entered “.js” which tells WebPagetest to ignore every HTTP request with a URL that contains that string. Each website was loaded three times and the median page load time was recorded. I then found the median of all these median page load times.
The median page load with JavaScript is 3.65 seconds. Without JavaScript the page load time drops to 2.487 seconds – a 31% decrease. (Here’s the data in WebPagetest: with JavaScript, without JavaScript.) It’s not a perfect analysis: Some script URLs don’t contain “.js” and inline script blocks are still executed. I think this is a good approximation and I hope to do further experiments to corroborate this finding.
Async Execution Order & Onload
The other new infobyte has to do with the async=true line from the GA async snippet. The purpose of this line is to cause the ga.js script to not block other async scripts from being executed. It turns out that some browsers preserve the execution order of scripts loaded using the insertBefore technique, which is the technique used in the GA snippet:
ga.type = ‘text/javascript’;
ga.async = true;
ga.src = (‘https:’ == document.location.protocol ? ‘https://ssl’ : ‘http://www’) + ‘.google-analytics.com/ga.js’;
var s = document.getElementsByTagName(‘script’)[0];
s.parentNode.insertBefore(ga, s);
Preserving execution order of async scripts makes the page slower. If the first async script takes a long time to download, all the other async scripts are blocked from executing, even if they download sooner. Executing async scripts immediately as they’re downloaded results in a faster page load time. I knew old versions of Firefox had this issue, and setting async=true fixed the problem. But I wanted to see if any other browsers also preserved execution order of async scripts loaded this way, and whether setting async=true worked.
To answer these questions I created a Browserscope user test called Async Script Execution Order. I tweeted the test URL and got 348 results from 60+ different browsers. (Thanks to all the people that ran the test! I still need results from more mobile browsers so please run the test if you have a browser that’s not covered.) Here’s a snapshot of the results:
The second column shows the results of loading two async scripts with the insertBefore pattern AND setting async=true. The third column shows the results if async is NOT set to true. Green means the scripts execute immediately (good) and red indicates that execution order is preserved (bad).
The results show that Firefox 3.6, OmniWeb 622, and all versions of Opera preserve execution order. Setting async=true successfully makes the async scripts execute immediately in Firefox 3.6 and OmniWeb 622, but not in Opera. Although this fix only applies to a few browsers, its small cost makes it worthwhile. Also, if we get results for more mobile browsers I would expect to find a few more places where the fix is necessary.
I also tested whether these insertBefore-style async scripts block the onload event. The results, shown in the fourth column, are mixed if we include older browsers, but we see that newer browsers generally block the onload event when loading these async scripts – this is true in Android, Chrome, Firefox, iOS, Opera, Safari, and IE 10. This is useful to know if you wonder why you’re still seeing long page load times even after adopting async script loading. It also means that code in your onload handler can’t reliably assume async scripts are loaded because of the many browsers out there that do not block the onload event, including IE 6-9.
And a final shout out to the awesomeness of the Open Source community that makes tools like WebPagetest and Browserscope available – thanks Pat and Lindsey!
(lack of) Caching for iPhone Home Screen Apps
Yesterday’s post, Unexpected Reloads in WebKit, revealed an interesting behavior that affects caching in Safari:
When you load the same URL back-to-back in Safari, the second load is treated the same as hitting Reload.
This is bad for performance because the browser issues a Conditional GET request for each resource instead of using the cached resource.
It’s important to be aware of this behavior when testing the primed cache experience in Safari, so web performance engineers should take note. However, in the real world it’s unlikely this behavior has much of an impact on desktop users. Here’s the table from yesterday’s post that shows how this Reload-like behavior is triggered when re-requesting a page:
| way of loading URL again | like Reload? |
|---|---|
| hit RETURN in location field | yes |
| delete URL and type it again | yes |
| launch same URL via bookmark | yes |
| click link to same URL | yes |
| go to another URL then type 1st URL again | no |
| modify querystring | no |
| enter URL in a new tab | no |
| Table 1. Triggering reload behavior in Safari | |
It’s possible that real world users might type the same URL or open the same bookmark two times in a row in the same tab, but it probably doesn’t happen that often. So what’s the big deal?
So what’s the big deal?
Whenever I see strange performance behavior I think about where that behavior might have a significant impact. Is there any place where this back-to-back Safari Reload behavior could have a significant impact? A comment from yesterday’s post hints at the answer:
Why is this article named “Unexpected Reloads in WebKit”?
Chrome is based on Webkit and doesn’t has same issue. Perhaps it would be less confusing to name it “Unexpected Reloads in Safari”.
Other people gave me the same feedback on the backchannel – why did I say “WebKit” instead of “Safari”.
Here’s why: WebKit is used in a lot of browsers. Whenever I see a bug (or a feature) in one popular WebKit-based browser I wonder if it exists in others. The main WebKit-based browsers I focus on are Chrome, Safari, Android, and iPhone. As soon as I noticed this behavior in Safari my next step was to conduct the same tests in Chrome, Android, and iPhone. As the commenter noted, this unexpected Reload behavior does not happen in Chrome. And it does not happen on Android (tested on my Nexus S). But it does happen on iPhone.
While it’s true that iPhone users are unlikely to manually launch the same URL twice-in-a-row in the same tab, there is a situation when this happens automatically: when launching home screen apps.
Home screen apps are a powerful feature on iPhone and Android that allow users to save URLs to the home screen and launch them similar to native apps. Unfortunately, launching home screen apps on the iPhone triggers something similar to the Reload behavior we see in Safari – where resources aren’t read from cache and instead generate extra HTTP requests. Let’s take a look at a few examples of home screen apps, starting with simple to more complex.
Amazon: simple URL
Typing http://www.amazon.com/ into the iPhone browser displays a version of Amazon’s front page that is customized for mobile – there’s less content, the images are smaller, etc. However, there is not a prompt to save the URL to the home screen. We can do that anyway using the arrow function key at the bottom of the screen and selecting “Add to Home Screen”.
If you’ve used home screen apps you might have noticed that they always open in the same browser tab. Let’s run a little test to confirm this:
- Click the Amazon home screen icon. This opens Amazon in mobile Safari.
- Open another tab by clicking the “pages” function key and opening a “New Page”. Enter some non-Amazon URL in this new tab, for example http://fast.stevesouders.com/ (a very lightweight page I use for testing). At this point we have at least two tabs, one with Amazon and one with fast.stevesouders.com, and we’re looking at the fast.stevesouders.com tab.
- Go back to the home screen and click the Amazon icon again.
- Note that you’re taken back into mobile Safari to the first tab that contains Amazon.
We just opened the exact same URL back-to-back in the same tab. We didn’t do it intentionally – that’s the default behavior for iPhone home screen apps. Here’s a waterfall chart for this test. (You can view an interactive waterfall by loading the HAR file in pcapperf.)
The home screen app URL is http://www.amazon.com/gp/aw/h.html/187-9233150-9797455. The first time the home screen app is launched starts at the top with 187-9233150-9797455. Since the cache was empty all the subsequent resources have 200 responses. There are some 404s for icons followed by the request for fast.stevesouders.com.
The second launch of the Amazon home screen app (187-9233150-9797455 below fast.stevesouders.com) is where it gets interesting. When the Amazon home screen app is launched the second time, a Conditional GET request is made for all of the resources even though these resources are in the cache with a future expiration date.
All of the resources that are re-requested have an expiration date more than 10 years in the future. For example, the response headers for title_gradient._V233984477_.png are:
content-length: 291 expires: Tue, 06 May 2031 21:44:21 GMT last-modified: Mon, 10 Aug 2009 11:50:45 GMT cache-control: max-age=626560472 date: Wed, 29 Jun 2011 01:09:49 GMT content-type: image/png
We know it was cached because when the Amazon home screen app is launched the second time the Conditional GET request for title_gradient._V233984477_.png has an If-Modified-Since header that contains the last-modified date in the initial response:
if-modified-since: Mon, 10 Aug 2009 11:50:45 GMT
It appears that we’ve stumbled into the Reload-like behavior we saw in Safari on the desktop. Further evidence of this is if you launch the home screen app, then type a new URL over the Amazon URL, and launch the home screen app again the resources are read from cache instead of generating numerous Conditional GET requests. (Load this HAR file in pcapperf to see for yourself.)
Untappd: full screen app
Amazon was a simple home screen app – really just a bookmark on the home screen. Developers can do more with home screen apps to make them launch and look like native apps. As described in Apple’s How-To’s for Safari on iPhone, various parts of the home screen app user experience are customizable including the home screen icon, viewport, and zooming and scaling. Developers can also have their home screen app launch in “full screen mode” by hiding the Safari UI components, including the status bar and location bar. In this situation, every time the home screen app is launched it uses the same “tab” with the exact same URL – thus triggering the Reload behavior.
Let’s have a look at Untappd on the iPhone. The first time you navigate to http://untappd.com/ in iPhone’s browser you get a suggestion to add the web app to the home screen:
After which you’ll have a customized Untappd home screen icon:
Now let’s investigate how caching works for this home screen app. We start by clearing the cache then launching the home screen app. You’ll notice there is no location bar or other Safari controls. Then we go back to the home screen and launch the Untappd home screen app again. The waterfall chart is shown below. (Here’s the HAR file.)
The first time the Untappd home screen app is launched it loads seven HTTP requests. Three of these resources are cacheable: jquery.min.js (1 year), gears_init.js (1 hour), and ga.js (1 day). Loader.gif and ajax-loader.png don’t have a future expiration date, but they do have Last-Modified and ETag response headers that could be used in a Conditional GET request.
But we see that the second time Untappd is launched from the home screen, all of the resources are re-requested. To make matters worse, none of these are Conditional GET requests, so a 200 status code is returned with the full response body.
The punchline
It’s unfortunate that home screen apps suffer from this bad caching behavior on the iPhone. Thankfully, there is a workaround: application cache. I ran similar tests on other home screen apps that use application cache. The resources listed in the CACHE: section of the manifest file were used on the iPhone without generating Conditional GET requests.
I feel bad about recommending the use of application cache. This is an issue with the browser cache on mobile Safari (and to a lesser degree on desktop Safari) that should be fixed. It’s a significant amount of work for developers to adopt application cache. The plus side is that doing so achieves the ability to work offline.
After this lengthy analysis and numerous waterfalls, here’s the punchline in a nutshell:
Home screen apps on iPhone are slower because resources are re-requested even though they should be read from cache. Use application cache to avoid this performance problem.
Unexpected Reloads in WebKit
People who work on web performance often need to load the same URL over and over again. Furthermore, they need to do this while simulating a real user’s empty cache experience and primed cache experience. When I want to analyze the empty cache experience the flow is simple: go to about:blank, clear the browser cache, enter the URL, and hit RETURN.
But what’s the right way to fetch a page repeatedly when analyzing the primed cache experience?
The main goal when testing the primed cache version of a page is to see which resources are read from cache. The goal for better performance is to cache as many responses as possible thus reducing the number of requests made when the cache is primed. If a resource has an expiration date in the future, the browser uses the cached version and doesn’t have to make an HTTP request resulting in a faster page. If a resource is expired (the expiration date is in the past) the browser issues a Conditional GET request using the If-Modified-Since and If-None-Match request headers. If the resource hasn’t changed then the server returns a simple 304 status code with no body. This is faster (because there’s no response body) but still takes time to do the HTTP request. (See my article on ETags for examples of IMS and INM.)
One way to re-request a page is to hit the Reload button, but this doesn’t give an accurate portrayal of the typical primed cache user experience. Hitting Reload causes the browser to always make an IMS/INM request for resources in the page, even for cached resources that have an expiration date in the future. Normally these resources would be used without generating an HTTP request. Although users do occasionally hit the Reload button it’s more likely that they’ll navigate to a page via a link or the location field, both of which avoid the time consuming Conditional GET requests generated when hitting Reload.
The technique I adopted years ago for re-requesting a page when testing the primed cache is to click in the location field and hit RETURN. That’s a fine approach in IE, Firefox, Chrome, and Opera, but not in Safari. Let’s investigate why.
hitting RETURN in the location field
I’m using Untappd as an example. Untappd has 68 requests when loaded on the desktop. Figure 1 shows the waterfall chart for the first 31 requests when loaded in Firefox 4 with an empty cache:
- Figure 1. untappd.com – Firefox 4 – empty cache
Most of the resources shown in Figure 1 have an expiration date in the future and therefore won’t generate an HTTP request if the user has a primed cache. To test that I click in the location field and hit RETURN. The resulting waterfall chart is shown in Figure 2. Sure enough the number of HTTP requests drops from 68 to 4!
- Figure 2. untappd.com – Firefox 4 – primed cache
If you repeat this experiment in Chrome, Firefox, Internet Explorer, and Opera you’ll get similar results – empty cache generates 68 requests, primed cache generates 4 requests. However, the result is very different in Safari 5. It’s important to understand why.
Safari is different
This test shows that Untappd has done a good job of optimizing the primed cache experience – the number of HTTP requests made by the browser drops from 68 to 4. Running the same test in Safari 5 produces different results. Clearing the cache and loading untappd.com in Safari 5 loads 68 HTTP requests – just as before. To test the primed cache experience we click in the location field and hit RETURN. Instead of only 4 requests there are 68 HTTP requests.
Why are there 64 more HTTP requests in Safari 5 for the primed cache test? Looking at the HTTP request headers we see that these are all Conditional GET requests. Let’s use http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js as the example (it’s the 8th request in Figure 1). In the empty cache scenario the HTTP request headers are:
Accept: */* Cache-Control: max-age=0 Referer: http://untappd.com/ User-Agent: Mozilla/5.0 (Macintosh; [snip...] Safari/533.20.27
The HTTP status code returned for that empty cache request is 200 OK.
In the primed cache test when we hit RETURN in the location field we see that the request for jquery.min.js contains an extra header:
Accept: */* Cache-Control: max-age=0 If-Modified-Since: Mon, 15 Feb 2010 23:30:12 GMT Referer: http://untappd.com/ User-Agent: Mozilla/5.0 (Macintosh; [snip...] Safari/533.20.27
The header that’s added in the primed cache test is If-Modified-Since. This is a Conditional GET request. The HTTP status code that’s returned is 304 Not Modified. Even though all I did was hit RETURN in the location field, Safari treated that like hitting the Reload button.
unexpected “reload” in Webkit
Unlike other browsers, Safari 5 treats hitting RETURN in the location field the same as clicking the Reload button. When else does this happen? Assuming you’ve loaded a URL in Safari and are looking at that page, this table lists various ways to load that URL again. For each technique I show whether loading the URL this way generates extra Conditional GET requests similar to clicking Reload.
| way of loading URL again | like Reload? |
|---|---|
| hit RETURN in location field | yes |
| delete URL and type it again | yes |
| launch same URL via bookmark | yes |
| click link to same URL | yes |
| go to another URL then type 1st URL again | no |
| modify querystring | no |
| enter URL in a new tab | no |
| Table 1. Triggering reload behavior in Safari | |
This black box testing indicates that whenever the same URL is loaded back-to-back in the same tab, Safari 5 treats it as a Reload. I was describing this behavior to Jay Freeman (saurik) at Foo Camp. He pointed me to this code from WebCore:
else if (sameURL) // Example of this case are sites that reload the same URL with a different cookie // driving the generated content, or a master frame with links that drive a target // frame, where the user has clicked on the same link repeatedly. m_loadType = FrameLoadTypeSame;
Searching in that same file for FrameLoadTypeSame we find this code:
case FrameLoadTypeReload: case FrameLoadTypeReloadFromOrigin: case FrameLoadTypeSame: case FrameLoadTypeReplace: history()->updateForReload(); m_client->transitionToCommittedForNewPage(); break;
This code doesn’t account for the behavior, but it does show that FrameLoadTypeSame and FrameLoadTypeReload are treated as similar cases in this context, and perhaps that’s why IMS/INM requests are generated.
One important takeaway from this is: don’t hit RETURN in the location field to test primed cache experience in Safari. Instead, go to a different URL and then type the test URL in the location field, or open a new tab and type the URL.
There’s a second more important takeaway from this. I’ll cover that in tomorrow’s post. If you know the answer, please don’t spoil it. Oh what the heck – if you think you know the answer go ahead and add a comment.
Evolution of Script Loading
Velocity China starts tomorrow morning. I’m kicking it off with a keynote and am finishing up my slides. I’ll be talking about progressive enhancement and smart script loading. As I reviewed the slides it struck me how the techniques for loading external scripts have changed significantly in the last few years. Let’s take a look at what we did, where we are now, and try to see what the future might be.
Scripts in the HEAD
Just a few years ago most pages, even for the top websites, loaded their external scripts with HTML in the HEAD of the document:
<head> <script src=”core.js” type=”text/javascript”></script> <script src=”more.js” type=”text/javascript”></script> </head>
Developers in tune with web performance best practices cringe when they see scripts loaded this way. We know that older browsers (now that’s primarily Internet Explorer 6 & 7) load scripts sequentially. The browser downloads core.js and then parses and executes it. Then it does the same with more.js – downloads the script and then parses and executes it.
In addition to loading scripts sequentially, older browsers block other downloads until this sequential script loading phase is completed. This means there may be a significant delay before these browsers even start downloading stylesheets, images, and other resources in the page.
These download issues are mitigated in newer browsers (see my script loading roundup). Starting with IE 8, Firefox 3.6, Chrome 2, and Safari 4 scripts generally get downloaded in parallel with each other as well as with other types of resources. I say “generally” because there are still quirks in how browsers perform this parallel script loading. In IE 8 and 9beta, for example, images are blocked from being downloaded during script downloads (see this example). In a more esoteric use case, Firefox 3 loads scripts that are document.written into the page sequentially rather than in parallel.
Then there’s the issue with rendering being blocked by scripts: any DOM elements below a SCRIPT tag won’t be rendered until that script is finished loading. This is painful. The browser has already downloaded the HTML document but none of those Ps, DIVs, and ULs get shown to the user if there’s a SCRIPT above them. If you put the SCRIPT in the document HEAD then the entire page is blocked from rendering.
There are a lot of nuances in how browsers load scripts especially with the older browsers. Once I fully understood the impact scripts had on page loading I came up with my first recommendation to improve script loading:
Move Scripts to the Bottom
Back in 2006 and 2007 when I started researching faster script loading, all browsers had the same problems when it came to loading scripts:
- Scripts were loaded sequentially.
- Loading a script blocked all other downloads in the page.
- Nothing below the SCRIPT was rendered until the script was done loading.
The solution I came up with was to move the SCRIPT tags to the bottom of the page.
... <script src=”core.js” type=”text/javascript”></script> <script src=”more.js” type=”text/javascript”></script> </body>
This isn’t always possible, for example, scripts for ads that do document.write can’t be moved – they have to do their document.write in the exact spot where the ad is supposed to appear. But many scripts can be moved to the bottom of the page with little or no work. The benefits are immediately obvious – images download sooner and the page renders more quickly. This was one of my top recommendations in High Performance Web Sites. Many websites adopted this change and saw the benefits.
Load Scripts Asynchronously
Moving scripts to the bottom of the page avoided some problems, but other script loading issues still existed. During 2008 and 2009 browsers still downloaded scripts sequentially. There was an obvious opportunity here to improve performance. Although it’s true that scripts (often) need to be executed in order, they don’t need to be downloaded in order. They can be downloaded in any order – as long as the browser preserves the original order of execution.
Browser vendors realized this. (I like to think that I had something to do with that.) And newer browsers (starting with IE8, Firefox 3.6, Chrome 2, and Safari 4 as mentioned before) started loading scripts in parallel. But back in 2008 & 2009 sequential script loading was still an issue. I was analyzing MSN.com one day and noticed that their scripts loaded in parallel – even though this was back in the Firefox 2.0 days. They were using the Script DOM Element approach:
var se = document.createElement("script");
se.src = "core.js";
document.getElementsByTagName("head")[0].appendChild(se);
I’ve spent a good part of the last few years researching asynchronous script loading techniques like this. These async techniques (summarized in this blog post with full details in chapter 4 of Even Faster Web Sites) achieve parallel script loading in older browsers and avoid some of the quirks in newer browsers. They also mitigate the issues with blocked rendering: when a script is loaded using an async technique the browser charges ahead and renders the page while the script is being downloaded. This example has a script in the HEAD that’s loaded using the Script DOM Element technique. This script is configured to take 4 seconds to download. When you load the URL you’ll see the page render immediately, proving that rendering proceeds when scripts are loaded asynchronously.
Increased download parallelism and faster rendering – what more could you want? Well…
Async + On-demand Execution
Loading scripts asynchronously speeds up downloads (more parallelism) and rendering. But – when the scripts arrive at the browser rendering stops and the browser UI is locked while the script is parsed and executed. There wouldn’t be any room for improvement here if all that JavaScript was needed immediately, but websites don’t use all the code that’s downloaded – at least not right away. The Alexa US Top 10 websites download an average of 229 kB of JavaScript (compressed) but only execute 29% of those functions by the time the load event fires. The other 71% of code is cruft, conditional blocks, or most likely DHTML and Ajax functionality that aren’t used to render the initial page.
This discovery led to my recommendation to split the initial JavaScript download into the code needed to render the page and the code that can be loaded later for DHTML and Ajax features. (See this blog post or chapter 3 of EFWS.) Websites often load the code that’s needed later in the window’s onload handler. The Gmail Mobile team found wasn’t happy with the UI locking up when that later code arrived at the browser. After all, this DHTML/Ajaxy code might not even be used. They’re the first folks I saw who figured out a way to separate the download phase from the parse-execute phase of script loading. They did this by wrapping all the code in a comment, and then when the code is needed removing the comment delimiters and eval’ing. Gmail’s technique uses iframes so requires changing your existing scripts. Stoyan has gone on to explore using image and object tags to download scripts without the browser executing them, and then doing the parse-execute when the code is needed.
What’s Next?
Web pages are getting more complex. Advanced developers who care about performance need more control over how the page loads. Giving developers the ability to control when an external script is parsed and executed makes sense. Right now it’s way too hard. Luckily, help is on the horizon. Support for LINK REL=”PREFETCH” is growing. This provides a way to download scripts while avoiding parsing and execution. Browser vendors need to make sure the LINK tag has a load event so developers can know whether or not the file has finished downloading. Then the file that’s already in the browser’s cache can be added asynchronously using the Script DOM Element approach or some other technique only when the code is actually needed.
We’re close to having the pieces to easily go to the next level of script loading optimization. Until all the pieces are there, developers on the cutting edge will have to do what they always do – work a little harder and stay on top of the latest best practices. For full control over when scripts are downloaded and when they’re parsed and executed I recommend you take a look at the posts from Gmail Mobile and Stoyan.
Based on the past few years I’m betting there are more improvements to script loading still ahead.
Render first. JS second.
Let me start with the takeaway point:
The key to creating a fast user experience in today’s web sites is to render the page as quickly as possible. To achieve this JavaScript loading and execution has to be deferred.
I’m in the middle of several big projects so my blogging rate is down. But I got an email today about asynchronous JavaScript loading and execution. I started to type up my lengthy response and remembered one of those tips for being more productive: “type shorter emails – no one reads long emails anyway”. That just doesn’t resonate with me. I like typing long emails. I love going into the details. But, I agree that an email response that only a few people might read is not the best investment of time. So I’m writing up my response here.
It took me months to research and write the “Loading Scripts Without Blocking” chapter from Even Faster Web Sites. Months for a single chapter! I wasn’t the first person to do async script loading – I noticed it on MSN way before I started that chapter – but that work paid off. There has been more research on async script loading from folks like Google, Facebook and Meebo. Most JavaScript frameworks have async script loading features – two examples are YUI and LABjs. And 8 of today’s Alexa Top 10 US sites use advanced techniques to load scripts without blocking: Google, Facebook, Yahoo!, YouTube, Amazon, Twitter, Craigslist(!), and Bing. Yay!
The downside is – although web sites are doing a better job of downloading scripts without blocking, once those scripts arrive their execution still blocks the page from rendering. Getting the content in front of the user as quickly as possible is the goal. If asynchronous scripts arrive while the page is loading, the browser has to stop rendering in order to parse and execute those scripts. This is the biggest obstacle to creating a fast user experience. I don’t have scientific results that I can cite to substantiate this claim (that’s part of the big projects I’m working on). But anyone who disables JavaScript in their browser can attest that sites feel twice as fast.
My #1 goal right now is to figure out ways that web sites can defer all JavaScript execution until after the page has rendered. Achieving this goal is going to involve advances from multiple camps – changes to browsers, new web development techniques, and new pieces of infrastructure. I’ve been talking this up for a year or so. When I mention this idea these are the typical arguments I hear for why this won’t work:
In response to this argument I point to Opera’s Delayed Script Execution feature. I encourage you to turn it on, surf around, and try to find a site that breaks. Even sites like Gmail and Facebook work! I’m sure there are some sites that have problems (perhaps that’s why this feature is off by default). But if some sites do have problems, how many sites are we talking about? And what’s the severity of the problems? We definitely don’t want errors, rendering problems, or loss of ad revenue. Even though Opera has had this feature for over two years (!), I haven’t heard much discussion about it. Imagine what could happen if significant resources focused on this problem.
What are the next steps?
- Browsers should look at Opera’s behavior and implement the SCRIPT ASYNC and DEFER attributes.
- Developers should adopt asynchronous script loading techniques and avoid rendering the initial page view with JavaScript on the client.
- Third party snippet providers, most notably ads, need to move away from document.write.
Redirect caching deep dive
I was talking to a performance guru today who considers redirects one of the top two performance problems impacting web pages today. (The other was document.write.) I agree redirects are an issue, so much so that I wrote a chapter on avoiding redirects in High Performance Web Sites. What makes matters worse is that, even though redirects are cacheable, most browsers (even the new ones) don’t cache them.
Another performance guru, Eric Lawrence, sent me an email last week pointing out how cookies, status codes, and response headers affect redirect caching. Even though there are a few redirect tests in Browserscope, they don’t test all of these conditions. I wanted a more thorough picture of the state of redirect caching across browsers.
The Redirect Caching Tests page provides a test harness for exercising different redirect caching scenarios.
You can use the “Test Once” button to test a specific scenario, but if you choose “Test All” the harness runs through all the tests and offers to post the results (anonymously) to Browserscope. Here’s a snapshot of the results:
I realize you can’t read the results, but suffice it to say red is bad. If you click on the table you’ll go to the full results. They’re broken into two tables: redirects that should be cached, and redirects that should not be cached. For example, a 301 response with an expiration date in the future should be cached, but a 302 redirect with an expiration date in the past shouldn’t be cached. The official ruling can be found in RFC 2616. (Also, discussions Eric had with Mark Nottingham, chair of the Httpbis working group, indicate that 303 redirects with a future expiration date should be cached.)
Chrome, iPhone, and Opera 10.60 are doing the best job, but there’s still a lot of missed opportunities. IE 9 platform preview 3 still doesn’t cache any redirects, but Eric’s blog post, Caching Improvements in Internet Explorer 9, describes how that’ll be in the final version of IE9. If you use redirects that don’t change, make sure to use a 301 status code and set an expiration date in the future. That will ensure they’re cached in Chome, Firefox, iPhone, Opera, and IE9.
Please help out by running the test on your browser (especially mobile!) and contributing the results back to Browserscope so we can help browser vendors know what needs to be fixed.
Mobile cache file sizes
Mobile is big, but knowledge about how to make a mobile web site fast is lacking. The state of mobile web performance is in a similar place as desktop was six years ago when I started my performance deep dive. How many of us are happy with the speed of web pages on mobile? (I’m not.) How many of us know the top 10 best practices for building high performance mobile web sites? (I don’t.)
I’ve been focusing more on mobile in order to help build this set of best practices. Browserscope is a valuable resource since it measures all browsers, both mobile and desktop. The Network category for popular mobile browsers shows information about max connections per hostname, parallel script loading, redirect caching, and more. Since Browserscope’s data is crowdsourced it’s easy to get coverage on a wide variety of mobile devices. The table below shows the results from Browserscope for some popular mobile devices.
One thing I’ve wanted to measure on mobile is the browser’s cache. Caching on mobile devices is a cause for concern. In my experience a page I visited just a few minutes ago doesn’t seem to be cached when I visit it again. A few months ago I started creating tests for measuring the browser’s cache.
That’s why I was especially excited to see Ryan Grove’s post on Mobile Browser Cache Limits. I noticed his results were quite different from mine, so I commented on his blog post and invited him to contact me. Which he did! It’s great to find someone to collaborate with, especially when designing tests like this where another pair of eyes is a big help.
Ryan and I created a new test design. He’s deployed his under the name cachetest on GitHub. My implementation is called Max Cache File Size. I’m hosting it so you can run it immediately. I’ve integrated it with Browserscope as a User Test. Anyone who runs my hosted version has the option to post their results (anonymously) to Browserscope and contribute to building a repository for script cache sizes for all browsers.
Here’s a link to the Max Cache File Size results on Browserscope. A summary of the results with some other findings follows:
| Browser | Max Cached Script Size Same Session |
Cache Cross Lock/Unlock |
Cache Cross Power Cycle |
|---|---|---|---|
| Android 2.1 | 4 MB | yes | yes |
| Android 2.2 | 2 MB | yes | yes |
| iPad | 4 MB | yes | no |
| iPhone 3 | 4 MB | yes | no |
| iPhone 4 | 4 MB | yes | no |
| Palm Pre | 1 MB | yes | yes |
My Max Cache File Size test measures the largest script that’s cached in the same session (going from one page to another page). Many mobile devices reach the maximum size tested – 4MB. It’s interesting to see that in the recent upgrade from Android 2.1 to 2.2, the maximum cached script size drops from 4MB to 2MB. The Palm Pre registers at 320kB – much smaller than the others but large enough to handle many real world situations. Note that these sizes are the script’s uncompressed size.
Knowing the cache size that applies during a single session is valuable, but users often revisit pages after locking and unlocking their device, and some users might even power cycle their device between visits. Ryan and I manually tested a few devices under these scenarios – the results are shown in the previous table. Although results are mixed for the power cycling case, the cached items persist across lock/unlock. For me personally, this is the more typical case. (I only power cycle when I’m on the plane or need to reset the device.)
These results show that the 15 kB and 25 kB size limit warnings for a single resource are no longer a concern for mobile devices. However, even though the test went as high as 4 MB (uncompressed), I dearly hope you’re not even close to that size. (I saw similar results for stylesheets, but removed them from the automated test because stylesheets over ~1 MB cause problems on the iPhone.)
It’s great to have this data, and have it verified by different sources. But this is only testing the maximum size of a single script and stylesheet that can be cached. I believe the bigger issue for mobile is the maximum cache size. A few months ago I wrote a Call to improve browser caching. I wrote that in the context of desktop browsers, where I have visibility into the browser’s cache and available disk space. I think the size of mobile caches is even smaller. If you have information about the size of the browser cache on mobile devices, or tests to determine that, please share them in the comments below.
Finally, please run the Max Cache File Size test and add more data to the results.
Many thanks to Ryan Grove for working on this caching test – check out his updated post: Mobile Browser Cache Limits, Revisited. And thanks to Lindsey Simon for making Browserscope such a great framework for crowdsourcing browser performance data.
Frontend SPOF
My evangelism of high performance web sites started off in the context of quality code and development best practices. It’s easy for a style of coding to permeate throughout a company. Developers switch teams. Code is copied and pasted (especially in the world of web development). If everyone is developing in a high performance way, that’s the style that will characterize how the company codes.
This argument of promoting development best practices gained traction in the engineering quarters of the companies I talked to, but performance improvements continued to get backburnered in favor of new features and content that appealed to the business side of the organization. Improving performance wasn’t considered as important as other changes. Everyone assumed users wanted new features and that’s what got the most attention.
It became clear to me that we needed to show a business case for web performance. That’s why the theme for Velocity 2009 was “the impact of performance on the bottom line”. Since then there have been numerous studies released that have shown that improving performance does improve the bottom line. As a result, I’m seeing the business side of many web companies becoming strong advocates for Web Performance Optimization.
But there are still occasions when I have a hard time convincing a team that focusing on web performance, specifically frontend performance, is important. Shaving off hundreds (or even thousands) of milliseconds just doesn’t seem worthwhile to them. That’s when I pull out the big guns and explain that loading scripts and stylesheets in the typical way creates a frontend single point of failure that can bring down the entire site.
Examples of Frontend SPOF
The thought that simply adding a script or stylesheet to your web page could make the entire site unavailable surprises many people. Rather than focusing on CSS mistakes and JavaScript errors, the key is to think about what happens when a resource request times out. With this clue, it’s easy to create a test case:
<html> <head> <script src="http://www.snippet.com/main.js" type="text/javascript"> </script> </head> <body> Here's my page! </body> </html>
This HTML page looks pretty normal, but if snippet.com is overloaded the entire page is blank waiting for main.js to return. This is true in all browsers.
Here are some examples of frontend single points of failure and the browsers they impact. You can click on the Frontend SPOF test links to see the actual test page.
| Frontend SPOF test | Chrome | Firefox | IE | Opera | Safari |
|---|---|---|---|---|---|
| External Script | blank below | blank below | blank below | blank below | blank below |
| Stylesheet | flash | flash | blank below | flash | blank below |
| inlined @font-face | delayed | flash | flash | flash | delayed |
| Stylesheet with @font-face | delayed | flash | totally blank* | flash | delayed |
| Script then @font-face | delayed | flash | totally blank* | flash | delayed |
* Internet Explorer 9 does not display a blank page, but does “flash” the element.
The failure cases are highlighted in red. Here are the four possible outcomes sorted from worst to best:
- totally blank – Nothing in the page is rendered – the entire page is blank.
- blank below – All the DOM elements below the resource in question are not rendered.
- delayed – Text that uses the @font-face style is invisible until the font file arrives.
- flash – DOM elements are rendered immediately, and then redrawn if necessary after the stylesheet or font has finished downloading.
Web Performance avoids SPOF
It turns out that there are web performance best practices that, in addition to making your pages faster, also avoid most of these frontend single points of failure. Let’s look at the tests one by one.
- External Script
- All browsers block rendering of elements below an external script until the script arrives and is parsed and executed. Since many sites put scripts in the HEAD, this means the entire page is typically blank. That’s why I believe the most important web performance coding pattern for today’s web sites is to load JavaScript asynchronously. Not only does this improve performance, but it avoids making external scripts a possible SPOF.
- Stylesheet
- Browsers are split on how they handle stylesheets. Firefox and Opera charge ahead and render the page, and then flash the user if elements have to be redrawn because their styling changed. Chrome, Internet Explorer, and Safari delay rendering the page until the stylesheets have arrived. (Generally they only delay rendering elements below the stylesheet, but in some cases IE will delay rendering everything in the page.) If rendering is blocked and the stylesheet takes a long time to download, or times out, the user is left staring at a blank page. There’s not a lot of advice on loading stylesheets without blocking page rendering, primarily because it would introduce the flash of unstyled content.
- inlined @font-face
- I’ve blogged before about the performance implications of using @font-face. When the @font-face style is declared in a STYLE block in the HTML document, the SPOF issues are dramatically reduced. Firefox, Internet Explorer, and Opera avoid making these custom font files a SPOF by rendering the affected text and then redrawing it after the font file arrives. Chrome and Safari don’t render the customized text at all until the font file arrives. I’ve drawn these cells in yellow since it could cause the page to be unusable for users using these browsers, but most sites only use custom fonts on a subset of the page.
- Stylesheet with @font-face
- Inlining your @font-face style is the key to avoiding having font files be a single point of failure. If you inline your @font-face styles and the font file takes forever to return or times out, the worst case is the affected text is invisible in Chrome and Safari. But at least the rest of the page is visible, and everything is visible in Firefox, IE, and Opera. Moving the @font-face style to a stylesheet not only slows down your site (by requiring two sequential downloads to render text), but it also creates a special case in Internet Explorer 7 & 8 where the entire page is blocked from rendering. IE 6 is only slightly better – the elements below the stylesheet are blocked from rendering (but if your stylesheet is in the HEAD this is the same outcome).
- Script then @font-face
- Inlining your @font-face style isn’t enough to avoid the entire page SPOF that occurs in IE. You also have to make sure the inline STYLE block isn’t preceded by a SCRIPT tag. Otherwise, your entire page is blank in IE waiting for the font file to arrive. If that file is slow to return, your users are left staring at a blank page.
SPOF is bad
Five years ago most of the attention on web performance was focused on the backend. Since then we’ve learned that 80% of the time users wait for a web page to load is the responsibility of the frontend. I feel this same bias when it comes to identifying and guarding against single points of failure that can bring down a web site – the focus is on the backend and there’s not enough focus on the frontend. For larger web sites, the days of a single server, single router, single data center, and other backend SPOFs are way behind us. And yet, most major web sites include scripts and stylesheets in the typical way that creates a frontend SPOF. Even more worrisome – many of these scripts are from third parties for social widgets, web analytics, and ads.
Look at the scripts, stylesheets, and font files in your web page from a worst case scenario perspective. Ask yourself:
- Is your web site’s availability dependent on these resources?
- Is it possible that if one of these resources timed out, users would be blocked from seeing your site?
- Are any of these single point of failure resources from a third party?
- Would you rather embed resources in a way that avoids making them a frontend SPOF?
Make sure you’re aware of your frontend SPOFs, track their availability and latency closely, and embed them in your page in a non-blocking way whenever possible.
AutoHead – my first Browserscope user test
In the comments from my last blog post (appendChild vs insertBefore) someone asked which browsers do and don’t automatically create a HEAD element. This is important when you’re deciding how to dynamically add scripts to your web page. I used this question as the motivation for creating my first Browserscope user test. Here’s the story behind this new feature in Browserscope and the answer to the automatically create HEAD question. (You can run the AutoHead test to skip ahead and see the results.)
Level, Meta-level, Meta-meta-level
Level1: When Chrome was launched in 2008 I started a project called UA Profiler to analyze the performance characteristics of browsers. The key concept was to crowdsource gathering the data – publish the test framework and encourage the web community to run the tests on their browser of choice. There are numerous benefits to this approach:
- a wider variety of browsers are tested (more than I could possibly install in a test lab)
- results for new browsers happen immediately (often before the browser is officially released)
- tests are performed under real world conditions
Level 2: I teamed up with Lindsey Simon to take UA Profiler to the next level. The result was the launch of Browserscope. In addition to making this a functioning open source project, the framework was opened up to include multiple test categories. In addition to performance (renamed “Network“), other test categories were added: Security, Acid3, Selectors API, and Rich Text.
Level 3: A few weeks ago Lindsey took Browserscope to the next level with the addition of the User Tests feature. Now, anyone can add a test to Browserscope. In this early alpha version of the feature, users create one or more test pages on their own server, register the test with Browserscope, and embed a JavaScript snippet at the end of their test to send the results back to Browserscope for storing. The benefit for the test creator is that Browserscope stores all the data, parses the User-Agent strings for proper categorization, and provides a widget for viewing the results.
Even though Lindsey is careful to call this an alpha, it went very smoothly for me. Once I had my test page, it took less than 15 minutes to integrate with Browserscope and start gathering results. So let’s take a look at my test…
the test – AutoHead
In my appendChild vs insertBefore blog post I talk about why this code generates bugs:
document.getElementsByTagName('head')[0].appendChild(...)
The context was using this pattern in 3rd party code snippets – where you don’t have any control of the main page. It turns out that some web pages out in the wild wild web don’t use the HEAD tag. Luckily, most browsers automatically create a HEAD element if one isn’t specified in the page. Unfortunately, not all browsers do this.
In the comments on that blog post Andy asked, “What browsers are we talking about here?”
How can I possibly attempt to answer that question? It would require running a test on many different versions of many different browsers, including mobile devices. I’m not equipped with a setup to do that.
Then the light bulb lit up. 
I can do this with a Browserscope User Test!
Creating the test was easy. My HTML page doesn’t have a HEAD tag. I put a script at the bottom that checks if the page contains a head element:
bHead = document.getElementsByTagName('head').length;
I have to store the result in a specific variable that Browserscope looks for:
var _bTestResults = {
'autohead': bHead
};
This data structure is slurped up by Browserscope via this snippet (as shown on the User Tests Howto page):
(function() {
var _bTestKey = '<YOUR-TEST-ID-GOES-HERE>';
var _bScript = document.createElement('script');
_bScript.src = 'http://www.browserscope.org/user/beacon/'
+ _bTestKey;
_bScript.setAttribute('async', 'true');
var scripts = document.getElementsByTagName('script');
var lastScript = scripts[scripts.length - 1];
lastScript.parentNode.insertBefore(_bScript, lastScript);
})();
Voila! You’re done. Well, almost done. You still have to promote your test.
Promoting your test
With this small amount of work I’m now ready to ask the web community to help me gather results. For me personally, I accomplish this by writing this blog post asking for help:
Help me out by running the AutoHead test. Thanks!
I’ve embedded the Browserscope results snippet in the iframe below, so you can see results as they come in. So far iPhone and Safari 3.2 are the only browsers that don’t automatically create the HEAD element.
If you want to avoid bugs when dynamically adding scripts, you might want to use one of the more solid patterns mentioned in my appendChild vs insertBefore blog post. If you want to gather data on some browser test that interests you, read the Browserscope User Test Howto and go for it. If you have problems, contact the Browserscope mailing list. If you have success, contact me and I’ll tweet your test to drive more traffic to it. This is still in alpha, but I’m very excited about the possibilities. I can’t wait to see the kinds of tests you come up with.
Update: After just one day, thanks to all of you who ran the test, I’ve collected 400 measurements on 20 different browsers and 60 unique versions. The results show that the following browsers do NOT automatically create a HEAD element: Android 1.6, Chrome 5.0.307 (strange), iPhone 3.1.3, Nokia 90, Opera 8.50, Opera 9.27, and Safari 3.2.1. This adds up to over 1% of your users, so it’s important to keep this in mind when adding scripts dynamically.
I also had some comments I wanted to pass on about my Browserscope user test. In hindsight, I wish I had chosen a better “test_key” name for the _bTestResults object. I didn’t realize this would appear as the column header in my results table. Rather than “autohead” I would have done “Automatically Creates HEAD”. Also, rather than return 0 or 1, I wish I had returned “no” and “yes”. Finally, I wish there was a way to embed the results table widget besides using an iframe. I’ll file bugs requesting better documentation for these items.
Call to improve browser caching
Over Christmas break I wrote Santa my browser wishlist. There was one item I neglected to ask for: improvements to the browser disk cache.
In 2007 Tenni Theurer and I ran an experiment to measure browser cache stats from the server side. Tenni’s write up, Browser Cache Usage – Exposed, is the stuff of legend. There she reveals that while 80% of page views were done with a primed cache, 40-60% of unique users hit the site with an empty cache at least once per day. 40-60% seems high, but I’ve heard similar numbers from respected web devs at other major sites.
Why do so many users have an empty cache at least once per day?
I’ve been racking my brain for years trying to answer this question. Here are some answers I’ve come up with:
- first time users – Yea, but not 40-60%.
- cleared cache – It’s true: more and more people are likely using anti-virus software that clears the cache between browser sessions. And since we ran that experiment back in 2007 many browsers have added options for clearing the cache frequently (for example, Firefox’s privacy.clearOnShutdown.cache option). But again, this doesn’t account for the 40-60% number.
- flawed experiment – It turns out there was a flaw in the experiment (browsers ignore caching headers when an image is in memory), but this would only affect the 80% number, not the 40-60% number. And I expect the impact on the 80% number is small, given the fact that other folks have gotten similar numbers. (In a future blog post I’ll share a new experiment design I’ve been working on.)
- resources got evicted – hmmmmm
OK, let’s talk about eviction for a minute. The two biggest influencers for a resource getting evicted are the size of the cache and the eviction algorithm. It turns out, the amount of disk space used for caching hasn’t kept pace with the size of people’s drives and their use of the Web. Here are the default disk cache sizes for the major browsers:
- Internet Explorer: 8-50 MB
- Firefox: 50 MB
- Safari: everything I found said there isn’t a max size setting (???)
- Chrome: < 80 MB (varies depending on available disk space)
- Opera: 20 MB
Those defaults are too small. My disk drive is 150 GB of which 120 GB is free. I’d gladly give up 5 GB or more to raise the odds of web pages loading faster.
Even with more disk space, the cache is eventually going to fill up. When that happens, cached resources need to be evicted to make room for the new ones. Here’s where eviction algorithms come into play. Most eviction algorithms are LRU-based – the resource that was least recently used is evicted. However, our knowledge of performance pain points has grown dramatically in the last few years. Translating this knowledge into eviction algorithm improvements makes sense. For example, we’re all aware how much costlier it is to download a script than an image. (Scripts block other downloads and rendering.) Scripts, therefore, should be given a higher priority when it comes to caching.
It’s hard to get access to gather browser disk cache stats, so I’m asking people to discover their own settings and share them via the Browser Disk Cache Survey form. I included this in my talks at JSConf and jQueryConf. ~150 folks at those conferences filled out the form. The data shows that 55% of people surveyed have a cache that’s over 90% full. (Caveats: this is a small sample size and the data is self-reported.) It would be great if you would take time to fill out the form. I’ve also started writing instructions for finding your cache settings.
I’m optimistic about the potential speedup that could result from improving browser caching, and fortunately browser vendors seem receptive (for example, the recent Mozilla Caching Summit). I expect we’ll see better default cache sizes and eviction logic in the next major release of each browser. Until then, jack up your defaults as described in the instructions. And please add comments for any browsers I left out or got wrong. Thanks.