JavaScript Performance

January 13, 2012 10:09 pm | 19 Comments

Last night I spoke at the San Francisco JavaScript Meetup. I gave a brand new talk called JavaScript Performance that focuses on script loading and async snippets. The snippet example I chose was the Google Analytics async snippet. The script-loading part of that snippet is only six lines, but a lot of thought and testing went into it. It’s a great prototype to use if you’re creating your own async snippet. I’ll tweet if/when the video of my talk comes out, but in the meantime the slides (Slideshare, pptx) do a good job of relaying the information.

There are two new data points from the presentation that I want to call out in this blog post.

Impact of JavaScript

The presentation starts by suggesting that JavaScript is typically the #1 place to look for making a website faster. My anecdotal experience supports this hypothesis, but I wanted to try to do some quantitative verification. As often happens, I turned to WebPagetest.

I wanted to test the Alexa Top 100 URLs with and without JavaScript. To load these sites withOUT JavaScript I used WebPagetest’s “block” feature. I entered “.js” which tells WebPagetest to ignore every HTTP request with a URL that contains that string. Each website was loaded three times and the median page load time was recorded. I then found the median of all these median page load times.

The median page load with JavaScript is 3.65 seconds. Without JavaScript the page load time drops to 2.487 seconds – a 31% decrease. (Here’s the data in WebPagetest: with JavaScript, without JavaScript.) It’s not a perfect analysis: Some script URLs don’t contain “.js” and inline script blocks are still executed. I think this is a good approximation and I hope to do further experiments to corroborate this finding.

Async Execution Order & Onload

The other new infobyte has to do with the async=true line from the GA async snippet. The purpose of this line is to cause the ga.js script to not block other async scripts from being executed. It turns out that some browsers preserve the execution order of scripts loaded using the insertBefore technique, which is the technique used in the GA snippet:

var ga = document.createElement(‘script’);
ga.type = ‘text/javascript’;
ga.async = true;
ga.src = (‘https:’ == document.location.protocol ? ‘https://ssl’ : ‘http://www’) + ‘.google-analytics.com/ga.js’;
var s = document.getElementsByTagName(‘script’)[0];
s.parentNode.insertBefore(ga, s);

Preserving execution order of async scripts makes the page slower. If the first async script takes a long time to download, all the other async scripts are blocked from executing, even if they download sooner. Executing async scripts immediately as they’re downloaded results in a faster page load time. I knew old versions of Firefox had this issue, and setting async=true fixed the problem. But I wanted to see if any other browsers also preserved execution order of async scripts loaded this way, and whether setting async=true worked.

To answer these questions I created a Browserscope user test called Async Script Execution Order. I tweeted the test URL and got 348 results from 60+ different browsers. (Thanks to all the people that ran the test! I still need results from more mobile browsers so please run the test if you have a browser that’s not covered.) Here’s a snapshot of the results:

The second column shows the results of loading two async scripts with the insertBefore pattern AND setting async=true. The third column shows the results if async is NOT set to true. Green means the scripts execute immediately (good) and red indicates that execution order is preserved (bad).

The results show that Firefox 3.6, OmniWeb 622, and all versions of Opera preserve execution order. Setting async=true successfully makes the async scripts execute immediately in Firefox 3.6 and OmniWeb 622, but not in Opera. Although this fix only applies to a few browsers, its small cost makes it worthwhile. Also, if we get results for more mobile browsers I would expect to find a few more places where the fix is necessary.

I also tested whether these insertBefore-style async scripts block the onload event. The results, shown in the fourth column, are mixed if we include older browsers, but we see that newer browsers generally block the onload event when loading these async scripts – this is true in Android, Chrome, Firefox, iOS, Opera, Safari, and IE 10. This is useful to know if you wonder why you’re still seeing long page load times even after adopting async script loading. It also means that code in your onload handler can’t reliably assume async scripts are loaded because of the many browsers out there that do not block the onload event, including IE 6-9.

And a final shout out to the awesomeness of the Open Source community that makes tools like WebPagetest and Browserscope available – thanks Pat and Lindsey!

19 Comments

Silk, iPad, Galaxy comparison

December 1, 2011 9:51 am | 11 Comments

In my previous blog post I announced Loadtimer – a mobile test harness for measuring page load times. I was motivated to create Loadtimer because recent reviews of the Kindle Fire lacked the quantified data and reliable test procedures needed to compare browser performance.

Most performance evaluations of Silk that have come out since its launch have two conclusions:

  1. Silk is faster when acceleration is turned off.
  2. Silk is slow compared to other tablets.

Let’s poke at those more rigorously using Loadtimer.

Test Description

In this test I’m going to compare the following tablets: Kindle Fire (with acceleration on and off), iPad 1, iPad 2, Galaxy 7.0, and Galaxy 10.1.

The test is based on how long it takes for web pages to load on each device. I picked 11 URLs that are top US websites:

Some popular choices (Google, YouTube, and Twitter) weren’t selected because they have framebusting code and so don’t work in Loadtimer’s iframe-based test harness.

The set of 11 URLs were loaded 9 times on each device. The set of URLs was randomized for each run. All the tests were conducted on my home wifi over a Comcast cable modem. (Check out this photo of my test setup.) All the tests were done at the same time of day over a 3 hour period. I did one test at a time to avoid bandwidth contention, and rotated through the devices doing one run at a time. I cleared the cache between each run.

Apples and Oranges

The median page load time for each URL on each device is shown in the Loadtimer Results page. It’s a bit complicated to digest. The fastest load time is shown in green and the slowest is red – that’s easy. The main complication is that not every device got the same version of a given URL. Cells in the table that are shaded with a gray background were cases where the device received a mobile version of the URL. Typically (but not always) the mobile version is lighter than the desktop version (fewer requests, fewer bytes, less JavaScript, etc.) so it’s not valid to do a heads up comparison of page load times between desktop and mobile versions.

Out of 11 URLs, the Galaxy 7.0 received 6 that were mobile versions. The Galaxy 10.1 and Silk each received 2 mobile versions, and the iPads each had only one mobile version across the 11 URLs.

In order to gauge the difference between the desktop and mobile versions, the results table shows the number of resources in each page. eBay, for example, had 64 resources in the desktop version, but only 18-22 in the mobile version. Not surprisingly, the three tablets that received the lighter mobile version had the fastest page load times. (If a mobile version was faster than the fastest desktop version, I show it in non-bolded green with a gray background.)

This demonstrates the importance of looking at the context of what’s being tested. In the comparisons below we’ll make sure to keep the desktop vs mobile issue in mind.

Silk vs Silk

Let’s start making some comparisons. The results table is complicated when all 6 rows are viewed. The checkboxes are useful for making more focused comparisons. The Silk (accel off) and Silk (accel on) results show that indeed Silk performed better with acceleration turned off for every URL. This is surprising, but there are some things to note.

First, this is the first version of Silk. Jon Jenkins, Director of Software Development for Silk, spoke at Velocity Europe a few weeks back. In his presentation he shows different places where the split in Silk’s split architecture could happen (slides 26-28). He also talked about the various types of optimizations that are part of the acceleration. Although he didn’t give specifics, it’s unlikely that all of those architectural pieces and performance optimizations have been deployed in this first version of Silk. The test results show that some of the obvious optimizations, such as concatenating scripts, aren’t happening when acceleration is on. I expect we’ll see more optimizations rolled out during the Silk release cycle, just as we do for other browsers.

A smaller but still important issue is that although the browser cache was cleared between tests, the DNS cache wasn’t cleared. When acceleration is on there’s only one DNS lookup needed – the one to Amazon’s server. When acceleration is off Silk has to do a DNS lookup for every unique domain – an average of 13 domains per page. Having all of those DNS lookups cached gives an unfair advantage to the “acceleration off” page load times.

I’m still optimistic about the performance gains we’ll see as Silk’s split architecture matures, but for the remainder of this comparison we’ll use Silk with acceleration off since that performed best.

Silk vs iPad

I had both an iPad 1 and iPad 2 at my disposal so included both in the study. The iPad 1 was the slowest across all 11 URLs so I restricted the comparison to Silk (accel off) and iPad 2.

The results are mixed with iPad 2 being faster for most but not all URLs. The iPad 2 is fastest in 7 URLs. Silk is fastest in 3 URLs. One URL (eBay) is apples and oranges since Silk gets a mobile version of the site (18 resources compared to 64 resources for the desktop version).

Silk vs Galaxy

Comparing the Galaxy 7.0 to any other tablet is not fair since Galaxy 7.0 receives a lighter mobile version in 6 of 11 URLs. The Galaxy 7.0 has the slowest page load time in 3 of the 4 URLs where it, Galaxy 10.1, and Silk all receive the desktop version. Since it’s slower head-to-head and has mobile versions in the other URLs, I’ll focus on comparing Silk to the Galaxy 10.1.

Silk has the fastest page load time in 7 URLs. The Galaxy 10.1 is faster in 3 URLs. One URL is mixed as Silk gets a mobile version (18 resources) while the Galaxy 10.1 gets a desktop version (64 resources).

 Takeaways

These results show that, as strange as it might sound, Silk appears to be faster when acceleration is turned off. Am I going to turn off acceleration on my Kindle Fire? No. I don’t want to miss out on the next wave of performance optimizations in Silk. The browser is sound. It holds its own compared to other tablet browsers. Once the acceleration gets sorted out I expect it’ll do even better.

More importantly, it’s nice to have some real data and to have Loadtimer to help with future comparisons. Doing these comparisons to see which browser/tablet/phone is fastest makes for entertaining reading and heated competition. But all of us should expect more scientific rigor in the reviews we read, and push authors and ourselves to build and use better tools for measuring performance. I hope Loadtimer is useful. Loadtimer plus pcapperf and the Mobile Perf bookmarklet are the start of a mobile performance toolkit. Between the three of them I’m able to do most of what I need for analyzing mobile performance. It’s still a little clunky, but just as it happened in the desktop world we’ll see better tools with increasingly powerful features across more platforms as the industry matures. It’s still early days.

 

11 Comments

Loadtimer: a mobile test harness

December 1, 2011 1:35 am | 14 Comments

Measuring mobile performance is hard

When Amazon announced their Silk browser I got excited reading about the “split architecture”. I’m not big on ereaders but I pre-ordered my Kindle Fire that day. It arrived a week or two ago. I’ve been playing with it trying to find a scientific way to measure page load times for various websites. It’s not easy.

  • Since it’s a new browser and runs on a tablet we don’t have plugins like Firebug.
  • It doesn’t (yet) support the Navigation Timing spec, so even though I can inspect pages using Firebug Lite (via the Mobile Perf bookmarklet) and Weinre (I haven’t tried it but I assume it works), there’s no page load time value to extract.
  • Connecting my Fire to a wifi hotspot on my laptop running tcpdump (the technique evangelized by pcapperf) doesn’t work in accelerated mode because Silk uses SPDY over SSL. This technique works when acceleration is turned off, but I want to see the performance optimizations.

While I was poking at this problem a bunch of Kindle Fire reviews came out. Most of them talked about the performance of Silk, but I was disappointed by the lack of scientific rigor in the testing. Instead of data there were subjective statements like “the iPad took about half as long [compared to Silk]” and “the Fire routinely got beat in rendering pages but often not by much”. Most of the articles did not include a description of the test procedures. I contacted one of the authors who confided that they used a stopwatch to measure page load times.

If we’re going to critique Silk and compare its performance to other browsers we need reproducible, unbiased techniques for testing performance. Using a stopwatch or loading pages side-by-side and doing a visual comparison to determine which is faster are not reliable methods for measuring performance. We need better tools.

Introducing Loadtimer

Anyone doing mobile web development knows that dev tools for mobile are lacking. Firebug came out in 2006. We’re getting close to having that kind of functionality in mobile browsers using remote debuggers, but it’s pretty safe to say the state of mobile dev tools is 3-5 years behind desktop tools. It might not be sexy, but there’s a lot to be gained from taking tools and techniques that worked on the desktop and moving them to mobile.

In that vein I’ve been working the last few days to build an iframe-based test harness similar to one I built back in 2003. I call it Loadtimer. (I was shocked to see this domain was available – that’s a first.) Here’s a screenshot:

The way it works is straightforward:

  • It’s preloaded with a list of popular URLs. The list of URLs can be modified.
  • The URLs are loaded one-at-a-time into the iframe lower in the page.
  • The iframe’s onload time is measured and displayed on the right next to each URL.
  • If you check “record load times” the page load time is beaconed to the specified URL. The beacon URL defaults to point to loadtimer.org, but you can modify it if, for example, you’re testing some private pages and want the results to go to your own server.
  • You can’t test websites that have “framebusting” code that prevents them from being loaded in an iframe, such as Google, YouTube, Twitter, and NYTimes.

There are some subtle optimizations worth noting:

  • You should clear the cache between each run (unless you explicitly want to test the primed cache experience). There’s no way for the test harness to clear the cache, but it does have a check that helps remind you to clear the cache. (It loads a script that is known to take 3 seconds to load – if it takes less than 3 seconds it means the cache wasn’t cleared.)
  • It’s possible that URL 1′s unload time could make URL 2′s onload time be longer than it actually should be. To avoid this about:blank is loaded between each URL.
  • The order of the preset URLs is randomized to mitigate biases across URLs, for example, where URL 1 loads resources used by URL 2.

Two biases that aren’t addressed by Loadtimer:

  • DNS resolutions aren’t cleared. I don’t think there’s a way to do this on mobile devices short of power cycling. This could be a significant issue when comparing Silk with acceleration on and off. When acceleration is on there’s only one DNS lookup, whereas when acceleration is off there’s a DNS lookup for each hostname in the page (13 domains per page on average). Having the DNS resolutions cached gives an advantage to acceleration being off.
  • Favicons aren’t loaded for websites in iframes. This probably has a negligible impact on page load times.

 Have at it

The nice thing about the Loadtimer test harness is that it’s web-based – nothing to install. This ensures it’ll work on all mobile phones and tablets that support JavaScript. The code is open source. There’s a forum for questions and discussions.

There’s also a results page. If you select the “record load times” checkbox you’ll be helping out by contributing to the crowdsourced data that’s being gathered. Getting back to what started all of this, I’ve also been using Loadtimer the last few days to compare the performance of Silk to other tablets. Those results are the topic of my next blog post – see you there.

 

14 Comments

frontend SPOF survey

October 13, 2011 9:10 am | 3 Comments

Pat Meenan had a great blog post yesterday, Testing for Frontend SPOF. “SPOF” means single point of failure. I coined the term frontend SPOF to describe the all-too-likely situation where the HTML document returns successfully, but some other resource (a stylesheet, script, or font file) blocks the entire website from loading. This typically manifests itself as a blank white screen that the user stares out for 20 seconds or longer.

Frontend SPOF happens most frequently with third party content. If the HTML document returns successfully, then all the resources from the main website are likely to return successfully, as well. Third party content, however, isn’t controlled by the main website and thus could be suffering an outage or overload while the main website is working fine. As a result, the uptime of a website is no greater than the availability of the third party resources it uses that are in a position to cause frontend SPOF.

In my blog post of the same name I describe how Frontend SPOF happens and ways to avoid it, but I don’t provide a way for website owners to determine which third party resources may cause frontend SPOF. This is where Pat comes in. He’s created a public blackhole server: blackhole.webpagetest.org with the static IP address 72.66.115.13. Pointing your third party resources to this blackhole and reloading the page tells you if those resources cause frontend SPOF. Since Pat is the creator of WebPagetest.org, he has integrated this into the scripting capabilities of that tool so website owners can load their website and determine if any third party resources cause frontend SPOF.

/etc/hosts

I took a different approach outlined by Pat: I added the following lines to my /etc/hosts file (your location may vary) mapping these third party hostnames to point to the blackhole server:

72.66.115.13 apis.google.com
72.66.115.13 www.google-analytics.com
72.66.115.13 connect.facebook.net
72.66.115.13 platform.twitter.com
72.66.115.13 s7.addthis.com
72.66.115.13 l.addthiscdn.com
72.66.115.13 cf.addthis.com
72.66.115.13 api-public.addthis.com
72.66.115.13 widget.quantcast.com
72.66.115.13 ak.quantcast.com
72.66.115.13 assets.omniture.com
72.66.115.13 www.omniture.com
72.66.115.13 scripts.omniture.com
72.66.115.13 b.voicefive.com
72.66.115.13 ar.voicefive.com
72.66.115.13 c.statcounter.com
72.66.115.13 www.statcounter.com
72.66.115.13 www-beta.statcounter.com
72.66.115.13 js.revsci.net

After restarting my browser all requests to these hostnames will timeout. Pat’s blog post mentions 20 seconds for a timeout. He was running on Windows. I’m running on my Macbook where the timeout is 75 seconds! Any website that references third party content on these hostnames in a way that produces frontend SPOF will be blank for 75 seconds – an easy failure to spot.

survey says

THE GOOD: At this point I started loading the top 100 US websites. I was pleasantly surprised. None of the top 20 websites suffered from frontend SPOF. There were several that loaded third party content from these hostnames, but they had safeguarded themselves:

  • MSN makes a request to ar.voicefive.com, but does it asynchronously using a document.write technique.
  • AOL references platform.twitter.com, but puts the SCRIPT tag at the very bottom of the BODY so page rendering isn’t blocked.
  • IMDB uses the async version of Google Analytics, and puts the platform.twitter.com widget in an iframe.
  • LiveJournal goes above and beyond by wrapping the Google +1 and Facebook widgets in a homegrown async script loader.

THE BAD: Going through the top 100 I found five websites that had frontend SPOF:

  1. CNET loads http://platform.twitter.com/widgets.js in the HEAD as a blocking script.
  2. StumbleUpon loads http://connect.facebook.net/en_US/all.js at the top of BODY as a blocking script.
  3. NFL loads http://connect.facebook.net/en_US/all.js in the HEAD as a blocking script.
  4. Hulu, incredibly, loads Google Analytics in the HEAD as a blocking script. Please use the async snippet!
  5. Expedia loads http://connect.facebook.net/en_US/all.js as a blocking script in the middle of the page, so the right half of the page is blocked from rendering.

These results, although better than I expected, are still alarming. Although I only found five websites with frontend SPOF, that’s 5% of the overall sample. The percentage will likely grow as the sample size grows because best practices are more widely adopted by the top sites. Also, my list of third party hostnames is a small subset of all widgets and analytics available on the Web. And remember, I didn’t even look at ads.

Is it really worth blocking your site’s entire page for a widget button or analytics beacon – especially when workarounds exist? If you’re one of the five sites that faltered above, do yourself and your users a favor and find a way to avoid frontend SPOF. And if you’re outside the top 100, test your site using Pat’s blackhole server by editing /etc/hosts or following Pat’s instructions for testing frontend SPOF on WebPagetest.org.

3 Comments

App cache & localStorage survey

September 26, 2011 9:51 pm | 24 Comments

In preparation for my talk at HTML5 Dev Conf I surveyed the Alexa US Top 10 websites to see which ones use app cache and localStorage. I mostly focus on mobile these days so it’s natural to think I ran these tests from a mobile browser, which I did. But I also tested with a desktop browser.

Some people might ask, Why a desktop browser?

To which I would reply, Why not a desktop browser?

I blogged previously about how Google and Bing use localStorage on mobile devices. It’s a powerful performance technique. They break up their JavaScript and CSS into smaller blocks and save them in localStorage. Simultaneously they set a cookie so that the server knows not to send that payload on subsequent searches, a savings of 150-170 kB before gzipping.

In the comments on that post Simon asked:

I’m curious why the techniques with LocalStorage are used for the mobile versions of the search sites but not for the standard desktop versions – I would think that this technique would work well [on] all clients, no?

I agree – this and other HTML5 web storage techniques make sense for the desktop, too. There are some reasons why we’re seeing these used first on mobile:

  • Mobile latencies are higher and connection speeds are lower, so clientside caching is more important on mobile.
  • Mobile disk cache sizes are smaller than desktop sizes, so a better alternative is needed for mobile.
  • There are still desktop browsers with significant market share that are missing many HTML5 capabilities, whereas mobile browsers have more support for HTML5.

Even though the motivation for app cache and localStorage are stronger for mobile, they should also be used when the website is accessed from a desktop browser. I did a quick survey to see which of the top 10 websites were using app cache or localStorage on either mobile or desktop. Here are the results:

Website Mobile Desktop
app cache localStorage app cache localStorage
Google Search no yes no yes [1]
Bing no yes no no
Facebook no yes no no
Yahoo! Front Page no yes [2] no no
YouTube no yes no no
Amazon no no no yes [3]
Twitter yes yes no yes
LinkedIn no no no no
eBay no no no no
MSN.com no no no no
[1] Google Search on the desktop uses sessionStorage, not localStorage.
[2] Yahoo! Front Page only stores two numbers.
[3] Amazon on the desktop only stores a ~30 character string.

Ignoring the small uses of localStorage, 5 of these top 10 websites use localStorage on mobile, but only 2 out of 10 use localStorage (or sessionStorage) on desktop. None of them use app cache, either on mobile or desktop except for Twitter on mobile. I’m surprised no one is using app cache. It’s not appropriate for all applications, such as search, but I would enjoy catching up on Twitter, Facebook, and LinkedIn on the plane – potentially from my laptop in addition to my mobile device. App cache not only brings this offline capability, but provides better caching than the browser’s disk cache.

I’ll repeat this survey in a few months to track the progress. I expect we’ll see the use of localStorage and app cache increase, and for desktop to catch up to mobile.

 

24 Comments

(lack of) Caching for iPhone Home Screen Apps

June 28, 2011 10:14 pm | 10 Comments

Yesterday’s post, Unexpected Reloads in WebKit, revealed an interesting behavior that affects caching in Safari:

When you load the same URL back-to-back in Safari, the second load is treated the same as hitting Reload.

This is bad for performance because the browser issues a Conditional GET request for each resource instead of using the cached resource.

It’s important to be aware of this behavior when testing the primed cache experience in Safari, so web performance engineers should take note. However, in the real world it’s unlikely this behavior has much of an impact on desktop users. Here’s the table from yesterday’s post that shows how this Reload-like behavior is triggered when re-requesting a page:

way of loading URL again like Reload?
hit RETURN in location field yes
delete URL and type it again yes
launch same URL via bookmark yes
click link to same URL yes
go to another URL then type 1st URL again no
modify querystring no
enter URL in a new tab no
Table 1. Triggering reload behavior in Safari

It’s possible that real world users might type the same URL or open the same bookmark two times in a row in the same tab, but it probably doesn’t happen that often. So what’s the big deal?

So what’s the big deal?

Whenever I see strange performance behavior I think about where that behavior might have a significant impact. Is there any place where this back-to-back Safari Reload behavior could have a significant impact? A comment from yesterday’s post hints at the answer:

Why is this article named “Unexpected Reloads in WebKit”?

Chrome is based on Webkit and doesn’t has same issue. Perhaps it would be less confusing to name it “Unexpected Reloads in Safari”.

Other people gave me the same feedback on the backchannel – why did I say “WebKit” instead of “Safari”.

Here’s why: WebKit is used in a lot of browsers. Whenever I see a bug (or a feature) in one popular WebKit-based browser I wonder if it exists in others. The main WebKit-based browsers I focus on are Chrome, Safari, Android, and iPhone. As soon as I noticed this behavior in Safari my next step was to conduct the same tests in Chrome, Android, and iPhone. As the commenter noted, this unexpected Reload behavior does not happen in Chrome. And it does not happen on Android (tested on my Nexus S). But it does happen on iPhone.

Update June 29: In a comment on yesterday’s post, Libo Song correctly pointed out that this back-to-back Reload-like behavior does happen on Android. He tested on Nexus One and I confirmed on Nexus S. Although Android does exhibit the Reload-like behavior when the same URL is entered back-to-back in the same tab, this doesn’t happen very often. The more important issue that is the focus of this post is how this Reload-like behavior slows down the launching of home screen apps. In this regard Android does not exhibit the Reload-like behavior when home screen apps are launched. Here’s a HAR file showing Amazon being loaded twice from the home screen with fast.stevesouders.com in between. The second launch doesn’t generate any HTTP requests.

While it’s true that iPhone users are unlikely to manually launch the same URL twice-in-a-row in the same tab, there is a situation when this happens automatically: when launching home screen apps.

Home screen apps are a powerful feature on iPhone and Android that allow users to save URLs to the home screen and launch them similar to native apps. Unfortunately, launching home screen apps on the iPhone triggers something similar to the Reload behavior we see in Safari – where resources aren’t read from cache and instead generate extra HTTP requests. Let’s take a look at a few examples of home screen apps, starting with simple to more complex.

Amazon: simple URL

Typing http://www.amazon.com/ into the iPhone browser displays a version of Amazon’s front page that is customized for mobile – there’s less content, the images are smaller, etc. However, there is not a prompt to save the URL to the home screen. We can do that anyway using the arrow function key at the bottom of the screen and selecting “Add to Home Screen”.

If you’ve used home screen apps you might have noticed that they always open in the same browser tab. Let’s run a little test to confirm this:

  1. Click the Amazon home screen icon. This opens Amazon in mobile Safari.
  2. Open another tab by clicking the “pages” function key and opening a “New Page”. Enter some non-Amazon URL in this new tab, for example http://fast.stevesouders.com/ (a very lightweight page I use for testing). At this point we have at least two tabs, one with Amazon and one with fast.stevesouders.com, and we’re looking at the fast.stevesouders.com tab.
  3. Go back to the home screen and click the Amazon icon again.
  4. Note that you’re taken back into mobile Safari to the first tab that contains Amazon.

We just opened the exact same URL back-to-back in the same tab. We didn’t do it intentionally – that’s the default behavior for iPhone home screen apps. Here’s a waterfall chart for this test. (You can view an interactive waterfall by loading the HAR file in pcapperf.)

The home screen app URL is http://www.amazon.com/gp/aw/h.html/187-9233150-9797455. The first time the home screen app is launched starts at the top with 187-9233150-9797455. Since the cache was empty all the subsequent resources have 200 responses. There are some 404s for icons followed by the request for fast.stevesouders.com.

The second launch of the Amazon home screen app (187-9233150-9797455 below fast.stevesouders.com) is where it gets interesting. When the Amazon home screen app is launched the second time, a Conditional GET request is made for all of the resources even though these resources are in the cache with a future expiration date.

All of the resources that are re-requested have an expiration date more than 10 years in the future. For example, the response headers for title_gradient._V233984477_.png are:

content-length: 291
expires: Tue, 06 May 2031 21:44:21 GMT
last-modified: Mon, 10 Aug 2009 11:50:45 GMT
cache-control: max-age=626560472
date: Wed, 29 Jun 2011 01:09:49 GMT
content-type: image/png

We know it was cached because when the Amazon home screen app is launched the second time the Conditional GET request for title_gradient._V233984477_.png has an If-Modified-Since header that contains the last-modified date in the initial response:

if-modified-since: Mon, 10 Aug 2009 11:50:45 GMT

It appears that we’ve stumbled into the Reload-like behavior we saw in Safari on the desktop. Further evidence of this is if you launch the home screen app, then type a new URL over the Amazon URL, and launch the home screen app again the resources are read from cache instead of generating numerous Conditional GET requests. (Load this HAR file in pcapperf to see for yourself.)

Untappd: full screen app

Amazon was a simple home screen app – really just a bookmark on the home screen. Developers can do more with home screen apps to make them launch and look like native apps. As described in Apple’s How-To’s for Safari on iPhone, various parts of the home screen app user experience are customizable including the home screen icon, viewport, and zooming and scaling. Developers can also have their home screen app launch in “full screen mode” by hiding the Safari UI components, including the status bar and location bar. In this situation, every time the home screen app is launched it uses the same “tab” with the exact same URL – thus triggering the Reload behavior.

Let’s have a look at Untappd on the iPhone. The first time you navigate to http://untappd.com/ in iPhone’s browser you get a suggestion to add the web app to the home screen:

After which you’ll have a customized Untappd home screen icon:

Now let’s investigate how caching works for this home screen app. We start by clearing the cache then launching the home screen app. You’ll notice there is no location bar or other Safari controls. Then we go back to the home screen and launch the Untappd home screen app again. The waterfall chart is shown below. (Here’s the HAR file.)

The first time the Untappd home screen app is launched it loads seven HTTP requests. Three of these resources are cacheable: jquery.min.js (1 year), gears_init.js (1 hour), and ga.js (1 day). Loader.gif and ajax-loader.png don’t have a future expiration date, but they do have Last-Modified and ETag response headers that could be used in a Conditional GET request.

But we see that the second time Untappd is launched from the home screen, all of the resources are re-requested. To make matters worse, none of these are Conditional GET requests, so a 200 status code is returned with the full response body.

The punchline

It’s unfortunate that home screen apps suffer from this bad caching behavior on the iPhone. Thankfully, there is a workaround: application cache. I ran similar tests on other home screen apps that use application cache. The resources listed in the CACHE: section of  the manifest file were used on the iPhone without generating Conditional GET requests.

I feel bad about recommending the use of application cache. This is an issue with the browser cache on mobile Safari (and to a lesser degree on desktop Safari) that should be fixed. It’s a significant amount of work for developers to adopt application cache. The plus side is that doing so achieves the ability to work offline.

After this lengthy analysis and numerous waterfalls, here’s the punchline in a nutshell:

Home screen apps on iPhone are slower because resources are re-requested even though they should be read from cache. Use application cache to avoid this performance problem.

Update Oct 12: Home screen apps in iOS 5 do not exhibit this problem. Blaze.io reports that home screen apps use caching as expected. They also have faster JS likely do to the integration of the Nitro engine.

10 Comments

Unexpected Reloads in WebKit

June 27, 2011 4:47 pm | 9 Comments

People who work on web performance often need to load the same URL over and over again. Furthermore, they need to do this while simulating a real user’s empty cache experience and primed cache experience. When I want to analyze the empty cache experience the flow is simple: go to about:blank, clear the browser cache, enter the URL, and hit RETURN.

But what’s the right way to fetch a page repeatedly when analyzing the primed cache experience?

The main goal when testing the primed cache version of a page is to see which resources are read from cache. The goal for better performance is to cache as many responses as possible thus reducing the number of requests made when the cache is primed. If a resource has an expiration date in the future, the browser uses the cached version and doesn’t have to make an HTTP request resulting in a faster page. If a resource is expired (the expiration date is in the past) the browser issues a Conditional GET request using the If-Modified-Since and If-None-Match request headers. If the resource hasn’t changed then the server returns a simple 304 status code with no body. This is faster (because there’s no response body) but still takes time to do the HTTP request. (See my article on ETags for examples of IMS and INM.)

One way to re-request a page is to hit the Reload button, but this doesn’t give an accurate portrayal of the typical primed cache user experience. Hitting Reload causes the browser to always make an IMS/INM request for resources in the page, even for cached resources that have an expiration date in the future. Normally these resources would be used without generating an HTTP request. Although users do occasionally hit the Reload button it’s more likely that they’ll navigate to a page via a link or the location field, both of which avoid the time consuming Conditional GET requests generated when hitting Reload.

The technique I adopted years ago for re-requesting a page when testing the primed cache is to click in the location field and hit RETURN. That’s a fine approach in IE, Firefox, Chrome, and Opera, but not in Safari. Let’s investigate why.

hitting RETURN in the location field

I’m using Untappd as an example. Untappd has 68 requests when loaded on the desktop. Figure 1 shows the waterfall chart for the first 31 requests when loaded in Firefox 4 with an empty cache:

Figure 1. untappd.com – Firefox 4 – empty cache

Most of the resources shown in Figure 1 have an expiration date in the future and therefore won’t generate an HTTP request if the user has a primed cache. To test that I click in the location field and hit RETURN. The resulting waterfall chart is shown in Figure 2. Sure enough the number of HTTP requests drops from 68 to 4!

Figure 2. untappd.com – Firefox 4 – primed cache

If you repeat this experiment in Chrome, Firefox, Internet Explorer, and Opera you’ll get similar results – empty cache generates 68 requests, primed cache generates 4 requests. However, the result is very different in Safari 5. It’s important to understand why.

Safari is different

This test shows that Untappd has done a good job of optimizing the primed cache experience – the number of HTTP requests made by the browser drops from 68 to 4. Running the same test in Safari 5 produces different results. Clearing the cache and loading untappd.com in Safari 5 loads 68 HTTP requests – just as before. To test the primed cache experience we click in the location field and hit RETURN. Instead of only 4 requests there are 68 HTTP requests.

Why are there 64 more HTTP requests in Safari 5 for the primed cache test? Looking at the HTTP request headers we see that these are all Conditional GET requests. Let’s use http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js as the example (it’s the 8th request in Figure 1). In the empty cache scenario the HTTP request headers are:

Accept: */*
Cache-Control: max-age=0
Referer: http://untappd.com/
User-Agent: Mozilla/5.0 (Macintosh; [snip...] Safari/533.20.27

The HTTP status code returned for that empty cache request is 200 OK.

In the primed cache test when we hit RETURN in the location field we see that the request for jquery.min.js contains an extra header:

Accept: */*
Cache-Control: max-age=0
If-Modified-Since: Mon, 15 Feb 2010 23:30:12 GMT
Referer: http://untappd.com/
User-Agent: Mozilla/5.0 (Macintosh; [snip...] Safari/533.20.27

The header that’s added in the primed cache test is If-Modified-Since. This is a Conditional GET request. The HTTP status code that’s returned is 304 Not Modified. Even though all I did was hit RETURN in the location field, Safari treated that like hitting the Reload button.

unexpected “reload” in Webkit

Unlike other browsers, Safari 5 treats hitting RETURN in the location field the same as clicking the Reload button. When else does this happen? Assuming you’ve loaded a URL in Safari and are looking at that page, this table lists various ways to load that URL again. For each technique I show whether loading the URL this way generates extra Conditional GET requests similar to clicking Reload.

way of loading URL again like Reload?
hit RETURN in location field yes
delete URL and type it again yes
launch same URL via bookmark yes
click link to same URL yes
go to another URL then type 1st URL again no
modify querystring no
enter URL in a new tab no
Table 1. Triggering reload behavior in Safari

This black box testing indicates that whenever the same URL is loaded back-to-back in the same tab, Safari 5 treats it as a Reload. I was describing this behavior to Jay Freeman (saurik) at Foo Camp. He pointed me to this code from WebCore:

else if (sameURL)
   // Example of this case are sites that reload the same URL with a different cookie
   // driving the generated content, or a master frame with links that drive a target
   // frame, where the user has clicked on the same link repeatedly.
   m_loadType = FrameLoadTypeSame;

Searching in that same file for FrameLoadTypeSame we find this code:

case FrameLoadTypeReload:
case FrameLoadTypeReloadFromOrigin:
case FrameLoadTypeSame:
case FrameLoadTypeReplace:
   history()->updateForReload();
   m_client->transitionToCommittedForNewPage();
   break;

This code doesn’t account for the behavior, but it does show that FrameLoadTypeSame and FrameLoadTypeReload are treated as similar cases in this context, and perhaps that’s why IMS/INM requests are generated.

One important takeaway from this is: don’t hit RETURN in the location field to test primed cache experience in Safari. Instead, go to a different URL and then type the test URL in the location field, or open a new tab and type the URL.

There’s a second more important takeaway from this. I’ll cover that in tomorrow’s post. If you know the answer, please don’t spoil it. Oh what the heck – if you think you know the answer go ahead and add a comment.

9 Comments

HTTP Archive: servers and most 404s

May 9, 2011 5:11 am | 5 Comments

I launched the HTTP Archive about a month ago. The reaction has been positive including supportive tweets from Tim O’Reilly, Werner Vogels, Robert Scoble, and John Resig. I’m also excited about the number of people that have already started contributing to the project. Two new stats charts are available thanks to patches from open source contributors.

James Byers contributed the patch for generating the Most Common Servers pie chart. This chart is similar to BuiltWith’s Web Server chart. BuiltWith shows a higher presence of IIS than shown here. Keep in mind the sample sets are different – the HTTP Archive hits the world’s top ~17K URLs while BuiltWith is covering 1M URLs.

The other new chart comes from Carson McDonald. It shows pages with the most 404s. Definitely a list you don’t want to find your website on.

l’ve added some other features I’ll blog about tomorrow and am planning a bigger announcement later this week, so stay tuned for some more HTTP Archive updates.

5 Comments

HTTP Archive: max-age

April 18, 2011 9:40 pm | 10 Comments

There’s a long list of interesting stats to be added to the HTTP Archive. I’m planning on knocking those off at about one a week. (If someone wants to help that’d be great – contact me. Familiarity with MySQL and Google Charts API is a plus.)

Last week I added an interesting stat looking at the cache lifetime being specified for resources – specifically the value set in the Cache-Control: max-age response header. As a reminder, the HTTP Archive is currently analyzing the top ~17K websites worldwide. Across those websites a total of ~1.4M resources are requested. The chart below shows the distribution of max-age values across all those resources.

56% of the resources don’t have a max-age value and 3% have a zero or negative value. That means only 41% of resources are cacheable. In more concrete terms, the average number of resources downloaded per page is 81. 33 of those are cacheable, but the other 48 will likely generate an HTTP request on every page view. Ouch! That’s going to slow things down. Only 24% of resources are cacheable for more than a day. Adding caching headers is an obvious performance win that needs wider adoption.

10 Comments

Storager case study: Bing, Google

March 28, 2011 9:26 pm | 23 Comments

Storager

Last week I posted my mobile comparison of 11 top sites. One benefit of analyzing top websites is finding new best practices. In that survey I found that the mobile version of Bing used localStorage to reduce the size of their HTML document from ~200 kB to ~30 kB. This is a good best practice in general and makes even more sense on mobile devices where latencies are higher, caches are smaller, and localStorage is widely supported.

I wanted to further explore Bing’s use of localStorage for better performance. One impediment is that there’s no visibility into localStorage on a mobile device. So I created a new bookmarklet, Storager, and added it to the Mobile Perf uber bookmarklet. (In other words, just install Mobile Perf – it bundles Storager and other mobile bookmarklets.)

Storager lets you view, edit, clear, and save localStorage for any web page on any browser – including mobile. Viewing localStorage on a 320×480 screen isn’t ideal, so I did the obvious next step and integrated Storager with Jdrop. With these pieces in place I’m ready to analyze how Bing uses localStorage.

Bing localStorage

My investigation begins by loading Bing on my mobile device – after the usual redirects I end up at the URL http://m.bing.com/?mid=10006. Opening Storager from the Mobile Perf bookmarklet I see that localStorage has ~10 entries. Since I’m not sure when these were written to localStorage I clear localStorage (using Storager) and hit reload. Opening Storager again I see the same ~10 entries and save those to Jdrop. I show the truncated entries below. I made the results public so you can also view the Storager results in Jdrop.

BGINFO: {"PortraitLink":"http://www.bing.com/fd/hpk2/Legzira_EN-US262...
CApp.Home.FD66E1A3: #ContentBody{position:relative;overflow:hidden;height:100%;-w...
CUX.Keyframes.B8625FE...: @-webkit-keyframes scaleout{from{-webkit-transform:scale3d(1,...
CUX.Site.18BDD936: *{margin:0;padding:0}table{border-collapse:separate;border-sp...
CUX.SiteLowRes.C8A1DA...: .blogoN{background-image:url(data:image/png;base64,iVBORw0KGg...
JApp.Home.DE384EBF: (function(){function a(){Type.registerNamespace("SS");SS.Home...
JUX.Compat.0907AAD4: function $(a){return document.getElementById(a)}var FireEvent...
JUX.FrameworkCore.A39...: (function(){function a(){Type.registerNamespace("BM");AjaxSta...
JUX.MsCorlib.172D90C3: window.ss={version:"0.6.1.0",isUndefined:function(a){return a...
JUX.PublicJson.540180...: if(!this.JSON)this.JSON={};(function(){function c(a){return a...
JUX.UXBaseControls.25...: (function(){function a(){Type.registerNamespace("UXControls")...
RMSM.Keys: CUX.Site.18BDD936~CUX.Keyframes.B8625FEE~CApp.Home.FD66E1A3~C...

These entries are written to localStorage as part of downloading the Bing search page. These entries add up to ~170 kB in size (uncompressed). This would explain the large size of the Bing HTML document on mobile. We can verify that these keys are downloaded via the HTML document by searching for a unique string from the data such as “FD66E1A3″. We find this string in the Bing document source (saved in Jdrop) as the id of a STYLE block:

<style data-rms="done" id="CApp.Home.FD66E1A3" rel="stylesheet" type="text/css">
#ContentBody{position:relative;overflow:hidden;height:100%;-webkit-tap-highlight-color:...

Notice how the content of this STYLE block matches the data in localStorage. The other localStorage entries also correspond to SCRIPT and STYLE blocks in the initial HTML document. Bing writes these blocks to localStorage and then on subsequent page views reads them back and inserts them into the document resulting in a much smaller HTML document download size. The Bing server knows which blocks are in the client’s localStorage via a cookie, where the cookie is comprised of the localStorage keys delimited by “~”:

RMSM=JApp.Home.DE384EBF~JUX.UXBaseControls.252CB7BF~JUX.FrameworkCore.A39F6425~
JUX.PublicJson.540180A4~JUX.Compat.0907AAD4~JUX.MsCorlib.172D90C3~CUX.SiteLowRes.C8A1DA4E~
CApp.Home.FD66E1A3~CUX.Keyframes.B8625FEE~CUX.Site.18BDD936~;

Just to be clear, everything above happens during the loading of the blank Bing search page. Once a query is issued the search results page downloads more keys (~95 kB additional data) and expands the cookie with the new key names.

Google localStorage

Another surprise from last week’s survey was that the mobile version of Google Search had 68 images in the results HTML document as data: URIs, compared to only 10 for desktop and iPad. Mobile browsers open fewer TCP connections and these connections are typically slower compared to desktop, so reducing the number of HTTP requests is important.

The additional size from inlining data: URIs doesn’t account for the large size of the Google Search results page, so perhaps localStorage is being seeded here, too. Using Storager we see over 130 entries in localStorage after a search for flowers. Here’s a sample. (As before, the key names and values may be truncated.)

 mres.-8Y5Dw_nSfQztyYx: <style>a{color:#11c}a:visited{color:#551a8b}body{margin:0;pad...
 mres.-Kx7q38gfNkQMtpx: <script> //<![CDATA[ var Zn={},bo=function(a,b){b&&Zn[b]||(ne...
 mres.0kH3gDiUpLA5DKWN: <style>.zl9fhd{padding:5px 0 0}.sc59bg{clear:both}.pyp56b{tex...
 mres.0thHLIQNAKnhcwR4: <style>.fdwkxt{width:49px;height:9px;background:url("data:ima...
 mres.36ZFOahhhEK4t3WE: <script> //<![CDATA[ var kk,U,lk;(function(){var a={};U=funct...
 mres.3lEpts5kTxnI2I5S: <script> //<![CDATA[ var Ec,Fc,Gc=function(a){this.Jl=a},Hc="...
 mres.4fbdvu9mdAaBINjE: <script> //<![CDATA[ u("_clOnSbt",function(){var a=document.g...
 mres.5QIb-AahnDgEGlYP: <script> //<![CDATA[ var cb=function(a){this.Cc=a},db=/\s*;\s...
 mres:time.-8Y5Dw_nSfQ...: 1301368541872
 mres:time.-Kx7q38gfNk...: 1301368542755
 mres:time.0kH3gDiUpLA...: 1301368542257
 mres:time.0thHLIQNAKn...: 1301368542223
 mres:time.36ZFOahhhEK...: 1301368542635
 mres:time.3lEpts5kTxn...: 1301368542579
 mres:time.4fbdvu9mdAa...: 1301368542720
 mres:time.5QIb-AahnDg...: 1301368542856

Searching the search results docsource for a unique key such as “8Y5D” we find:

<style id="r:-8Y5Dw_nSfQztyYx" type="text/css">
a{color:#11c}a:visited{color:#551a8b}body{margin:0;padding:0}...

Again we see that multiple SCRIPT and STYLE blocks are being saved to localStorage totaling 154 kB. On subsequent searches the HTML document size drops from the initial size of 220 kB uncompressed (74 kB compressed) to 67 kB uncompressed (16 kB compressed). In addition to the key names being saved in a cookie, it appears that an epoch time (in milliseconds) is associated with each key.

Conclusion

Bing and Google Search make extensive use of localStorage for stashing SCRIPT and STYLE blocks that are used on subsequent page views. None of the other top sites from my previous post use localStorage in this way. Are Bing and Google Search onto something? Yes, definitely. As I pointed out in my previous post, this is another example of a performance best practice that is used on a top mobile site but is not in the recommendations from Page Speed or YSlow. Many of the performance best practices that I’ve evangelized over the last six years for desktop apply to mobile, but I believe there are specific mobile best practices that we’re just beginning to identify. I’ve started using “High Performance Mobile” as the title of future presentations. Another book? hmmm….

23 Comments