High Performance Web Sites blog

I co-hosted a great web performance meetup yesterday called SpeedGeeks LA. The idea started when Phil Dixon invited me down to Shopzilla to give a talk. (Phil and I have stayed in touch after his amazing session at Velocity 2009.) Phil suggested we invite some other performance-minded companies in the area, and then we started talking about expanding it to include more speakers. We ended up with over 3 hours of presentations and nearly 150 people.

We had a great mix of talks (I hope to post all slides soon):

• In Search of Speed – I kicked it off talking about the importance of progressive enhancement as an enabler of progressive rendering. (slides)
• You Get What You Measure – A reprise of Phil’s talk from Velocity that includes the oft-cited stats about a 7-12% increase in revenue and 50% drop in server load thanks to faster performance.
• A Holistic Approach to Building a High Performance Web Site – Paddy Hannon (Edmunds) – A no-nonsense approach to instilling performance into the development process.
• Latency Improvements for PicasaWeb – Gavin Doughtie (Google) – Great tips from a web guru about what makes PicasaWeb fast.
• Performance-as-a-service – Morten Begai (Heroku) – Performance built-in to their Rails hosting environment.
• MSFast – Jeremy Custenborder, Yadid Ramot (MySpace) – Demo of MSFast plus a preview of what looks to be an awesome Flash profiler. Watch for it!
• Trace Down JavaScript & AJAX Performance in IE – Andreas Grabner (dynaTrace) – Check out their new dynaTrace Ajax Edition tool.
• Application Architecture Determines Application Performance – Randy Stafford (Oracle) – A reminder that sometimes you need to focus on backend performance, too.

Oracle was kind enough to sponsor lunch for everyone and raffle off a Wii. How awesome is that?!

A huge shoutout to Phil and the folks at Shopzilla, especially Lauren Moser. They hosted everyone at their headquarters and even handed out swag bags. The space was setup perfectly, with room for folks to group into smaller conversations during breaks. Just a top notch operation on all fronts.

I’m calling this the inaugural event for SpeedGeeks. My hope is that LA is just the first of many cities where SpeedGeeks gather to share knowledge about make the web faster. I’d like to see SpeedGeeks expand in a similar way as how Ignite events take place worldwide. If you’d like to pull together such an event in your city, contact me and we can brainstorm. I can rally performance gurus in the area and, who knows, I might even be able to drop-in and help co-host.

SpeedGeeks of the world unite!

A few years ago, I set a goal to foster the creation of an Internet Performance Archive. The idea is similar to the Internet Archive. But whereas the Internet Archive’s Wayback Machine provides a history of the Web’s state of HTML, IPA would provide a history of the Web’s state of performance: total bytes downloaded, number of images|scripts|stylesheets, use of compression, etc. as well as results from performance analysis tools like YSlow and Page Speed. Early versions of this idea can be seen in Show Slow and Cesium.

I realized that a key ingredient to this idea was a way to capture the page load experience, basically, a way to save what you see in a packet sniffer. Wayback Machine archives HTML, but it doesn’t capture critical parts of the page load experience like the page’s resources (scripts, stylesheets, etc.) and their HTTP headers. It’s critical to capture this information so that the performance results can be viewed in the context of what was actually loaded and analyzed.

What’s needed is an industry standard for archiving HTTP information. The first step in establishing that industry standard took place today with the announcement of HttpWatch and Firebug joint support of the HTTP Archive format.

HttpWatch has long supported exporting HTTP information. That’s one of the many reasons why it’s the packet sniffer I use almost exclusively. Earlier this year, as part of the Firebug Working Group, I heard that Firebug wanted to add an export capability for Net Panel. I suggested that, rather than create yet another proprietary format, Firebug team up with HttpWatch to develop a common format, and drive that forward as a proposal for an industry standard. I introduced Simon Perkins (HttpWatch) and Jan “Honza” Odvarko (main Net Panel developer), then stepped back as they worked together to produce today’s announcement.

The HTTP Archive format (HAR for short – that was my contribution ;-) is in JSON format. You can see it in action in HttpWatch 6.2, released today. HAR has been available in Firebug for a month or so. You need Firebug 1.5 alpha v26 or later and Honza’s NetExport add-on (v0.7b5 or later).

Here’s what the end-to-end workflow looks like. After installing NetExport, the “Export” button is added to Firebug Net Panel. Selecting this, I can save the HTTP information for my Google flowers search to a file called “google-flowers.har”.

After saving the file, it’s automatically displayed in Honza’s HTTP Archive Viewer web page:

I can then open the file in HttpWatch:

I’m incredibly excited to see this milestone reached, thanks to the work of Honza and Simon. I encourage other vendors and projects to add support for HAR files. The benefits aren’t limited to performance analysis. Having a format for sharing HTTP data across tools and browsers is powerful for debugging and testing, as well. One more block added to the web performance foundation. Thank you HttpWatch and Firebug!

A contingent of web dev gurus are on their way to Europe the first week of November for some awesome conferences. If you haven’t signed up, check into it soon – seats are going fast.

The gurus I’ve been coordinating with include John Resig, Doug Crockford, Ben Galbraith, and Dion Almaer. But there’s more! Other speakers/friends who I’m excited to techdown with are Christian Heilmann, PPK, Nicole Sullivan, Richard Campbell, Kyle Simpson, and Tom Hughes-Croucher.

The conferences (in chronological order) are:

• Øredev (Nov 2-6, Malmö, Sweden) – Malmö is right across the channel from Copenhagen. I’ve never been to Denmark or Sweden, so am psyched to check this out (and bring back some legos).
• Fronteers (Nov 5-6, Amsterdam) – This is what started it all for me. I heard the reviews of Fronteers 2008 and swore I was going to go this year. I have a lot of respect for PPK’s work, and am honored he asked me to present. I visited Amsterdam a few times and loved it, but it’s been 15 years so I’m excited to get back. Unfortunately, Fronteers 2009 is already sold out! If you didn’t get a ticket, check out one of the other conferences.
• JSConf.eu (Nov 7-8, Berlin) – I heard great things about JSConf in DC and wanted to get to know this group. It’s great that it follows so closely after Fronteers. My ancestors came from Germany, so I look forward to visiting Germany again.

I’ll be presenting at all three conferences, giving away a few books, and sitting down with developers in these countries to discuss the tech challenges they face, and hopefully pick up some best practices, especially with regard to performance.

I am so psyched for this trip! My wife and kids are bummed they’re not coming, so I can’t show any excitement at home. (Luckily, my wife doesn’t read my blog – don’t tell her how excited I am!) All three of these conferences are going to be great. I hope to see you there!

Last week I was reading Ajaxian (my favorite blog) and saw the post about @font-face. I had been wondering for a few months about how font files impact web performance, so I followed the story back to Zoltan Hawryluk’s original post: @font-face in Depth. A great read on the aesthetics, mechanics, and compatibility issues of fonts, but not much about performance. I added a comment:

How does this impact performance? Do font file downloads happen in parallel with other resources? Are they downloaded at declaration (”@font-face {}”), when used in a style (”body { font-family: }”), or when a DOM element uses a style that uses the font? Do they follow the normal HTTP caching behavior? Can they be gzipped?

I started doing some research to answer these questions, but during that time there have been a number of great posts about @font-face performance issues:

• Paul Irish: Fighting the @font-face FOUT
• Stoyan Stefanov: Gzip your @font-face files
• Zoltan Hawryluk (again): More @font-face fun

This blog post summarizes Paul, Stoyan, and Zoltan’s findings plus some very important discoveries of my own.

FOUT: Flash of Unstyled Text

Paul refers to FOUT as the “flash of unstyled text”. This is a nod to the term FOUC coined by David Hyatt for “flash of unstyled content”.  FOUC occurs when stylesheets are loaded late in the page. (FOUC is the justification for Rule 5 – Put Stylesheets at the Top from High Performance Web Sites.)

Paul comments on two versions of FOUT:

• re-rendered text – Firefox renders text using a default font while the @font-face file is being downloaded. Once the font file is downloaded, Firefox re-renders the text using the correct font.
• invisible text – Most other browsers don’t draw the text at all until the font is downloaded.

I found an even worse FOUT issue:

• IE doesn’t render anything in the page until the font file is done downloading.

You can see this using my FOUT Test where the font file takes six seconds to download – in IE the entire page is white until six seconds pass, even though the HTML text and other parts of the page already arrived at the browser. (This is critical when considering outage scenarios – see the @font-face Outage section that follows.)

All of these FOUT situations are bad for the user experience. It’s jarring for the user to have parts of the page appearing or changing while they view it. And it’s not just the @font-face text that’s the problem – all browsers except IE will likely have to re-render other parts of the page if the default font and @font-face font are different sizes.

FOUT doesn’t impact web page performance in terms of absolute load times, but it does affect the user’s perception of page speed, and that’s what we’re really trying to optimize.

Blocking Other Downloads

When it comes to the question of whether font files block other downloads, the short answer is “no”. I created the Blocking Test to measure whether font files block other resources from being downloaded. The test page contains some text that uses a font downloaded via @font-face. This text is followed by an image, an iframe, a stylesheet, and an external script. Each of these four resources as well as the font file are configured to take two seconds to download. In all major browsers, the page loads in two seconds, meaning no files were blocked.

Note that font files are subject to the same connections-per-hostname limits as other resources. Run the Per Hostname Test to see the limits for your browser. In the Blocking Test, I sharded the resources across multiple domains to avoid any connection limits. But if font files were served from the same domain as other resources, then blocking could occur if the connections-per-hostname limit was exceeded.

Browser Busy Indicators

Although font files don’t block other downloads, they do trigger the browser’s busy indicators. This has a negative effect on the user’s perception of page speed because it gives the impression that the page takes a long time to load. The busy indicators that are triggered vary by browser, according to the following table.

Note that font files block the window’s onload event from firing in IE and Firefox, but not Safari nor Chrome. This is seen by running the FOUT Test – the page load time is six seconds for IE and Firefox, a few hundred milliseconds for Safari and Chrome.

Wasteful Downloads

Paul credits Garrick from Kernest for pointing out that IE is a little jumpy when it comes to downloading fonts. IE starts downloading the font file as soon as it encounters the @font-face declaration. This means IE downloads the font file even if no elements in the page use the font.

You can verify this with my Unused Font Test. This page has an @font-face declaration, but nothing in the page uses the font. In this page, the font file is configured to take six seconds to download. Sure enough, the overall page load time in IE is six seconds, indicating that the font file was downloaded even though it’s not used. Perhaps IE adopted this approach to avoid the FOUT problem, but given the likelihood of @font-face occurring even if the font isn’t used in the current page, this behavior is likely to result in many wasteful downloads and an overall degradation in performance.

Compression

Stoyan’s post about gzipping font files is a must read. Paul initially thought this wasn’t possible, and I followed suit. Thank goodness Stoyan did the legwork to show that, in fact, you should compress font files. His survey results show a savings of 40% or more for TTF, OTF, and EOT files. I now have Apache configured to compress my font files.

Caching

Font files are cacheable, just like any other resource. You can see this in the Cache Test. The font file is configured to take six seconds to download, so the first time you visit the page, the load time is a little over six seconds. But the font file has a far future Expires header, so clicking on the Cache Test link again happens more quickly since the font file is read from the browser’s cache.

@font-face Outage

Given the FOUT and browser busy issues caused by font files, we need to think about the worst case scenario: What if there’s an issue with the server hosting the font file – what happens if it times out or takes an extremely long time to respond? The Outage Test shows what happens in this situation, and it’s not pretty. The font file in this test is configured to take 20 seconds to download. Here are the results:

• IE: Recall from the FOUT Test that in IE, nothing in the entire page is rendered until the font file is done downloading. This is true even if the @font-face style is below everything else in the page. If you load the Outage Test in IE, nothing will render for 20 seconds. Ouch!
• In Firefox, the text is drawn in a default font, then redrawn once the font file is downloaded after 20 seconds.
• In most other browsers, the text isn’t drawn for 20 seconds, but the rest of the page is rendered.

Prompted by Annie Sullivan, I tried to find the maximum time a browser would spend trying to download a font file. The maximum download time I tested was 10 minutes. Safari did the best: after 60 seconds it bailed and rendered the stylized text with a default font. IE did the worst: with a 10 minute download time, IE still showed a blank white page. Firefox drew the text in a default font immediately (its normal behavior), but its busy indicators continued to agitate for 10 minutes. Chrome never rendered the text, and its busy indicators continued for the full 10 minutes.

These slow response and outage issues raise significant concerns for anyone considering using @font-face. It’s different from what happens when an image times out and the page ends up showing the broken image icon. If a font file fails to return, the page is blocked in IE, the text isn’t displayed in Chrome, and the browser’s busy indicators never stop in IE, Firefox, and Chrome (at least for 10 minutes).

@font-face Performance Advice

My first piece of advice is to avoid using @font-face unless it’s critical to the page.

The primary reason for this advice is the risk that font files will block rendering of the entire page in IE until they’re done downloading if there is a SCRIPT tag above the @font-face declaration. Stylesheets also have this blocked rendering problem. But stylesheets provide styling for all aspects of the entire page, whereas font files only add one thing – a custom font.

If you’re not deterred by the FOUT and outage issues, and want to push on to use @font-face, I recommend deferring the font file download until after the page has rendered, as shown in the Lazy Load Test. This solves the issues in IE – the page renders and then the font is downloaded in the background and enhances the stylized text once it arrives. This technique has benefits in other browsers, as well. By lazy loading the font file, most of the browser busy indicators aren’t triggered. The lazy load code looks like this:

function lazyload() {
	var sRule1 =
		"@font-face {" +
		"  font-family: 'Yanone';" +
		"  src: url('/bin/resource.cgi?type=font&sleep=6');" +
		"  src: local('Yanone'), " +
                "url('/bin/resource.cgi?type=font&sleep=6') " +
                "format('truetype');" +
		"}";

	var style1 = document.styleSheets[0];
	if ( "function" === typeof(style1.insertRule) ) {
		// Firefox, Safari, Chrome
		style1.insertRule(sRule1, 0);
	}
	else if ( "string" === typeof(style1.cssText) ) {
		// IE
		style1.cssText = sRule1;
	}
}

This is a prototype, not a hardened solution. There are shortcuts here (styleSheets[0] is an empty style block in the page). In Safari and Chrome there is a slight flash of the stylized text. I talk about some possible workarounds in the test page itself.

In Paul’s post he talks about prefetching font files, but notes that the technique doesn’t apply to IE. I consider IE to be the most significant problem to solve. Also, I would tend to put stylesheets and scripts, and in some cases images, at a higher priority to download than font files. It depends on the page and how the font file is used.

I don’t recommend using data: URIs in place of the font files in stylesheets. That could double the amount of data that was downloaded, since the stylesheet would have to contain the data for both the EOT and TTF files. This would also make stylesheets take longer to download, and stylesheets block rendering in almost all browsers.

Summing up:

• Only use @font-face is you’re absolutely certain you need it.
• Put your @font-face declaration above all SCRIPT tags.
• If you have multiple font files, consider sharding them across multiple domains.
• Don’t include unused @font-face declarations – IE will download them whether they’re used or not.
• Gzip the font files and give them a future Expires header.
• Consider lazy loading the font files, at least in IE.

Given the high number of posts about @font-face recently, I expect we’ll see more performance issues and workarounds as we grapple with how to use this feature in today’s browsers while ensuring our users get the fast experience they want and deserve.

Alistair Croll adds more evidence to the business case for high performance web sites in his blog post Proof that speeding up websites improves online business. This was the primary theme of Velocity 2009, and major web destinations shared their business case studies, including Bing, Google, and Shopzilla. Alistair rounds out the stats by answering the question: How big an impact does performance optimization have on the business metrics of a typical media or e-commerce site?

Alistair worked with his friends at Strangeloop Networks to gather the data. Strangeloop makes acceleration appliances that automatically optimize dynamic web apps in real time. They left the appliance off for half of the visitors and turned it on for the other half, and compared the differences. Some highlights from what they found:

• pages per visit grew from 11.04 to 15.64
• time spent on the site went from 23:50 to 30:10
• conversion rate increased 16.07%
• order value increased 5.51%

In addition to these metrics, we have data that shows improving web performance reduces operating costs. In his talk about Shopzilla’s performance improvements (video, slides), Phil Dixon mentions that the number of servers required to run their site dropped in half. Netflix’s outbound traffic dropped almost 50% as a result of the work done by Bill Scott and his team.

Web Performance Optimization improves user and business metrics. WPO also decreases operating costs by reducing hardware requirements and bandwidth, which in turn reduces carbon footprint. It’s a win on all fronts. We’re going to see even more case studies on the positive impact of performance optimization, and as a result, the interest in learning more about this field will continue to grow. In addition to speaking at Øredev, Fronteers, and JSConf.eu, I’m organizing a few other performance meetups in the next few months. Watch my blog for announcements.

I love evangelizing high performance best practices and engaging with developers who are focused on delivering a fast user experience. Helping to optimize the Web appeals to the (speed)geek in me, and my humanitarian side feels good about creating a better user experience.

There are many companies and developers out there that go the extra mile to deliver a fast web page. My hat’s off to you all. It can be hard to maintain a high performance development process day-in-and-day-out. Heck, it can be hard to even understand all the performance optimizations that are possible, and which ones make sense for you. I savor the challenge this presents, for the same reason that I resonate with what Jimmy Dugan (Tom Hanks) says to Dottie Hinson (Geena Davis) in A League of Their Own when she complains about how hard playing baseball is:

It’s supposed to be hard. If it wasn’t hard, everyone would do it. The hard… is what makes it great.

As much as I love the skills it takes to build fast web sites, I also want a majority of sites on the Web to get a YSlow A. Right now, the bar is just too high. For fast web sites to become pervasive, web development needs to be fast by default. Performance best practices need to be baked into the most popular development frameworks and happen automatically. We’re seeing this with browsers (parallel script loading, faster JavaScript engines, etc.) and in some frameworks (such as SproutCore and WordPress). I predict we’ll see more frameworks and tools that deliver this fast by default promise. And that’s where Aptimize comes in.

I’ve been tracking Aptimize for about a year since they contacted me about their Website Accelerator. I was psyched to have them present at and sponsor Velocity. Website Accelerator changes web pages in real time and injects many of the performance best practices from my books, plus some others that aren’t in my books. It’s a server-side module that runs on Microsoft Sharepoint, ASP.NET, and Linux/Apache. One of their more high profile users wrote up the results last week in this blog post: How we did it: Speeding up SharePoint.Microsoft.com.

Here are some of the before-vs-after stats:

• # of HTTP requests empty cache went from 96 to 35
• # of HTTP requests primed cache went from 50 to 9
• empty cache page load times were reduced 46-64%
• primed cache page load times were reduced 15-53%

These results were achieved by automatically applying the following changes to the live pages as they left the server:

• concatenate scripts together (goes from 18 to 11 requests)
• concatenate stylesheets together (goes from 17 to 5 requests)
• create CSS sprites and inline images using data: URIs (goes from 38 to 13 requests)
• add a far future Expires header
• remove duplicate scripts and stylesheets
• minify (remove whitespace, etc.) in scripts and stylesheets

You can see the results by visiting sharepoint.microsoft.com. (Compare the optimized page to the before page by adding “?wax=off” to the URL. Get it? “wax=on”, “wax=off”?? The Aptimize team is from New Zealand, so they’re always up for a good laugh. Note that once you use the querystring trick you might have to clear your cookies to move between versions.) You know you’re looking at the optimized page if you see URLs that start with “http://sharepoint.microsoft.com/wax.axd/…”.

With my recent work on SpriteMe, I was most interested in how they did CSS sprites and used data: URIs. Here’s one of the sprites Aptimize Website Accelerator created automatically. They don’t create it on-the-fly for every page. They create the sprite image and cache it, then reuse it on subsequent pages.

I was especially impressed with the work they did using data: URIs. Data: URIs are a technique for avoiding the extra HTTP request for files (typically images) by embedding the encoded response data directly in the HTML document. The main problem with data: URIs is that they’re not supported in IE7 and earlier. Recent blogs from web dev performance rock stars Stoyan Stefanov and Hedger Wang discuss a workaround for early versions of IE using MHTML. Aptimize has already incorporated these techniques into their product.

To see this in action, go to sharepoint.microsoft.com in a browser other than IE. I’m using Firefox with Firebug. Inspect the printer icon next to “print friendly” at the bottom of the page and you’ll notice the background-image style looks like this:

url(data:image/gif;base64,R0lGODlhGAAaALMJA...

That’s the base64-encoded image. It’s declared in the MSCOMSP-print rule inside this Aptimize-generated stylesheet.

Inspect the same “print friendly” icon in IE7 and you’ll see this background-image:

url("mhtml:http://sharepoint.microsoft.com/wax.axd/cache/2fpw31-aG1G4JD2$a$MeCg.mhtx!I_aWWORG5UHNr6iB8dIowgoA")

This is worth understanding: “mhtml” tells IE this is MIME HTML. The URL, http://sharepoint.microsoft.com/wax.axd/cache/2fpw31-aG1G4JD2$a$MeCg.mhtx, points to multipart response. The location, I_aWWORG5UHNr6iB8dIowgoA, identifies which part of the multipart response to use. There we find the same base64-encoding of the printer icon.

Most sites avoid data: URIs because of lack of cross-browser support. The MHTML workaround has been known for awhile, but I don’t know of many web sites that understand this technique, let alone use it. And here we have a tool that adds this technique to your web site automatically. Well done, Aptimize.