frontend SPOF survey

October 13, 2011 9:10 am | 3 Comments

Pat Meenan had a great blog post yesterday, Testing for Frontend SPOF. “SPOF” means single point of failure. I coined the term frontend SPOF to describe the all-too-likely situation where the HTML document returns successfully, but some other resource (a stylesheet, script, or font file) blocks the entire website from loading. This typically manifests itself as a blank white screen that the user stares out for 20 seconds or longer.

Frontend SPOF happens most frequently with third party content. If the HTML document returns successfully, then all the resources from the main website are likely to return successfully, as well. Third party content, however, isn’t controlled by the main website and thus could be suffering an outage or overload while the main website is working fine. As a result, the uptime of a website is no greater than the availability of the third party resources it uses that are in a position to cause frontend SPOF.

In my blog post of the same name I describe how Frontend SPOF happens and ways to avoid it, but I don’t provide a way for website owners to determine which third party resources may cause frontend SPOF. This is where Pat comes in. He’s created a public blackhole server: blackhole.webpagetest.org with the static IP address 72.66.115.13. Pointing your third party resources to this blackhole and reloading the page tells you if those resources cause frontend SPOF. Since Pat is the creator of WebPagetest.org, he has integrated this into the scripting capabilities of that tool so website owners can load their website and determine if any third party resources cause frontend SPOF.

/etc/hosts

I took a different approach outlined by Pat: I added the following lines to my /etc/hosts file (your location may vary) mapping these third party hostnames to point to the blackhole server:

72.66.115.13 apis.google.com
72.66.115.13 www.google-analytics.com
72.66.115.13 connect.facebook.net
72.66.115.13 platform.twitter.com
72.66.115.13 s7.addthis.com
72.66.115.13 l.addthiscdn.com
72.66.115.13 cf.addthis.com
72.66.115.13 api-public.addthis.com
72.66.115.13 widget.quantcast.com
72.66.115.13 ak.quantcast.com
72.66.115.13 assets.omniture.com
72.66.115.13 www.omniture.com
72.66.115.13 scripts.omniture.com
72.66.115.13 b.voicefive.com
72.66.115.13 ar.voicefive.com
72.66.115.13 c.statcounter.com
72.66.115.13 www.statcounter.com
72.66.115.13 www-beta.statcounter.com
72.66.115.13 js.revsci.net

After restarting my browser all requests to these hostnames will timeout. Pat’s blog post mentions 20 seconds for a timeout. He was running on Windows. I’m running on my Macbook where the timeout is 75 seconds! Any website that references third party content on these hostnames in a way that produces frontend SPOF will be blank for 75 seconds – an easy failure to spot.

survey says

THE GOOD: At this point I started loading the top 100 US websites. I was pleasantly surprised. None of the top 20 websites suffered from frontend SPOF. There were several that loaded third party content from these hostnames, but they had safeguarded themselves:

• MSN makes a request to ar.voicefive.com, but does it asynchronously using a document.write technique.
• AOL references platform.twitter.com, but puts the SCRIPT tag at the very bottom of the BODY so page rendering isn’t blocked.
• IMDB uses the async version of Google Analytics, and puts the platform.twitter.com widget in an iframe.
• LiveJournal goes above and beyond by wrapping the Google +1 and Facebook widgets in a homegrown async script loader.

THE BAD: Going through the top 100 I found five websites that had frontend SPOF:

1. CNET loads http://platform.twitter.com/widgets.js in the HEAD as a blocking script.
2. StumbleUpon loads http://connect.facebook.net/en_US/all.js at the top of BODY as a blocking script.
3. NFL loads http://connect.facebook.net/en_US/all.js in the HEAD as a blocking script.
4. Hulu, incredibly, loads Google Analytics in the HEAD as a blocking script. Please use the async snippet!
5. Expedia loads http://connect.facebook.net/en_US/all.js as a blocking script in the middle of the page, so the right half of the page is blocked from rendering.

These results, although better than I expected, are still alarming. Although I only found five websites with frontend SPOF, that’s 5% of the overall sample. The percentage will likely grow as the sample size grows because best practices are more widely adopted by the top sites. Also, my list of third party hostnames is a small subset of all widgets and analytics available on the Web. And remember, I didn’t even look at ads.

Is it really worth blocking your site’s entire page for a widget button or analytics beacon – especially when workarounds exist? If you’re one of the five sites that faltered above, do yourself and your users a favor and find a way to avoid frontend SPOF. And if you’re outside the top 100, test your site using Pat’s blackhole server by editing /etc/hosts or following Pat’s instructions for testing frontend SPOF on WebPagetest.org.