Unexpected Reloads in WebKit

June 27, 2011 4:47 pm | 9 Comments

People who work on web performance often need to load the same URL over and over again. Furthermore, they need to do this while simulating a real user’s empty cache experience and primed cache experience. When I want to analyze the empty cache experience the flow is simple: go to about:blank, clear the browser cache, enter the URL, and hit RETURN.

But what’s the right way to fetch a page repeatedly when analyzing the primed cache experience?

The main goal when testing the primed cache version of a page is to see which resources are read from cache. The goal for better performance is to cache as many responses as possible thus reducing the number of requests made when the cache is primed. If a resource has an expiration date in the future, the browser uses the cached version and doesn’t have to make an HTTP request resulting in a faster page. If a resource is expired (the expiration date is in the past) the browser issues a Conditional GET request using the If-Modified-Since and If-None-Match request headers. If the resource hasn’t changed then the server returns a simple 304 status code with no body. This is faster (because there’s no response body) but still takes time to do the HTTP request. (See my article on ETags for examples of IMS and INM.)

One way to re-request a page is to hit the Reload button, but this doesn’t give an accurate portrayal of the typical primed cache user experience. Hitting Reload causes the browser to always make an IMS/INM request for resources in the page, even for cached resources that have an expiration date in the future. Normally these resources would be used without generating an HTTP request. Although users do occasionally hit the Reload button it’s more likely that they’ll navigate to a page via a link or the location field, both of which avoid the time consuming Conditional GET requests generated when hitting Reload.

The technique I adopted years ago for re-requesting a page when testing the primed cache is to click in the location field and hit RETURN. That’s a fine approach in IE, Firefox, Chrome, and Opera, but not in Safari. Let’s investigate why.

hitting RETURN in the location field

I’m using Untappd as an example. Untappd has 68 requests when loaded on the desktop. Figure 1 shows the waterfall chart for the first 31 requests when loaded in Firefox 4 with an empty cache:

Figure 1. untappd.com – Firefox 4 – empty cache

Most of the resources shown in Figure 1 have an expiration date in the future and therefore won’t generate an HTTP request if the user has a primed cache. To test that I click in the location field and hit RETURN. The resulting waterfall chart is shown in Figure 2. Sure enough the number of HTTP requests drops from 68 to 4!

Figure 2. untappd.com – Firefox 4 – primed cache

If you repeat this experiment in Chrome, Firefox, Internet Explorer, and Opera you’ll get similar results – empty cache generates 68 requests, primed cache generates 4 requests. However, the result is very different in Safari 5. It’s important to understand why.

Safari is different

This test shows that Untappd has done a good job of optimizing the primed cache experience – the number of HTTP requests made by the browser drops from 68 to 4. Running the same test in Safari 5 produces different results. Clearing the cache and loading untappd.com in Safari 5 loads 68 HTTP requests – just as before. To test the primed cache experience we click in the location field and hit RETURN. Instead of only 4 requests there are 68 HTTP requests.

Why are there 64 more HTTP requests in Safari 5 for the primed cache test? Looking at the HTTP request headers we see that these are all Conditional GET requests. Let’s use http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js as the example (it’s the 8th request in Figure 1). In the empty cache scenario the HTTP request headers are:

Accept: */*
Cache-Control: max-age=0
Referer: http://untappd.com/
User-Agent: Mozilla/5.0 (Macintosh; [snip...] Safari/533.20.27

The HTTP status code returned for that empty cache request is 200 OK.

In the primed cache test when we hit RETURN in the location field we see that the request for jquery.min.js contains an extra header:

Accept: */*
Cache-Control: max-age=0
If-Modified-Since: Mon, 15 Feb 2010 23:30:12 GMT
Referer: http://untappd.com/
User-Agent: Mozilla/5.0 (Macintosh; [snip...] Safari/533.20.27

The header that’s added in the primed cache test is If-Modified-Since. This is a Conditional GET request. The HTTP status code that’s returned is 304 Not Modified. Even though all I did was hit RETURN in the location field, Safari treated that like hitting the Reload button.

unexpected “reload” in Webkit

Unlike other browsers, Safari 5 treats hitting RETURN in the location field the same as clicking the Reload button. When else does this happen? Assuming you’ve loaded a URL in Safari and are looking at that page, this table lists various ways to load that URL again. For each technique I show whether loading the URL this way generates extra Conditional GET requests similar to clicking Reload.

way of loading URL again like Reload?
hit RETURN in location field yes
delete URL and type it again yes
launch same URL via bookmark yes
click link to same URL yes
go to another URL then type 1st URL again no
modify querystring no
enter URL in a new tab no
Table 1. Triggering reload behavior in Safari

This black box testing indicates that whenever the same URL is loaded back-to-back in the same tab, Safari 5 treats it as a Reload. I was describing this behavior to Jay Freeman (saurik) at Foo Camp. He pointed me to this code from WebCore:

else if (sameURL)
   // Example of this case are sites that reload the same URL with a different cookie
   // driving the generated content, or a master frame with links that drive a target
   // frame, where the user has clicked on the same link repeatedly.
   m_loadType = FrameLoadTypeSame;

Searching in that same file for FrameLoadTypeSame we find this code:

case FrameLoadTypeReload:
case FrameLoadTypeReloadFromOrigin:
case FrameLoadTypeSame:
case FrameLoadTypeReplace:

This code doesn’t account for the behavior, but it does show that FrameLoadTypeSame and FrameLoadTypeReload are treated as similar cases in this context, and perhaps that’s why IMS/INM requests are generated.

One important takeaway from this is: don’t hit RETURN in the location field to test primed cache experience in Safari. Instead, go to a different URL and then type the test URL in the location field, or open a new tab and type the URL.

There’s a second more important takeaway from this. I’ll cover that in tomorrow’s post. If you know the answer, please don’t spoil it. Oh what the heck – if you think you know the answer go ahead and add a comment.

9 Responses to Unexpected Reloads in WebKit

  1. IME Safari has a very broken cache.

    For example, last time I checked, they’d send If-Modified-Since even if the previous response didn’t include a Last-Modified header; they’d use the previous Date value.

  2. Thank you.
    Et voilà pourquoi!
    For a long time I do not understand these
    differences in behavior between browsers. And yes! Safra is my browser of development! And yes Chrome is definitly better! Now I think I can be able to program these f… “last-modified header” that were inconsistent. Every time I tried, I gave up believing crazy! Steve you are our light! (French translated with Google)

  3. Why is this article named “Unexpected Reloads in WebKit”?

    Chrome is based on Webkit and doesn’t has same issue. Perhaps it would be less confusing to name it “Unexpected Reloads in Safari”.

  4. Hitting Cmd+Return (open same URL in another tab) will work here?

  5. if i understand your article we must implement a second way to cache files with both future expire headers and also etags?

  6. @Morgan: Please refer to today’s blog post to understand why I referred to WebKit instead of Safari. I’ll post that tonight.

    @Maciej: Yes, Cmd+Return also works, i.e., it opens the URL a second time in a new tab and does NOT act like a Reload.

    @nicolas: ETags won’t help in this situation – Safari will issue both If-Modified-Since and If-None-Match in the case of hitting RETURN in the location field (and the other Reload-like scenarios).

  7. It can be a big issue for favorite’s website, for example… Google, no?

  8. I discovered the following behavior a few days ago. I created a simple html page that contains a cacheable PNG,
    and a non-cacheable PNG. Here is my finding:

    iPhone 4
    1) pressing html page URL enter
    – conditional GET with If-Modified-Since header
    2) pressing refresh button
    – same as pressing URL enter
    3) navigating from a hyperlink to a page containing the resource
    – no GET if the cached resource is still valid

    Android 2.3.4 Nexus one:
    1) pressing html page URL enter
    – conditional GET with If-Modified-Since header
    2) pressing refresh button
    – hard refresh, unconditional GET (without If-Modified-Since header)
    3) navigating from a hyperlink to a page containing the resource
    – no GET if the cached resource is still valid

    The above tests are all on WiFi.

  9. This looks more like the culprit:

    void FrameLoader::load(DocumentLoader* newDocumentLoader)
    ResourceRequest& r = newDocumentLoader->request();
    FrameLoadType type;

    if (shouldTreatURLAsSameAsCurrent(newDocumentLoader->originalRequest().url())) {
    type = FrameLoadTypeSame;

    Do you have a reduced test case you could add to bugs.webkit.org?