Flushing the Document Early

May 18, 2009 10:02 pm | 23 Comments

This post is based on a chapter from Even Faster Web Sites, the follow-up to High Performance Web Sites. Posts in this series include: chapters and contributing authors, Splitting the Initial Payload, Loading Scripts Without Blocking, Coupling Asynchronous Scripts, Positioning Inline Scripts, Sharding Dominant Domains, Flushing the Document Early, Using Iframes Sparingly, and Simplifying CSS Selectors.

 

The Performance Golden Rule reminds us that for most web sites, 80-90% of the load time is on the front end. However, for some pages, the time it takes the back end to generate the HTML document is more than 10-20% of the overall page load time. Even if the HTML document is generated quickly on the back end, it may still take a long time before it’s received by the browser for users in remote locations or with slow connections. While the HTML document is being generated on the back end and sent over the wire, the browser is waiting idly. What a waste! Instead of letting the browser sit there doing nothing, web developers can use flushing to jumpstart page loading even before the HTML document response is fully received.

Flushing is when the server sends the initial part of the HTML document to the client before the entire response is ready. All major browsers start parsing the partial response. When done correctly, flushing results in a page that loads and feels faster. The key is choosing the right point at which to flush the partial HTML document response. The flush should occur before the expensive parts of the back end work, such as database queries and web service calls. But the flush should occur after the initial response has enough content to keep the browser busy. The part of the HTML document that is flushed should contain some resources as well as some visible content. If resources (e.g., stylesheets, external scripts, and images) are included, the browser gets an early start on its download work. If some visible content is included, the user receives feedback sooner that the page is loading.

Most HTML templating languages, including PHP, Perl, Python, and Ruby, contain a “flush” function. Getting flushing to work can be tricky. Problems arise when output is buffered, chunked encoding is disabled, proxies or anti-virus software interfere, or the flushed response is too small. Scanning the comments in PHP’s flush documentation shows that it can be hard to get all the details correct. Perhaps that’s why most of the U.S. top 10 sites don’t flush the document early. One that does is Google Search.

Google Search flushing earlyGoogle Search flushing early

The HTTP waterfall chart for Google Search shows the benefits of flushing the document early. While the HTML document response (the first bar) is still being received, the browser has already started downloading one of the images used in the page, nav_logo4.png (the second bar). By flushing the document early, you make your pages start downloading resources and rendering content more quickly. This is a benefit that all users will appreciate, especially those with slow Internet connections and high latency.

23 Responses to Flushing the Document Early

  1. Apache filters like mod_deflate or mod_gzip can have their own buffer and makes sometimes things difficult if we want to flush early. Do you have any solution for that ?

  2. FYI, for ASP the Response.Flush Method exists.
    http://msdn.microsoft.com/en-us/library/ms525560.aspx

    Interestingly, using this method will result in the server (IIS) not honoring Keep-Alive.

    This means there is a trade-off to be made. Anybody have experience in making this trade-off?

    txs,
    Aaron

  3. @Eric: The most recent versions of Apache don’t have an issue with this. For earlier versions, you might have to add padding to get 8K of compressed content in the partial response. There’s a page or two about this in the chapter.

    @Aaron: Thanks for pointing this out. Keep-alive should only be necessary for HTTP/1.0 clients. For HTTP/1.1 (the majority of clients), persistent connections doesn’t rely on the Keep-Alive header. The documentation at the URL is limited, so it’d be great if someone running IIS could test this and comment.

  4. I was never able to use flushing properly. I had issues with IE if I am right. At least I can remember some browsers don’t care about the flush, they have their own rules do decide how much data they need before displaying the page or part of it.
    I really need some good examples of how to use it. Also gzipping HTML seems not to be compatible with flush.

    Richard
    http://sili.co.nz/blog

  5. Flushing can play nicely with gzip, depending on how the server is set up, and how gzipping is set up. “Chunked” transfer encoding lets pages be sent in pieces, and you can flush between pieces.

    In PHP, often you have to call ob_flush(); and then flush(); for it to work.

  6. It can be useful to flash head section — usually it contains static (or almost static) data. But this will force browsers to download all blocking (CSS/JS) files.

    But I haven’t seen any 100% working solution to flush content. Only 100 words about ‘flush is cool’ :(

  7. Any research done on flushing partial response for an XHR call?

    Most of the libraries out there wouldn’t invoke a callback till readystate 4 is reached.

  8. @Richard – Some browsers have a minimum size before they start parsing a partial response: IE (255 bytes), Safari (1K), Chrome (2K). I’m finishing up the code examples that go with the book – those will be available by June 15.

    @Sudhee – I’m familiar with XHR streaming – it’s not supported in all browsers. Dylan Schiemann’s chapter on Comet in Even Faster Web Sites discusses this in more detail.

  9. There are several Yahoo! sites that do this as well. The new homepage is one of them.

  10. Excellent info i was stuck with this since week.

    Cheers,
    Innovative Consulting
    http://www.groupiconsulting.com

  11. Seems to me that to keep connections alive, one needs a Content-Length header unless the message length (or end) can be determined otherwise (http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.4).

    When you start flushing early, it won’t be possible to send the Content-Length header. SO IIS seems to have a point here.

  12. The addition of Chunked Encoding to HTTP/1.1 specifies how the server can indicate the end of the response. In section 3.6.1 it says “The chunked encoding is ended by any chunk whose size is zero, followed by the trailer, which is terminated by an empty line.”

    And just to clarify, HTTP/1.1 uses persistent connections by default, not Keep-Alive.

  13. Does anyone know why I can only flush() in PHP if I sleep() afterwards? Currently, my site looks better but actually loads more slowly.

  14. Great info! But I’m still out of luck.
    I use mod deflate and so I’m using ob_flush(); followed by flush();. I’ve tried placing it everywhere in the document!
    My Apache version is Apache/2.2.3 (CentOS) so I have set DeflateBufferSize 1. (setting 0 gave me an error that it has to be positive)
    But this is still not flushing anything to the browser. I’ve tried in Chrome and Firefox, and can clearly see nothing is flushed when inspecting Firebug. IE actually now takes 5 times as long!

    Is there a solution, or am I doomed to never being able to use mod deflate?
    Strangely, before using mod_deflate, the content would flush automatically without a flush() command in the code…

    Thanks Steve, your advice as well as your tools are such a great help!

  15. @Melvin: Test it without gzip and make sure you’re sending enough data. If it still doesn’t work, post a test URL and I’ll take a look.

  16. Steve, without Gzip the file gets flushed early, without even using flush();
    Here’s the page. http://tinyurl.com/yfbf7y2 please note that if you test more than once, you need to change the dates on the left, as the results gets cached in the SID.
    Also here are some results from Firebug’s “NET” tab that I found interesting http://tinyurl.com/yzmmfx6

    Thanks!

  17. Excellent post! Can we have more posts on easy-to-fix back-end optimizations please? I just realized my website’s time-to-first-byte is pretty high.

  18. All,

    I’d appreciate your help.

    I have a case where the flush() works fine, but using the Trailer header does not. PHP throws the “Cannot modify header information – headers already sent by” error.

    Test page: http://www.aaronpeters.nl/sandbox/centralpoint/flush/alex/test-no-smarty-trailer-header-2.php
    Source code: http://www.aaronpeters.nl/sandbox/centralpoint/flush/alex/test-no-smarty-trailer-header-2.txt

    Any tips is highly appreciated!

    (Apache 2.2.8, PHP 5.2.4, mod_deflate is on)

  19. @Aaron: Trailers are only supported in Opera: http://www.browserscope.org/?category=network&v=top . Are you doing this just for testing? I’m not surprised that PHP would not allow it. You could look at the Browserscope code for this test and see how it was done. Note: I didn’t write that code so you’ll have to talk to the creator for Q&A.

  20. Only supported in Opera?
    That’s just too bad.

    Yes, doing tests with this, with the objective to then implement it at a client (so they don’t have to refactor the app).
    Aborting now, due to lack of browser support :(

    txs for reply

  21. @bob
    Apache doesn’t necessarily send a chunk of content down to the client every time you call flush() from PHP… calling flush() just makes the content available to Apache, which may wait until it has a decent sized chunk to send.

    I think this may be why you only see the flush() calls working like you expect when you sleep() after.

    I could be wrong though, while since I did any PHP.

  22. i had no idea about the “flush” function. I use it at least once a day outside of php tho ;)

  23. Something else you need to be aware of when doing this on IIS6 is that you can’t do it if you are using GZIP for dynamic content and the compression level is too high, we found that it will not work if the compression level is higher than 3, if the compression level is higher then the buffer isn’t flushed until the entire page is processed.