SteveSouders.com

A: There are two main causes: slow network connections and proxies. Some of the tests rely on timing information. If you're on a really slow network connection, it could cause the test to falsely fail. For example, to test if scripts block downloads two 4-second scripts are loaded. If the page takes more than 8 seconds to load, the assumption is the first script blocked the second script. However, it's possible that the script didn't block, and instead it took over 8 seconds just to download the scripts. Some proxies (most notably Squid) downgrade traffic to HTTP/1.0. Many browsers use a different number of connections for HTTP/1.0, so those tests may differ.

A: I wrote the browser detection code myself. I especially wanted to do this so that new browsers, like Chrome and Android, were recognized. Every week or so, I'll look at the browsers classified as "unknown" and map them to a real browser. Eventually, I hope to integrate a library to do browser detection from User-Agent.

A: Here's a short description of each test.

Check Latency

This isn't actually a test. Many of the tests measure how long it takes for a resource to load, but the load time can vary greatly depending on the user's network latency. This page measures the average latency to the UA Profiler server, and then adjusts the timing thresholds throughout the remaining test pages accordingly. If you have high latency (slow network connection), the tests take longer to load. If you have low latency (fast network connection), the tests are run faster.

Connections per Hostname

When HTTP/1.1 was introduced with persistent connections enabled by default, the suggestion was that browsers open only two connections per hostname. Pages that had 10 or 20 resources served from a single hostname loaded slowly because the resources were downloaded two-at-a-time. Browsers have been increasing the number of connections opened per hostname, for example, IE went from 2 in IE7 to 6 in IE8. This test measures how many HTTP/1.1 connections are opened for the browser being tested.

Max Connections

The previous test measures maximum connections for a single hostname. This test measures the maximum number of connections a browser will open total - across all hostnames. The upper limit is 60, so if a browser actually supports more than that it'll still show up as 60.

Parallel Scripts

When most browsers start downloading an external script, they wait until the script is done downloading, parsed, and executed before starting any other downloads. Although parsing and executing scripts in order is important for maintaining code dependencies, it's possible to safely download scripts in parallel with other resources in the page (including other scripts). This test determines if scripts can be downloaded in parallel with other resources in the page.

Parallel Stylesheets

Similar to scripts, some browsers block all downloads once they start downloading a stylesheet. This test determines if stylesheets can be downloaded in parallel with other resources in the page.

Parallel Stylesheet and Inline Script

A lesser known performance problem is the problems caused when a stylesheet is followed by an inline script block. If a browser doesn't block when downloading stylesheets (as measured by the previous test), then a stylesheet followed by an image could both be downloaded in parallel. But suppose an inline script block was placed between the stylesheet's LINK tag and the image IMG tag. The result, for some browsers, is that the image isn't downloaded until the stylesheet finishes. The reason is that the image download must occur after the inline script block is executed (in case the script block itself inserts images or in some other way manipulates the DOM), and the inline script block doesn't execute until after the stylesheet is downloaded and parsed (in case the inline script block depends on CSS rules in the stylesheet). It's important to preserve the order of the stylesheet rules being applied to the page, followed by executing the inline script block, but there's no reason other resources shouldn't be downloaded in parallel and not applied to the page until after the inline script block is executed. A subtlety of this test is that if the test is determined to be a failure if the inline script is executed before the stylesheet is done downloading - although this is faster it could lead to unexpected behavior.

Prime the Cache

This page isn't a test - it merely loads resources that will be used in the next few pages to test caching.

Cache Expires

This test determines if a resource with a future expiration date is correctly read from the browser's cache, or issues an unnecessary HTTP request. This is really testing the browser's memory cache.

Cache Redirects

Many pages use redirects to send users from one page to another, for example http://google.com/ redirects to http://www.google.com/. Unfortunately, most browsers don't pay attention to the cache headers of these redirects, and force the user to endure the redirect over and over again. This test measures if a redirect for the page is cached when it has a future expiration date.

Cache Resource Redirects

Whereas the previous test measures redirect caching for the main page, this test measures redirect caching for resources in the page.

Link Prefetch

This test determines if the prefetch keyword for the link tag works. (See the link prefetch description in this MDC FAQ and in a working draft of the HTML 5 spec.) Prefetch is an easy way for web developers to download resources they know the user will need in the future.

Compression Supported

Compressing text responses typically reduces the number of bytes transmitted by approximately 70%. This test measures if the browser sends an Accept-Encoding header announcing support for compression.

data: URLs

A "data:" URL (aka an inline image), is a technique for embedding other resources directly into the main HTML document. Doing this avoids an extra HTTP request. This test checks if an image inserted using a "data:" URL is rendered correctly.

A: I wanted to focus on performance, so I don't have tests to validate adherence to the HTML or ECMAScript standards. JavaScript execution speed is certainly part of performance, but there are experts working in this area, such as John Resig, with existing benchmarks. I'm most familiar with these browser traits. Since there was no test measuring and recording these performance characteristics, I built UA Profiler to do it.

A: After saving ten test results for a given browser in a day, subsequent test results are not saved.

A: The following tests can be run in isolation: Connections per Hostname, Max Connections, Parallel Scripts, Parallel Stylesheets, Parallel Stylesheet and Inline Script, Compression Supported, and data: URLs. The other tests are multi-page tests and only work correctly when run as part of the overall test suite.

A: The overall score is the number of tests that passed out of the total number of tests, for example, 8 out of 11 ("8/11"). For an individual test for a given browser, the result is the median across all test results. Let's look at Firefox 3 as an example. The result for "Connections per Hostname" is the median of all the results for that test. Across all test results for Firefox 3 suppose the values for this test were 2, 6, 6, 6, 7. The overall result would be the median: "6". The outlier results, 2 and 7, might have been generated from a test done through a proxy connection or with a slow Internet connection. But those outliers are outweighed by the larger number of typical test results. The yes/no results, such as Parallel Scripts, are stored as 1 and 0, so the same median approach works there, too.

A: By default, the results table only shows browsers that have at least 4 tests from at least 2 different IP addresses. If you check "show sparse results", all browsers are shown regardless of the number of tests and IP addresses.

A: I want to add tests for whether https resources are cached, when the browser busy indicators are triggered, how DNS lookups are cached, support for pipelining, number of connections per hostname and maximum connections when HTTP/1.0 is used, and support for persistent connections (mostly over HTTP/1.0).

A:

• break out results by OS
• detect if the user is coming through a proxy and notify them that this might affect their results
• progress bar for tests, eg, "test 4 of 11"
• improve the GUI - anyone want to help? please?

A: Yes, but I haven't put the code up in a open repository. If enough people have interest in contributing to the code base, I'll do that.

A: Feel free to contact me. To avoid spam I've setup a form that is guaranteed to reach me: contact Steve