There's no doubt the latest crop of stable browsers from Google, Microsoft, and Mozilla are the best the companies have ever produced. But how do they perform when tested under identical conditions?
CNET put the latest stable versions of Firefox, Chrome, and Internet Explorer through a gauntlet of benchmarks that considered JavaScript and HTML5 performance, as well as boot times and memory usage. (Opera and Safari were not tested because they have not been updated recently, and neither has yet implemented hardware acceleration close to the level that the other three browsers have.) Note that these charts are at best a snapshot in time, and are dependent on the hardware being used and any extensions installed. The full charts are below, followed by analysis and an explanation of our methodology.
(Credit: Chart by Seth Rosenblatt/CNET) 
(Credit: Chart by Seth Rosenblatt/CNET) 
(Credit: Chart by Seth Rosenblatt/CNET) 
(Credit: Chart by Seth Rosenblatt/CNET) *JSGamebench was conducted by Facebook developers. The test was included because it's a publicly available test of real-world gameplay, though we opted to use Facebook's published data for simplicity's sake. The hardware acceleration using WebGL results were not included because only Firefox 4 and Chrome 11 were included in the test group, and Chrome 11 was not tested by CNET this round because it's still in beta.
(Credit: Chart by Seth Rosenblatt/CNET) 
(Credit: Chart by Seth Rosenblatt/CNET)
Though the competition is extremely close in some cases--especially JavaScript rendering--Firefox 4 is strongly favored by HTML5 processing, boot time, and memory usage. Overall, I'd judge from these results that Firefox 4 is the winner this time around.
Chrome, however, is absolutely killing it on Google's V8 benchmark. Expect the next version of Chrome to perform much better on the JSGamebench test, once hardware acceleration has been fully enabled. You currently have to toggle a few switches in about:flags to get it all. Also expect Chrome's boot time and memory performance to improve--Google has said it plans to spend more time working on Chrome's memory hogginess in the coming versions.
Given the renewed resurgence in Internet Explorer, it's also hard to imagine that the IE development team isn't already working on making the browser better.
Also of interest is that the SunSpider results are extremely close. The gulf between 250 milliseconds and 290 milliseconds is just not going to be that detectable by the average person.
How we tested
Our test machine was a Lenovo T400, with an Intel Core 2 Duo T9400 chip running at 2.53GHz, with 3GB of RAM, using Windows 7 x86. We used four publicly available tests: WebKit SunSpider 0.9.1, Mozilla Kraken 1.0, Google V8 version 6, and JSGameBench 0.3. All tests except for JSGamebench were conducted using a "cold boot" of the browser, that is, both the computer and the browser being tested were restarted before each test. Each test was performed three times, and the results you see are the averages. Browsers had all extensions and add-ons deactivated for the tests.
We opened five Web sites for all tests, in addition to any test site. These were: talkingpointsmemo.com, aol.com, youtube.com, newyorktimes.com, giantbomb.com, cnettv.cnet.com.
The boot time benchmarks were conducted by manually starting a stopwatch when clicking on the browser's taskbar icon, and then hitting stop when the last tab's resolving indicator stopped rotating. One half-second was subtracted from Internet Explorer 9's pre-averaged times to account for the extra time it took to hit the Reload previous session link, since the browser doesn't support that feature the way Firefox 4 and Chrome 10 do.
The memory test was conducted by opening the aforementioned set of tabs and looking at Google Chrome's memory manager. You can access it by typing "about:memory" into the Chrome location bar. The figure we used is the Private Memory, which only totals memory used by the browser that's not shared by other processes. It's also useful because it tallies all of Chrome's open tab memory usage into one convenient number.
CNET put the latest stable versions of Firefox, Chrome, and Internet Explorer through a gauntlet of benchmarks that considered JavaScript and HTML5 performance, as well as boot times and memory usage. (Opera and Safari were not tested because they have not been updated recently, and neither has yet implemented hardware acceleration close to the level that the other three browsers have.) Note that these charts are at best a snapshot in time, and are dependent on the hardware being used and any extensions installed. The full charts are below, followed by analysis and an explanation of our methodology.
(Credit: Chart by Seth Rosenblatt/CNET) (Credit: Chart by Seth Rosenblatt/CNET) (Credit: Chart by Seth Rosenblatt/CNET) (Credit: Chart by Seth Rosenblatt/CNET) (Credit: Chart by Seth Rosenblatt/CNET) (Credit: Chart by Seth Rosenblatt/CNET) | Chrome 10 | Internet Explorer 9 | Firefox 4 | |
|---|---|---|---|
| SunSpider 0.9.1 (ms) | 336.20 | 250.60 | 292.37 |
| Kraken (ms) | 8,806.30 | 15,606.77 | 7265.13 |
| V8 v6 (higher is better) | 5,173.67 | 2,235.33 | 3540.33 |
| JSGamebench 0.3* (higher is better) | 322.00 | 1,156.00 | 1,482.00 |
| Boot time (s) | 26.22 | 21.86 | 17.80 |
| Memory (kb) | 390,532 | 205,616 | 148,020 |
Chrome, however, is absolutely killing it on Google's V8 benchmark. Expect the next version of Chrome to perform much better on the JSGamebench test, once hardware acceleration has been fully enabled. You currently have to toggle a few switches in about:flags to get it all. Also expect Chrome's boot time and memory performance to improve--Google has said it plans to spend more time working on Chrome's memory hogginess in the coming versions.
Given the renewed resurgence in Internet Explorer, it's also hard to imagine that the IE development team isn't already working on making the browser better.
Also of interest is that the SunSpider results are extremely close. The gulf between 250 milliseconds and 290 milliseconds is just not going to be that detectable by the average person.
How we tested
Our test machine was a Lenovo T400, with an Intel Core 2 Duo T9400 chip running at 2.53GHz, with 3GB of RAM, using Windows 7 x86. We used four publicly available tests: WebKit SunSpider 0.9.1, Mozilla Kraken 1.0, Google V8 version 6, and JSGameBench 0.3. All tests except for JSGamebench were conducted using a "cold boot" of the browser, that is, both the computer and the browser being tested were restarted before each test. Each test was performed three times, and the results you see are the averages. Browsers had all extensions and add-ons deactivated for the tests.
We opened five Web sites for all tests, in addition to any test site. These were: talkingpointsmemo.com, aol.com, youtube.com, newyorktimes.com, giantbomb.com, cnettv.cnet.com.
The boot time benchmarks were conducted by manually starting a stopwatch when clicking on the browser's taskbar icon, and then hitting stop when the last tab's resolving indicator stopped rotating. One half-second was subtracted from Internet Explorer 9's pre-averaged times to account for the extra time it took to hit the Reload previous session link, since the browser doesn't support that feature the way Firefox 4 and Chrome 10 do.
The memory test was conducted by opening the aforementioned set of tabs and looking at Google Chrome's memory manager. You can access it by typing "about:memory" into the Chrome location bar. The figure we used is the Private Memory, which only totals memory used by the browser that's not shared by other processes. It's also useful because it tallies all of Chrome's open tab memory usage into one convenient number.
No comments:
Post a Comment