Inconclusive tests paint Android browser 52 percent faster than iPhone's, but what about Safari's Nitro engine? (UPDATED: the testing is flawed)

According to a Blaze study stemming from 45,000 Android and iOS tests, the Android browser on average loads web pages 52 percent faster than mobile Safari. The results are inconclusive, however, because it’s unclear whether Blaze’s measurements take into account the new Nitro JavaScript engine that comes with iOS 4.3. The report was completed before this complaint was made public and Blaze is arguing that the lack of Nitro boost can “slightly” skew the results given that “JavaScript only accounts for a small percentage of the total load time.”

The devices used for testing include a Nexus One running Android 2.3 and an iPhone 4 with, presumably, iOS 4.3. Both operating systems sport WebKit-based browsers.

On average, Android came in 52 percent faster with an average page load time of 2.144 second versus 3.254 seconds for the iPhone. Android apparently outperformed Apple’s handset on 84 of the tested websites, including those that are not optimized for mobile access. On mobile sites, however, Android’s lead dropped to just three percent.

Regarding that Nitro issue, a follow-up note on Blaze’s site says the following:

Some wonder whether the new Nitro JavaScript engine was used in our measurements. We’re still investigating this issue, as the report was completed before it was made known. So far we’ve seen indications in both directions, so we can’t say for sure it’s being applied. That said, the results from measuring Android show that JavaScript only accounts for a small percentage of the total load time, about 15% on average. This implies that even if Nitro is not in use, it likely can only slightly narrow the gap. We’ll follow up with any additional info.

UPDATE [March 17, 2011 7:30pm Eastern]

A spokesperson for Apple confirmed that “the embedded web viewer does not take advantage of Safari’s web performance optimizations.” Put simply, the new Nitro engine only kicks in the Safari app, not when viewing web pages in third-party apps that use the embedded web viewer. Meanwhile, the blogosphere exposed the Blaze survey as flawed because they actually tested the web controls of each platform using custom apps rather than actual browsers. Plus, Blaze technology chief Guy Podjarny admitted to operating under flawed assumptions:

This test leveraged the embedded browser which is the only available option for iPhone applications. Blaze was under the assumption that Apple would apply the same updates to their embedded browser as they would their regular browser. If this is not the case and according to Apple’s response, it’s certainly possible the embedded browser might produce different results. If Apple decides to apply their optimizations across their embedded browser as well, then we would be more than willing to create a new report with the new performance results.

Topping it all off , Apple spokesperson Natalie Kerris told The Loop:

Their testing is flawed because they didn’t actually test the Safari web browser on the iPhone. Instead they only tested their own proprietary app which uses an embedded web viewer that doesn’t take advantage of Safari’s web performance optimizations. Despite this fundamental testing flaw, they still only found an average of a second difference in loading web pages.