Skip to main content

Report finds almost all Android OEMs, not just Samsung, cheat on benchmarks

Android-Benchmarks-cheaters

When Apple SVP Phil Schiller pointed us to a story earlier this week that Samsung was artificially inflating benchmark scores for its new Galaxy Note 3, many were quick to point out it wasn’t the first time Samsung had been caught engaged in such a practice. The same issue was discovered by AnandTech for the Galaxy S4 back in July, and today the site has an extensive report showing that almost every Android smartphone manufacturer is shipping devices that do the same.

As pictured in the chart above, that includes the HTC One, HTC One mini, LG G2, Galaxy Tab 10.1, and many others. In fact, the only companies that appear to not be using the method is Apple and Motorola, as well as Google with its Nexus 4 and Nexus 7 devices:

We started piecing this data together back in July, and even had conversations with both silicon vendors and OEMs about getting it to stop. With the exception of Apple and Motorola, literally every single OEM we’ve worked with ships (or has shipped) at least one device that runs this silly CPU optimization. It’s possible that older Motorola devices might’ve done the same thing, but none of the newer devices we have on hand exhibited the behavior. It’s a systemic problem that seems to have surfaced over the last two years, and one that extends far beyond Samsung…  None of the Nexus do, which is understandable since the optimization isn’t a part of AOSP. This also helps explain why the Nexus 4 performed so slowly when we reviewed it – this mess was going on back then and Google didn’t partake.

As noted in the report, the gains that OEMs are experiencing from the inflated scores are probably not worth the press they’ve been receiving. AnandTech points out that most of the inflated scores provide under a 10% increase in GPU and CPU performance benchmarks:

The hilarious part of all of this is we’re still talking about small gains in performance. The impact on our CPU tests is 0 – 5%, and somewhere south of 10% on our GPU benchmarks as far as we can tell. I can’t stress enough that it would be far less painful for the OEMs to just stop this nonsense and instead demand better performance/power efficiency from their silicon vendors.

You can check out the full report here, which offers in-depth analysis on the optimizations it found for several devices across various benchmark tests.

FTC: We use income earning auto affiliate links. More.

You’re reading 9to5Mac — experts who break news about Apple and its surrounding ecosystem, day after day. Be sure to check out our homepage for all the latest news, and follow 9to5Mac on Twitter, Facebook, and LinkedIn to stay in the loop. Don’t know where to start? Check out our exclusive stories, reviews, how-tos, and subscribe to our YouTube channel

Comments

  1. 311sie - 11 years ago

    Shocking… just shocking.

  2. stuffradio (@stuffradio) - 11 years ago

    Won’t someone please think of the children!?

  3. Hamza Sheikh - 11 years ago

    This is really shocking for everyone in smartphone industry. People may have lost their trust over Android OEMs after this post going viral on the internet. I have seen many Apple trolls getting silent on this.

  4. Lee (@leemahi) - 11 years ago

    I love my iPad. I saw that at least 5 times a week. The first year I said it several times a day! It’s all about the fluidity and speed of iOS, not necessarily the chip.

  5. Khürt Williams - 11 years ago

    Crickets.

  6. i2fun - 11 years ago

    Phil “the shill” Shiller? lol… just being true to his very nature. I bet he was the kid in his grade school classes that was always telling on others for doing what he’d just done and got away with. Then getting patted on the back by the teachers, before leaving campus and getting beat to a pulp for being a RAT! …..and now he still looks like one too!

Author

Avatar for Jordan Kahn Jordan Kahn

Jordan writes about all things Apple as Senior Editor of 9to5Mac, & contributes to 9to5Google, 9to5Toys, & Electrek.co. He also co-authors 9to5Mac’s Logic Pros series.


Manage push notifications

notification icon
We would like to show you notifications for the latest news and updates.
notification icon
You are subscribed to notifications
notification icon
We would like to show you notifications for the latest news and updates.
notification icon
You are subscribed to notifications