Remember all the hubbub that cropped up yesterday about Samsung cheating in benchmark tests? And then earlier today it denying such accusations? Well, AnandTech has been doing some research since reports of Samsung’s “cheating” came out in July, and discovered something very interesting in regards to benchmark tests among Android OEMs. As you can see from the chart above, almost all Android OEMs cheat at one or more benchmark tests, boosting the speeds of their respective CPUs and GPUs when a benchmark has been detected. LG does it, HTC does it, Samsung does it, and even ASUS does it. Something interesting, however, that AnandTech found, is that Motorola seems to be the only major Android OEM that – at least for modern devices – does not “cheat” on benchmark tests.
Of course, it was also found that Nexus devices do not have this built in because it is not a feature of AOSP, only a feature that OEMs have to build into the software. AnandTech said this after it had worked with multiple devices from multiple OEMs:
With the exception of Apple and Motorola, literally every single OEM we’ve worked with ships (or has shipped) at least one device that runs this silly CPU optimization…It’s a systemic problem that seems to have surfaced over the last two years, and one that extends far beyond Samsung.
It seems that when Samsung was called out on this back in July, it shouldn’t have been the only one named, nor does it seem to have stopped it from doing it yet again in the newer devices (aka the Note 3). Google has stayed clear of this by not implementing such thing into either AOSP or its Nexus device lineup (which is actually manufactured by other OEMs). It also pointed out that the Google Play Edition of the HTC One and Galaxy S4 aren’t clean from this, as they aren’t really straight AOSP devices as are the Nexus lineup. Nonetheless, this seems to be a widespread problem over on the Android side of things…so I’ll just kick back, relax, and keep using my iPhone 5s (and unaffected Nexus 7).
The entire article over on AnandTech is quite lengthy, but certainly worth the read if you want more information on how all this was figured out. Check it out here: “They’re (Almost) All Dirty: The State of Cheating in Android Benchmarks.”
What do you think? Is this taking things too far? Do you think this has always been going on? Let us know in the comments, or tweet me @TiP_Kyle.