Leadership Scorecard: The Top Six Telecom Vendors
People no doubt are shocked—shocked!—to learn that most of the major smartphone vendors have apparently cheated on performance tests of their devices.
Ars Technica and AnandTech found that smartphone makers were using a trick to speed up their devices when they were being clocked by benchmarking software. Though it didn’t uncover the workaround, ReadWrite explained it very well:
Research by technology publications Ars Technica and AnandTech have shown that almost all major smartphone manufacturers have ways to make their smartphones look faster than they really are. Samsung, HTC, ASUS, LG, NVIDIA have all employed a technique in their devices that will register the presence of a benchmarking app and then dial up the device to maximum output to trick the test into thinking the device is faster than it would normally be. Of the tests done by AnandTech, only Apple and Google-owned Motorola do not try to game most popular benchmarking apps.
Ars Technica’s explanation of the step by step forensics its editors employed to discover what it claims the Samsung Galaxy Note 3 is doing is fun to read.
There seems to be disagreement on the amount of speed the vendors added when benchmarking was under way. This is from Ars Technica:
The difference is remarkable. In Geekbench’s multicore test, the Note 3’s benchmark mode gives the device a 20 percent boost over its “natural” score. With the benchmark boosting logic stripped away, the Note 3 drops down to LG G2 levels, which is where we initially expected the score to be, given the identical SoCs.
AnandTech suggested that the tweaking was a bit less dramatic. After crediting a researcher who uses the name AndreiF7, the site’s Anand Lal Shimpi and Brian Klug looked at several devices. Only those from Motorola and Apple were clear, according to the authors. They point to a smaller cheat than the researchers at Ars Technica:
The hilarious part of all of this is we’re still talking about small gains in performance. The impact on our CPU tests is 0 – 5%, and somewhere south of 10% on our GPU benchmarks as far as we can tell.
The lessons to be drawn from this are obvious: In a highly competitive marketplace, vendors are likely to lie if they are confident that they can get away with it. Luckily, the technology media and the people with whom they work are extremely clever and eventually will catch on.
The best approach of those who really want to know how their devices are operating is to follow Ronald Reagan’s simple dictum (which, apparently, is a Russian proverb): Trust, but verify.
