On Samsung and benchmarks

Due to time zone differences, and my general unwillingness to sit in front of a computer after the clock strikes 17:00, I’m probably the last tech journalist to talk about the discovery AnandTech made yesterday. For those who have no idea what I’m talking about, here’s the one sentence summary: The graphics processor in Samsung’s flagship phone overclocks itself by 10% when it detects that it’s running a certain set of benchmarking tools.

Is this bad? Absolutely. What they’ve done is indefensible. That being said, I’m having massive flashbacks to my earlier career in the PC gaming industry. About a decade ago, I used to work for a boutique PC maker called “HyperSonic PC Systems”. My job was to order the latest components, benchmark them, and then tell my boss what we should and shouldn’t sell to our customers.

While I can’t speak about NVIDIA’s business practices today, back then they used to email me a new set of graphics card drivers seemingly every week. If a new game came out, or a benchmarking tool received a software update, they wanted to make sure that their card generated the biggest number. Whether it be a frame rate figure or a synthetic score, NVIDIA wanted to be on top, and they did just about anything possible to make that happen.

It saddens me that the same thing is now happening to mobile phones, but I’m honestly having a hard time caring. Smartphones have reached a point right now where most people buy them for reasons you and I would consider to be idiotic. Which company paid for the biggest billboard on the highway? Which company signed the most lucrative operator deals? Which company came out with the pink color variant first?

I’m not going to lie to you and say benchmarks don’t matter, because they do, and I love a good AnandTech review as much as the next geek, but with time they matter less and less to an ever shrinking number of people. Be honest with yourself, do you buy one phone over the other because it has a higher AnTuTu score?

No.

You take the holistic package into consideration. Does this phone have a good camera? Will this phone get software updates during the next two years? Will this phone feel good in my hand? Will this phone, with its massive screen, even fit in my pocket?

Like I said earlier, I’m not defending Samsung, and I applaud Brian Klug for the work he’s done; he’s practically my role model. But I think we, the media, have overblown this “news” to scary levels.

Update 1: Ok, now I’m mad. Samsung just published a statement on their official blog, in Korean, that says nothing fishy is going on here. Unfortunately, that’s total bullshit. Brian Klug looked at the code that dictates clock speeds on the GS4, and the string used to describe the 10% overclock of the GPU is called “BenchmarkBooster”. If that’s not enough evidence, I don’t know what is. Just man up and say you’re caught, Samsung, don’t lie to people.

Update 2: It looks like the statement in Korean was lost in translation. Samsung has now issued the statement in English to clarify the reason for tweaking the code (HT: The Guardian):

Under ordinary conditions, the GALAXY S4 has been designed to allow a maximum GPU frequency of 533MHz. However, the maximum GPU frequency is lowered to 480MHz for certain gaming apps that may cause an overload, when they are used for a prolonged period of time in full-screen mode. Meanwhile, a maximum GPU frequency of 533MHz is applicable for running apps that are usually used in full-screen mode, such as the S Browser, Gallery, Camera, Video Player, and certain benchmarking apps, which also demand substantial performance.

The maximum GPU frequencies for the GALAXY S4 have been varied to provide optimal user experience for our customers, and were not intended to improve certain benchmark results.

Samsung Electronics remains committed to providing our customers with the best possible user experience.

Like this post? Share it!

  • lalala

    i have agree 100% its been overblown.90% of consumers buying a phone will not even know or care what a benchmark is when they buy a phone.im pretty sure almost every company in any industry does something like this.using benchmarks to sell a phone is as stupid as nokia telling ppl to buy their phones simply because of their camera and not anything else!

  • Qbancelli

    Not only you rag on everything and everyone BUT Samsung, but now you’re making excuses for them.

    I have often wonder if you’re just a fanboy or Samsung is funding this site.

  • The news is, obviously, that a company acted deceptively. I’m not sure what it has to do with purchasing behavior.

    • Stefan Constantinescu

      The point I’m trying to get across is that they acted deceptively with the intent to influence the purchasing behavior of an extremely small minority of power users. It’s still a very poor thing to do, but as cynical as this may sound, it’s unsurprising. Samsung’s marketing department is a prime example of this type of thinking, calling a processor with eight physical application cores an “octa-core” chip, despite the fact that only four of them can be on at any given time.

      Would Apple ever do this? No, of course not, but people don’t buy iPhones based on what sort of 3D Mark score it gets.

      • KirkBurgess

        Actually, basically every review I read of the S4 when it was launched mentioned it was “the fastest most powerful android phone yet”. This was in mainstream media reviews, and a not insignificant amount of mainstream consumers shelling out for a top end phone simply wanted the latest and greatest device, and they chose the S4 because that’s what reviews said was “the latest and greatest.”

        Turns out it wasn’t the greatest apparently.

        • I disagree. Few reviewers said that. Most reviewed the Snapdragon 600 version, which was similar to the HTC One in terms of specs, anyway. I doubt it that this rigging had a meaningful impact for Samsung in terms of sales.
          Still, it’s condemnable and really stupid.

          • KirkBurgess

            Not in my country at least. Was the snapdragon USA only?

  • Chaka10

    You mean “don’t [continue to] lie to people [over and over, even after you’ve been caught]”?