UL Benchmarks Kicks Huawei Devices from its Database over Cheating

[btarunr]

UL Benchmarks de-listed several popular Huawei devices from its database over proof of cheating in its benchmarks. Over the month, it was found that several of Huawei's devices, such as P20 Pro, Nova 3, and Play; overclocked their SoCs while ignoring all power and thermal limits, to achieve high benchmark scores, when it detected that a popular benchmark such as 3DMark, was being run. To bust this, UL Benchmarks tested the three devices with "cloaked" benchmarks, or "private benchmarks" as they call it. These apps are identical in almost every way to 3DMark, but lack the identification or branding that lets Huawei devices know when to overclock themselves to cheat the test.

The results were startling. When the devices have no clue that a popular benchmark is being run (or if has no way of telling that 3DMark is being run), it chugs along at its "normal" speed, which is 35% to 36% lower. The rules that bind device manufacturers from advertising UL's 3DMark scores explicitly state that the device must not detect the app and optimize its hardware on the fly to ace the test. Huawei responded to UL by stating that it will unlock a new "performance mode" to users that lets them elevate their SoCs to the same high clocks for any application.



View at TechPowerUp Main Site

 

[First Strike]

It's so typical of Huawei, and they never apologize for it.

Edit: and they have every convenience to do that, because they are the only manufacturer on earth that uses Kirin SoCs. If they decided to cheat, then no single Kirin on the planet would be honest.

 

[champsilva]

It's so typical of Huawei, and they never apologize for it.

They apologize by saying: others do, why cant we?

 

[bubbly1724]

Huawei, how about just put more Mali GPU cores in your SoCs instead of cheating? Your SoCs have always been severely underpowered in the GPU segment.

 

[DeathtoGnomes]

It's so typical of Huawei, and they never apologize for it.

typical of a company willing to do anything to make a buck or two. Asian companies do things like this more often, some might say normal business practice.

 

[Prima.Vera]

Actually Anandtech did a very good investigation regarding this, even went to an interview with a cheater's Executive.
Very good reading below:

https://www.anandtech.com/show/13318/huawei-benchmark-cheating-headache

Also it looks like Huawei are the biggest cheats in the Mobile industry:
https://www.engadget.com/2018/08/20/huawei-caught-passing-off-dlsr-pictures-as-phone-camera-samples/

 

[DeathtoGnomes]

Actually Anandtech did a very good investigation regarding this, even went to an interview with a cheater's Executive.
Very good reading below:

https://www.anandtech.com/show/13318/huawei-benchmark-cheating-headache

oh my! they can benchmark bananas there! (@robot zombie )

 

[sepheronx]

typical of a company willing to do anything to make a buck or two. Asian companies do things like this more often, some might say normal business practice.

What about the car manufacturers from Germany?

Corporate cheating isn't secluded to Asian countries.

TBH, I am not surprised by this. To be able to get ahead, is usually done by unscrupulous activities. The good thing is, after this coming to light, it may force companies to be more honest in their future devices. May also be a good reason to start reducing prices of their products they lied about.

Edit: I am a terrible speller.

 

[blobster21]

it's only a matter of renaming quake.exe to quike.exe to reveal this dirty trick.

 

[R0H1T]

What about the car manufacturers from Germany?

Corporate cheating isn't secluded to Asian countries.

TBH, I am not surprised by this. To be able to get ahead, is usually done by unscrupulous activities. The good thing is, after this coming to light, it may force companies to be more honest in their future devices. May also be a good reason to start reducing prices of their products they lied about.

Edit: I am a terrible speller.

Do we count Monsanto, Apple or certain banks from you know when? I know the point you're trying to make, but there's not much point bringing car makers in this debate. As for the larger point about Asian companies, yeah everyone does & they're all wrong ~ corporate greed is wrong!

Nope, the tax dodgers will always get away with it, one way to keep them in check is to make corporations pay through their noses when caught in a scam or lie. It rarely happens for a domestic company like Apple in US or Huawei in China, but it should & that'll hopefully set a precedent.

 

[DeathtoGnomes]

What about the car manufacturers from Germany?

Corporate cheating isn't secluded to Asian countries.

TBH, I am not surprised by this. To be able to get ahead, is usually done by unscrupulous activities. The good thing is, after this coming to light, it may force companies to be more honest in their future devices. May also be a good reason to start reducing prices of their products they lied about.

Edit: I am a terrible speller.

you're so right it isnt, but it is more common there to see cheating, espionage, even sabotage. If there were awards for being unscrupulous, chances are an Asian company would win before a German corporation would. There were more wars started over Tea and Suishi than Beer and a Brezel.

 

[Assimilator]

What about the car manufacturers from Germany?

Corporate cheating isn't secluded to Asian countries.

What about ATI, an American company, who did this in the Quake III benchmark way back in the 8500 Pro days? Companies everywhere are run by unethical weasels, that's why we need free press to help keep them accountable.

This sort of benchmark gaming is going to become a massive thing in the mobile market where heat and power constraints are so much more vital. It's why I don't even bother with mobile "GPUs" (calling them GPUs is an insult to real GPUs that NVIDIA and AMD produce).

 

[btarunr]

What about the car manufacturers from Germany?

Corporate cheating isn't secluded to Asian countries.

Volkswagen did cough up $8 billion for TDIgate. What's Huawei paying its customers/regulators?

 

[Prima.Vera]

it's only a matter of renaming quake.exe to quike.exe to reveal this dirty trick.

I like quak.exe better!!

 

[IceShroom]

UL need to ban Apple also.

 

[Vayra86]

Volkswagen did cough up $8 billion for TDIgate. What's Huawei paying its customers/regulators?

I don't know, but I do know Huawei phones aren't responsible for cancerous amounts of NOx (versus cheating a few imaginary benchmark points).. I know its a minor detail, but hey...

And I also know Volkswagen & co. were happily continuing the practice too. All of their subsidiaries fell one after the other. It was so common they even forgot about it. Never mind the fact they tried to lobby their way out first as well. (We 'can't' reduce emissions by thát much, make it half pls!')

Honestly that is a kind of brutality Huawei isn't even coming close to here... It's completely wrong to say 'Oh its another Asian company, see, they do it again'... when Western companies do it, they don't even apologize, they actually take it another step further and find new ways to continue going at it.

 

[Caring1]

Volkswagen did cough up $8 billion for TDIgate. What's Huawei paying its customers/regulators?

VW was a scapegoat, all vehicle manufacturers fudge emission and fuel economy figures, if you purchase based on them you may as well wear a bright yellow jacket so they can see you coming.

 

[sepheronx]

Volkswagen did cough up $8 billion for TDIgate. What's Huawei paying its customers/regulators?

What about VW? They paid cause they were first to get caught. Rest didn't. Add to that, how much has GM been given by the same government that caught VW (BTW, it wasn't Germany. It was US) in terms of handouts just to keep it alive? How much has GM paid for cheating? US hates competition. They love control. To obtain control is to blame others of evil doings that they themselves do. They will go as far as bomb and kill people over it too!

Do we count Monsanto, Apple or certain banks from you know when? I know the point you're trying to make, but there's not much point bringing car makers in this debate. As for the larger point about Asian companies, yeah everyone does & they're all wrong ~ corporate greed is wrong!

Nope, the tax dodgers will always get away with it, one way to keep them in check is to make corporations pay through their noses when caught in a scam or lie. It rarely happens for a domestic company like Apple in US or Huawei in China, but it should & that'll hopefully set a precedent.

There is every point to add car makers to this debate. Much like bringing in any example of corporate cheating by western companies who may do even worst (as you said, Monsanto) compared to this Chinese phone maker yet get away with it, while everyone's attention is on evil easterners and their corporate greed. That's my point. My point is rather aimed at someone blaming an entire region of doing what we do as well.

Just pointing out hypocrisy. That is all.

 

[Gasaraki]

Comparing this to the VW sandal is stupid. VW cheated government regulations. Every auto manufacture that cheats on regulations have/are being fined, it's just not as big of a news because it doesn't affect milions on cars worldwide over many years like VW's scandal.

UL is just a benchmark software. Samsung and other brands have cheated on benchmarking software before and when the press and public call them out on it, they stop. This is the whole point of reviews and UL banning them from their list. Good job UL.

 

[sepheronx]

Comparing this to the VW sandal is stupid. VW cheated government regulations. Every auto manufacture that cheats on regulations have/are being fined, it's just not as big of a news because it doesn't affect milions on cars worldwide over many years like VW's scandal.

UL is just a benchmark software. Samsung and other brands have cheated on benchmarking software before and when the press and public call them out on it, they stop. This is the whole point of reviews and UL banning them from their list. Good job UL.

And a benchmark does? Does it somehow prevent the phone from making phone calls or playing some crappy "Run" knockoff game (Since thats pretty much all on a mobile phone as it is), or somehow prevent one from reading stuff off of it or watching stupid videos online? No.

Before calling a point stupid; it may benefit you to go back and re-read the basis point of what was being discussed. Especially when someone outright goes near racist in trying point out something that is rather hypocritical because it is done by nearly everyone else (corporate cheating and lying).

Think before typing.

 

[DR4G00N]

Well it's not like these mobile benchmarks are reliable in any way since they are so easy to cheat in the first place.
On android simply change Simulate color space to monochrome and watch your graphics scores improve by 10-20%.

 

[R-T-B]

it's only a matter of renaming quake.exe to quike.exe to reveal this dirty trick.

Pepperidge farm remembers.

Volkswagen did cough up $8 billion for TDIgate. What's Huawei paying its customers/regulators?

A free performance mode they didn't have before? That's something. They didn't violate the law, afterall...

 

[Ferrum Master]

What about ATI, an American company.

They were canucks, keep your facts straight .

 

[dozenfury]

UL need to ban Apple also.

That was apples (no pun intended) and oranges to what Huawei did. Huawei is directly manipulating benchmarks through cheating. Apple had code that slowed phones after years to try to extend the battery. Totally different since Apple wasn't manipulating benchmarks to try to influence purchasing decisions. And more importantly there are very practical reasons to try to reduce battery usage on older phones, as anyone who's ever had a phone with a battery over a year or two old knows. They took some heat for the design decision, but that's not comparable to outright fraud. And I would call inserting code to specifically detect 3dMark running and then oc'ing to the hilt (along with other steps) to manipulate benchmark results as crooked as it comes. Maybe it's a good thing they are on the "bad" list from a US legal standpoint, since sounds like they need to stay on it...

 

[IceShroom]

That was apples (no pun intended) and oranges to what Huawei did. Huawei is directly manipulating benchmarks through cheating. Apple had code that slowed phones after years to try to extend the battery. Totally different since Apple wasn't manipulating benchmarks to try to influence purchasing decisions. And more importantly there are very practical reasons to try to reduce battery usage on older phones, as anyone who's ever had a phone with a battery over a year or two old knows. They took some heat for the design decision, but that's not comparable to outright fraud. And I would call inserting code to specifically detect 3dMark running and then oc'ing to the hilt (along with other steps) to manipulate benchmark results as crooked as it comes. Maybe it's a good thing they are on the "bad" list from a US legal standpoint, since sounds like they need to stay on it...

I am talking about how Apple get high score. Apple clearly optimize(like not running full load, less quality) for benchmark so that they look powerful than others(Benifit of close source).