AMD Expresses its Displeasure Over Intel's PT Benchmarks for 9th Gen Core

[btarunr]

AMD gave its first major reaction to the Principled Technologies (PT) controversy, in which it came out strongly against the questionable methods PT employed, in its performance comparison between the Core i9-9900K and AMD Ryzen 7 2700X, in addition to certain other Ryzen Threadripper series products. In its response, AMD made its official position on controversy clear - it is not happy with PT.

AMD prepared a long list of flaws with PT's original testing, and the areas where it did not correct the mistakes in its second testing. The company also put out a list of its own "best practices" for comparative benchmarking, which prescribes "sanitizing the operating system," "sanitizing the platform" for stock vs. overclocked testing, "sanitizing the data," and to not create a vast disconnect between the test environment and the real-world.



View at TechPowerUp Main Site

 

[atomicus]

Their orginal methodologies were so flawed it really begs the question how they've surivived this long as a company. Such basic inept errors were made, it would surely call in to question all their previous work over the years. If I were a client of theirs I'd certainly be looking elsewhere. It's beyond farce.

 

[R0H1T]

Yeah no doubt about it, it's like the conclusions were preordained, sadly such practices are far too common & swept under the rug with the excuse that everyone does it

 

[Slizzo]

Go get 'em AMD.

And to think, AMD has contracted PT in the past to do testing on their stuff as well...

https://www.principledtechnologies.com/portfolio-marketing.php#AMD

 

[HTC]

I'd like if AMD would, with whatever the next flagship CPU it launches would make 2 separate testings:

- the right way, as described in the 2nd pic of the OP
- the wrong way, as made in version one of PT's review but, in this occasion, favoring AMD instead of Intel, obviously

Would be interesting to see the comparison ...

 

[Prince Valiant]

I'd like if AMD would, with whatever the next flagship CPU it launches would make 2 separate testings:

- the right way, as described in the 2nd pic of the OP
- the wrong way, as made in version one of PT's review but, in this occasion, favoring AMD instead of Intel, obviously

Would be interesting to see the comparison ...

I'd like to think AMD's PR team is intelligent enough not to make themselves hypocrites.

 

[Casecutter]

Wow ... Spectre, cheating, and running hot like a new i9 9900K even though Intel gave them solder!
The show must go on...

 

[Lightning]

Who cares about some rando website ? TPU review is up.

 

[Ubersonic]

"Enclosed thermal environment unsuitable for comparative testing" - What the systems had cases as opposed to open rigs? IMO that's a plus as it's mimics real world use..


Testing between AMD/Intel CPUs was so much easier when they used the same socket/platform lol, such a shame Intel had to block that >.>

 

[eidairaman1]

Intel, of course bribing.

AMD should have people doing a slew of tests live with specs all posted on Youtube.
Obvious they can't trust PT for their unprofessional style.

 

[Darmok N Jalad]

PT was used to give Intel an out. It’s not false advertising, as an independent company did the testing. Everyone will quickly forget the company “Principled Technologies,” just like they did with CTS-Lab, a company that still hasn’t published the documents on the AMD flaws it found, and it has done no other research since.

 

[jsjones008]

"Enclosed thermal environment unsuitable for comparative testing" - What the systems had cases as opposed to open rigs? IMO that's a plus as it's mimics real world use..


Testing between AMD/Intel CPUs was so much easier when they used the same socket/platform lol, such a shame Intel had to block that >.>

I think they're referring more to the specific case PT was using. The case they were using has very little direct airflow, and they were using the stock downdraft cooler on the AMD benches. This created a scenario where the CPU was not being properly cooled for the setup.

 

[cdawall]

I mean amd isn't exactly the king of fair in pr events on their own side, but this webpages delivery was a trashcan fire.

 

[jsjones008]

PT was used to give Intel an out. It’s not false advertising, as an independent company did the testing. Everyone will quickly forget the company “Principled Technologies,” just like they did with CTS-Lab, a company that still hasn’t published the documents on the AMD flaws it found, and it has done no other research since.

The thing with PT is like you said, they were hired by Intel.

My curiosity on the matter is if PT's original testing was carried out as Intel asked, purposefully giving them the numbers they wanted, and then the follow up was OK'd by Intel to try and stop the bleeding.

I know Steve from Gamer's Nexus interviewed one of the founders of PT, and according to him they tested it how they wanted and how they thought was appropriate, but if a big corporation is paying you big money for something, you're not going to throw said corporation under the bus.

 

[Upgrayedd]

Still tho "Gaming Mode" .. seriously..you call it game mode then get pissed when they use it for games? Why even have it then?

 

[opteron]

Go get 'em AMD.

And to think, AMD has contracted PT in the past to do testing on their stuff as well...

https://www.principledtechnologies.com/portfolio-marketing.php#AMD

Well, $$$ talks louder. PT will surely get banned by AMD for sure, given the high fees $$$ paid by Intel.


"sanitizing the operating system," "sanitizing the platform" for stock vs. overclocked testing, "sanitizing the data,"

let's santize PT...

 

[moproblems99]

Still tho "Gaming Mode" .. seriously..you call it game mode then get pissed when they use it for games? Why even have it then?

Game Mode is for Threadripper. Why AMD gave it to Ryzen, who knows.

 

[crazyeyesreaper]

Still tho "Gaming Mode" .. seriously..you call it game mode then get pissed when they use it for games? Why even have it then?

ITs because the game mode is meant for legacy titles. Some games won't even run if more than 4 cores are present or have issues. Supreme Commander comes to mind.

 

[Vayra86]

Well constructed response. I hope it gets enough exposure...

 

[HTC]

I'd like to think AMD's PR team is intelligent enough not to make themselves hypocrites.

They don't need PT for that: remember Polaris launch, where they claimed 2 cards surpassed the 1080 in AotS @ a much lower cost?

How did that turn out?

 

[Fluffmeister]

Why get upset anyway? This duopoly can't fail whatever the other party does.

 

[Aquinus]

Why get upset anyway? This duopoly can't fail whatever the other party does.

So long as everyone is content to rely on x86 for running their applications, it's likely going to remain that way. It's not like there are really any good options, even if the software support is there (like with a lot of ARM variants.)

 

[Pewzor]

I think they're referring more to the specific case PT was using. The case they were using has very little direct airflow, and they were using the stock downdraft cooler on the AMD benches. This created a scenario where the CPU was not being properly cooled for the setup.

While Lyle trying to blow the dust off the running 2700x bench using a heat gun during benchmarking session.

They don't need PT for that: remember Polaris launch, where they claimed 2 cards surpassed the 1080 in AotS @ a much lower cost?

How did that turn out?

You mean Raja who said it... and you know how that turned out? Intel hired Raja with offer he can't refuse.
Says a lot about Intel eh.
Raja probably botched Vega because Intel told him to do that, and if he do it right Intel will cut him a fat check and give him a management position so Intel can take on much weaker GPU maker AMD before taking on Nvidia...
Everything makes sense now.

 

[uuuaaaaaa]

This speaks for itself:

Software and workloads used in performance tests may have been optimized for performance only on Intel® microprocessors. Performance tests, such as SYSmark* and MobileMark*, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit https://www.intel.com/benchmarks.

Intel is a sponsor and member of the BenchmarkXPRT Development Community, and was the major developer of the XPRT family of benchmarks. Principled Technologies is the publisher of the XPRT family of benchmarks.

Source:
https://www.intel.com/content/www/us/en/benchmarks/intel-product-performance.html

 

[HTC]

You mean Raja who said it... and you know how that turned out? Intel hired Raja with offer he can't refuse.
Says a lot about Intel eh.
Raja probably botched Vega because Intel told him to do that, and if he do it right Intel will cut him a fat check and give him a management position so Intel can take on much weaker GPU maker AMD before taking on Nvidia...
Everything makes sense now.

Apparently, Raja was deprived of 2 / 3 of the engineers assigned to work on Vega early during Vega's development, which were re-assigned to work with one of CPU projects (dunno which Zen version), or so i've read: can't seem to find where i read that, atm. This is the reason Vega took way way longer to be introduced VS the planned time and that may, quite possibly, have a direct impact in it's current performance.