Wednesday, October 10th 2018
Principled Technologies' Response to Allegations of Horse Manure Data Disingenuous
Principled Technologies Wednesday published its first response to allegations of flawed and misleading "independent" comparison between the $319 AMD Ryzen 7 2700X and the 66% pricier $530 (pre-order price) Intel Core i9-9900K, which Intel used in its launch event to woo gamers and investors. In its response, the company elaborated on the reasons why it tested the AMD chip with memory and cooler settings reputed hardware reviewers found sub-optimal. "One goal of this study was to test the CPUs and their graphics subsystems, not the GPUs, so we ran the tests at the most common gaming resolution (62.06%), 1920×1080," reads the response, touting a foregone conclusion that gamers with $500 8-core processors still game at 1920 x 1080. We get that they, like every CPU reviewer, are trying to simulate a CPU-limited scenario, but to justify their settings with Steam Hardware Survey data as "the most common resolution," is a disingenuous argument.
We next see Principled Technologies justify the use of NH-U14S TR4-SP3 cooler on the Ryzen Threadripper 2990WX. Noctua, in its own TDP Guide for this cooler, states that 250W TDP (which also happens to be the TDP of the 2990WX), is the design limit of this air cooler, and goes as far as to mention that an additional NF-A15 fan, which is not included with the cooler, is recommended to be able to "increase Precision Boost headroom," implying that out of the box, the cooler is already bottlenecking the 2990WX. The Core i9-9980XE, on the other hand, has a rated TDP of 165W, and Noctua provides no additional guidance for 165W TDP Core X family processors, such as the Core i9-7980XE. Principled Technologies' reasoning for memory configuration proves they either continue to lack basic knowledge on AMD Ryzen memory controller limitations, or are deliberately disregarding it in an attempt to cripple AMD chips.The Principled Technologies response contains no explanation as to why they used four memory modules, or a 4-module + dual-rank module configuration for mainstream desktop processors that feature dual-channel memory controllers. It has been common knowledge for close to two years now, that AMD Ryzen processors offer stunted performance when saddled with dual-rank memory modules, more so when all four memory slots of the motherboard are populated.
Simply mentioning the DRAM clock as 2933 MHz doesn't even paint half the picture. AMD platform motherboard BIOSes are programmed to automatically find stable memory timings for a given memory clock. Principled Technologies is quick to justify using the Wraith Prism cooler for the Ryzen 7 2700X "because AMD says so," but the same AMD document (Reviewer's Guide) that recommends Wraith Prism also provides a detailed account on the limitations of Ryzen's memory controllers. The controllers cannot handle 4-module + dual-rank configurations above DDR4-1866, 4-module single-rank (effectively dual-rank) above DDR4-2400, without severely loosening up memory timings to compensate for lack of stability.
2-module single-rank is not only an AMD-recommended configuration, but also 16 GB remains an overwhelmingly more popular memory amount than 32 GB vide Steam Hardware Survey (the same survey they used to justify 1080p game testing). In 2-module single-rank, a Ryzen processor will let you spool memory clocks all the way up to DDR4-3400, and memory modules exist that achieve that clock with reasonably tight timings.It's good that Principled Technologies admitted the crime they committed testing the Ryzen 7 2700X with half its cores effectively disabled using the "game mode" toggle in Ryzen Master that no CPU reviewer tests with, and offer to re-test the processor. We wish that while they're at it, they also update their Ryzen Master utility to re-test the Ryzen Threadripper 2990WX with Dynamic Local Mode. The comparison with the re-badged Core i9-9980XE significantly will change.
Comparing a $319 processor with a 66% pricier $530 processor is disingenuous to begin with, and the only way you'll find out if the i9-9900K offers that much more performance is to wait for reviews from independent sources. So far there's none.
We next see Principled Technologies justify the use of NH-U14S TR4-SP3 cooler on the Ryzen Threadripper 2990WX. Noctua, in its own TDP Guide for this cooler, states that 250W TDP (which also happens to be the TDP of the 2990WX), is the design limit of this air cooler, and goes as far as to mention that an additional NF-A15 fan, which is not included with the cooler, is recommended to be able to "increase Precision Boost headroom," implying that out of the box, the cooler is already bottlenecking the 2990WX. The Core i9-9980XE, on the other hand, has a rated TDP of 165W, and Noctua provides no additional guidance for 165W TDP Core X family processors, such as the Core i9-7980XE. Principled Technologies' reasoning for memory configuration proves they either continue to lack basic knowledge on AMD Ryzen memory controller limitations, or are deliberately disregarding it in an attempt to cripple AMD chips.The Principled Technologies response contains no explanation as to why they used four memory modules, or a 4-module + dual-rank module configuration for mainstream desktop processors that feature dual-channel memory controllers. It has been common knowledge for close to two years now, that AMD Ryzen processors offer stunted performance when saddled with dual-rank memory modules, more so when all four memory slots of the motherboard are populated.
Simply mentioning the DRAM clock as 2933 MHz doesn't even paint half the picture. AMD platform motherboard BIOSes are programmed to automatically find stable memory timings for a given memory clock. Principled Technologies is quick to justify using the Wraith Prism cooler for the Ryzen 7 2700X "because AMD says so," but the same AMD document (Reviewer's Guide) that recommends Wraith Prism also provides a detailed account on the limitations of Ryzen's memory controllers. The controllers cannot handle 4-module + dual-rank configurations above DDR4-1866, 4-module single-rank (effectively dual-rank) above DDR4-2400, without severely loosening up memory timings to compensate for lack of stability.
2-module single-rank is not only an AMD-recommended configuration, but also 16 GB remains an overwhelmingly more popular memory amount than 32 GB vide Steam Hardware Survey (the same survey they used to justify 1080p game testing). In 2-module single-rank, a Ryzen processor will let you spool memory clocks all the way up to DDR4-3400, and memory modules exist that achieve that clock with reasonably tight timings.It's good that Principled Technologies admitted the crime they committed testing the Ryzen 7 2700X with half its cores effectively disabled using the "game mode" toggle in Ryzen Master that no CPU reviewer tests with, and offer to re-test the processor. We wish that while they're at it, they also update their Ryzen Master utility to re-test the Ryzen Threadripper 2990WX with Dynamic Local Mode. The comparison with the re-badged Core i9-9980XE significantly will change.
Comparing a $319 processor with a 66% pricier $530 processor is disingenuous to begin with, and the only way you'll find out if the i9-9900K offers that much more performance is to wait for reviews from independent sources. So far there's none.
22 Comments on Principled Technologies' Response to Allegations of Horse Manure Data Disingenuous
Reminds me of CTS-labs debacle. Go visit their website. Under available publications, it only has “AMDFlaws (3/12/2018) - Publication Summary Coming Soon...”
tl;dr: Intel is required to say that benchmarks that favor their stuff unfairly with certain words per FCC settlement of AMD v Intel. Said phrase is found at the end of the PT document. It wasn't an accident. It was Intel up to their old shit again.
That's the official word for "turd", or better..."bull shit" ? :roll:
Off topic using this as reference, if you play at any higher res than 1080p (specially 4K), then you don't need a $500 CPU since the GPU will be the bottleneck and having an expensive CPU wont give you more FPS right? As the res increases you should spend more in GPU and less in CPU right? Does it make any sense or I got it wrong?
And since Intel processors doesn't ship with coolers anymore, they used a Noctua cooler lol.
Have a guess which processors were throttling because of thermal limits of a stock coolers.
Even Steam Survey still showing an overwhelming (!) amount of sub 1440p gamers is 'not relevant enough' somehow. And here's the kicker: on Steam Survey, many of the 'ultrawide' resolutions you see are people who use two separate screens side by side and only game on one ;) Its also completely not true that virtually every gaming oriented monitor out these days is first and foremost high refresh rate even before high resolution... :D
Oh well. Its an editorial disguised as news, let's just keep it at that I guess... The gist of it is still sound.
Having Game Mode even available for 2700X is an interesting decision on AMD's part though.
I believe that a lot of gamers play in 60 Hz TVs (either 1080p or 4K) than in high refresh rates monitors, specially in the third world these monitors are not popular at all. Would be a nice idea that the next TPU poll were if you play above 60 Hz or not. 2080 is the second fastest GPU and at 1440p is not a bottleneck for CPU sometimes.
Do you have a high refresh rate monitor? If not, what is the point of higher FPS.
Most techie folks understand that just about any freshman college IT student could fairly easily run a benchmark. It's not rocket science. About the only hurdle to anyone starting their own benchmark company is just financial access to acquire (at least temporarily) the hardware to test. Setting up a PC to bios defaults, running a benchmark, and recording the results, is something I could have easily done at 13-14 years old as just a semi-techie kid. So the bar is pretty low. And with anything, when you hire what claims to be a 16 year professional corporation to perform what is a fairly simple task, they open themselves up to a lot of negative attention when they mess up. And there were at least 3 or 4 bad oversights/questionable decisions in this testing. Probably not a company I'll focus on when looking at benchmarks for my next CPU purchasing decision.
Intel shares some of this blame too. What it looks they were aiming to do in the first place is disappointing.
Is PT owned by Intel? Of course not! They just supply PT with all the software engineers.