Is CPU game benchmarking methodology (currently) flawed?

[HTC]

After watching this video, i'm not so sure anymore:

He claims that the "ancient" FX8350 became faster the the I5 2500K simply by testing with a much much faster graphics card and has shown that, with each increase in GPU performance, the distance to the 2500K narrows, until it's reversed. To be fair, i personally have some doubts because there are some variables he didn't take into account, such as motherboards differences, memory differences, storage speed differences, used drivers differences that may or may not contribute to the overall result, not to mention that in the final part, he used the results of the FX8370 instead (due to not having the FX8350 on the benchmarks).

This finding contradicts the current thinking that in order to remove the GPU out of the equation, one needs to test @ lower resolutions / details so that one can say X processor is better then Y processor @ gaming with the results one gets after the tests, and any changes made to the graphics card will never change the outcome. I understand the logic ... but is this really true?

You may think: why am i even thinking about this with "ancient" CPUs? Because you can use the same principle on current CPUs: if the testing methodology is indeed wrong, then all those referencing reviews with it are being misled (even though not intentionaly) and this means a new way to test a gaming CPU must be found and this one scrapped.

And so i propose that our very own @W1zzard tests this (@ his convenience) and answer this question so that there's zero doubts. Use a 2500K and a 8350 and pair it with a 1080Ti, a 980Ti, a 680Ti (2 generation gap between each card, i think): only change the boards and CPUs while trying to keep the rest, if @ all possible (including drivers), so that there are less variables to interfere with the final results.


EDIT

I think this topic can be closed now because someone has tested this and found the 2500K to be faster on all but 1 title (of 16 tested):

It seems the methodology still holds: as such, no point in keeping this topic open, i think.

EDIT #2

The plot thickens ...

I do admit that i missed the very important bit that different games being tested was the point. I'm also to blame in the sense that i took HUB's numbers as a "gospel" (like Adored calls it) in order to "prove" that the methodology wasn't flawed after all.

His more recent video shows that current CPU benchmarking indeed is flawed (in this title @ the very least). Let me however say that what i'm trying to show is not Intel VS AMD CPU performance bit but the difference you get in a supposedly CPU bottlenecked game when changing from a nVidia card to an AMD one on BOTH CPUs:

View attachment 85800

You can't have it both ways: either the CPU is being bottlenecked or it isn't. Adored showed that both Intel and AMD benefited from changing from nVidia to AMD in DX12 and both lost in DX11. That the gap shrunk in DX12 is not the issue i'm trying to address: that there is a gap is the issue i'm trying to address. This proved that the CPU wasn't being bottlenecked after all, or there wouldn't have been an increase in both CPUs: there was another variable that wasn't being accounted for.

But there are variables here, because of which i think more testing is definitely required: he tested crossfire VS single card and that introduces another variable that doesn't have to be present: CF scaling. It seems RotTR isn't the only game because it happens in DX12's The Division, that i know of so far.

In fact, you don't even need to use 2 CPUs to test if this is true or not, but you do need an nVidia and an AMD cards, as well as RotTR game: just run RotTR using DX11 and DX12 with settings you're absolutely sure will bottleneck the CPU with both cards @ stock and then with the highest overclock on the cards you can get.

If the CPU is bottlenecked "properly", then going from a stock nVidia card to an OCed one should yield margin of error differences and the same should be true for stock AMD card VS OCed one but, if comparisons between manufacturers are allot higher then margin of error, then you'll have your proof right there.

There's also this video that i found very interesting about CPU overhead in nVidia VS AMD:

 

[Aenra]

Aaah, saw that. Yes, i actually believe/agree with it; to an extent that i'd need proper, hard data to be convinced otherwise (ie not the youtube/google kind).
Why? I lack the technical knowledge, granted, but for starters it's way too logical for me to dismiss it and also, empiricity. Have had many AMD chips, have sure worn them out testing things (games included). Can definitely attest to a marked improvement in later years.

Also, why are you surprised? Because Intel told us that going back in time to 2009* is best? And never mind our ignoring almost a decade of technological progress?

*that's when the first quad core came out. Yeap.

edit: the only thing i will not comment on is the TPU mention in this video. Because i am unsure as to whether there was copy-pasting involved in that article, or the OP actually believed what he was typing. It's hard to analyze intentions or infer meaning from just a few lines; i read, do my own thinking and move on.

 

[toilet pepper]

I'm using an oced fx8320 for years and it still works but I know I can get more coz I can feel some hiccups. It really depends on the engine used and testing methodology. Scenes where cpu process is important are really hard to replicate. Games like GTA V, WD2 or any multiplayer games can't be benchmarked. No idea whats the beat way to test it.

 

[MrGenius]

Very eye-opening. Thanks for sharing that with us.

 

[qubit]

He seems to be conflating two things: low res testing and performance improvement with older AMD CPUs over time to make for a bogus argument.

Now, it's 5am here and I'm too tired to watch the video again and analyze things in detail, but it seems that the improved AMD FX performance is coming from newer games which are better optimized for it (and no doubt better drivers and perhaps OS patches) and using that as an argument to claim that low res testing is wrong. That's bullshit. It's obvious that if you want to know what framerate a CPU can achieve, then the graphics card must be taken out of the picture and hence tested at low resolution.

Heck, I remember Unreal Tournament 2003 had a special benchmarking mode which really did take the graphics card out of the picture. It simulated an infinitely fast graphics card by not actually sending the draw calls to it, instead just returning immediately, so all you saw was a static picture while the game benchmark ran. AFAIK this is the only game to have this feature and more games should have it too.

What he should be doing is testing those same games that were used when FX was initially benchmarked, but with modern graphics cards and drivers run at low resolution, but he doesn't do that. That's not comparing like with like and therefore invalid. Judging by those YouTube comments, it looks like he's fooled a lot of people and managed to gain thousands of subscribers, but not me.

He sounds like a real AMD apologist and fanboy.

 

[HTC]

He seems to be conflating two things: low res testing and performance improvement with older AMD CPUs over time to make for a bogus argument.

Now, it's 5am here and I'm too tired to watch the video again and analyze things in detail, but it seems that the improved AMD FX performance is coming from newer games which are better optimized for it (and no doubt better drivers and perhaps OS patches) and using that as an argument to claim that low res testing is wrong. That's bullshit. It's obvious that if you want to know what framerate a CPU can achieve, then the graphics card must be taken out of the equation and hence tested at low resolution.

Heck, I remember Unreal Tournament 2003 had a special benchmarking mode which really did take the graphics card out of the picture. It simulated an infinitely fast graphics card by not actually sending the draw calls to it, instead just returning immediately, so all you saw was a static picture while the game benchmark ran. AFAIK this is the only game to have this feature and more games should have it too.

What he should be doing is testing those same games that were used when FX was initially benchmarked, but with modern graphics cards and drivers run at low resolution, but he doesn't do that. That's not comparing like with like and therefore invalid. Judging by those YouTube comments, it looks like he's fooled a lot of people and managed to gain thousands of subscribers, but not me.

Actually, he didn't do any tests @ all: what he did was grab the results of tests done on the same website and notice how it changed from being X slower to Y faster, by trading the card for much faster ones. BUT, because this wasn't done using the same equipment (boards, storage, games, memory speeds) there's a chance that this could be totally missleading OR could be actually even more pronounced: that's what this review would be hoping to answer, definitively.

You do however have a point when you say newer games benefit better from more cores / threads VS older games and it's also an aspect @W1zzard should consider if / when he makes this review: use older and newer titles in an effort to show if this happens only on more multithreaded games or accross the board.

 

[qubit]

Haha, guess I put it a little ambiguously lol, my bad. Indeed he didn't run any of those benchmarks himself. I meant to say that if he really wants to confirm this for himself then he should run a set of benchmarks himself to properly compare. If not, then at least find comparable reviews, but that's hard to do because things change over time, such as drivers etc, therefore he should really do it himself under controlled conditions. I strongly suspect that if he did this then the 2500K would still come out on top, but that's not what he wants to tell us.

Think, just how many thousands of enthusiasts there are out there motivated to compare products. They would start to notice the FX improvements, which would get picked up by the tech press such as TPU, but it's not happened. Therefore, what he's saying is a crock and I think he's got an agenda to make it look like AMD's products are getting hard done by Intel and everyone else, when the real blame lies with the products themselves.

Heck, just look at the Ryzen launch, all full of hype and promise. AMD and their partners, including Microsoft have had years to work with them to ensure a reasonably bug-free launch, but what do we see? BIOS issues preventing memory modules from working properly and a Windows 10 bug lowering overall performance due to SMT issues. How did Microsoft and more importantly AMD, not notice something so blatantly obvious before launch and just get it fixed? Especially since Windows 7 doesn't have this problem.

I tell you, if I'd dropped a grand+ plus on a Ryzen upgrade (CPU, mobo, RAM, cooler) I'd be pissed at seeing problems like this and might ask for my money back. I expect the thing to work properly out of the box. This is typical AMD and why I have no confidence in them. Funny how Intel CPUs haven't dropped in price much since Ryzen's launch and its "incredible" value proposition, isn't it? Yes, I think Ryzen does have that potential, but the launch has been poorly executed, ruining it. Looks like we'll have to wait for Ryzen v2 for an improved IPC that can compete head-on with Intel in games, but it's a good start, bugs notwithstanding.

There's also another thing that makes him quite the AMD apologist. When buying hardware, one should almost always buy what performs best right now, rather than a promise of better performance in the future. It's not guaranteed, there's no timescale and by the time that the improvement comes, if it does, the hardware will be out of date anyway. So, who cares if the FX works better than a 2500K now? They're both obsolete and can't even be purchased new any more.

I have literally put this philosophy into practice today. You might remember that my Palit GTX 1080 died a couple of months ago and I got a refund on it. Well, I was waiting for the 1080 Ti to come out before buying its replacement. What do I see? A card that costs an obscene £700+, is "only" 35% faster than a 1080 and an overclocked 7700K isn't fast enough to max the card out at 1080p a lot of the time (check TPU's review). Now, I still have a 2500K (at a rather non-enthusiast stock speed, but don't tell anyone ) which is a lot slower than TPU's test rig, so I passed on the 1080 Ti and bought another 1080 today, taking advantage of the cheaper prices.

I've actually bought two models, as I can't quite tell which one I'm gonna keep just from the reviews and will return one to Amazon. Noise is all-important for me, so it's a toss-up between the Palit GTX 1080 GameRock (same as I had before) at £500 or the Zotac GTX 1080 AMP! Extreme at a cool £600 which @W1zzard said is the best GTX 1080 he's ever tested, in his review. Yes, it's somewhat more than I wanted to pay for a 1080 right now, but I really like it, so I'm biased towards keeping it, despite that steep £100 premium. If the coil whine performance is better on the Zotac then keeping it is a no-brainer.

 

[RejZoR]

That's not entirely true. Some people buy systems long term and for those it does matter how well hardware ages. And for those people DX12/Vulkan performance does matter. And so does the number of cores. You can be assured R7 1800X will age way better than 7700K. Clock per clock difference is minimal, but Ryzen is rocking twice as many cores. It doesn't require a genius to predict that it'll perform better for longer.

 

[Vya Domus]

Many games run in engine limitations and quirks which literally no one mentions. Also running lower resolutions is hardly a good method to show CPU performance in games , or rather it just simply doesn't tell you anything. It's the number of draw calls that hammer down a processor , running a lower resolution simply puts less stress on the GPU , you get no extra information whatsoever about the CPU than if you were running the game on 3x4K panels.

Sure you might say that by doing this the CPU is preparing as many frames as it can , true but then you run in those engine limitations I talked about , so you still don't know the whole story. Take a look at Doom , it's capped at 200fps , yet because it is very well written it will happily prepare you those 200 frames even on slower CPU's.

So in the end the only thing you have to make sure when testing is to max out the game , which some reviewers oddly enough don't do it all the time

 

[the54thvoid]

I'm assuming that's the Adored person? Enough said.

Can people stop posting links to revenue generating guff from non experts with opinions they get paid for?

Summarise a point and please give a simple link. Stop embedding YT links and giving no further info, it's a pain in the ass.

 

[IRQ Conflict]

it's a pain in the ass.

So? Move along.

 

[Aenra]

While qubit makes valid points, the point remains.
- The more the software moves towards the direction we'd been expecting it to move, the more these old 'dinosaurs' / "horrible" (read:AMD) CPUs get the juice squeezed out of them, it being the point all along.
- The more "gamerzzzz" end up defining this market, the more skewed and convoluted said direction will be. If not halted that is, in terms of mentality. Case in point being the re-re-rehashing of 4cores, ad infinitum. Not to exclude AMD from this mind, they too are planning to sell those.
(when progress is defined by the needs of 15yr olds using software made for 15yr olds, this is what you get, or can get)

I do not think he missed the point or that he lacks the mental capacity to understand that he'd need proper testing for this. I think he just knows his audience, knows his platform (youtube) and he accommodates.
Let's face it, half a page long text and already people think oh, wall of text. Attention span issues abound.

edit: also, there is just enough flavor to this to justify it all. He's using exactly what the parrots have been using; except to showcase other, contradictory points. I do enjoy seeing that happen. Is he 100% right? No. Is he closer to an empirical truth than the others are? Why yes

 

[HTC]

While qubit makes valid points, the point remains.
- The more the software moves towards the direction we'd been expecting it to move, the more these old 'dinosaurs' / "horrible" (read:AMD) CPUs get the juice squeezed out of them, it being the point all along.
- The more "gamerzzzz" end up defining this market, the more skewed and convoluted said direction will be. If not halted that is, in terms of mentality. Case in point being the re-re-rehashing of 4cores, ad infinitum. Not to exclude AMD from this mind, they too are planning to sell those.
(when progress is defined by the needs of 15yr olds using software made for 15yr olds, this is what you get, or can get)

I do not think he missed the point or that he lacks the mental capacity to understand that he'd need proper testing for this. I think he just knows his audience, knows his platform (youtube) and he accommodates.
Let's face it, half a page long text and already people think oh, wall of text. Attention span issues abound.

edit: also, there is just enough flavor to this to justify it all. He's using exactly what the parrots have been using; except to showcase other, contradictory points. I do enjoy seeing that happen. Is he 100% right? No. Is he closer to an empirical truth than the others are? Why yes

IMHO, it's important to know if the current way to measure a gaming CPU's performance is flawed or not. If it's not, the discussion ends here but if it is, then steps need to be taken to come up with a way to benchmark it that is not flawed.

Haha, guess I put it a little ambiguously lol, my bad. Indeed he didn't run any of those benchmarks himself. I meant to say that if he really wants to confirm this for himself then he should run a set of benchmarks himself to properly compare. If not, then at least find comparable reviews, but that's hard to do because things change over time, such as drivers etc, therefore he should really do it himself under controlled conditions. I strongly suspect that if he did this then the 2500K would still come out on top, but that's not what he wants to tell us.

Think, just how many thousands of enthusiasts there are out there motivated to compare products. They would start to notice the FX improvements, which would get picked up by the tech press such as TPU, but it's not happened. Therefore, what he's saying is a crock and I think he's got an agenda to make it look like AMD's products are getting hard done by Intel and everyone else, when the real blame lies with the products themselves.

He's not a reviewer and doesn't have the necessary funds to get the required material to make a review himself, AFAIK. Different sites test with different hardware so it's not really viable to "find comparable reviews" so he did the "next best thing" by just using one site.

That's why i ask if @W1zzard can do it because if a reputable site such as TPU comes to the conclusion that the methodology is flawed (if indeed it is flawed), then the rest of the reviewers will pay attention and strive to come up with a way that's NOT flawed.

 

[Shihab]

Many games run in engine limitations and quirks which literally no one mentions. Also running lower resolutions is hardly a good method to show CPU performance in games , or rather it just simply doesn't tell you anything. It's the number of draw calls that hammer down a processor , running a lower resolution simply puts less stress on the GPU , you get no extra information whatsoever about the CPU than if you were running the game on 3x4K panels.

Resolution limiting as an attempt to shift bottlenecks, it doesn't necessarily put more theoretical load on the CPU (i.e. it won't create different data to process), though practically, it does.
Draw calls for each frame remain fixed no matter what resolution you run at, however, at higher resolution the GPU's ability to implement the calls it receives becomes slower, so we might -and often do- reach a point where the CPU sits idle while the GPU is busy rendering the frame from the previous batch of calls. Lower the resolution and -as you said- the stress on the GPU, and it becomes a keep-em-coming scenario: The greater the CPU IPC/Clock, the more calls it sends in a time unit, the more overall framerate you get, the less frame time variance you end up with. The last two are the only metrics for a CPU's performance in games (aside from PCIe bandwidth, which hasn't really been an issue since PCIe 2.0 days).

Do these metrics have any worth for real world use (@1080p+) though? That's a different matter.

 

[Aenra]

IMHO, it's important to know if the current way to measure a gaming CPU's performance is flawed or not

And? Must you undergo tests on a professional level to grasp whether it is or not?
Those very tests (which to varying degrees we all rely on, let's be honest), all by themselves, show you that a "horrid" Bulldozer cpu was actually not really that horrid, but rather that it wasn't made to perform in the way a dual core did (assuming one couldn't grasp that much on their own).
That even more to the point (you care about gaming performance right?), when games actually -started- using 5, 6 and 7 cores, said 'horrid' CPUs turned out to outperform that amazing, uber, et al CPU everybody was telling you to buy. Seen anyone admit that out in the open? Nope? Wonder why?

Put differently, we have been using a comparison process that neglected the software limitations (or took them for granted, failing to even mention them) and called for absolute truths.
As in, buy not "because with the current monkeys (sorry i meant software programmers) it will run a game faster". For now. Which it did.
But rather, "buy because it's better". Absolute.

Which leads to what started all this. Not the 480 versus 720p, no. What started all this was questioning whether we directed an entire market's technological prowess into regressing back into what the monkeys were capable/willing of. Ergo whether, in answer to your question, our standards of "better" were actually rather flawed; or not, situation depending of course.
It's why i said no, he's not 100% right (IPC, frequency, they will always have a role to play). But yes, he has a point. Thing is, no one allowed us to see/work with that point, until very recently (last what? Couple of years? In terms of games?). Certain conclusions can now be made, regarding hype, direction, mentality and how this industry of satellites (read: blogs/"press", the bigger the market, the more of them and nevermind if they're worthy of a voice) functions.

Good stuff any day of the week i'd say

edit: as stated, self-evident to some. Am here to remind however that as it turns out, this wasn't that obvious to many, many people out there. And now that we have enough multi-core games, it is. Assuming they're willing to listen.

 

[HTC]

And? Must you undergo tests on a professional level to grasp whether it is or not?
Those very tests (which to varying degrees we all rely on, let's be honest), all by themselves, show you that a "horrid" Bulldozer cpu was actually not really that horrid, but rather that it wasn't made to perform in the way a dual core did (assumiong one couldn't grasp that much on their own).
That even more to the point, when games actually -started- using 5 and 6 cores, said 'horrid' CPU turned out to outperform that amazing, uber, et al CPU everybody was telling you to buy. Seen anyone admit that out in the open? Nope? Wonder why?

Put differently, we have been using a comparison process that neglected the software limitations (or took them for granted, failing to even mention them) and called for absolute truths.
As in, buy not "because with the current monkeys (sorry i meant software programmers) it will run a game faster". For now. Which it did.
But rather, "buy because it's better". Absolute.

Which leads to what started all this. Not the 480 versus 720p, no. What started all this was questioning whether we directed an entire market's technological prowess into regressing back into what the monkeys were capable/willing of. Ergo whether, in answer to your question, our standards of "better" were actually rather flawed; or not, situation depending of course.
It's why i said no, he's not 100% right (IPC, frequency, they will always have a role to play). But yes, he has a point. Thing is, no one allowed us to see/work with that point, until very recently (last what? Couple of years? In terms of games?). Certain conclusions can now be made, regarding hype, direction, mentality and how this industry of satellites (read: blogs/"press", the bigger the market, the more of them and nevermind if they're worthy of a voice) functions.

Good stuff any day of the week i'd say

edit: as stated, self-evident to some. Am here to remind however that as it turns out, this wasn't that obvious to many, many people out there. And now that we have enough multi-core games, it is. Assuming they're willing to listen.

To remove doubts, yes: i'd say "professional level" testing is required to make sure this is or is not an issue.

A while back, everybody was using FRAPS and other such FPS measuring tools but then came PCPer (i think it was) and told us these programs were all working incorrectly and what they were showing was not what was being experienced, and so they came up with a way to properly show this. We became aware that something we were taking for granted was flawed all along ... sound familiar?

I'm not saying the current methodology is flawed but i AM saying that steps must be taken to ensure it's not.

 

[JalleR]

Of cause you cant bench a CPU to see what is good in the future, you can see what is good now, and that the way it is.
Is it pretty known that in moden games an old 8 core will be faster that a old 4 core cpu, and in older games, good Single threaded performance is better ..... I dont see the issue with the way it is done with the lov resolution, in GOW4 in UHD my GTX1080 is Working its ass off to delever fps and my Cpu is ideling, but in 1080P my cpu is Working its ass of and my GTX1080 is chilling.... so yes today Ryzen doesnt have the BEST gaming performance IF WE ONLY LOOK AT THE CPU's gaming performance, but in the future it will be better that the 7700K because Cores will be more important because games will maybe bee developed to use more cores, NOTHING NEW IN THAT.

If you want to see Run the GOV4 benchmark if you have GOV4. GOV = Gears of ware

 

[TheoneandonlyMrK]

After watching this video, i'm not so sure anymore:

He claims that the "ancient" FX8350 became faster the the I5 2500K simply by testing with a much much faster graphics card and has shown that, with each increase in GPU performance, the distance to the 2500K narrows, until it's reversed. To be fair, i personally have some doubts because there are some variables he didn't take into account, such as motherboards differences, memory differences, storage speed differences, used drivers differences that may or may not contribute to the overall result, not to mention that in the final part, he used the results of the FX8370 instead (due to not having the FX8350 on the benchmarks).


This finding contradicts the current thinking that in order to remove the GPU out of the equation, one needs to test @ lower resolutions / details so that one can say X processor is better then Y processor @ gaming with the results one gets after the tests, and any changes made to the graphics card will never change the outcome. I understand the logic ... but is this really true?

You may think: why am i even thinking about this with "ancient" CPUs? Because you can use the same principle on current CPUs: if the testing methodology is indeed wrong, then all those referencing reviews with it are being misled (even though not intentionaly) and this means a new way to test a gaming CPU must be found and this one scrapped.

And so i propose that our very own @W1zzard tests this (@ his convenience) and answer this question so that there's zero doubts. Use a 2500K and a 8350 and pair it with a 1080Ti, a 980Ti, a 680Ti (2 generation gap between each card, i think): only change the boards and CPUs while trying to keep the rest, if @ all possible (including drivers), so that there are less variables to interfere with the final results.

you would find your self quite wrong and him quite right , my 8350 (when its an i7 for intel)is still the recomended amd cpu in AAA games , ive had many an argument over this and at the end of the day i can still play any new games max settings on my Ancient cpu ,some dont get cicuitry like others and one hard fact can be enough to focus on to derail all the good, my cpu still has chops at high res gaming apparently , who knew.



i couldnt possibly agree with the video more.

 

[Aenra]

To remove doubts, yes: i'd say "professional level" testing is required to make sure this is or is not an issue

I understand your reasoning and i too would like some concrete data.
I need also state however that already, we have more than enough to understand how it -is- flawed. Now obviously, we are not talking about 100% flawed, because equally obviously, there is no one PC rig that is "bestest" for everything, just as there is no "one" gaming test to rule them all (diffrent APIs, different generations, different engines, etc).

Until a time when more will be availabe, employ some personal judgement, employ some observational skills.
How often do you see this discussion (like never*), why is it that so few people 'bother' with this (when say, we've had fanboying megathreads about Ryzens before they were even a thing), why is it that repetition by mouth has ended up favoring certain elements (or brands) of this equation over others, etc.

Where there's smoke..

*yes, like never.. we're talking about "bloggers", "press", word of mouth, gamerz, and where it's all lead us; not about threads discussing a 0.0000012% variation in speeds. Just to be clear.

 

[mroofie]

After watching this video, i'm not so sure anymore:

He claims that the "ancient" FX8350 became faster the the I5 2500K simply by testing with a much much faster graphics card and has shown that, with each increase in GPU performance, the distance to the 2500K narrows, until it's reversed. To be fair, i personally have some doubts because there are some variables he didn't take into account, such as motherboards differences, memory differences, storage speed differences, used drivers differences that may or may not contribute to the overall result, not to mention that in the final part, he used the results of the FX8370 instead (due to not having the FX8350 on the benchmarks).

This finding contradicts the current thinking that in order to remove the GPU out of the equation, one needs to test @ lower resolutions / details so that one can say X processor is better then Y processor @ gaming with the results one gets after the tests, and any changes made to the graphics card will never change the outcome. I understand the logic ... but is this really true?

You may think: why am i even thinking about this with "ancient" CPUs? Because you can use the same principle on current CPUs: if the testing methodology is indeed wrong, then all those referencing reviews with it are being misled (even though not intentionaly) and this means a new way to test a gaming CPU must be found and this one scrapped.

And so i propose that our very own @W1zzard tests this (@ his convenience) and answer this question so that there's zero doubts. Use a 2500K and a 8350 and pair it with a 1080Ti, a 980Ti, a 680Ti (2 generation gap between each card, i think): only change the boards and CPUs while trying to keep the rest, if @ all possible (including drivers), so that there are less variables to interfere with the final results.

He forgot to mention that the 2500K will beat the fx8350 when oced

 

[the54thvoid]

So? Move along.

Just to be clear, I will once i reply to you. My point is verified by your blunt reply. I was 'tricked' into a thread about a topic i am interested in only to be met with no info and a YT link. Had I known the thread required viewing a video of a guy whose work i don't value, i could have avoided the thread.

So i have moved along. Though watching people blatantly ignoring Ryzens deficit is quite depressing. It's an amazing CPU but it is weaker in gaming. I've looked at 1440p reviews where it still loses out. Adored lives to be 'contentious'. It gets more hits that way.

But it's gutter work.

Out.

 

[Aenra]

Out

No offense, but you missed the point here. You've let a couple of biased opinions and/or your own (not that i blame you, i don't use Youtube or Facebook either) color your judgement.
This is about how a product is perceived, who and how has a role to play in that and what it means for the market as a whole. Forget Adored, forget what he has to do to get or maintain his subscribers.

Lastly, 'cause momma taught me to be a fair redneck, Adored wasn't saying or implying that one is 'better' than the other. And he's not refuting the i7 770K's current superiority in FPS; he even illustrates it by showing videos of others so we can see said FPS difference for ourselves.
Stick to the forest and forget about the tree

 

[TheoneandonlyMrK]

He forgot to mention that the 2500K will beat the fx8350 when oced

you can oc said 8350 upto 5ghz most modern games and gtaV love that though the elecy bill man also loves that too.
honesty rocks..

Haha, guess I put it a little ambiguously lol, my bad. Indeed he didn't run any of those benchmarks himself. I meant to say that if he really wants to confirm this for himself then he should run a set of benchmarks himself to properly compare. If not, then at least find comparable reviews, but that's hard to do because things change over time, such as drivers etc, therefore he should really do it himself under controlled conditions. I strongly suspect that if he did this then the 2500K would still come out on top, but that's not what he wants to tell us.

Think, just how many thousands of enthusiasts there are out there motivated to compare products. They would start to notice the FX improvements, which would get picked up by the tech press such as TPU, but it's not happened. Therefore, what he's saying is a crock and I think he's got an agenda to make it look like AMD's products are getting hard done by Intel and everyone else, when the real blame lies with the products themselves.

Heck, just look at the Ryzen launch, all full of hype and promise. AMD and their partners, including Microsoft have had years to work with them to ensure a reasonably bug-free launch, but what do we see? BIOS issues preventing memory modules from working properly and a Windows 10 bug lowering overall performance due to SMT issues. How did Microsoft and more importantly AMD, not notice something so blatantly obvious before launch and just get it fixed? Especially since Windows 7 doesn't have this problem.

I tell you, if I'd dropped a grand+ plus on a Ryzen upgrade (CPU, mobo, RAM, cooler) I'd be pissed at seeing problems like this and might ask for my money back. I expect the thing to work properly out of the box. This is typical AMD and why I have no confidence in them. Funny how Intel CPUs haven't dropped in price much since Ryzen's launch and its "incredible" value proposition, isn't it? Yes, I think Ryzen does have that potential, but the launch has been poorly executed, ruining it. Looks like we'll have to wait for Ryzen v2 for an improved IPC that can compete head-on with Intel in games, but it's a good start, bugs notwithstanding.

There's also another thing that makes him quite the AMD apologist. When buying hardware, one should almost always buy what performs best right now, rather than a promise of better performance in the future. It's not guaranteed, there's no timescale and by the time that the improvement comes, if it does, the hardware will be out of date anyway. So, who cares if the FX works better than a 2500K now? They're both obsolete and can't even be purchased new any more.

I have literally put this philosophy into practice today. You might remember that my Palit GTX 1080 died a couple of months ago and I got a refund on it. Well, I was waiting for the 1080 Ti to come out before buying its replacement. What do I see? A card that costs an obscene £700+, is "only" 35% faster than a 1080 and an overclocked 7700K isn't fast enough to max the card out at 1080p a lot of the time (check TPU's review). Now, I still have a 2500K (at a rather non-enthusiast stock speed, but don't tell anyone ) which is a lot slower than TPU's test rig, so I passed on the 1080 Ti and bought another 1080 today, taking advantage of the cheaper prices.

I've actually bought two models, as I can't quite tell which one I'm gonna keep just from the reviews and will return one to Amazon. Noise is all-important for me, so it's a toss-up between the Palit GTX 1080 GameRock (same as I had before) at £500 or the Zotac GTX 1080 AMP! Extreme at a cool £600 which @W1zzard said is the best GTX 1080 he's ever tested, in his review. Yes, it's somewhat more than I wanted to pay for a 1080 right now, but I really like it, so I'm biased towards keeping it, despite that steep £100 premium. If the coil whine performance is better on the Zotac then keeping it is a no-brainer.

your perspective is skewed by your fat wallet mate, if id picked for now 5 years ago my pc wouldnt still be doing all that i am now and right now i dont know what ill be doing with my pc in 3 years , vr maybe , point is dont shove your perspective so hard down my throat.
some of us are more subjective about whats a fault.

can i game at 550+fps low settings 1080p , dont give a damn
can i game at 4k ultra settings ,do care



but thats letting my perspective cloud the issue.

 

[HTC]

No offense, but you missed the point here. You've let a couple of biased opinions and/or your own (not that i blame you, i don't use Youtube or Facebook either) color your judgement.
This is about how a product is perceived, who and how has a role to play in that and what it means for the market as a whole. Forget Adored, forget what he has to do to get or maintain his subscribers.

Lastly, 'cause momma taught me to be a fair redneck, Adored wasn't saying or implying that one is 'better' than the other. And he's not refuting the i7 770K's current superiority in FPS; he even illustrates it by showing videos of others so we can see said FPS difference for ourselves.
Stick to the forest and forget about the tree

Currently, whenever a new CPU is launched and reviewed, it's tested by multiple sites using low resolution / details in an effort to remove the GPU as a factor in whatever results they get. So, in theory, if you were to swap the card in the reviews for a much faster one, and then an even much faster one, the results of the CPU gaming benchmarks would not change and whatever placing in the reviews by X, Y and Z processors would be kept.

But in the video, Adored shows that not only this is incorrect but, in this specific case, it actually gets reversed, and i'm not talking about 1% or 2% here. That raises the question of whether or not the methodology of gaming CPU benchmarking is correct, which is why i created the topic to ask @W1zzard if he could answer this question.

There are however several variables and i address them in the OP because i have no idea if any of them has an impact on the testing, which is why i'm not sure myself if this method is indeed flawed or not.

 

[qubit]

your perspective is skewed by your fat wallet mate, if id picked for now 5 years ago my pc wouldnt still be doing all that i am now and right now i dont know what ill be doing with my pc in 3 years , vr maybe , point is dont shove your perspective so hard down my throat.
some of us are more subjective about whats a fault.

can i game at 550+fps low settings 1080p , dont give a damn
can i game at 4k ultra settings ,do care



but thats letting my perspective cloud the issue.

1 I don't have a fat wallet (I wish I did). I'm just a bit better off than I used to be so can buy some nice things once in a while
2 Not shoving my perspective down anyone's throat. Sounds like you're butthurt because you've got an FX8350 and I've given AMD some well deserved criticism. Remember, it's them I'm having a go at, not you

It's not actually clear what you're getting at, anyway.