How to do Power Consumption Testing in CPU Reviews?

[W1zzard]

There has been some discussion about power testing methodologies with Alder Lake.

Thoughts, requests, ideas, suggestions how to change this for the future?

Obviously I cannot spend multiple hours on testing power consumption alone.

I'd very much continue measuring full-system wall-power, as I have a great data collection pipeline already setup for it. Data is collected digitally, over Ethernet, processed, and it spits out TPU-style charts for me, with minimal user interaction.

I do have definite plans to add "Gaming Power Consumption", using Cyberpunk 2077 at highest settings, v-sync off

 

[R0H1T]

Do probably at least 1 real world (application) use, like Cinebench. Then maybe a synthetic test through something like Thread Racer ~

 

[W1zzard]

Do probably at least 1 real world (application) use, like Cinebench

How is Cinebench a real world application?

 

[R0H1T]

I said real world application use, besides isn't Cinebench doing basically what Cinema4D does? Outside of built in benchmarks, within the application itself, there's probably nothing like it.

 

[Selaya]

[ ... ]

I'd very much continue measuring full-system wall-power, as I have a great data collection pipeline already setup for it. Data is collected digitally, over Ethernet, processed, and it spits out TPU-style charts for me, with minimal user interaction.

[ ... ]

That's fine and should be kept, but maybe another dataset at the EPS12V? Would be extra useful since those two datasets will also reveal the memory, GPU et cetera overhead.

EDIT: Just glanced over the thread that led to this spinoff. Would still stand by the previous (if it's not too much of a hassle), but maybe add an uhm inverse test of some popular benchmark (CPU-Z, Cinebench or something) where you normalise for power? (Ie lock all processors to 65W or something, then bench them to see which is the most efficient and also how much performance is lost since power/performance scaling is not linear)

 

[HD64G]

For real world power consumption testing I would prefer one test showing 7zip encoding and another decoding. The average values are more than enough. CB23 can be used to check the max power limits and temps more than anything else. As for gaming, Timespy is a standard, fast and reliable benchmark which makes full use of CPU also. My 5c.

 

[R0H1T]

Ie lock all processors to 65W

That's not enforceable probably expect for Intel's locked chips, you can lock their clock speeds though so there's that.

 

[RandallFlagg]

I would think you would need to model, and then script, a use scenario that runs at human speeds.

There could be more than one scenario, probably should be.

For example, one scenario might be office worker 8 hours loading up spreadsheets and word docs, some OCR as that is common with document conversion in offices, some web activity to represent the work day. Throw in a couple of hours of video conference, since that is a big thing (even for those not WFH, they have to communicate with those who are). Obviously this would mostly be idle time, since office workers spend 90% of their time entering something into a spreadsheet cell or word document, not just doing recalcs. Then 4 hours of web / media / gaming for an avid PC user. That would give you a 12 hour loop.

I want to say the above looks a lot like PCMark10. It does, except PCMark10 is a benchmark at max speed, not a real world use. In the real world, we move a lot slower than their script.

Then you've got content creators, scripted activity on things like Adobe Photoshop, Lightroom, along with some mix of web/browser use, conferencing and so on.

Then plug it into a Kill-A-Watt meter and let it run the script for a day.

So creating such scripted activities that move at real human speeds and do real human activities would be the difficult part. If it were a car, you'd just drive it through a course for 4 hours.

I'm actually kind of shocked that the industry hasn't built something like that. I would bet that Intel and AMD have these kind of tools, based on whatever they model user activities as.

 

[W1zzard]

For example, one scenario might be office worker 8 hours

This will pretty much be identical to the "idle" result

Then you've got content creators, scripted activity

All scripted tests that I'm aware of run without any "thinking time" in-between

 

[The King]

On the aspect of PSUs being used. If the same PSU cannot be used in all test.
Using two different PSUs that have the same OEM and rating can have different components inside them
which can make said PSU fail their rating. Pull more wattage!

Please keep this in mind when you do these CPU consumption tests. If readings are done at the AC outlet.
If readings are taken form the board probe points then this should be a none issue

 

[chrcoluk]

There has been some discussion about power testing methodologies with Alder Lake.

Thoughts, requests, ideas, suggestions how to change this for the future?

Obviously I cannot spend multiple hours on testing power consumption alone.

I'd very much continue measuring full-system wall-power, as I have a great data collection pipeline already setup for it. Data is collected digitally, over Ethernet, processed, and it spits out TPU-style charts for me, with minimal user interaction.

I do have definite plans to add "Gaming Power Consumption", using Cyberpunk 2077 at highest settings, v-sync off

As I understand you tested due to cpu specification right? The new Alder Lake, if I am right now no longer has the traditional TDP as part of its spec and is not out of spec with sustained high power usage?

To answer I need to understand what the supported mode of usage is by intel, basically my answer is to test to cpu manufacturer specifications, disable MCE etc.

If your question is related to workloads then I have no issues with how the testing was done, whilst gaming workloads today might not place much of a demand on the chip, we dont know if that will be the case in years to come. Putting the cpu under AVX prime95 for power draw testing is not realistic, but I think rendering is a reasonable test. There is people who stream games using x264 rendering whilst gaming all under one PC. I play games whilst recording using x264 rendering.

There is a option to consider adding a new test in addition to the standard test which is to test a chip without turbo boost (or XFR for AMD), e.g. I run my ryzen rig with XFR off as its way more efficient. That last 10% performance is inefficient on most cpus/gpus.

I do get the arguments made though, as an example the RTX 3080 is power hungry, however because its more powerful if you dont fully stress it, then it can use less power than older generations because it never goes into high inefficient clocks. The extra test I suggested disabling turbo clocks might cover this scenario.

 

[GerKNG]

i think it's not relevant enough to make a list of "CPU X pulls X Watts here and there"

Cinebench R23 to see how far it goes while being compareable to other CPUs.
every game, graphics settings (lower or higher CPU load) changes it anyways.

 

[RandallFlagg]

This will pretty much be identical to the "idle" result

Actually I'm pretty sure it wont.

Here is me signing into Office 365 - just signing in, that's it, browser already opened - on my personal desktop a 10850K. That's those spikes :

I could give a mess of other examples however, when I'm idle my CPU shows 1-2% usage. When I'm using my PC it looks like the above, maybe half the time. Even scrolling through a document like a PPT or XLS takes some CPU.

That's what that video was showing on the Kill-A-Watt meter. +20-40W power draw just from opening a folder.

All scripted tests that I'm aware of run without any "thinking time" in-between

That's the problem the entire PC review industry has, tests like PCMark10 and naturally Cinebench and so on are effectively max performance benchmarks. If its a car, I want to know what happens when I drive an hour in mixed highway / city stop-n-go traffic at normal speed.

 

[W1zzard]

On the aspect of PSUs being used. If the same PSU cannot be used in all test.

I am using the same PSU of course. On the same setup, with the same fan type and fans, no RGB, etc.

As I understand you tested due to cpu specification right?

I always review at spec, with MCE disabled. For ADL spec is PL1=PL2=241 W
Additional test runs show power limits removed, overclocked, and possible special scenarios, like I did "E-Cores Disabled" for 12900K

 

[dirtyferret]

I am using the same PSU of course. On the same setup, with the same fan type and fans, no RGB, etc.

I've spotted the problem

 

[W1zzard]

I've spotted the problem

Not sure if serious. If you are, please make your argument

 

[cadaveca]

I do have definite plans to add "Gaming Power Consumption", using Cyberpunk 2077 at highest settings, v-sync off

Gaming tests, sure, but how well do they actually load the system. Can we get something like Blender as well? Or something like Vegas or final cut render. I use Timespy stability test and aida64 in laptop reviews, but should probably follow your lead...

But I don't own Cyberpunk.

 

[W1zzard]

Can we get something like Blender as well?

I suspect cinebench nT will be pretty much identical to Blender?

But I don't own Cyberpunk.

Any modern GPU limited game should be similar

 

[cadaveca]

I suspect cinebench nT will be pretty much identical to Blender?

To a degree, but PL2 effectively is eliminated generally speaking with longer blender render, if board behaves. Not everyone understands PL1=PL2 = 241W. How do you point that out effectively?

Any modern GPU limited game should be similar

But game costs are write-offs.... LOL.

 

[W1zzard]

To a degree, but PL2 effectively is eliminated generally speaking with longer blender render, if board behaves. Not everyone understands PL1=PL2 = 241W.

Actually my Blender run is relatively short, so it'll include some turbo power. CB R23 will run for 10 minutes before taking results, so it'll run at PL1 exclusively in pre-ADL.

 

[dirtyferret]

Not sure if serious. If you are, please make your argument

not serious just found it funny that anyone would even assume you tested with RGB on.

 

[dragontamer5788]

I feel like most standard CPU-benchmarks would "just" need to be tweaked into "Joules used per benchmark".

For example: Cinebench, Blender, Linux compiles are all "jobs". Cinebench just renders one image. Blender renders one image (multiple choice here: 4 or 5 images to choose from). Linux compile might have more I/O used but should still be mostly CPU power.

From there, you collect data on Watts each-second, then integrate watts over time to get Joules.

------

Even for "rate-based" benchmarks (like SPECjbb), which run for a fixed time + return a rate of processing, you still care about the Joules used per benchmark. If SPECjbb reports 10,000 transactions processed, and your power meter reports 20,000 joules used, then you report to us 0.5 joules per transaction.

-------

I'm frankly unsure if anything you do needs to be changed, aside from spending some effort on making an accurate Joule-counter. Can anyone think of a situation where the methodology I just described fails to work out mathematically? The math has a lot of "divisions / reciprocals" going on and instinctively, I know I'm pretty bad at those and make mistakes all the time in logic when reciprocals are involved. But... my instinct leans towards just calculating Joules and ignoring everything else.

 

[TheoneandonlyMrK]

I feel like most standard CPU-benchmarks would "just" need to be tweaked into "Joules used per benchmark".

For example: Cinebench, Blender, Linux compiles are all "jobs". Cinebench just renders one image. Blender renders one image (multiple choice here: 4 or 5 images to choose from). Linux compile might have more I/O used but should still be mostly CPU power.

From there, you collect data on Watts each-second, then integrate watts over time to get Joules.

------

Even for "rate-based" benchmarks (like SPECjbb), which run for a fixed time + return a rate of processing, you still care about the Joules used per benchmark. If SPECjbb reports 10,000 transactions processed, and your power meter reports 20,000 joules used, then you report to us 0.5 joules per transaction.

-------

I'm frankly unsure if anything you do needs to be changed, aside from spending some effort on making an accurate Joule-counter. Can anyone think of a situation where the methodology I just described fails to work out mathematically? The math has a lot of "divisions / reciprocals" going on and instinctively, I know I'm pretty bad at those and make mistakes all the time in logic when reciprocals are involved. But... my instinct leans towards just calculating Joules and ignoring everything else.

They could do with retaining a degree of simplicity, some won't know what a joule is.

@W1zzard
For me what you do is fine just add gaming only power for those that believe it's the be all and end all.

 

[dragontamer5788]

They could do with retaining a degree of simplicity, some won't know what a joule is.

I guess "kilowatt-hours" would be the unit of energy quoted in the USA the most. But scientists would typically use Joules (aka watt-seconds) instead.

Hmmm... some discussion / polling on what makes sense to more people probably should be done. I would personally understand Joules. But maybe more people are familiar with "kilowatt-hours" instead??

-------

A Watt is literally a "Joule-per-second". So a Kilowatt-hour is "1000 Joules-per-second hours", which is more confusing IMO, despite being the commonly quoted figure in the USA. Also, given how much energy these benchmarks use (probably 300 seconds @ 100 Watts or so, or 30,000 Joules or so), that's "too small" to be quoting kilowatt hours (aka: a kilowatt hour is 3600000 Joules. So a typical benchmark would be 0.008 kilowatt-hours of energy)

What would look better in an article? 30000 Joules or 0.008 kw-hr ??

Or maybe we go with 8.3 watt-hours? Or 30-kilojoules ?

 

[basco]

my take on this is how will motherboard implementations affect this- like asus multicore enhancement or how it´s called today.
like does the mainboard you are using for the review use PL1 or whatever - i think ya got my point
like on asus motherboards it uses the 241watt from load defaults not 125 watt - is what i can gather from the reviews.

or is it - what wattage is the norm for that 12900k 125 or 241 or the nearly 400watts its using during load on heavy multitask but not games?
thanks very much for your work and time!