Testing ARROW lake - power limited

[Taraquin]

You're also still missing the point (the fab node is a small factor in the efficiency equation), but that's your right I guess.

I know that, I just think Intel could have done better. There are several issues with Arrow lake. Insanely high latency may be the worst.

 

[bug]

I know that, I just think Intel could have done better.

When have we had a fresh architecture/design when that assertion wasn't true?

There are several issues with Arrow lake. Insanely high latency may be the worst.

Not sure what the Arrow Lake issues are, I haven't read any in-depth review of the architecture yet.

 

[Taraquin]

When have we had a fresh architecture/design when that assertion wasn't true?

Not sure what the Arrow Lake issues are, I haven't read any in-depth review of the architecture yet.

Zen 4, Intel 12th gen for example. Major improvements and good utilization of nrw node/arcitecture

 

[BoggledBeagle]

I made this comparison of P and E core performance at 1 GHz of Raptor and Arrow lake CPUs.



I do not think any of this is a new information and I will not comment about the numbers and analyse them.

Only one remark: you may think that abandoning HT is a huge disadvantage, when Raptor P core with 2 threads can do 520 points and Arrow with 1 thread only 412, but in reality, when the threads are assigned work in order of their strenght, the HT advantage will start manifest itself only for loads of 25 and more threads.

Here is a comparison of the performance of 24 core CPUs, I also simulated Arrow lake with HT.


If you average HT benefit over the whole load thread count range, HT benefit is only 2%. Most consumers in most use scenarios will not even notice that HT is missing.

 

[DavidC1]

If you average HT benefit over the whole load thread count range, HT benefit is only 2%. Most consumers in most use scenarios will not even notice that HT is missing.

Nice analysis. I want to see a more comprehensive E vs P core data, but it seems every website in all countries want to just copy each other. Nothing unique about them.

Hyperthreading, or the non-marketing term Simultaneous Multi-Threading has potentially a much bigger issue. You see, die size and transistor wise, the impact is fairly small. Various numbers are thrown around, 5-10% of the core size.

The real problem is increased validation difficulty, and more recently security. It's making sure everything is split between two threads without execution errors. Making sure the firmware and BIOS is super finely tuned so you don't lose performance is a big deal too.

Let's say the increased difficulty adds just 1 months to development. Over 5 generations, that's 5 months over architectures that do not have SMT. The rumbling is that the Lion Cove P core in Arrowlake couldn't have HT because the team is fumbling really bad and couldn't keep it in.

 

[BoggledBeagle]

Hyperthreading, or the non-marketing term Simultaneous Multi-Threading has potentially a much bigger issue. You see, die size and transistor wise, the impact is fairly small. Various numbers are thrown around, 5-10% of the core size.

The real problem is increased validation difficulty, and more recently security. It's making sure everything is split between two threads without execution errors. Making sure the firmware and BIOS is super finely tuned so you don't lose performance is a big deal too.

I am not sure Intel fully translated the benefit of not having HT circuitry on the chip/not having to validate it into P core performance improvement. I am not saying that 8% improvement is insignificant, but in comparison with E core improvement it seems a bit underwhelming.

 

[LittleBro]

When have we had a fresh architecture/design when that assertion wasn't true?

Not sure what the Arrow Lake issues are, I haven't read any in-depth review of the architecture yet.

Unlike Zen5, Arrow Lake has serious stability issues especially on W11 2H24, meaning OS freezes, BSODs. Especially the iGPU has a lot of problems. Performance gains aside, Zen5 was at least stable at the date of release, not causing any serious (system breaking) issues.

As Taraquin pointed out, going from 10nm to 3nm while achieving only 24% perf. per watt improvement is really bad. It should have been much much more. They are absolutely not leveraging process node capabilities. This points out to the fact that the whole architecture since Raptor Lake is inefficient and needs serious optimizations.

Unfortunately, poor effectivity is not just a problem of mainstream Intel CPUs. Same applies to their "AI" accelerators and server CPUs, and discrete GPUs.

Btw, Arrow Lake might have been much better with HT. Thanks to enlarging cache on E-cores they have considerable IPC gain (30+%), without it Arrow Lake would be total disaster.

 

[DavidC1]

Btw, Arrow Lake might have been much better with HT. Thanks to enlarging cache on E-cores they have considerable IPC gain (30+%), without it Arrow Lake would be total disaster.

It's not cache why E-cores are doing better. They have dramatically improved and expanded every part of the uarch.

The E core team makes bigger changes every 2 years than the P core team did in their best days(Pentium M, Core 2), and they did it for past 10 years rather consistently. They are now being brought into the spotlight because now the absolute performance is very good.

-The original Atom was in-order.
-The Silvermont successor was OoOE and took out HT, at the same transistor count. 50% per clock gain
-Goldmont went from 2 to 3-wide decode and went OoOE with FP. 30% per clock gain
-Goldmont Plus widened the backend, and quadrupled the L2 predecode cache to 64KB. 30% per clock gain
-Tremont introduced clustered decode, doubling it to 6-wide in some scenarios, 128KB L2 predecode cache, 30% per clock gain
-Gracemont enhanced clustered decode, so it can now execute 6-wide everywhere. Took out the 128KB L2 predecode cache and replaced it with OD-ILD, which performed better. Supports FMA, meaning AVX2, doubling FP performance with AVX2
-Skymont has triple 3-wide decode, so it's 9-wide. Doubled number of FP units, meaning it straight up improved performance without recompiling.

Nevermind every generation the branch predictor has improved, units added, buffers expanded. And they made careful decisions rather than just expanding everything. Replacing the L2 predecode to OD-ILD was a risk, it was successful. Clustered decode was a risk, it was successful. Now AMD is using it with Zen 5, although Zen 5's cluster only works with SMT, not with single thread.

The E core team has every chance of becoming the next top Intel team. This is Pentium 4 vs Pentium M all over again.

 

[LittleBro]

Yes and what's the point of P-cores then? If so called efficient cores are so good nowadays ...

If you average HT benefit over the whole load thread count range, HT benefit is only 2%. Most consumers in most use scenarios will not even notice that HT is missing.

Check out Phoronix. They made tests on effect of HT/SMT in apps. Zen 4 gets nice IPC boost with SMT while having same consumption.

If there is is no benefit with HT on Intel, maybe the implementantion is wrong.

 

[BoggledBeagle]

Zen 4 gets nice IPC boost with SMT

100% of cores on AMD CPUs can do SMT. Only 33% of cores on 24 core Intel CPU can do HT. Also cheaper AMD CPUs have only 6 cores. HT is differently important for the products of both companies.

 

[DemonicRyzen666]

Intel 1 P core 16 E core | Ultra 9 285​

source: https://www.reddit.com/r/intel/comments/1gf33lz

This image makes me remember that a Slice of L3 cache is shared between 4 E-cores.
This looks like the E-cores are small enough to keep data in the fast slice of L3 cache, so they don't have to constantly go back to DRAM at a lot.
A single P-core takes up that whole slice of L3 cache, then has to go to the ring bus if it falls off that slice, doesn't it?
This would be why P-cores seem to be jumping around in windows is, because the other P-cores are picking the data that was supposed to be processed by another P-core that ended up in its L3 cache.

This almost makes it look like e-cores could make really good use of L3 3D cache increase.
P-cores however seem to need a lot more bandwidth as seen with CU-DIMMs, I don't think they'd gain anything from a larger L3 cache either from their design.

 

[evernessince]

I got myself a testing PC with 265K, Strix E board and ordinary RAM (32GB, 6000 MHz, CL36) and tested the CPU with two power limits 100 and 160W, and compared it to 13900KS with 4 E cores disabled, both with HT on and off.

View attachment 368981

View attachment 368980

As you can see, the results are pretty consistent, both apps in both power limits mean 24% performance improvement of Arrow lake compared to Raptor lake with HT disabled and 11% with HT enabled.

Most of the power inefficiency of Raptor lake is gone when it is so much power limited - Raptor lake is extremely efficient with 100W power limit. I believe that the result is pretty positive for Arrow lake with 20 threads against Raptor lake with 28 threads.

13900KS could be a little bit better than 14900K, but that is down to silicon lottery, I do not think it has a significant advantage compared to standard 14th gen CPUs.

I don't think it's really a fair comparison to take a balls to the walls full power special edition flagship part and compare it against a lower end part. The former absolutely is not tuned for efficiency and is the worst in that regard save only for the 14900KS.

We can't draw conclusions on architectural efficiency based on this data alone as you are doing. There's the problem I mentioned above but there's also the fact that it's well known that Intel pushed most of it's 13th and 14th gen parts well out of their sweet spot. TPU's power scaling review of the 13th and 14th gen reviews show significant efficiency gains before even delving into undervolting (which also helps given how much voltage intel pushes out of box). The 13400 is a good example of what the architecture is capable of efficiency wise.

This is absolutely a best case scenario for Arrow Lake (which says a lot given it's only 24%). I suspect in reality once we see an actual like for like efficiency comparison, Arrow Lake will end up being much like Zen 5's efficiency improvements. In otherwords, -5 to +5% and only looks good in comparison to power hungry stock settings.

 

[BoggledBeagle]

I don't think it's really a fair comparison to take a balls to the walls full power special edition flagship part and compare it against a lower end part.

When power limited, these CPUs run completely differently - at much lower frequencies, more efficiently. They have nothing to do with how they perform out of the box.

 

[evernessince]

When power limited, these CPUs run completely differently - at much lower frequencies, more efficiently. They have nothing to do with how they perform out of the box.

When power limited the 13900KS is still using an over aggressive voltage table. Power limiting just forces it to pick a lower value on that table. Preferably you'd want to tune the voltage for both to see where they land but you'd want to do that in an Apples to Apples comparison as possible. Flagship vs flagship for example. Disabling cores is only simulating it and thus carries with it a lower level of assurance.

 

[BoggledBeagle]

When power limited the 13900KS is still using an over aggressive voltage table. Power limiting just forces it to pick a lower value on that table. Preferably you'd want to tune the voltage for both to see where they land but you'd want to do that in an Apples to Apples comparison as possible. Flagship vs flagship for example. Disabling cores is only simulating it and thus carries with it a lower level of assurance.

What do you mean??? I am comparing two completely different generations of products made on different processes.

Or do you believe that 13900KS is radically better than 14900K and when power limited it performs much better than 14900K would, because it requires lower voltage and can run at higher frequency? I personally believe that in that scenario even one of the best 13900KS would perform just low single digits % better than the dog 14900K.

BTW if you want to buy me 14900K and 285K chips to test, you can, of course. Will you???

 

[evernessince]

What do you mean??? I am comparing two completely different generations of products made on different processes.

Hence why it's important that when we want to demonstrate which architecture is more efficient, we try and put each into it's sweet spot and eliminate any variables that we can. We want to see what the architecture is capable of in an ideal scenario. Arrow lake launches in a more ideal power / voltage scenario (I assume, haven't seen a power scaling review of it yet) while the 13900KS does not.

or do you believe that 13900KS is radically better than 14900K and when power limited it performs much better than 14900K would, because it requires lower voltage and can run at higher frequency? I personally believe that in that scenario even one of the best 13900KS would perform just low single digits % better than the dog 14900K.

That wasn't what I was trying to say but I agree with you that the two are going to be about the same. I was just pointing out that the issues with the voltage table. It's well known that the high end 13th and 14th gen CPUs use too much voltage at any of their higher frequencies (4.8 GHz and above). This is due to the way these CPUs were tuned by Intel and variability in silicon quality.

BTW if you want to buy me 14900K and 285K chips to test, you can, of course. Will you???

You don't necessarily need these chips to make a better comparison. You can UV the 13900KS and 265K into their sweet spots in addition to your power limiting.

Honestly ideally you'd want 5 samples to see variability but yeah that's a tall ask. It's why having multiple review outlets is important. A good example is the variability of power consumption and needed voltages for lower end Zen 1 parts. Awhile back Derbauer made a video showing a notable difference (20-30w) in power consumption of AMD's mid-range Ryzen 1600X. With all variables controlled it turned about to just be sample variation.

 

[LittleBro]

100% of cores on AMD CPUs can do SMT. Only 33% of cores on 24 core Intel CPU can do HT. Also cheaper AMD CPUs have only 6 cores. HT is differently important for the products of both companies.

It's not about importance, rather about implementation. I'm gonna quote myself here:

If there is is no benefit with HT on Intel, maybe the implementantion is wrong.

I understand Intel had so many problems with their HT security-wise that it lost all the meaning in the end and yield no performance gains.
Yet again, it's about implementation. AMD struggled to implement HT-like feature for a decade, but it looks like they implemented it later, but better.

 

[BoggledBeagle]

Hence why it's important that when we want to demonstrate which architecture is more efficient, we try and put each into it's sweet spot and eliminate any variables that we can. ...

How are these sweet spots defined?

... It's well known that the high end 13th and 14th gen CPUs use too much voltage at any of their higher frequencies (4.8 GHz and above). This is due to the way these CPUs were tuned by Intel and variability in silicon quality.

If you mean KS models, they on average require LESS voltage at each given frequency than standard models. For example, my current 13900KS required 50mV less than my old 14900K at 5500 MHz.

 

[evernessince]

How are these sweet spots defined?

It's the range in which the highest ratio of power to performance can be had for a given architecture.

I did a quick chart just as an explainer.

The X axis is performance
The Y is voltage (or power, either would do in this instance).
The highlighted red area is the sweet spot.

The sweet spot's start and end is mapped to two notable drops in perf per watt, providing a range in which users can tune voltages and power to depending on their performance requirements. A person who lower favors power consumption can pick the point all the way to the left of the sweet spot, which would provide the highest efficiency possible. A person who wants as much performance as possible while maintaining reasonable efficiency would pick the point all the way to the right. In general CPUs using the same architecture should have the same sweet spot, with some variance due to silicon quality.

If the 14900K or 13900K was mapped to this chart, the tail end to the right of the sweet spot would be very longs as these architectures see small gains above 125w, so it's a small amount of performance dragged over a large power increase.

If you mean KS models, they on average require LESS voltage at each given frequency than standard models. For example, my current 13900KS required 50mV less than my old 14900K at 5500 MHz.

Not referring specificially to the KS models hence why I made a broad statement regarding 13th and 14th gen but my statement definitely applies to them. Yes they can in general get a higher frequency at a lower voltage compared to stock models but the point of my argument was to point out the voltage headroom Intel left on the table. This problem still exists for the KS models.

 

[ShrimpBrime]

It's the range in which the highest ratio of power to performance can be had for a given architecture.

I did a quick chart just as an explainer.

The X axis is performance
The Y is voltage (or power, either would do in this instance).
The highlighted red area is the sweet spot.

View attachment 370229
The sweet spot's start and end is mapped to two notable drops in perf per watt, providing a range in which users can tune voltages and power to depending on their performance requirements. A person who favors power consumption can pick the point all the way to the left of the sweet spot, which would provided the highest efficiency possible. A person who wants as much performance as possible while maintaining reasonable efficiency would pick the point all the way to the right. In general CPUs using the same architecture should have the same sweet spot, with some variance due to silicon quality.

If the 14900K or 13900K was mapped to this chart, the tail end to the right of the sweet spot would be very longs as these architectures see small gains above 125w, so it's a small amount of performance dragged over a large power increase.



Not referring specificially to the KS models hence why I made a broad statement regarding 13th and 14th gen but my statement definitely applies to them. Yes they can in general get a higher frequency at a lower voltage compared to stock models but the point of my argument was to point out the voltage headroom Intel left on the table. This problem still exists for the KS models.

I had realized the testing is no good after figuring out cutting e-cores on a flagship and reducing power to 140w so that his cooling is adequate for full load testing. Neither of which anyone would be doing on a normal regular basis.

There's no point. I watch, but there's nothing to really see here.

 

[BoggledBeagle]

The sweet spot's start and end is mapped to two notable drops in perf per watt, providing a range in which users can tune voltages and power to depending on their performance requirements. A person who lower favors power consumption can pick the point all the way to the left of the sweet spot, which would provide the highest efficiency possible.

You know, I got a 13900K early and I played with it and tested it with different power limits.

Here are the actual results:



THERE IS NO SWEET SPOT!!! The only information you get is the less power, the more efficient the CPU is. There is no maximal value on the Perf/W curve, other than the starting point at 12W, below that the CPU did not work at all. You have to make a decision, how to run the CPU based on your priorities.

You want max performance? Run it at 350W. cpu dies quickly
You want max efficiency? Run it at 12W. stupid
You want max performance, but your cooler can handle just 160W? Run it at 160W.
You want good efficiency, but in the same time want at least 30K points from it? Run it at 110W.
You want max performance and you want perf/w be at least 400? Run it at 50W.

etc.

You may include temperature, voltage and frequency itself as your deciding factors, if you have some information about the values of these parameters which will prevent fast degradation.

 

[evernessince]

You know, I got a 13900K early and I played with it and tested it with different power limits.

Here are the actual results:

View attachment 370258

THERE IS NO SWEET SPOT!!! The only information you get is the less power, the more efficient the CPU is. There is no maximal value on the Perf/W curve, other than the starting point at 12W, below that the CPU did not work at all. You have to make a decision, how to run the CPU based on your priorities.

You want max performance? Run it at 350W. cpu dies quickly
You want max efficiency? Run it at 12W. stupid
You want max performance, but your cooler can handle just 160W? Run it at 160W.
You want good efficiency, but in the same time want at least 30K points from it? Run it at 110W.
You want max performance and you want perf/w be at least 400? Run it at 50W.

No, there's a pretty obvious sweet spot there 125 - 250w:

I'm not sure why you decided to do two more charts with just power tuning and no voltage tuning. We can already see the exact same data in TPU's power scaling review, save for your example scaling a tad better above 200w (well within sample variance). I suppose if your point was to validate their data than you did good. Otherwise it's a missed opportunity to provide something we haven't already seen.

 

[JustBenching]

I know that, I just think Intel could have done better. There are several issues with Arrow lake. Insanely high latency may be the worst.

The IMC is on a different tile so yeah...sad for gamers.

Hence why it's important that when we want to demonstrate which architecture is more efficient, we try and put each into it's sweet spot and eliminate any variables that we can. We want to see what the architecture is capable of in an ideal scenario. Arrow lake launches in a more ideal power / voltage scenario (I assume, haven't seen a power scaling review of it yet) while the 13900KS does not.



That wasn't what I was trying to say but I agree with you that the two are going to be about the same. I was just pointing out that the issues with the voltage table. It's well known that the high end 13th and 14th gen CPUs use too much voltage at any of their higher frequencies (4.8 GHz and above). This is due to the way these CPUs were tuned by Intel and variability in silicon quality.



You don't necessarily need these chips to make a better comparison. You can UV the 13900KS and 265K into their sweet spots in addition to your power limiting.

Honestly ideally you'd want 5 samples to see variability but yeah that's a tall ask. It's why having multiple review outlets is important. A good example is the variability of power consumption and needed voltages for lower end Zen 1 parts. Awhile back Derbauer made a video showing a notable difference (20-30w) in power consumption of AMD's mid-range Ryzen 1600X. With all variables controlled it turned about to just be sample variation.

From testing both a 13900k and a 14900k I don't think this is remotely true. You don't have much room for undervolting in either one of them, which means they are already running at the lowest voltage possible. Compared to 12900k, which I could easily drop from the 1.27 stock voltage to 1.06.

@BoggledBeagle Could you test again at 65 watts? Supposedly that's where ARL peaks in efficiency compared to RL, I just wanna see if that's remotely true.

 

[BoggledBeagle]

@BoggledBeagle Could you test again at 65 watts? Supposedly that's where ARL peaks in efficiency compared to RL, I just wanna see if that's remotely true.

I cannot, I disassembled the test system. Note that the 160 and 100W results were remarkably similar to each other and since 65W is very close to 100W, I think it is safe to presume, that the performance difference at 65W would be very similar to the other results

 

[N/A]

No, there's a pretty obvious sweet spot there 125 - 250w:



I'm not sure why you decided to do two more charts with just power tuning and no voltage tuning. We can already see the exact same data in TPU's power ...

This is a sweet range, not a spot. But I agree that some voltage tuning had to take place.