Testing NPU in Core Ultra CPU with which applications?

[Shazamy]

Hi all,

Just got a new Core Ultra 7 165H laptop. I'm used to benchmarking CPUs and GPUs, but now with the NPU being part of the die, I want to know what applications actually take advantage of it as of today? Is there a benchmarking tool as well? How do I see any real world benefits?

Thanks in advance!

 

[pressing on]

Hi all,

Just got a new Core Ultra 7 165H laptop. I'm used to benchmarking CPUs and GPUs, but now with the NPU being part of the die, I want to know what applications actually take advantage of it as of today? Is there a benchmarking tool as well? How do I see any real world benefits?

Thanks in advance!

Here's an Intel link that may help. The only app I have heard of that uses the Meteor Lake NPU is Zoom with its AI Assistant.

 

[Ware]

I want to know what applications actually take advantage of it as of today? Is there a benchmarking tool as well? How do I see any real world benefits?

Look at OpenVINO.
If you want a quick and easy example then try Audacity. There are OpenVINO plugins for Audacity that can use CPU,GPU, and NPU for AI effects.
You could make a small audio file and compare the time it takes to complete the effects on different CPU/GPU/NPU, and maybe even combinations of each.

 

[Shazamy]

Thanks for the replies guys! That's pretty much what I expected, I heard about Zoom using it, even for the background effects, but they work fine as is, so not sure how to test if the NPU makes them work any better, or takes load off the CPU (which might be the more important use case).
I guess now that NPUs are becoming more accessible we'll start seeing them being used more soon

 

[Ware]

not sure how to test if the NPU makes them work any better, or takes load off the CPU

 

[Shazamy]

Thanks Ware! I'll be sure to check it out.

I'm also wondering, is the NPU so unique that it can't be used for the same tasks as the CPU? Or can it?
Basically, in a hypothetical scenario where all the P+E cores are at %100 utilization, would the 2 NPU cores kick in and try to help (assuming there's no NPU specific workload also going on)?

 

[Solaris17]

Thanks Ware! I'll be sure to check it out.

I'm also wondering, is the NPU so unique that it can't be used for the same tasks as the CPU? Or can it?
Basically, in a hypothetical scenario where all the P+E cores are at %100 utilization, would the 2 NPU cores kick in and try to help (assuming there's no NPU specific workload also going on)?

No they wouldn’t, the entire point is that workloads that are programmed specifically on the npu run on the NPU because of the speed. They are work load specific though, or else NPUs wouldn’t exist.

 

[Shazamy]

That what I was thinking @Solaris17.
I only asked because Intel counts the 2 NPU cores on the ARK page for the Core Ultra CPUs as if they're part of the CPU portion of the package, yet it counts the Arc GPU cores separately.
It seems to me that the NPU should have it's own section and distinction.

 

[Ware]

Intel counts the 2 NPU cores

You seem to think the LP E Cores are the NPU. They are x86 cores.
AFAIK NPU it's own thing, like an evolution of GNA.

edit:
AFAIK the LP E Cores are meant to save power by handling low performance/priority tasks, and the Thread Director should handle them as needed.

is the NPU so unique that it can't be used for the same tasks as the CPU? Or can it?

You can see in Audacity that CPU,GPU, and NPU can do the same work load.
The difference is mostly how long it takes, and how much power it takes to do the job.
Some OpenVINO benchmarks are about efficiency.
GNA could be used for noise reduction at lower power consumption than CPU, but also lower quality.
It was a way to offload the CPU to save battery, not outperform CPU.
I don't expect NPU to outperform GPU, but to do some of the same(AI) work more efficiently.