It looks like a single CCD has an MTr/mm^2 of 128.2. That's the same as the 5090. 4 nm confirmed!

AMD told Chips and Cheese that these are literally the same CCDs that go into Ryzen 9000, which is indeed using TSMC N4P.

DavenIt looks like a single CCD has an MTr/mm^2 of 128.2. That's the same as the 5090. 4 nm confirmed!

The process is called 4n, and the N stands for Nvidia. That's just the original 5nm with some Nvidia mods, just like Ada was. Its Nvidia 5nm confirmed.

SquaredAMD told Chips and Cheese that these are literally the same CCDs that go into Ryzen 9000, which is indeed using TSMC N4P.

In the same interview, they say this chip uses an organic substrate to connect the CCDs with lower latency and power than the desktop parts.

The IO looks like a regular GPU. Even USB controllers look fine. Modern graphics cards are equipped with USB-C ports.

Bruno VieiraIn the same interview, they say this chip uses an organic substrate to connect the CCDs with lower latency and power than the desktop parts.

Desktop parts use organic substrate, and the huge number of wires is the reason IOD can't be placed closer to CCDs. This APU seems to include something more advanced that enables the chips to sit next to each other. It could be Local Si Interconnect (LSI), which is approximately the TSMC's version of EMIB.

AMD absolutely nailed the first generation with minimal setbacks. I never expected an MCM APU with 16 fucking cores to deliver such solid battery life while going toe-to-toe with Apple; despite using an inferior process node and lacking mac OS-level optimization. I can’t help but wonder how powerful the "Halo next-gen" will be, especially given the overwhelmingly positive reception.

Bruno Vieiran the same interview, they say this chip uses an organic substrate to connect the CCDs with lower latency and power than the desktop parts.

Yes, a new version of Fanout Infinity Link with organic interposer. They used something similar on Navi 31 to connect MCDs to GCD. Infinity Link brings a lot more density and bandwidth than Infinity Fabric.



It looks like they are trialing this interconnect on Strix Halo and they will implement it across Zen6 chiplets and IOD as a new high-bandwidth, low latency and high-efficiency interconnect standard. Quite exciting, indeed.

AlB80The IO looks like a regular GPU. Even USB controllers look fine. Modern graphics cards are equipped with USB-C ports.

No. Regular GPU die has less diverse logic and functions than IOD.
Some modern GPUs have USB-C port as an interface for DP video signal, and not for carrying USB or PCIe data.

On Strix Halo IOD, USB3 and USB4 PHY are additional pieces of logic, as well as NPU, which are not present on GPU die.

Imagine they got rid of the NPU and give up 8 more CU's.

Tek-CheckYes, a new version of Fanout Infinity Link with organic interposer. They used something similar on Navi 31 to connect MCDs to GCD. Infinity Link brings a lot more density and bandwidth than Infinity Fabric.

Is this the same as RDL (Re-Distribution Layer)? In my (limited) understanding, RDL is a thin and dense multilayer PCB, which is bonded as a whole, or maybe built layer by layer, on top of the organic substrate, under the chiplets. AMD didn't want to disclose if they used it in Navi 31 or MI300, at least not initially.

DenverAMD absolutely nailed the first generation with minimal setbacks.

It is 1st gen but a lot of experience building MCMs has been funneled into this product. AMD could call it "Instinct AI 101", it would seem right.

Sound_CardImagine they got rid of the NPU and give up 8 more CU's.

Just an idea: CPU thread scheduling is so hard that no one can get it right these days, and "AI" has the potential to help here. If AMD and MS wanted of course.

Sound_CardImagine they got rid of the NPU and give up 8 more CU's.

Not now. NPU is needed for AI workloads, power management, other productivity tasks, etc. Its size takes space of 12 CUs. They need NPU on a chip like this for further development, as the next gen of NPU should go over 100 TOPS. As the logic shrinks in the next gen, they will be able to add more CUs and IO on a die.

It took them four iterations to come to this maximum size design for the package used. It's better to have such product in the market than wait another year or so for yet another lab chip iteration to be perfected. It's more practical the way it is.

WirkoIs this thread same as RDL (Re-Distribution Layer)? In my (limited) understanding, RDL is a thin and dense multilayer PCB, which is bonded as a whole, or maybe built layer by layer, on top of the organic substrate, under the chiplets. AMD didn't want to disclose if they used it in Navi 31 or MI300, at least not initially.

Final MI300 is CoWoS-S, though they did have CoWoS-R as test chip. Navi 31 uses InFO-R/oS packaging with 4 RDL layers.
semianalysis.com/2023/06/12/amd-mi300-taming-the-hype-ai-performance/

For Strix Halo, we don't know, until we know. It's either InFO-R or InFO-L, or another interation.

I want this for desktop so much.

Tek-CheckNo. Regular GPU die has less diverse logic and functions than IOD.
Some modern GPUs have USB-C port as an interface for DP video signal, and not for carrying USB or PCIe data.

On Strix Halo IOD, USB3 and USB4 PHY are additional pieces of logic, as well as NPU, which are not present on GPU die.

nv tensor cores are not NPU?

ymdhisI want this for desktop so much.

Transplanting that notebook mobo to a tower case should be possible.

WirkoTransplanting that notebook mobo to a tower case should be possible.

No need, ITX mobos featuring this chip will probably be announced within this year.

AlB80nv tensor cores are not NPU?

Tensor cores are found in SMs. Each SM integrates specialized hardware, including tensor cores, RT cores, texture units, etc., whereas NPU is a separate, dedicated logic on mobility APUs for neural processing. Two different designs.

Tek-CheckNot now. NPU is needed for AI workloads, power management, other productivity tasks, etc. Its size takes space of 12 CUs. They need NPU on a chip like this for further development, as the next gen of NPU should go over 100 TOPS. As the logic shrinks in the next gen, they will be able to add more CUs and IO on a die.

Is the NPU actually, technically needed zo ? AFAIK power management has never needed a dedicated NPU, nor did AI tasks.

It seems to me that it "needs" to be there only due to Microsoft's commercial tantrum.

WirkoDesktop parts use organic substrate, and the huge number of wires is the reason IOD can't be placed closer to CCDs. This APU seems to include something more advanced that enables the chips to sit next to each other. It could be Local Si Interconnect (LSI), which is approximately the TSMC's version of EMIB.

the first picture shows the IFOP on CCDs and IOD perfectly aligned, something that would be impossible with the N6 IOD on desktop. This alone could what allows thedies to be put next to each other

TPUniqueIs the NPU actually, technically needed zo ? AFAIK power management has never needed a dedicated NPU, nor did AI tasks.
It seems to me that it "needs" to be there only due to Microsoft's commercial tantrum.

It is needed if equivalent AI acceleration, similar in function to tensor cores, cannot be found integrated within GPU cores.

TPUniqueIs the NPU actually, technically needed zo ? AFAIK power management has never needed a dedicated NPU, nor did AI tasks.

It seems to me that it "needs" to be there only due to Microsoft's commercial tantrum.

Jim Kelly has an interview somewhere since he has been at Tenstorrent. I remember the convo being something in regards to him talking to a PSU engineer, and even they could benefit from a micro ML processor that costs a couple bucks. NPUs will eventually be in all kinds of devices.

Do you technically need NPU capabilities? Sure don't. Just like you technically don't need RT capabilities to run RT stuff. You can do it all on a GPU with RT cores or even a CPU. It'll just be slower and use more energy. NPU can do something the CPU could do, but it'll do it much faster and use a lot less power. It's kinda why general processors can have multiple fixed function processors/engines on the same package.

Tek-CheckIt is needed if equivalent AI acceleration, similar in function to tensor cores, cannot be found integrated within GPU cores.

Tensor cores are more general than NPUs tho, as in data types, and they require more power than a bare NPU as well.
AMD's software stack is still lacking when it comes to using their GPUs for ML acceleration, I guess that's why they are shoving an NPU in this product.

Tek-CheckNot now. NPU is needed for AI workloads, power management, other productivity tasks, etc. Its size takes space of 12 CUs. They need NPU on a chip like this for further development, as the next gen of NPU should go over 100 TOPS. As the logic shrinks in the next gen, they will be able to add more CUs and IO on a die.

NPUs have their uses, but it looks to me like Microsoft went crazy for hammers, and now every problem looks like a nail to them. But most of those problems obviously aren't nails, and everyone else is using a different and better tool to solve the problem.
Most NPUs in computers don't do anything because almost nothing uses them. Even Copilot, which is what these NPUs are supposed to be for, doesn't use them (yet?).
NPUs are useful for some AI productivity tasks, but most people who need powerful AI accelerators for productivity buy graphics cards with more powerful capabilities than any integrated NPU.
A typical user's AI needs are more efficiently handled by cloud services which can dynamically distribute load across thousands of NPUs/TPUs, not by having their own NPU that does literally nothing 99%+ of the time.
Where are NPUs being used for power management, and what does an NPU do in those situations that can't be done by a basic microcontroller?

Maybe NPUs will end up great in the long run, but it would require a major shift in both how people use their computers and how software uses NPU hardware. I'm expecting it will end up the same way as "smart"/IOT devices, and that these NPUs in everything will be about as useful as wifi connections for fridges and toasters - maybe useful for a few people with specialised needs, but stupid and irrelevant for everyone else.

SpeedyblupiNPUs are useful for some AI productivity tasks, but most people who need powerful AI accelerators for productivity buy graphics cards with more powerful capabilities than any integrated NPU.

NPUs can even accelerate GPU, both in gaming and in productivity. Intel showed this a few days ago on Arrow Lake HX. GPUs can offload some taxing tasks to NPU and this could bring a few percentages in gaming by freeing resources on GPU. We need to see NPU as an enhancer of performance, both for hardware and software.

SpeedyblupiA typical user's AI needs are more efficiently handled by cloud services which can dynamically distribute load across thousands of NPUs/TPUs, not by having their own NPU that does literally nothing 99%+ of the time.

We need to have NPU hardware in the first place on die in order to develop software ecosystem that will use it. We are in the process of developing software and this will only get better. At the end of the say, not all CPUs are sold with NPU and that's fine. Mobility chips need it more at the moment, but make no mistake, more powerful NPUs will eventually come to desktop chips too.

SpeedyblupiWhere are NPUs being used for power management, and what does an NPU do in those situations that can't be done by a basic microcontroller?

Some tasks with parallel processing will save more battery power if offloaded to NPU than done on traditional CPU sequentially.

One example of more flexible data flow through NPU

SpeedyblupiMaybe NPUs will end up great in the long run, but it would require a major shift in both how people use their computers and how software uses NPU hardware. I'm expecting it will end up the same way as "smart"/IOT devices, and that these NPUs in everything will be about as useful as wifi connections for fridges and toasters - maybe useful for a few people with specialised needs, but stupid and irrelevant for everyone else.

We will be using NPUs without even been aware of it, unless you look for it into Task Manager. It's software developers who need to improve applications we commonly use in order to make better use of NPUs. It will take some time. Microsoft is deploying Copilot across their applications, some more meaningful than others, but it will be down to individual applications to improve.

For example, during an average Zoom call in future version of it, NPU will handle denoising, improving picture and sound. It will be a better experience, but you will never think it's because there is NPU. Hence this impression that NPU does not do anything obvious and tangible. It will be also taking care of background things that we usually do not think about.

Tek-CheckNPUs can even accelerate GPU, both in gaming and in productivity. Intel showed this a few days ago on Arrow Lake HX. GPUs can offload some taxing tasks to NPU and this could bring a few percentages in gaming by freeing resources on GPU. We need to see NPU as an enhancer of performance, both for hardware and software.

It does not enhance performance. Intel's example was about running a game AND an assistant (likely a small LLM) in tandem. The assistant was offloaded to the NPU (exactly what this kind of thing is meant for), leaving the GPU free for the game alone.
But the performance of just running the game does not improve if you suddenly plug an NPU in there. NPUs are meant for inference of models, period.

Tek-CheckSome tasks with parallel processing will save more battery power if offloaded to NPU than done on traditional CPU sequentially

Not "some tasks", only model inference, which there aren't many in the desktop world that you run locally at the moment.

igormpIt does not enhance performance. Intel's example was about running a game AND an assistant (likely a small LLM) in tandem. The assistant was offloaded to the NPU (exactly what this kind of thing is meant for), leaving the GPU free for the game alone. But the performance of just running the game does not improve if you suddenly plug an NPU in there. NPUs are meant for inference of models, period.

I am not into semantic disputes just because I did not mention an assistant. NPU can and does enhance the performance of other components if tasks can be offloaded to it. There is no doubt about it. Whether additional application needs to be used for such gains, such as assistant, it does not really matter. What matters is end result.

igormpNot "some tasks", only model inference, which there aren't many in the desktop world that you run locally at the moment.

There are not many, you are right. I enjoy LM Studio and there are plenty of models that can be loaded into it. My system is good enough for 32B models. 70B is already slow. I do not have NPU, to help a bit my GPU. NPUs are not only meant for inference of models, as there are other applications gradually enabling more functions, such as specific editing and content creation tools in Photoshop, Premiere Pro, Lightroom, DaVinci, etc. There are plug-ins that already work and others that are being developed. It will take time to develop entire ecosystem.

Tek-CheckNPU can and does enhance the performance of other components if tasks can be offloaded to it. There is no doubt about it. Whether additional application needs to be used for such gains, such as assistant, it does not really matter. What matters is end result.

The point is that such "tasks" are only related to inference of low precision models.
If your application doesn't make use of such, there's nothing to be gained.
Again, within Intel's example it did not boost the game performance. They first hindered the game's performance by running a new task that used the GPU, and then offloaded it later. Similar fashion to people moving streaming stuff into a different device, it doesn't increase performance per se, just offloads stuff, and given that you added a new piece of hardware, this is kinda expected to happen.

Tek-CheckI do not have NPU, to help a bit my GPU

It would not help, but rather hinder the performance if you have a dedicated GPU.
Reminder that NPUs can only run stuff in either INT4 or INT8 at best, and they are tied to the CPU memory, which limits its performance nonetheless.

Tek-CheckNPUs are not only meant for inference of models, as there are other applications gradually enabling more functions, such as specific editing and content creation tools in Photoshop, Premiere Pro, Lightroom, DaVinci, etc. There are plug-ins that already work and others that are being developed. It will take time to develop entire ecosystem.

The examples you gave are still related to model inference. They run a model to do such tasks, in a similar fashion to your example of webcam effects and whatnot.
Model inference can mean lots of things, not only LLMs.
But yeah, not many desktop stuff making use of such. On mobiles you have tons of smaller models running on-device already, from image gallery stuff, to camera processing, to audio transcription. I'm not sure if those will even make sense on a desktop, but we shall see.