AMD Shows Instinct MI300 Exascale APU with 146 Billion Transistors

[AleksandarK]

During its CES 2023 keynote, AMD announced its latest Instinct MI300 APU, a first of its kind in the data center world. Combining the CPU, GPU, and memory elements into a single package eliminates latency imposed by long travel distances of data from CPU to memory and from CPU to GPU throughout the PCIe connector. In addition to solving some latency issues, less power is needed to move the data and provide greater efficiency. The Instinct MI300 features 24 Zen4 cores with simultaneous multi-threading enabled, CDNA3 GPU IP, and 128 GB of HBM3 memory on a single package. The memory bus is 8192-bit wide, providing unified memory access for CPU and GPU cores. CLX 3.0 is also supported, making cache-coherent interconnecting a reality.

The Instinct MI300 APU package is an engineering marvel of its own, with advanced chiplet techniques used. AMD managed to do 3D stacking and has nine 5 nm logic chiplets that are 3D stacked on top of four 6 nm chiplets with HBM surrounding it. All of this makes the transistor count go up to 146 billion, representing the sheer complexity of a such design. For performance figures, AMD provided a comparison to Instinct MI250X GPU. In raw AI performance, the MI300 features an 8x improvement over MI250X, while the performance-per-watt is "reduced" to a 5x increase. While we do not know what benchmark applications were used, there is a probability that some standard benchmarks like MLPerf were used. For availability, AMD targets the end of 2023, when the "El Capitan" exascale supercomputer will arrive using these Instinct MI300 APU accelerators. Pricing is unknown and will be unveiled to enterprise customers first around launch.



View at TechPowerUp Main Site

 

[ARF]

Instead of focusing on already super developed market segments which do not need so much attention, better AMD focus on the discrete graphics business. Very very lame !

 

[Mysteoa]

Instead of focusing on already super developed market segments which do not need so much attention, better AMD focus on the discrete graphics business. Very very lame !

You must be new. AMD is going where the money is.

 

[ARF]

You must be new. AMD is going where the money is.

There is also 10 times more money in the graphics market ! 8% today market share with outlook for probable bankruptcy, or 80% market share as is case with nvidia !

 

[TheoneandonlyMrK]

There is also 10 times more money in the graphics market ! 8% today market share with outlook for probable bankruptcy, or 80% market share as is case with nvidia !

They wouldn't get anywhere if they didn't diversify.
Cash in equals more cash for rnd.

And I would love to mess with this.

This unironically is the future, and the one Nvidia is most worried about.

And Intel is also working towards.

 

[terroralpha]

Instead of focusing on already super developed market segments which do not need so much attention, better AMD focus on the discrete graphics business. Very very lame !

as an AMD shareholder, i'm so glad they don't listen to random people on the internet.

 

[b4psm4m]

Relavitely speaking, I don't believe there is much money in a GPU market

A 4090 has a die size of 608 mm^2, wheras each chiplet die on a Zen 4 has an area of 70 mm^2. Not to mention all the power delivery circuitry, RAM, PCB and cooling that goes along with producing a GPU.

Of course AMD must keep a hand in the GPU market to stay relevant in that sector and other sectors, but I don't believe it is a big money maker for them.

 

[Daven]

as an AMD shareholder, i'm so glad they don't listen to random people on the internet.

Also as an AMD shareholder I concur. Ironically the MI300 is the ultimate accumulation of CPU, GPU and RAM technologies tailored for the most lucrative big data markets. Go AMD!

 

[Mysteoa]

There is also 10 times more money in the graphics market ! 8% today market share with outlook for probable bankruptcy, or 80% market share as is case with nvidia !

You are talking about Gaming GPUs market, which doesn't bring as much money as the server space and professional. Just look at Nvidia selling the same gaming gpu die to professionals or server for 10x the price.

Nvidia makes 58% of it revenue from Compute & Networking and 42% from Graphics. Graphics include Gaming GPU and streaming, professional Quadro/RTX GPUs and Visualization software, Automotive. So if we remove the non gaming stuff, the revenue just from Gaming is even less, it's about 26% from NVidia Q3 2022 Financial Results. Why would AMD aim for a market that brings only 26% of NVIDIA revenue when it can go for the bigger one?

 

[HBSound]

I am truly excited about this move. This is expressed to me the next time I build a nice PC.
The PC could be based on a DUAL Processor that supports CPU/GPU without using an external heater core (GPU).
From my vantage point, CPUs have a longer life cycle. Sad, the 5090 is on the way, and the 4090 just showed up.

This is exciting.

 

[spnidel]

There is also 10 times more money in the graphics market ! 8% today market share with outlook for probable bankruptcy, or 80% market share as is case with nvidia !

ever heard the phrase "don't put all your eggs in one basket"?

 

[Vya Domus]

Intel has a similar solution with Ponte Vecchio, which I don't know if it ever actually saw the light of day and it was announced like 2 years ago. The matter of the fact is AMD has way more experience making these kinds of things and knowing how they price their datacenter products compared to Intel they're gonna walk all over them.

 

[Leiesoldat]

Ponte Vecchio is inside the Aurora supercomputer which is still on track, albeit slowly.

 

[nexus290]

Instead of focusing on already super developed market segments which do not need so much attention, better AMD focus on the discrete graphics business. Very very lame !

If I was AMD I don't see why I shouldn't double down on a very profitable segment, and I don't see where AMD has lost focus on discrete graphics business.

There is also 10 times more money in the graphics market ! 8% today market share with outlook for probable bankruptcy, or 80% market share as is case with nvidia !

?????? you do realize that AMD and nvidia have almost the same revenue. Last quarter AMD $5.565 billion net income $1.095 billion nvidia $5.931 billion net income $1.456 billion not to mention AMD's revenue increased while nvidia's revenue shrank quarter on quarter and year on year. AMD as a company is literally on fire. Plus there is more revenue in data center then the consumer P.C. graphics which has shrunk to the lowest level in the last 20 years. So no there is not 10 times more money in graphics.

 

[AnarchoPrimitiv]

There is also 10 times more money in the graphics market ! 8% today market share with outlook for probable bankruptcy, or 80% market share as is case with nvidia !

The x86 T.A.M. when enterprise and consumer are combined is larger than the T.A.M. for graphics

Back on the topic of the article, I REALLY want something like this to trickle down to consumer. I know it probably won't because the ability to upgrade components separately trumps the advantages offered by combining CPU/GPU/Memory, but I'd love if AMD offered a desktop APU with 8 Zen4 cores, at least 24-32 RDNA3 CUs, and 8-16GB of HBM2e/3 (to be buttressed by additional DDR5 slots).....man, the SFF builds would be EPIC!!!

 

[nexus290]

Intel has a similar solution with Ponte Vecchio, which

Ponte Vecchio is inside the Aurora supercomputer which is still on track, albeit slowly.

Aurora was first announced in 2015 and to be finished in 2018.

 

[windwhirl]

Intel has a similar solution with Ponte Vecchio, which I don't know if it ever actually saw the light of day and it was announced like 2 years ago. The matter of the fact is AMD has way more experience making these kinds of things and knowing how they price their datacenter products compared to Intel they're gonna walk all over them.

Argentine government just bought a supercomputer with that.

According to press release from the government, it has 296 Ponte Vecchio accelerators, and 5120 Sapphire Rapids HBM cores paired with 1.66 PB of RAM (?)

 

[john_]

Instead of focusing on already super developed market segments which do not need so much attention, better AMD focus on the discrete graphics business. Very very lame !

There is also 10 times more money in the graphics market ! 8% today market share with outlook for probable bankruptcy, or 80% market share as is case with nvidia !

No. AMD did it's best, especially those last years, first with Polaris and then with it's RDNA1 and 2 offerings. But it was never enough because consumers are addicted to Nvidia's shiny sticker. And tech press are and where much friendlier to Nvidia. Anything bad from AMD is an excuse for huge titles about a disaster. Anything bad from Nvidia is something to be fixed if given the proper time.
The result was AMD to go from 30-40% to just 8%. Whatever AMD was offering wasn't good enough for people. People where always trying to justify paying for Nvidia hardware and Intel, to justify ignoring AMD's offerings. AMD's offerings where good enough only to force Intel and Nvidia to drop their prices, never good enough to be a real alternative.
That's the mentality of people for at least the last 10-15 years. It just got much worst lately.

So. Why invest a gazillion of dollars in a market where people just care about shiny stickers? The university/country/corporation, or the professional/scientist will care about performance not the sticker. So money invested in staff like MI300 will bring back profits. Money invested in gaming GPUs will just make a hole in AMD's financials. Even if the products are better value than the competition. So?

As an old and current AMD shareholder (and lately also Intel and Nvidia shareholder), I would say ***beep*** gamers and retail GPUs. They want Nvidia? Let them pay what Huang asks them. Keep investing in GPUs for products that will end up in servers, keep investing in GPUs for products that will go in laptops, keep investing in GPUs for APUs and products that will go in handheld gaming devices, keep investing in GPUs for securing the next generation of XBOX and PlayStation. But stop losing money in retail market. Sell directly to OEMs if necessary, as Intel is doing, but IGNORE THE ***BEEP*** retail market. It's a black hole. They are celebrating everytime Nvidia wins, they are celebrating everytime AMD is having a difficulty/bad luck.

Let's see what gamers will get from Huang if Nvidia is the only option in the market. And let's see if GPU hardware reviews will be getting the same numbers of viewers/readers if there is only ridiculously expensive options in the market from just one company.

 

[ARF]

The x86 T.A.M. when enterprise and consumer are combined is larger than the T.A.M. for graphics

AMD now has only very small part of those markets, not 100% of them - you can't compare them straight but only relatively.
What I am saying is AMD needs to focus on the market in which there are a lot of possibilities for growth.

If the graphics market is so undesired, then AMD leave it !

No. AMD did it's best, especially those last years, first with Polaris and then with it's RDNA1 and 2 offerings. But it was never enough because consumers are addicted to Nvidia's shiny sticker. And tech press are and where much friendlier to Nvidia. Anything bad from AMD is an excuse for huge titles about a disaster. Anything bad from Nvidia is something to be fixed if given the proper time.
The result was AMD to go from 30-40% to just 8%. Whatever AMD was offering wasn't good enough for people. People where always trying to justify paying for Nvidia hardware and Intel, to justify ignoring AMD's offerings. AMD's offerings where good enough only to force Intel and Nvidia to drop their prices, never good enough to be a real alternative.
That's the mentality of people for at least the last 10-15 years. It just got much worst lately.

So. Why invest a gazillion of dollars in a market where people just care about shiny stickers? The university/country/corporation, or the professional/scientist will care about performance not the sticker. So money invested in staff like MI300 will bring back profits. Money invested in gaming GPUs will just make a hole in AMD's financials. Even if the products are better value than the competition. So?

As an old and current AMD shareholder (and lately also Intel and Nvidia shareholder), I would say ***beep*** gamers and retail GPUs. They want Nvidia? Let them pay what Huang asks them. Keep investing in GPUs for products that will end up in servers, keep investing in GPUs for products that will go in laptops, keep investing in GPUs for APUs and products that will go in handheld gaming devices, keep investing in GPUs for securing the next generation of XBOX and PlayStation. But stop losing money in retail market. Sell directly to OEMs if necessary, as Intel is doing, but IGNORE THE ***BEEP*** retail market. It's a black hole. They are celebrating everytime Nvidia wins, they are celebrating everytime AMD is having a difficulty/bad luck.

Let's see what gamers will get from Huang if Nvidia is the only option in the market. And let's see if GPU hardware reviews will be getting the same numbers of viewers/readers if their is only ridiculusly expensive options in the market from just one company.

This is only AMD to blame. They have big partners like Apple which are inclined to use them instead of nvidia but outside they have very little presence. It's AMD's problem in creating the infrastructure and ecosystem for deliveries.
The average joe doesn't know - if you enforce on them AMD Radeon, they will buy it.
Today, they are enforcing nvidia, so yeah, there is no choice.

Don't forget how EVGA (or XFX years ago) which hold around 40% of the US market, simply jumped off ship.
These are the opportunities that AMD must focus in..

 

[Aquinus]

representing the sheer complexity of the such design.

Bad grammar. Bold spot should be something more like "of such a design."

Aside from the gramatical error, this sounds pretty awesome. No normal person will ever own one of these, but the fact that AMD is producing this means it's possible and feasible.

 

[john_]

This is only AMD to blame.

I haven't read the rest of your post. Will do it now. But you just prove me correct. With your first sentence.

Why invest in stuff only to hear "it's your fault I am not buying your stuff"?

Enjoy the Huang Era of gaming. You might need to get a consumer loan first.

PS. Read it. Apple is only a minimal part of the market, especially now they are moving to internal solutions. Other big companies use AMD as a secondary option. Nvidia and Intel are their primary options. Remember how OEMs where sabotaging their own AMD products a few years back? The average Joe will go to Nvidia because everyone sells/promotes Nvidia, no matter what. EVGA is just an AIB. It's not a huge retailer like for example Amazon, to push AMD's products to it's customers. EVGA's sticker value is directly connected with Nvidia's power in the market. EVGA selling AMD cards would had less success than Sapphire for example. For AMD EVGA's contribution would have been small.

PS2. Keep laughing.

 

[ARF]

I haven't read the rest of your post. Will do it now. But you just prove me correct. With your first sentence.

Why invest in stuff only to hear "it's your fault I am not buying your stuff"?

Enjoy the Huang Era of gaming. You might need to get a consumer loan first.

I don't work with nvidia
I said AMD is to blame because most of the time it doesn't deliver up to the expectations. The users want the fastest. AMD never delivers the fastest.
Actually, if RDNA 1 and before Vega 64, etc garbage, they simply ignored the top of the line market - the halo which sells the whole product lineup beneath it.

PS. Read it. Apple is only a minimal part of the market, especially now they are moving to internal solutions. Other big companies use AMD as a secondary option. Nvidia and Intel are their primary options. Remember how OEMs where sabotaging their own AMD products a few years back? The average Joe will go to Nvidia because everyone sells/promotes Nvidia, no matter what. EVGA is just an AIB. It's not a huge retailer like for example Amazon, to push AMD's products to it's customers. EVGA's sticker value is directly connected with Nvidia's power in the market. EVGA selling AMD cards would had less success than Sapphire for example. For AMD EVGA's contribution would have been small.

Exactly. AMD must concentrate on working relationships with those "OEM"s.
Or build its own ecosystem without them - direct sales, etc.
That's AMD to blame - because nvidia and Intel concentrate on buying the "OEM", so the result which you see in the stores is 99% Intel-nvidia exclusives.

 

[john_]

etc garbage

Keep proving me correct.

Exactly. AMD must concentrate on working relationships with those "OEM"s.

Yeah, they will invite them for tea. It's funny how you present relationships between corporations who sign contracts of billions of dollars. You make them look like they are so easy.

That's AMD to blame - because nvidia and Intel concentrate on buying the "OEM", so the result which you see in the stores is 99% Intel-nvidia exclusives.

Enjoy the Huang Era.

PS I am expecting a laughing smile, nothing less.

 

[siluro818]

There is also 10 times more money in the graphics market ! 8% today market share with outlook for probable bankruptcy, or 80% market share as is case with nvidia !

Does TPU give out some kind of cluelessness award on these forums that you're trying out for, or?

 

[AnotherReader]

I said AMD is to blame because most of the time it doesn't deliver up to the expectations. The users want the fastest. AMD never delivers the fastest.
Actually, if RDNA 1 and before Vega 64, etc garbage, they simply ignored the top of the line market - the halo which sells the whole product lineup beneath it.

I think you've forgotten the era before Pascal. Were the 7970, 290X, and Fury X not attempting to take the crown?