Intel Won't Compete Against NVIDIA's High-End AI Dominance Soon, Starts Laying Off Over 2,200 Workers Across US

[Nomad76]

Intel's taking a different path with its Gaudi 3 accelerator chips. It's staying away from the high-demand market for training big AI models, which has made NVIDIA so successful. Instead, Intel wants to help businesses that need cheaper AI solutions to train and run smaller specific models and open-source options. At a recent event, Intel talked up Gaudi 3's "price performance advantage" over NVIDIA's H100 GPU for inference tasks. Intel says Gaudi 3 is faster and more cost-effective than the H100 when running Llama 3 and Llama 2 models of different sizes.

Intel also claims that Gaudi 3 is as power-efficient as the H100 for large language model (LLM) inference with small token outputs and does even better with larger outputs. The company even suggests Gaudi 3 beats NVIDIA's newer H200 in LLM inference throughput for large token outputs. However, Gaudi 3 doesn't match up to the H100 in overall floating-point operation throughput for 16-bit and 8-bit formats. For bfloat16 and 8-bit floating-point precision matrix math, Gaudi 3 hits 1,835 TFLOPS in each format, while the H100 reaches 1,979 TFLOPS for BF16 and 3,958 TFLOPS for FP8.



In an interview with CRN, Anil Nanduri, head of Intel's AI acceleration office, stated that purchasing decisions for AI training infrastructure have primarily focused on performance rather than cost.

"And if you think in that context, there is an incumbent benefit, where all the frontier model research, all the capabilities are developed on the de facto platform where you're building it, you're researching it, and you're, in essence, subconsciously optimizing it as well. And then to make that port over [to a different platform] is work.
The world we are starting to see is people are questioning the [return on investment], the cost, the power and everything else. This is where—I don't have a crystal ball—but the way we think about it is, do you want one giant model that knows it all?", Anil Nanduri, the head of Intel's AI acceleration office.

Intel believes that for many businesses, the answer is "no" and they will instead opt for smaller models based on tasks with less performance demands. Nanduri said that while the Gaudi 3 can't "catch up" to NVIDIA's latest GPUs, from a head-to-head performance standpoint, Gaudi 3 chips are ideal to enable the right systems to run task-based and open source models.

On a different subject, Intel has announced major job cuts in several states as part of its wider plan to shrink its workforce. The company will eliminate 1,300 positions in Oregon, 385 in Arizona, 319 in California, and 251 in Texas. Intel has a workforce of over 23,000 in Oregon, 12,000 in Arizona, 13,500 in California, and 2,100 in Texas. The layoffs are set to take place over a 14-day period starting November 15.

View at TechPowerUp Main Site | Source

 

[TheLostSwede]

Does this mean that none of the new promised fabs will be built then? Also, does that mean Intel has to pay back all that free government money they got?

 

[Eternit]

This is rather obvious they won compete in high end AI and GPU. It's getting obvious they won't compete in high end fabs against TSMC. a few more years and they won't compete in high end CPU design. Probably they will end up as a patent troll.

 

[Nomad76]

Does this mean that none of the new promised fabs will be built then? Also, does that mean Intel has to pay back all that free government money they got?

I presume we can expect something like... "We need to build fabs to supply DoD and Pentagon contracts." it's a $3.5B or $4B contract

 

[Chaitanya]

This is rather obvious they won compete in high end AI and GPU. It's getting obvious they won't compete in high end fabs against TSMC. a few more years and they won't compete in high end CPU design. Probably they will end up as a patent troll.

They already seem to be getting their latest CPUs fab at TSMC.

 

[Zareek]

So layoff some of the plebs that built the empire and give the executives raises and bonuses for their pain and suffering. Modern big business is Feudalism rediscovered!

 

[Wirko]

Probably they will end up as a patent troll.

That's the business world equivalent of a neutron star?

 

[[XC] Oj101]

This is rather obvious they won compete in high end AI and GPU. It's getting obvious they won't compete in high end fabs against TSMC. a few more years and they won't compete in high end CPU design. Probably they will end up as a patent troll.

It's hard to believe that at one stage they had dominance is basically every market they were involved with, as well as untouchable fabs.

 

[Daven]

I'm taking this to mean that Falcon Shores and all future high end Intel GPU compute products are dead. This would also call into question all of Intel's GPU plans.

Intel probably just wants to provide server CPUs inside high end AI compute boxes wherever possible and open up a new market of entry level AI products with the Gaudi product series.

 

[64K]

The DoD money they will get anyway. That's a separate deal but I think Intel saw the writing on the wall with the CHIPS Act money not coming so cut, cut, cut.

 

[Nomad76]

The DoD money they will get anyway. That's a separate deal but I think Intel saw the writing on the wall with the CHIPS Act money not coming so cut, cut, cut.

Qualcomm still thinks about Intel, it is said to tell more after the next revenue report..

 

[SOAREVERSOR]

It's hard to believe that at one stage they had dominance is basically every market they were involved with, as well as untouchable fabs.

It's gone back and forth. As it has with AMD, nvidia, and more. They rise and they fall and on and on this goes.

 

[igormp]

Does this mean that none of the new promised fabs will be built then? Also, does that mean Intel has to pay back all that free government money they got?

I believe that's still going to happen since it's a matter of the US having a fab to compete with the ones in Asia, so the government will be backing this anyway.

If the fabs will be advanced enough to make the best products, that's an entirelly different question.

Anyhow, the inference space is getting pretty packed. Not only Nvidia is in there, but AMD and many other players are entering this area. Even companies in China have built their own accelerators for that.

 

[TheLostSwede]

I believe that's still going to happen since it's a matter of the US having a fab to compete with the ones in Asia, so the government will be backing this anyway.

What about the fab they were going to build in Germany? They've obviously already canned whatever they promised Italy. Intel already has a bunch of fabs in the US, which they most likely could swap out the equipment in. Intel isn't exactly leading the industry these days so...

If the fabs will be advanced enough to make the best products, that's an entirelly different question.

Obviously that's already not the case, or they wouldn't outsource to TSMC.

 

[Dirt Chip]

Green for the top, red and blue will fight the leftovers of mid-low.
AMD need to target intel as long they are relatively weak and push them out of consumer GPU’s.

A very good year for us non top end gamers is coming…

 

[SOAREVERSOR]

I believe that's still going to happen since it's a matter of the US having a fab to compete with the ones in Asia, so the government will be backing this anyway.

If the fabs will be advanced enough to make the best products, that's an entirelly different question.

Anyhow, the inference space is getting pretty packed. Not only Nvidia is in there, but AMD and many other players are entering this area. Even companies in China have built their own accelerators for that.

It's not entirely about competing with Asia though that is part of it. There are two larger factors driving this.

1.) Taiwan is at risk from China. In the chance of another Trump or Trump like conservative getting into office there is high risk that China will take Taiwan knowing nothing will be done about it. So other countries cannot be stuck depending on Taiwan forever.
2.) COVID showed the massive failure points in the global supply chain and just how fragile it is. Having a domestic capability reduces risk exposure if the supply chain is disrupted again.

The US can't afford not do this no matter how much of the money ends up being wasted.

 

[Sarajiel]

What about the fab they were going to build in Germany?

At the moment, Germany and Poland, which is supposed to do integration for the German fab, are "delayed" for two years, which most likely means that they lose at least parts of the state and EU funding.

 

[80-watt Hamster]

AMD need to target intel as long they are relatively weak and push them out of consumer GPU’s.

Why would you want this?

 

[igormp]

What about the fab they were going to build in Germany?

No idea, but I believe this one isn't meant for leading nodes. IIRC, the major investments where in the Ohio one (lots of money being thrown into that one), and Israel (which is halted, afaik).

Intel already has a bunch of fabs in the US, which they most likely could swap out the equipment in. Intel isn't exactly leading the industry these days so...

I believe the existing fabs are in use, but not of them are capable of making new stuff. I think it's better for them to keep those going as they already are and get new top-notch stuff in some specific fabs, like the Ohio one, but idk about all the internals and finances of that, so I'm just guessing here lol

Obviously that's already not the case, or they wouldn't outsource to TSMC.

They have to make CPUs that compete with others, hence why they went for what's current best.
However, not everyone is making CPUs, and not everything requires the exact best at the moment. Nvidia went with Samsung's 8nm for Ampere, which was an already outdated node, but they managed to deliver enough and likely had good pricing.
If intel manages to make their fab have good enough pricing to compete, they'll likely snatch some sales from other fabs, specially in for the likes of microcontrollers and whatnot.

The US may also give incentives for US-companies to make use of an US-fab, such as Microsoft, Amazon, Google and whatnot, just to guarantee their political sovereignty.

Green for the top, red and blue will fight the leftovers of mid-low.
AMD need to target intel as long they are relatively weak and push them out of consumer GPU’s.

A very good year for us non top end gamers is coming…

Are you aware this is not related to GPUs, but rather dedicated accelerators? Those would be closer to AMD's Alveo rather than their Instinct lineup (ok, Alveo os closer to Agilex, but Intel seems to be doing jackshit with their FPGA division).

It's not entirely about competing with Asia though that is part of it. There are two larger factors driving this.

1.) Taiwan is at risk from China. In the chance of another Trump or Trump like conservative getting into office there is high risk that China will take Taiwan knowing nothing will be done about it. So other countries cannot be stuck depending on Taiwan forever.
2.) COVID showed the massive failure points in the global supply chain and just how fragile it is. Having a domestic capability reduces risk exposure if the supply chain is disrupted again.

The US can't afford not do this no matter how much of the money ends up being wasted.

Yeah, there's lots of politics involved that I'd rather not discuss here, but the point is that the US really wants to be independent in that front, and Intel is their best shot currently.

 

[Dr. Dro]

The US can't afford not do this no matter how much of the money ends up being wasted.

Sigh, I would write a rebuttal; but it'd only get us both thread banned for politics. It's true that the US - and all other Western nations need to bring the semiconductor industry home, though. But since it's a national security interest for the Americans, it's safe to say the government will back Intel here.

Why would you want this?

The usual benevolent image of a ruthless megacorporation I reckon.

 

[Daven]

It's gone back and forth. As it has with AMD, nvidia, and more. They rise and they fall and on and on this goes.

This is not correct. Intel's problems are far reaching across the company and two decades in the making. This is the first time since Intel came into being that it is threaten with at best spinning off its divisions or being bought out and at worst bankruptcy.

 

[Hardcore Games]

The AI BS does nothing to perk up gaming. Gaming needs maybe 12 TFLOPS for 9th gen right now, 8GB VRAM is being squeezed even in GTX 2000 cards and RTX 3000 is even worse for limited VRAM.

 

[Eternit]

It's not entirely about competing with Asia though that is part of it. There are two larger factors driving this.

1.) Taiwan is at risk from China. In the chance of another Trump or Trump like conservative getting into office there is high risk that China will take Taiwan knowing nothing will be done about it. So other countries cannot be stuck depending on Taiwan forever.
2.) COVID showed the massive failure points in the global supply chain and just how fragile it is. Having a domestic capability reduces risk exposure if the supply chain is disrupted again.

The US can't afford not do this no matter how much of the money ends up being wasted.

It is not about what you want, it is about what you can have. They can build the fabs, but they are unable to have there world's best process.

The AI BS does nothing to perk up gaming. Gaming needs maybe 12 TFLOPS for 9th gen right now, 8GB VRAM is being squeezed even in GTX 2000 cards and RTX 3000 is even worse for limited VRAM.

I won't be surprised if in a few years Apple will took over gaming. They are making great SoCs and soon they can create a gaming console competing with Sony and Microsoft. If that will be successful then it will be natural for games moving towards Mac as they will have the same architecture as console. Microsoft is slowly killing desktop applications and Java Script works on Linux and Mac the same way as on Windows so soon Windows wont be needed in business and personal ussage will follow. Linux and Java Script works well on ARM so there will be no need for x86.

 

[Dragokar]

Does this mean that none of the new promised fabs will be built then? Also, does that mean Intel has to pay back all that free government money they got?

Of course........NOT

 

[ZoneDymo]

actually kinda worried about Intel