Thursday, October 17th 2024
Intel Won't Compete Against NVIDIA's High-End AI Dominance Soon, Starts Laying Off Over 2,200 Workers Across US
Intel's taking a different path with its Gaudi 3 accelerator chips. It's staying away from the high-demand market for training big AI models, which has made NVIDIA so successful. Instead, Intel wants to help businesses that need cheaper AI solutions to train and run smaller specific models and open-source options. At a recent event, Intel talked up Gaudi 3's "price performance advantage" over NVIDIA's H100 GPU for inference tasks. Intel says Gaudi 3 is faster and more cost-effective than the H100 when running Llama 3 and Llama 2 models of different sizes.
Intel also claims that Gaudi 3 is as power-efficient as the H100 for large language model (LLM) inference with small token outputs and does even better with larger outputs. The company even suggests Gaudi 3 beats NVIDIA's newer H200 in LLM inference throughput for large token outputs. However, Gaudi 3 doesn't match up to the H100 in overall floating-point operation throughput for 16-bit and 8-bit formats. For bfloat16 and 8-bit floating-point precision matrix math, Gaudi 3 hits 1,835 TFLOPS in each format, while the H100 reaches 1,979 TFLOPS for BF16 and 3,958 TFLOPS for FP8.
In an interview with CRN, Anil Nanduri, head of Intel's AI acceleration office, stated that purchasing decisions for AI training infrastructure have primarily focused on performance rather than cost.
On a different subject, Intel has announced major job cuts in several states as part of its wider plan to shrink its workforce. The company will eliminate 1,300 positions in Oregon, 385 in Arizona, 319 in California, and 251 in Texas. Intel has a workforce of over 23,000 in Oregon, 12,000 in Arizona, 13,500 in California, and 2,100 in Texas. The layoffs are set to take place over a 14-day period starting November 15.
Sources:
CRN, Data Center Dynamics
Intel also claims that Gaudi 3 is as power-efficient as the H100 for large language model (LLM) inference with small token outputs and does even better with larger outputs. The company even suggests Gaudi 3 beats NVIDIA's newer H200 in LLM inference throughput for large token outputs. However, Gaudi 3 doesn't match up to the H100 in overall floating-point operation throughput for 16-bit and 8-bit formats. For bfloat16 and 8-bit floating-point precision matrix math, Gaudi 3 hits 1,835 TFLOPS in each format, while the H100 reaches 1,979 TFLOPS for BF16 and 3,958 TFLOPS for FP8.
"And if you think in that context, there is an incumbent benefit, where all the frontier model research, all the capabilities are developed on the de facto platform where you're building it, you're researching it, and you're, in essence, subconsciously optimizing it as well. And then to make that port over [to a different platform] is work.Intel believes that for many businesses, the answer is "no" and they will instead opt for smaller models based on tasks with less performance demands. Nanduri said that while the Gaudi 3 can't "catch up" to NVIDIA's latest GPUs, from a head-to-head performance standpoint, Gaudi 3 chips are ideal to enable the right systems to run task-based and open source models.
The world we are starting to see is people are questioning the [return on investment], the cost, the power and everything else. This is where—I don't have a crystal ball—but the way we think about it is, do you want one giant model that knows it all?", Anil Nanduri, the head of Intel's AI acceleration office.
On a different subject, Intel has announced major job cuts in several states as part of its wider plan to shrink its workforce. The company will eliminate 1,300 positions in Oregon, 385 in Arizona, 319 in California, and 251 in Texas. Intel has a workforce of over 23,000 in Oregon, 12,000 in Arizona, 13,500 in California, and 2,100 in Texas. The layoffs are set to take place over a 14-day period starting November 15.
48 Comments on Intel Won't Compete Against NVIDIA's High-End AI Dominance Soon, Starts Laying Off Over 2,200 Workers Across US
Intel probably just wants to provide server CPUs inside high end AI compute boxes wherever possible and open up a new market of entry level AI products with the Gaudi product series.
If the fabs will be advanced enough to make the best products, that's an entirelly different question.
Anyhow, the inference space is getting pretty packed. Not only Nvidia is in there, but AMD and many other players are entering this area. Even companies in China have built their own accelerators for that.
AMD need to target intel as long they are relatively weak and push them out of consumer GPU’s.
A very good year for us non top end gamers is coming…
1.) Taiwan is at risk from China. In the chance of another Trump or Trump like conservative getting into office there is high risk that China will take Taiwan knowing nothing will be done about it. So other countries cannot be stuck depending on Taiwan forever.
2.) COVID showed the massive failure points in the global supply chain and just how fragile it is. Having a domestic capability reduces risk exposure if the supply chain is disrupted again.
The US can't afford not do this no matter how much of the money ends up being wasted.
However, not everyone is making CPUs, and not everything requires the exact best at the moment. Nvidia went with Samsung's 8nm for Ampere, which was an already outdated node, but they managed to deliver enough and likely had good pricing.
If intel manages to make their fab have good enough pricing to compete, they'll likely snatch some sales from other fabs, specially in for the likes of microcontrollers and whatnot.
The US may also give incentives for US-companies to make use of an US-fab, such as Microsoft, Amazon, Google and whatnot, just to guarantee their political sovereignty.Are you aware this is not related to GPUs, but rather dedicated accelerators? Those would be closer to AMD's Alveo rather than their Instinct lineup (ok, Alveo os closer to Agilex, but Intel seems to be doing jackshit with their FPGA division).Yeah, there's lots of politics involved that I'd rather not discuss here, but the point is that the US really wants to be independent in that front, and Intel is their best shot currently.
All gamers need competition to keep prices low and more players means more competition. I'll take 3 competitors instead of 2, especially at mid and low end. And hopefully the 2 not competing at the top can do so the next generation because having a single company at the top end sucks for those who want top tier gpus to be at least vaguely affordable.