Thursday, October 17th 2024
Intel Won't Compete Against NVIDIA's High-End AI Dominance Soon, Starts Laying Off Over 2,200 Workers Across US
Intel's taking a different path with its Gaudi 3 accelerator chips. It's staying away from the high-demand market for training big AI models, which has made NVIDIA so successful. Instead, Intel wants to help businesses that need cheaper AI solutions to train and run smaller specific models and open-source options. At a recent event, Intel talked up Gaudi 3's "price performance advantage" over NVIDIA's H100 GPU for inference tasks. Intel says Gaudi 3 is faster and more cost-effective than the H100 when running Llama 3 and Llama 2 models of different sizes.
Intel also claims that Gaudi 3 is as power-efficient as the H100 for large language model (LLM) inference with small token outputs and does even better with larger outputs. The company even suggests Gaudi 3 beats NVIDIA's newer H200 in LLM inference throughput for large token outputs. However, Gaudi 3 doesn't match up to the H100 in overall floating-point operation throughput for 16-bit and 8-bit formats. For bfloat16 and 8-bit floating-point precision matrix math, Gaudi 3 hits 1,835 TFLOPS in each format, while the H100 reaches 1,979 TFLOPS for BF16 and 3,958 TFLOPS for FP8.
In an interview with CRN, Anil Nanduri, head of Intel's AI acceleration office, stated that purchasing decisions for AI training infrastructure have primarily focused on performance rather than cost.
On a different subject, Intel has announced major job cuts in several states as part of its wider plan to shrink its workforce. The company will eliminate 1,300 positions in Oregon, 385 in Arizona, 319 in California, and 251 in Texas. Intel has a workforce of over 23,000 in Oregon, 12,000 in Arizona, 13,500 in California, and 2,100 in Texas. The layoffs are set to take place over a 14-day period starting November 15.
Sources:
CRN, Data Center Dynamics
Intel also claims that Gaudi 3 is as power-efficient as the H100 for large language model (LLM) inference with small token outputs and does even better with larger outputs. The company even suggests Gaudi 3 beats NVIDIA's newer H200 in LLM inference throughput for large token outputs. However, Gaudi 3 doesn't match up to the H100 in overall floating-point operation throughput for 16-bit and 8-bit formats. For bfloat16 and 8-bit floating-point precision matrix math, Gaudi 3 hits 1,835 TFLOPS in each format, while the H100 reaches 1,979 TFLOPS for BF16 and 3,958 TFLOPS for FP8.
"And if you think in that context, there is an incumbent benefit, where all the frontier model research, all the capabilities are developed on the de facto platform where you're building it, you're researching it, and you're, in essence, subconsciously optimizing it as well. And then to make that port over [to a different platform] is work.Intel believes that for many businesses, the answer is "no" and they will instead opt for smaller models based on tasks with less performance demands. Nanduri said that while the Gaudi 3 can't "catch up" to NVIDIA's latest GPUs, from a head-to-head performance standpoint, Gaudi 3 chips are ideal to enable the right systems to run task-based and open source models.
The world we are starting to see is people are questioning the [return on investment], the cost, the power and everything else. This is where—I don't have a crystal ball—but the way we think about it is, do you want one giant model that knows it all?", Anil Nanduri, the head of Intel's AI acceleration office.
On a different subject, Intel has announced major job cuts in several states as part of its wider plan to shrink its workforce. The company will eliminate 1,300 positions in Oregon, 385 in Arizona, 319 in California, and 251 in Texas. Intel has a workforce of over 23,000 in Oregon, 12,000 in Arizona, 13,500 in California, and 2,100 in Texas. The layoffs are set to take place over a 14-day period starting November 15.
48 Comments on Intel Won't Compete Against NVIDIA's High-End AI Dominance Soon, Starts Laying Off Over 2,200 Workers Across US
People still like to user their browsers on Windows instead of ChromeOS or in an Android tablet tho.
Reason is one is what I want, the other is what will be good for AMD.
Jensen will just add another 10% to the price of the 5090.
so many bad tidings in less than 2 moths. A few weeks ago they release their new server chips and blam! ..AMD releases their latest that stomps intel by at least 40% performance and load improvement with the 9005.
Not a single victory (omitting fud and marketing spin). Only losses.
www.techpowerup.com/320571/intel-ohio-fab-opening-delayed-to-2027-2028
Arizona is the only thing coming any time soon.And they're going to have to keep using TSMC, since I doubt their Arizona fab is going to save them, since it always takes around 12 months to tune a new fab and a new node, so they're going to be stuck between a rock and TSMC for some time.
The issue here is that Intel isn't a foundry business, they don't have the experience of dealing with third parties and third party issues, unlike the likes of TSMC, Samsung, GloFo and UMC to name a few. Yes, there are other in-house foundries, but only Intel is pretending to offer foundry services. There are so many parts to being a foundry service that Intel has no clue about and it's going to take them at least a handful of years to catch up with the competition of they're serious about being a foundry business.
recall watching part of a video in youtube were intel execs were drilled by the congress (i think) about laying off people, getting money and the execs still getting bonuses.Sadly, I didnt watch the whole thing and cant find it.
Ignore that, wrong info. It was a video about Boeing exec.It sucks, but thats how corporate works.Karma is a beautiful and scary thing.Well, we apparently have zero issue in Ngreedia, sorry, nvidia having a monopoly on GPUs, so why not have one on CPU's?
I kid, I understand part of it, but not sure why nobody seems to care about the current GPU situation.
As long your favorite team wins I guess.
AMD has already been 'dead' for how many times now? I believe it has more lives than a cat at this point.
Intel will survive just the same. Too big to fail. If anything, we're seeing an empire WAY past its expiry date, topple. That's fine. Gotta break things down before you can really start anew. AMD will claw in market share. Intel won't lose it all. They'll have to revive themselves, but they have all the IP to do so.
If AMD can bounce back from Bulldozer, Intel can bounce back from out of the abyss of their Lakes. Maybe they should start making CPUs people actually want, instead of power hungry re-releases of the same junk over and over.
Quote from article:
"The world we are starting to see is people are questioning the [return on investment], the cost, the power and everything else. This is where—I don't have a crystal ball—but the way we think about it is, do you want one giant model that knows it all?", Anil Nanduri, the head of Intel's AI acceleration office."
Dude, do you even English? Thats not even a sentence. What are you actually saying? What did you even want to say? Or are you just randomly asking yourself questions in the mirror, trying to come off smart? This is even less content than your average politician's quote. 'Head of AI'...
Intel: their chips are not successful, they're stuck in x86, and they have a direct competitor with a better product right now. Additionally, you need a magnifying glass to find a remaining USP in any of their product lines.Do not expect me to upgrade anytime soon, is the only possible answer here.... somehow though I'm quite convinced I'm not the one that needs to sell a product or needs to buy one, but AMD and Intel really do. Its funny how the world's turned around on us, interesting perspective.
But... I think people are getting too many panties in bunches around the chip developments lately, the real elephant to me is AI and how people are rushing towards the next cliff, only to figure out its, indeed, a cliff that you can fall from, and the descent's pretty long. Like Lemmings, they're all going there though. Must run fastuuhhr or AI will destroy us! Its hilarious to see, and reminds me of the dotcom bubble and the crypto craze. How did that work out again...
Everything you see now is banking on the supposed payoff from AI, but in the consumer world, there is no payoff at all. There's only data collection, and AI is a perfect tool to keep that going when people stop frequenting social media.
What if it would turn into a Fab only company? just manufacturing as they used to do? the company will survive and thrive probably but for consumers a player in the field is gone.
Thanks for those that are keeping it clean.
EDIT:: nvm it was impossible.