• Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

Intel Won't Compete Against NVIDIA's High-End AI Dominance Soon, Starts Laying Off Over 2,200 Workers Across US

Status
Not open for further replies.

Nomad76

News Editor
Staff member
Joined
May 21, 2024
Messages
738 (3.32/day)
Intel's taking a different path with its Gaudi 3 accelerator chips. It's staying away from the high-demand market for training big AI models, which has made NVIDIA so successful. Instead, Intel wants to help businesses that need cheaper AI solutions to train and run smaller specific models and open-source options. At a recent event, Intel talked up Gaudi 3's "price performance advantage" over NVIDIA's H100 GPU for inference tasks. Intel says Gaudi 3 is faster and more cost-effective than the H100 when running Llama 3 and Llama 2 models of different sizes.

Intel also claims that Gaudi 3 is as power-efficient as the H100 for large language model (LLM) inference with small token outputs and does even better with larger outputs. The company even suggests Gaudi 3 beats NVIDIA's newer H200 in LLM inference throughput for large token outputs. However, Gaudi 3 doesn't match up to the H100 in overall floating-point operation throughput for 16-bit and 8-bit formats. For bfloat16 and 8-bit floating-point precision matrix math, Gaudi 3 hits 1,835 TFLOPS in each format, while the H100 reaches 1,979 TFLOPS for BF16 and 3,958 TFLOPS for FP8.



In an interview with CRN, Anil Nanduri, head of Intel's AI acceleration office, stated that purchasing decisions for AI training infrastructure have primarily focused on performance rather than cost.

"And if you think in that context, there is an incumbent benefit, where all the frontier model research, all the capabilities are developed on the de facto platform where you're building it, you're researching it, and you're, in essence, subconsciously optimizing it as well. And then to make that port over [to a different platform] is work.
The world we are starting to see is people are questioning the [return on investment], the cost, the power and everything else. This is where—I don't have a crystal ball—but the way we think about it is, do you want one giant model that knows it all?", Anil Nanduri, the head of Intel's AI acceleration office.

Intel believes that for many businesses, the answer is "no" and they will instead opt for smaller models based on tasks with less performance demands. Nanduri said that while the Gaudi 3 can't "catch up" to NVIDIA's latest GPUs, from a head-to-head performance standpoint, Gaudi 3 chips are ideal to enable the right systems to run task-based and open source models.

On a different subject, Intel has announced major job cuts in several states as part of its wider plan to shrink its workforce. The company will eliminate 1,300 positions in Oregon, 385 in Arizona, 319 in California, and 251 in Texas. Intel has a workforce of over 23,000 in Oregon, 12,000 in Arizona, 13,500 in California, and 2,100 in Texas. The layoffs are set to take place over a 14-day period starting November 15.

View at TechPowerUp Main Site | Source
 

TheLostSwede

News Editor
Joined
Nov 11, 2004
Messages
17,797 (2.42/day)
Location
Sweden
System Name Overlord Mk MLI
Processor AMD Ryzen 7 7800X3D
Motherboard Gigabyte X670E Aorus Master
Cooling Noctua NH-D15 SE with offsets
Memory 32GB Team T-Create Expert DDR5 6000 MHz @ CL30-34-34-68
Video Card(s) Gainward GeForce RTX 4080 Phantom GS
Storage 1TB Solidigm P44 Pro, 2 TB Corsair MP600 Pro, 2TB Kingston KC3000
Display(s) Acer XV272K LVbmiipruzx 4K@160Hz
Case Fractal Design Torrent Compact
Audio Device(s) Corsair Virtuoso SE
Power Supply be quiet! Pure Power 12 M 850 W
Mouse Logitech G502 Lightspeed
Keyboard Corsair K70 Max
Software Windows 10 Pro
Benchmark Scores https://valid.x86.fr/yfsd9w
Does this mean that none of the new promised fabs will be built then? Also, does that mean Intel has to pay back all that free government money they got?
 
Joined
Apr 26, 2023
Messages
129 (0.21/day)
This is rather obvious they won compete in high end AI and GPU. It's getting obvious they won't compete in high end fabs against TSMC. a few more years and they won't compete in high end CPU design. Probably they will end up as a patent troll.
 

Nomad76

News Editor
Staff member
Joined
May 21, 2024
Messages
738 (3.32/day)
Does this mean that none of the new promised fabs will be built then? Also, does that mean Intel has to pay back all that free government money they got?
I presume we can expect something like... "We need to build fabs to supply DoD and Pentagon contracts." it's a $3.5B or $4B contract
 
Joined
Mar 7, 2011
Messages
4,640 (0.92/day)
This is rather obvious they won compete in high end AI and GPU. It's getting obvious they won't compete in high end fabs against TSMC. a few more years and they won't compete in high end CPU design. Probably they will end up as a patent troll.
They already seem to be getting their latest CPUs fab at TSMC.
 
Joined
Apr 18, 2019
Messages
941 (0.45/day)
Location
The New England region of the United States
System Name Gaming Rig
Processor Ryzen 7 3800X
Motherboard Gigabyte X570 Aurus Pro Wifi
Cooling Noctua NH-D15 chromax.black
Memory 32GB(2x16GB) Patriot Viper DDR4-3200C16
Video Card(s) EVGA RTX 3060 Ti
Storage Samsung 970 EVO Plus 1TB (Boot/OS)|Hynix Platinum P41 2TB (Games)
Display(s) Gigabyte G27F
Case Corsair Graphite 600T w/mesh side
Audio Device(s) Logitech Z625 2.1 | cheapo gaming headset when mic is needed
Power Supply Corsair HX850i
Mouse Redragon M808-KS Storm Pro (Great Value)
Keyboard Redragon K512 Shiva replaced a Corsair K70 Lux - Blue on Black
VR HMD Nope
Software Windows 11 Pro x64
Benchmark Scores Nope
So layoff some of the plebs that built the empire and give the executives raises and bonuses for their pain and suffering. Modern big business is Feudalism rediscovered!
 
Joined
Jan 3, 2021
Messages
3,635 (2.50/day)
Location
Slovenia
Processor i5-6600K
Motherboard Asus Z170A
Cooling some cheap Cooler Master Hyper 103 or similar
Memory 16GB DDR4-2400
Video Card(s) IGP
Storage Samsung 850 EVO 250GB
Display(s) 2x Oldell 24" 1920x1200
Case Bitfenix Nova white windowless non-mesh
Audio Device(s) E-mu 1212m PCI
Power Supply Seasonic G-360
Mouse Logitech Marble trackball, never had a mouse
Keyboard Key Tronic KT2000, no Win key because 1994
Software Oldwin
Joined
Jan 23, 2012
Messages
374 (0.08/day)
Location
South Africa
Processor Pentium II 400 @ 516MHz
Motherboard AOpen AX6BC EZ
Cooling Stock
Memory 192MB PC-133
Video Card(s) 2x Voodoo 12MB in SLI, S3 Trio64V+
Storage Maxtor 40GB
Display(s) ViewSonic E90
Audio Device(s) Sound Blaster 16
Software Windows 98 SE
This is rather obvious they won compete in high end AI and GPU. It's getting obvious they won't compete in high end fabs against TSMC. a few more years and they won't compete in high end CPU design. Probably they will end up as a patent troll.
It's hard to believe that at one stage they had dominance is basically every market they were involved with, as well as untouchable fabs.
 
Joined
Dec 12, 2016
Messages
1,960 (0.67/day)
I'm taking this to mean that Falcon Shores and all future high end Intel GPU compute products are dead. This would also call into question all of Intel's GPU plans.

Intel probably just wants to provide server CPUs inside high end AI compute boxes wherever possible and open up a new market of entry level AI products with the Gaudi product series.
 

64K

Joined
Mar 13, 2014
Messages
6,773 (1.72/day)
Processor i7 7700k
Motherboard MSI Z270 SLI Plus
Cooling CM Hyper 212 EVO
Memory 2 x 8 GB Corsair Vengeance
Video Card(s) Temporary MSI RTX 4070 Super
Storage Samsung 850 EVO 250 GB and WD Black 4TB
Display(s) Temporary Viewsonic 4K 60 Hz
Case Corsair Obsidian 750D Airflow Edition
Audio Device(s) Onboard
Power Supply EVGA SuperNova 850 W Gold
Mouse Logitech G502
Keyboard Logitech G105
Software Windows 10
The DoD money they will get anyway. That's a separate deal but I think Intel saw the writing on the wall with the CHIPS Act money not coming so cut, cut, cut.
 

Nomad76

News Editor
Staff member
Joined
May 21, 2024
Messages
738 (3.32/day)
The DoD money they will get anyway. That's a separate deal but I think Intel saw the writing on the wall with the CHIPS Act money not coming so cut, cut, cut.
Qualcomm still thinks about Intel, it is said to tell more after the next revenue report..
 
Joined
Apr 13, 2022
Messages
1,202 (1.21/day)
It's hard to believe that at one stage they had dominance is basically every market they were involved with, as well as untouchable fabs.
It's gone back and forth. As it has with AMD, nvidia, and more. They rise and they fall and on and on this goes.
 
Joined
May 10, 2023
Messages
388 (0.65/day)
Location
Brazil
Processor 5950x
Motherboard B550 ProArt
Cooling Fuma 2
Memory 4x32GB 3200MHz Corsair LPX
Video Card(s) 2x RTX 3090
Display(s) LG 42" C2 4k OLED
Power Supply XPG Core Reactor 850W
Software I use Arch btw
Does this mean that none of the new promised fabs will be built then? Also, does that mean Intel has to pay back all that free government money they got?
I believe that's still going to happen since it's a matter of the US having a fab to compete with the ones in Asia, so the government will be backing this anyway.

If the fabs will be advanced enough to make the best products, that's an entirelly different question.

Anyhow, the inference space is getting pretty packed. Not only Nvidia is in there, but AMD and many other players are entering this area. Even companies in China have built their own accelerators for that.
 

TheLostSwede

News Editor
Joined
Nov 11, 2004
Messages
17,797 (2.42/day)
Location
Sweden
System Name Overlord Mk MLI
Processor AMD Ryzen 7 7800X3D
Motherboard Gigabyte X670E Aorus Master
Cooling Noctua NH-D15 SE with offsets
Memory 32GB Team T-Create Expert DDR5 6000 MHz @ CL30-34-34-68
Video Card(s) Gainward GeForce RTX 4080 Phantom GS
Storage 1TB Solidigm P44 Pro, 2 TB Corsair MP600 Pro, 2TB Kingston KC3000
Display(s) Acer XV272K LVbmiipruzx 4K@160Hz
Case Fractal Design Torrent Compact
Audio Device(s) Corsair Virtuoso SE
Power Supply be quiet! Pure Power 12 M 850 W
Mouse Logitech G502 Lightspeed
Keyboard Corsair K70 Max
Software Windows 10 Pro
Benchmark Scores https://valid.x86.fr/yfsd9w
I believe that's still going to happen since it's a matter of the US having a fab to compete with the ones in Asia, so the government will be backing this anyway.
What about the fab they were going to build in Germany? They've obviously already canned whatever they promised Italy. Intel already has a bunch of fabs in the US, which they most likely could swap out the equipment in. Intel isn't exactly leading the industry these days so...
If the fabs will be advanced enough to make the best products, that's an entirelly different question.
Obviously that's already not the case, or they wouldn't outsource to TSMC.
 
Joined
Jul 15, 2020
Messages
1,026 (0.63/day)
System Name Dirt Sheep | Silent Sheep
Processor i5-2400 | 13900K (-0.02mV offset)
Motherboard Asus P8H67-M LE | Gigabyte AERO Z690-G, bios F29e Intel baseline
Cooling Scythe Katana Type 1 | Noctua NH-U12A chromax.black
Memory G-skill 2*8GB DDR3 | Corsair Vengeance 4*32GB DDR5 5200Mhz C40 @4000MHz
Video Card(s) Gigabyte 970GTX Mini | NV 1080TI FE (cap at 50%, 800mV)
Storage 2*SN850 1TB, 230S 4TB, 840EVO 128GB, WD green 2TB HDD, IronWolf 6TB, 2*HC550 18TB in RAID1
Display(s) LG 21` FHD W2261VP | Lenovo 27` 4K Qreator 27
Case Thermaltake V3 Black|Define 7 Solid, stock 3*14 fans+ 2*12 front&buttom+ out 1*8 (on expansion slot)
Audio Device(s) Beyerdynamic DT 990 (or the screen speakers when I'm too lazy)
Power Supply Enermax Pro82+ 525W | Corsair RM650x (2021)
Mouse Logitech Master 3
Keyboard Roccat Isku FX
VR HMD Nop.
Software WIN 10 | WIN 11
Benchmark Scores CB23 SC: i5-2400=641 | i9-13900k=2325-2281 MC: i5-2400=i9 13900k SC | i9-13900k=37240-35500
Green for the top, red and blue will fight the leftovers of mid-low.
AMD need to target intel as long they are relatively weak and push them out of consumer GPU’s.

A very good year for us non top end gamers is coming…
 
Joined
Apr 13, 2022
Messages
1,202 (1.21/day)
I believe that's still going to happen since it's a matter of the US having a fab to compete with the ones in Asia, so the government will be backing this anyway.

If the fabs will be advanced enough to make the best products, that's an entirelly different question.

Anyhow, the inference space is getting pretty packed. Not only Nvidia is in there, but AMD and many other players are entering this area. Even companies in China have built their own accelerators for that.
It's not entirely about competing with Asia though that is part of it. There are two larger factors driving this.

1.) Taiwan is at risk from China. In the chance of another Trump or Trump like conservative getting into office there is high risk that China will take Taiwan knowing nothing will be done about it. So other countries cannot be stuck depending on Taiwan forever.
2.) COVID showed the massive failure points in the global supply chain and just how fragile it is. Having a domestic capability reduces risk exposure if the supply chain is disrupted again.

The US can't afford not do this no matter how much of the money ends up being wasted.
 
Joined
Apr 21, 2021
Messages
250 (0.19/day)
System Name Silicon Graphics O2
Processor R5000 / 180MHz
Cooling noisy fan
Memory 384 MB
Storage 4 GB
Case the one with the old logo and proud of it ;)
Software IRIX 6.5
What about the fab they were going to build in Germany?
At the moment, Germany and Poland, which is supposed to do integration for the German fab, are "delayed" for two years, which most likely means that they lose at least parts of the state and EU funding.
 
Joined
Aug 21, 2015
Messages
1,760 (0.51/day)
Location
North Dakota
System Name Office
Processor Ryzen 5600G
Motherboard ASUS B450M-A II
Cooling be quiet! Shadow Rock LP
Memory 16GB Patriot Viper Steel DDR4-3200
Video Card(s) Gigabyte RX 5600 XT
Storage PNY CS1030 250GB, Crucial MX500 2TB
Display(s) Dell S2719DGF
Case Fractal Define 7 Compact
Power Supply EVGA 550 G3
Mouse Logitech M705 Marthon
Keyboard Logitech G410
Software Windows 10 Pro 22H2
Joined
May 10, 2023
Messages
388 (0.65/day)
Location
Brazil
Processor 5950x
Motherboard B550 ProArt
Cooling Fuma 2
Memory 4x32GB 3200MHz Corsair LPX
Video Card(s) 2x RTX 3090
Display(s) LG 42" C2 4k OLED
Power Supply XPG Core Reactor 850W
Software I use Arch btw
What about the fab they were going to build in Germany?
No idea, but I believe this one isn't meant for leading nodes. IIRC, the major investments where in the Ohio one (lots of money being thrown into that one), and Israel (which is halted, afaik).
Intel already has a bunch of fabs in the US, which they most likely could swap out the equipment in. Intel isn't exactly leading the industry these days so...
I believe the existing fabs are in use, but not of them are capable of making new stuff. I think it's better for them to keep those going as they already are and get new top-notch stuff in some specific fabs, like the Ohio one, but idk about all the internals and finances of that, so I'm just guessing here lol
Obviously that's already not the case, or they wouldn't outsource to TSMC.
They have to make CPUs that compete with others, hence why they went for what's current best.
However, not everyone is making CPUs, and not everything requires the exact best at the moment. Nvidia went with Samsung's 8nm for Ampere, which was an already outdated node, but they managed to deliver enough and likely had good pricing.
If intel manages to make their fab have good enough pricing to compete, they'll likely snatch some sales from other fabs, specially in for the likes of microcontrollers and whatnot.

The US may also give incentives for US-companies to make use of an US-fab, such as Microsoft, Amazon, Google and whatnot, just to guarantee their political sovereignty.

Green for the top, red and blue will fight the leftovers of mid-low.
AMD need to target intel as long they are relatively weak and push them out of consumer GPU’s.

A very good year for us non top end gamers is coming…
Are you aware this is not related to GPUs, but rather dedicated accelerators? Those would be closer to AMD's Alveo rather than their Instinct lineup (ok, Alveo os closer to Agilex, but Intel seems to be doing jackshit with their FPGA division).

It's not entirely about competing with Asia though that is part of it. There are two larger factors driving this.

1.) Taiwan is at risk from China. In the chance of another Trump or Trump like conservative getting into office there is high risk that China will take Taiwan knowing nothing will be done about it. So other countries cannot be stuck depending on Taiwan forever.
2.) COVID showed the massive failure points in the global supply chain and just how fragile it is. Having a domestic capability reduces risk exposure if the supply chain is disrupted again.

The US can't afford not do this no matter how much of the money ends up being wasted.
Yeah, there's lots of politics involved that I'd rather not discuss here, but the point is that the US really wants to be independent in that front, and Intel is their best shot currently.
 
Joined
Dec 25, 2020
Messages
7,083 (4.83/day)
Location
São Paulo, Brazil
System Name "Icy Resurrection"
Processor 13th Gen Intel Core i9-13900KS Special Edition
Motherboard ASUS ROG Maximus Z790 Apex Encore
Cooling Noctua NH-D15S upgraded with 2x NF-F12 iPPC-3000 fans and Honeywell PTM7950 TIM
Memory 32 GB G.SKILL Trident Z5 RGB F5-6800J3445G16GX2-TZ5RK @ 7600 MT/s 36-44-44-52-96 1.4V
Video Card(s) ASUS ROG Strix GeForce RTX™ 4080 16GB GDDR6X White OC Edition
Storage 500 GB WD Black SN750 SE NVMe SSD + 4 TB WD Red Plus WD40EFPX HDD
Display(s) 55-inch LG G3 OLED
Case Pichau Mancer CV500 White Edition
Audio Device(s) Apple USB-C + Sony MDR-V7 headphones
Power Supply EVGA 1300 G2 1.3kW 80+ Gold
Mouse Microsoft Classic Intellimouse
Keyboard IBM Model M type 1391405 (distribución española)
Software Windows 11 IoT Enterprise LTSC 24H2
Benchmark Scores I pulled a Qiqi~
The US can't afford not do this no matter how much of the money ends up being wasted.

Sigh, I would write a rebuttal; but it'd only get us both thread banned for politics. It's true that the US - and all other Western nations need to bring the semiconductor industry home, though. But since it's a national security interest for the Americans, it's safe to say the government will back Intel here.

Why would you want this?

The usual benevolent image of a ruthless megacorporation I reckon.
 
Joined
Dec 12, 2016
Messages
1,960 (0.67/day)
It's gone back and forth. As it has with AMD, nvidia, and more. They rise and they fall and on and on this goes.
This is not correct. Intel's problems are far reaching across the company and two decades in the making. This is the first time since Intel came into being that it is threaten with at best spinning off its divisions or being bought out and at worst bankruptcy.
 
Joined
Jan 1, 2019
Messages
520 (0.24/day)
The AI BS does nothing to perk up gaming. Gaming needs maybe 12 TFLOPS for 9th gen right now, 8GB VRAM is being squeezed even in GTX 2000 cards and RTX 3000 is even worse for limited VRAM.
 
Joined
Apr 26, 2023
Messages
129 (0.21/day)
It's not entirely about competing with Asia though that is part of it. There are two larger factors driving this.

1.) Taiwan is at risk from China. In the chance of another Trump or Trump like conservative getting into office there is high risk that China will take Taiwan knowing nothing will be done about it. So other countries cannot be stuck depending on Taiwan forever.
2.) COVID showed the massive failure points in the global supply chain and just how fragile it is. Having a domestic capability reduces risk exposure if the supply chain is disrupted again.

The US can't afford not do this no matter how much of the money ends up being wasted.
It is not about what you want, it is about what you can have. They can build the fabs, but they are unable to have there world's best process.

The AI BS does nothing to perk up gaming. Gaming needs maybe 12 TFLOPS for 9th gen right now, 8GB VRAM is being squeezed even in GTX 2000 cards and RTX 3000 is even worse for limited VRAM.
I won't be surprised if in a few years Apple will took over gaming. They are making great SoCs and soon they can create a gaming console competing with Sony and Microsoft. If that will be successful then it will be natural for games moving towards Mac as they will have the same architecture as console. Microsoft is slowly killing desktop applications and Java Script works on Linux and Mac the same way as on Windows so soon Windows wont be needed in business and personal ussage will follow. Linux and Java Script works well on ARM so there will be no need for x86.
 
Joined
Mar 1, 2021
Messages
498 (0.36/day)
Location
Germany
System Name Homebase
Processor Ryzen 5 5600
Motherboard Gigabyte Aorus X570S UD
Cooling Scythe Mugen 5 RGB
Memory 2*16 Kingston Fury DDR4-3600 double ranked
Video Card(s) AMD Radeon RX 6800 16 GB
Storage 1*512 WD Red SN700, 1*2TB Curcial P5, 1*2TB Sandisk Plus (TLC), 1*14TB Toshiba MG
Display(s) Philips E-line 275E1S
Case Fractal Design Torrent Compact
Power Supply Corsair RM850 2019
Mouse Sharkoon Sharkforce Pro
Keyboard Fujitsu KB955
Does this mean that none of the new promised fabs will be built then? Also, does that mean Intel has to pay back all that free government money they got?
Of course........NOT :D
 
Joined
Feb 11, 2009
Messages
5,578 (0.96/day)
System Name Cyberline
Processor Intel Core i7 2600k -> 12600k
Motherboard Asus P8P67 LE Rev 3.0 -> Gigabyte Z690 Auros Elite DDR4
Cooling Tuniq Tower 120 -> Custom Watercoolingloop
Memory Corsair (4x2) 8gb 1600mhz -> Crucial (8x2) 16gb 3600mhz
Video Card(s) AMD RX480 -> RX7800XT
Storage Samsung 750 Evo 250gb SSD + WD 1tb x 2 + WD 2tb -> 2tb MVMe SSD
Display(s) Philips 32inch LPF5605H (television) -> Dell S3220DGF
Case antec 600 -> Thermaltake Tenor HTCP case
Audio Device(s) Focusrite 2i4 (USB)
Power Supply Seasonic 620watt 80+ Platinum
Mouse Elecom EX-G
Keyboard Rapoo V700
Software Windows 10 Pro 64bit
actually kinda worried about Intel
 
Status
Not open for further replies.
Top