NVIDIA Prepared to Offer Custom Chip Designs to AI Clients

[T0@st]

NVIDIA is reported to be setting up an AI-focused semi-custom chip design business unit, according to inside sources known to Reuters—it is believed that Team Green leadership is adapting to demands leveraged by key data-center customers. Many companies are seeking cheaper alternatives, or have devised their own designs (budget/war chest permitting)—NVIDIA's current range of AI GPUs are simply off-the-shelf solutions. OpenAI has generated the most industry noise—their alleged early 2024 fund-raising pursuits have attracted plenty of speculative/kind-of-serious interest from notable semiconductor personalities.

Team Green is seemingly reacting to emerging market trends—Jensen Huang (CEO, president and co-founder) has hinted that NVIDIA custom chip designing services are on the cusp. Stephen Nellis—a Reuters reporter specializing in tech industry developments—has highlighted select NVIDIA boss quotes from an incoming interview piece: "We're always open to do that. Usually, the customization, after some discussion, could fall into system reconfigurations or recompositions of systems." The Team Green chief teased that his engineering team is prepared to take on the challenge meeting exact requests: "But if it's not possible to do that, we're more than happy to do a custom chip. And the benefit to the customer, as you can imagine, is really quite terrific. It allows them to extend our architecture with their know-how and their proprietary information." The rumored NVIDIA semi-custom chip design business unit could be introduced in an official capacity at next month's GTC 2024 Conference.



View at TechPowerUp Main Site | Source

 

[cvaldes]

Amusingly, AI will probably assist in accelerating development of custom chip designs.

Anyhow this makes sense like a bakery branching out into custom cake orders.

The big question is when will AMD and Intel offer similar services to compete with this rumored Nvidia offering? Nvidia is already dominating marketshare right now. Staying with off-the-shelf designs probably isn't going to be competitive advantage.

 

[Denver]

Big corporations are silently working on their chips to avoid being in Nvidia's clutches forever, so it doesn't make sense for their strategy; except in cases like Mediatek or Nintendo working to integrate Nvidia GPUs into their SOCs.

 

[cvaldes]

Yeah, but how long will it take to ship a new chip? It took Apple 10+ years of development to ship their first in-house designs and it wasn't a completely new design.

They did this over time. Modifying Arm cores, eventually moving parts to their own design. Hell, Apple used Imagination PowerVR graphics cores while they had moved the CPU cores to their own design. Apple chips with their first ML cores came out about 5 years ago within a year of Nvidia's Tensor cores.

And Apple had a really great VLSI team because they were designing their own ASICs. You don't just wake up one day and say "let's make an AI chip" and see it tape out six months from now. Your snot-nosed 22-year old new hires aren't going to be rolling out AI ASICs anytime soon.

For now a lot of companies want an advantage while they develop their own in-house solutions. That's what Nvidia is addressing. And if Big Company A launches their new AI chip in 2030, what happens in 2025, 2026, 2027, 2028, 2029? And there will be new small companies that simply don't have the resources to launch into a multi-year chip development program with no guarantee that they will stay afloat.

 

[pk67]

Amusingly, AI will probably assist in accelerating development of custom chip designs.

Anyhow this makes sense like a bakery branching out into custom cake orders.

The big question is when will AMD and Intel offer similar services to compete with this rumored Nvidia offering? Nvidia is already dominating marketshare right now. Staying with off-the-shelf designs probably isn't going to be competitive advantage.

AMD should ask them to design RDNA 6 chips with AI and RT performance more efficient than next gen GPU after Blackwell . I'm curious what price they would ask for that.

 

[john_]

Amusingly, AI will probably assist in accelerating development of custom chip designs.

Anyhow this makes sense like a bakery branching out into custom cake orders.

The big question is when will AMD and Intel offer similar services to compete with this rumored Nvidia offering? Nvidia is already dominating marketshare right now. Staying with off-the-shelf designs probably isn't going to be competitive advantage.

AMD is already doing custom APUs. In AI is a little (too much) late, so they need time to catch up with Nvidia. Intel's focus is on manufacturing.
Nvidia is doing custom chips a little time now, to circumvent restrictions and keep selling to China.

 

[cvaldes]

One thing Nvidia can offer is a massive AI software development ecosystem right now to compliment their hardware offerings.

People designing their own silicon will have to find their own software solutions which also takes a lot of time, money and expertise. Just putting a bunch of transistors on a wafer is not the sole task at hand.

I know a lot of TPU commenters tend to ignore the software development aspects. It's not just a spec sheet or benchmark.

 

[mb194dc]

It'll take years to build extra data centers, by which time the likes of Google will probably have their own LLM chips.

You can't just click your fingers and scale it up.

It can take even longer than that depending on if you need to build power infrastructure as well. Which is probable.

No need for Nvidia and paying 80% nosebleed margins on their chips. So obviously they'll try to retain that business.

 

[Veseleil]

One thing Nvidia can offer is a massive AI software development ecosystem right now to compliment their hardware offerings.

People designing their own silicon will have to find their own software solutions which also takes a lot of time, money and expertise. Just putting a bunch of transistors on a wafer is not the sole task at hand.

I know a lot of TPU commenters tend to ignore the software development aspects. It's not just a spec sheet or benchmark.

Couldn't said it better. Software development and especially optimization takes a lot of time and it's really expensive, and one of the reasons all of them are pushing AI so hard. Just look at Intel and their ARC drivers. Software is now holding back the progress, not the hardware, not anymore.

 

[cvaldes]

Couldn't said it better. Software development and especially optimization takes a lot of time and it's really expensive, and one of the reasons all of them are pushing AI so hard. Just look at Intel and their ARC drivers. Software is now holding back the progress, not the hardware, not anymore.

Which is why they have crazy performance improvements every time Intel releases new graphics drivers.

This was discussed heavily when the ARC graphics cards debuted and I said several times that the silicon looked fine, the drivers needed work. Yet most people online fixated on core counts, transistors, clock speeds, etc. rather than take software into account.

Intel has a better software engineering team than AMD. They will have a better chance at taking a dent out of Nvidia's AI accelerator marketshare in the next few years.

AMD really needs to step up their software for any chance of staying relevant.

 

[Solaris17]

You don't just wake up one day and say "let's make an AI chip" and see it tape out six months from now

It’s about 1.5 years

 

[cvaldes]

It’s about 1.5 years

With a seasoned VLSI team that has years of experience working together after years of research, generation to generation releases. Not some brand new startup or new organization cobbled together within a larger corporate entity (especially where corporate politics might muck things up).

Look at Apple's acquisition of P.A. Semi. It took nearly ten years to get a shipping product from the result of this acquisition.

I worked at a place that designed semiconductors. You don't just wave a magic wand and have a design that tapes out in 1.5 years. That wasn't possible thirty years ago and it certainly isn't possible today.

That's why Nvidia is doing all of this. They know some of these potential customers may roll their own in a few years. But not tomorrow, not next month, not next year.

Let's say Nvidia signs up 50 customers for this new custom chip design business. I doubt if two of them will tape out their own chips in 1.5 years. And whatever does tape out may not be competitive enough to whatever Nvidia has in 2026.

Again, all of this chip design work will be accelerated by AI assistance. It takes computer chips to design computer chips in 2024. You still need someplace to start. The era of using a pencil, pad of paper and sliderule to design silicon is over.

Look at AMD. They've been in the GPU business for decades and they still can't take a major bite out of Nvidia. It takes enormous skill and resources to play in the semiconductor business today.

 

[Solaris17]

With a seasoned VLSI team that has years of experience working together after years of research, generation to generation releases. Not some brand new startup or new organization cobbled together within a larger corporate entity (especially where corporate politics might muck things up).

Look at Apple's acquisition of P.A. Semi. It took nearly ten years to get a shipping product from the result of this acquisition.

I worked at a place that designed semiconductors. You don't just wave a magic wand and have a design that tapes out in 1.5 years. That wasn't possible thirty years ago and it certainly isn't possible today.

That's why Nvidia is doing all of this. They know some of these potential customers may roll their own in a few years. But not tomorrow, not next month, not next year.

But for what these chips are used for for companies that have in house divisions it’s absolutely possible

 

[cvaldes]

But for what these chips are used for for companies that have in house divisions it’s absolutely possible

I guarantee you that most of these companies that sign up for Nvidia's custom chip business do not currently believe they have the resources (time, expertise, money) to do the design in house and ship in a reasonable amount of time.

The ones that do will not sign up for this. But that's just a handful of companies on the planet.

I live in a place where autonomous vehicles are heavily tested on public roads. Pretty much every single one of them has an Nvidia GPU onboard. This includes some of the largest car corporations on the planet. That's right, Toyota, Ford, GM, they all use Nvidia GPUs for this usage case because they don't have in-house silicon good enough to do the job.

And as I pointed out earlier, it's not just the silicon. There's the software side of things.

You guys seriously think that Nvidia will be offering something that no one wants? Doesn't it make sense that maybe Nvidia got enough feedback (privately) where they thought they could turning this into a business? Isn't that how a lot of businesses start?

 

[Solaris17]

Oh for sure the kind of people that want these obviously aren’t the kind that can pull it off I’m just saying there definitely exists places that can do it in a very short amount of time. With the shortest iv seen being 1.5

 

[Redwoodz]

With a seasoned VLSI team that has years of experience working together after years of research, generation to generation releases. Not some brand new startup or new organization cobbled together within a larger corporate entity (especially where corporate politics might muck things up).

Look at Apple's acquisition of P.A. Semi. It took nearly ten years to get a shipping product from the result of this acquisition.

I worked at a place that designed semiconductors. You don't just wave a magic wand and have a design that tapes out in 1.5 years. That wasn't possible thirty years ago and it certainly isn't possible today.

That's why Nvidia is doing all of this. They know some of these potential customers may roll their own in a few years. But not tomorrow, not next month, not next year.

Let's say Nvidia signs up 50 customers for this new custom chip design business. I doubt if two of them will tape out their own chips in 1.5 years. And whatever does tape out may not be competitive enough to whatever Nvidia has in 2026.

Again, all of this chip design work will be accelerated by AI assistance. It takes computer chips to design computer chips in 2024. You still need someplace to start. The era of using a pencil, pad of paper and sliderule to design silicon is over.

Look at AMD. They've been in the GPU business for decades and they still can't take a major bite out of Nvidia. It takes enormous skill and resources to play in the semiconductor business today.

Nope. That was the old days when Nvidia's stranglehold in compute software mattered a lot. It will matter little here, unless Jensen can trick everyone into believing his BS.There are so many more players in this ballgame the past is irrelevent. Your statements clearly show the fake dominance Nvidia has enjoyed in gpu, where comparible products are deemed less than because it's not made by team green. I hope you realise AMD has been building custom SoC's for many customers, many more than Nvidia yet you treat them like dirtbags.
Nvidia is the noob here when it comes to designing custom chips.

 

[cvaldes]

Oh for sure the kind of people that want these obviously aren’t the kind that can pull it off I’m just saying there definitely exists places that can do it in a very short amount of time. With the shortest iv seen being 1.5

Well, name some companies.

I was driving up 101 a few days ago and I didn't see any billboards proclaiming: "Custom AI silicon! Get a tailored design taped out in 18 months! Call now for a free quote!"

Nope. That was the old days when Nvidia's stranglehold in compute software mattered a lot. It will matter little here, unless Jensen can trick everyone into believing his BS.There are so many more players in this ballgame the past is irrelevent. Your statements clearly show the fake dominance Nvidia has enjoyed in gpu, where comparible products are deemed less than because it's not made by team green. I hope you realise AMD has been building custom SoC's for many customers, many more than Nvidia yet you treat them like dirtbags.
Nvidia is the noob here when it comes to designing custom chips.

I know AMD does custom SoC design work beyond Sony and Microsoft. I'm an indirect shareholder of AMD, Intel, and Nvidia, I read the occasional annual report.

But for sure, AMD's custom SoC design work has not materialized into major revenue.

Again, people here are ignoring the software side of things. In order to use AI chips you need software. Currently, Nvidia's AI software development environment greatly exceeds that of AMD, Intel, and others in terms of addressing a broad selection of interests.

Lots of companies will eventually roll their own but NOT TOMORROW, NOT NEXT MONTH, NOT NEXT YEAR.

If there are comparable products and services being offered in the custom AI chip design marketplace, who are those competitors? List six or seven companies. List their customers. And their quarterly revenues over the past 24-36 months. Don't worry, I can look up AMD, NVDA, and INTC quarterly reports online.

And let's not forget that there's a manufacturing component. Let's just say that magically I have an ASIC design ready to tape out, gifted by my fairy chip godmother. What chance do I have calling up TSMC and asking them to set aside some wafers on their most advanced node? Credit check? Oh yes, of course, here's my driver's license.

Nvidia already has that leverage. How much business has Nvidia been giving to TSMC and Samsung over the past ten years?

 

[Solaris17]

I was driving up 101 a few days ago and I didn't see any billboards proclaiming: "Custom AI silicon! Get a tailored design taped out in 18 months! Call now for a free quote!"

Big companies like MS, Meta and AWS aren’t making them for others

 

[cvaldes]

Big companies like MS, Meta and AWS aren’t making them for others

No one said all companies would use this new chip design service. You conveniently listed three companies all above $1 trillion in market capitalization. There are 497 other companies in the S&P 500 and only three others above $1T in market cap: AAPL, NVDA themselves, and GOOG.

Let's skip past Berkshire-Hathaway and look at the next few companies: Visa, JPMorganChase, UnitedHealth, Walmart, and Mastercard. Do you think any of these companies have a VLSI team that is ready to tape out a competitive AI accelator design in 1.5 years? Or Exxon Mobil, Johnson & Johnson, Proctor & Gamble, Home Depot, Merck?

Even Tesla has not weaned themselves off of Nvidia, AMD, and Intel silicon.

Let's face it, Nvidia doesn't have the resources to accept everyone knocking at their door. There will be other companies who will offer AI chip design services. But who are they?

I bet the top 20% of the S&P 500 makes about 85% of the revenue.

If Nvidia can handle something like 50 custom chip customers, they can pretty much focus on the S&P 100. What is going to happen is a temporary squeeze on engineering talent. These companies will be fighting to attract the right talent.

 

[Solaris17]

No one said all companies would use this new chip design service. You listed three companies all above $1 trillion in market capitalization. There are 497 other companies in the S&P 500 and only three others above $1T in market cap: AAPL, NVDA themselves, and GOOG.

Let's skip past Berkshire-Hathaway and look at the next few companies: Visa, JPMorganChase, UnitedHealth, Walmart, and Mastercard. Do you think any of these companies have a VLSI team that is ready to tape out a competitive AI accelator design in 1.5 years?

Not at all like I said in my previous post. I was just commenting on the statement you made regarding how long it takes to make them. Which I took as you saying it couldn’t be done sooner.

 

[cvaldes]

Not at all like I said in my previous post. I was just commenting on the statement you made regarding how long it takes to make them. Which I took as you saying it couldn’t be done sooner.

Some companies are better situated in creating custom chip designs on their own than others.

In the same way, if you asked ten people to start bakeries, not all of them would open on the same day with the same volume, same foot traffic, same revenue, etc.

Again, please list the companies that can go from concept to tapeout in 1.5 years. I would love to hear about these players. You're the one who brought it up.

 

[Redwoodz]

Well, name some companies.

I was driving up 101 a few days ago and I didn't see any billboards proclaiming: "Custom AI silicon! Get a tailored design taped out in 18 months! Call now for a free quote!"


I know AMD does custom SoC design work beyond Sony and Microsoft. I'm an indirect shareholder of AMD, Intel, and Nvidia, I read the occasional annual report.

But for sure, AMD's custom SoC design work has not materialized into major revenue.

Again, people here are ignoring the software side of things. In order to use AI chips you need software. Currently, Nvidia's AI software development environment greatly exceeds that of AMD, Intel, and others in terms of addressing a broad selection of interests.

Lots of companies will eventually roll their own but NOT TOMORROW, NOT NEXT MONTH, NOT NEXT YEAR.

If there are comparable products and services being offered in the custom AI chip design marketplace, who are those competitors? List six or seven companies. List their customers. And their quarterly revenues over the past 24-36 months. Don't worry, I can look up AMD, NVDA, and INTC quarterly reports online.

And let's not forget that there's a manufacturing component. Let's just say that magically I have an ASIC design ready to tape out, gifted by my fairy chip godmother. What chance do I have calling up TSMC and asking them to set aside some wafers on their most advanced node? Credit check? Oh yes, of course, here's my driver's license.

Nvidia already has that leverage. How much business has Nvidia been giving to TSMC and Samsung over the past ten years?

AMD MI300 Performance – Faster Than H100, But How Much?

MI400 Broadcom + AMD Anti-Nvidia Alliance Coming With UEC and Open XGMI Today MI300X is finally released and it’s coming out with a bang. There’s a lot of customers announced, which we discussed vo…

www.semianalysis.com

AMD Instinct MI300X Released at Opportune Moment. NVIDIA AI GPUs in Short Supply

LaminiAI appeared to be one of the first customers to receive an initial shipment of AMD's Instinct MI300X accelerators, as disclosed by their CEO posting about functioning hardware on social media late last week. A recent Taiwan Economic Daily article states that the "MI300X is rumored to have...

www.techpowerup.com

Intel and Ohio Supercomputer Center Double AI Processing Power with New HPC Cluster

A collaboration including Intel, Dell Technologies, Nvidia and the Ohio Supercomputer Center (OSC), today introduces Cardinal, a cutting-edge high-performance computing (HPC) cluster. Purpose-built to meet the increasing demand for HPC resources in Ohio across research, education and industry...

www.techpowerup.com

Arm Launches Next-Generation Neoverse CSS V3 and N3 Designs for Cloud, HPC, and AI Acceleration

Last year, Arm introduced its Neoverse Compute Subsystem (CSS) for the N2 and V2 series of data center processors, providing a reference platform for the development of efficient Arm-based chips. Major cloud service providers like AWS with Graviton 4 and Trainuium 2, Microsoft with Cobalt 100...

www.techpowerup.com

Intel Foundry Services (IFS) and Cadence Design Systems Expand Partnership on SoC Design

Intel Foundry Services (IFS) and Cadence Design Systems Inc. today announced a multiyear strategic agreement to jointly develop a portfolio of key customized intellectual property (IP), optimized design flows and techniques for Intel 18A process technology featuring RibbonFET gate-all-around...

www.techpowerup.com

GitHub - ray-project/llmperf-leaderboard

Contribute to ray-project/llmperf-leaderboard development by creating an account on GitHub.

github.com

From the last few days of news. Nvidia's lead is due to them selling AI equipment masked as something else, before anyone else. That lead will not last because they have general designs that are sure to be beat by custom SoC's. That's why Jensen is trying to entice them under his jacket.

 

[cvaldes]

Why wouldn't he? That's a good strategy to react to what the competition is doing. I wish AMD did the same with their Radeon products vis-a-vis Nvidia's GeForce products.

Your inclusion of the Arm Neoverse article is amusing since it quotes how Arm CPUs work in conjunction with Nvidia Grace and Nvidia Bluefield. Moreover, selling chip designs is Arm's business.

We already know that AMD and Intel have AI products, there's nothing that shows that these products are the result of some sort of custom chip design business. The AMD MI300 Instinct product appears to be off the shelf, just like Nvidia's current offerings.

Just tossing out random articles of other companies' off-the-shelf AI products does not negate the possible desirability for custom chip design services.

 

[Solaris17]

Some companies are better situated in creating custom chip designs on their own than others.

In the same way, if you asked ten people to start bakeries, not all of them would open on the same day with the same volume, same foot traffic, same revenue, etc.

Again, please list the companies that can go from concept to tapeout in 1.5 years. I would love to hear about these players. You're the one who brought it up.

I did. I could show you the racks I wrote tooling for and then racked in testing before deployment, but then I would probably get in trouble.

To each there own bro, you can go on beliving it takes a decade to make an ASIC. I just thought it would be friently to let you know it doesnt.

I will also throw in as a bonus just for you, that if you think any of these companies arent working every microsecond towards plaform diversification even if news outlets mention they are buying "X GPUs" from %manufacturer% you are being decived.

Nvidia has a market for custom order chips no doubt buddy, if you dont want to belive the rest of it...well I dont really care.

 

[cvaldes]

I did. I could show you the racks I wrote tooling for and then racked in testing before deployment, but then I would probably get in trouble.

To each there own bro, you can go on beliving it takes a decade to make an ASIC. I just thought it would be friently to let you know it doesnt.

I didn't say every ASIC. But a certain category of extremely complex ASIC from initial concept to marketable, competitive product, yes, it can take up to ten years. Not all ASICs have the same complexity.

Maybe you can release after five years with a less capable product. Or something that has no cost savings. Or something with a truncated feature list.

Remember that all of these companies (Intel, AMD, Nvidia, and others) have released products that were flops despite months (sometimes years) of work. Just because it tapes out doesn't mean it's good.

Ah, so you say you know companies that can tape out in 1.5 years but rather than point them out without comment, you leave an even blurrier and unverifiable story and bow out of coughing up names. That's very convenient. And if they are known for and took pride in having such quick turnaround, this wouldn't really be hiding anything, would it?

I will also throw in as a bonus just for you, that if you think any of these companies arent working every microsecond towards plaform diversification even if news outlets mention they are buying "X GPUs" from %manufacturer% you are being decived.

Nvidia has a market for custom order chips no doubt buddy, if you dont want to belive the rest of it...well I dont really care.

Oh, I believe that any company that signs up for Nvidia's rumored custom chip service will be looking into doing it themselves.

But that said, do you bake your own bread? Pasta? Cookies? Beer?

Look at your jeans. The fabric likely came from a loom run by another company. From cotton grown by farmers elsewhere.

Someone gives you a manual pasta machine, an egg, a cup of flour. Do you think you can pump out pasta as good as the boxed stuff from the supermarket? Or the pricier fresh stuff in the deli department's refrigerator? The first time? The twentieth time? How much time would it take? Can do this for a restaurant that serves 150 covers at dinner?

Let's say Home Depot has some programmers running AI models on inventory management on Nvidia AI accelerators. Do you think they'll be ready to tape out a competitive chip design in 1.5 years?