OpenAI Could Make Custom Chips to Power Next-Generation AI Models

[AusWolf]

lol, no that's not Dojo.

Enter Dojo: Tesla Reveals Design for Modular Supercomputer & D1 Chip

Two months ago, Tesla revealed a massive GPU cluster that it said was “roughly the number five supercomputer in the world,” and which was just a precursor to Tesla’s real supercomputing moonshot: the long-rumored, little-detailed Dojo system.

www.hpcwire.com

Hot Chips 34 – Tesla’s Dojo Microarchitecture

To say Tesla is merely interested in machine learning is an understatement.

chipsandcheese.com

Similar to RISC-V, but with Tesla's own custom instructions and four-wide SMT. Interesting.

 

[stimpy88]

What I find funny is the fact that every single serious player in the A.I. industry is talking about making their own chips, MS, Tesla, Google, OpenAI, Meta the list goes on. The collective hate for nGreedia and their business practices is palpable. Make hey while you can, you greedy little leather jacket man.

Price your products into the stratosphere nGreedia, and even the corps won't stick with you.

 

[kondamin]

You've clearly missed the point that these H100s cost $40K a piece and neural nets don't start working well till you get the gpu count over the thousands. Then there's the upkeep price which is insane. Any idea the cost to power and maintain a 10K gpu cluster buddy?

They want their own AI chip because they SEE that Elon's Dojo is providing the same performance as a 10K gpu cluster at 1/6th the cost and 1/4 the footprint.

People must think AI is free or something. There are huge costs for this level of compute and that's not even mentioning that the whole industry is compute constrained.

Open AI claimed it was spending about 3 million a month on operational costs that's both power infrastructure under microsoft Azure and wages.
It was doing 10million queries per day

So 3/300= 1cent that is still very expensive for a query but not 4

That hardware is bought and used over a period of years it's not that part of it is used up every query and that it needs to be replaced after a billion.
That cluster will be happy running all sorts of stuff a couple of years on.
Also power, yeah microsoft has a power bill in the billions but they are doing a shit ton more than just doing 10m queries per day.

Conclusion someone was adding a heck of a lot of fudge to the numbers.

 

[AusWolf]

What I find funny is the fact that every single serious player in the A.I. industry is talking about making their own chips

A 150 g pack of potato chips costs way over £2 now in most shops now, which is outright ridiculous, so I guess I'd better make my own as well.

 

[ARF]

What I find funny is the fact that every single serious player in the A.I. industry is talking about making their own chips, MS, Tesla, Google, OpenAI, Meta the list goes on. The collective hate for nGreedia and their business practices is palpable. Make hey while you can, you greedy little leather jacket man.

Price your products into the stratosphere nGreedia, and even the corps won't stick with you.

Wonder why AMD doesn't sell its radeons graphics cards, then? The collective hate towards AMD is outragous, too. 8-9% market share is very disturbing.

A 150 g pack of potato chips costs way over £2 now in most shops now, which is outright ridiculous, so I guess I'd better make my own as well.

Just buy a kilo potatos for 1.17£, make your own "chips" and call it a day

 

[AnotherReader]

No, we do not. However the original post points out that there are possible cost efficiencies in rolling your own AI silicon rather than buying at the market.

The US government certainly isn't telling Nvidia how much to price their AI accelerators.

Right now it's basically this:

AI customer: "How much is this?"
Nvidia: "How much you got?"
AI customer: "I don't like your attitude."
Nvidia: "We have competitors. Ask them about pricing and availability."
AI customer: "But your stuff is better. And you have a better dev platform. And our staff is already familiar with your products."
Nvidia: "Don't worry, we'll sell what we have to someone."

It's important to note that Apple makes their own machine learning silicon (the Neural Engine in their marketing speak), but for their own use. They have already done what the original post basically describes since about 2017. And they probably started working on in earnest back in 2010.

There are no Nvidia Tensor cores in my iPhone, iPad, and Mac. Apple is not affected by the market's demand for Nvidia's AI products.

Silicon Valley was built by people who left larger companies to start their own thing, hoping to do better than the status quo. There's nothing new about OpenAI's thoughts. Google, Meta, Amazon, they all have teams working on custom silicon designs.

Apple's neural engine is targeted at inference. They may claim in their marketing that it can do training too, but that isn't its raison d'etre. Machine learning silicon is rather simple when compared to Apple's CPUs so it's extremely unlikely that they started working on it in 2010. AlexNet woke up the wider computing world to the feasibility of machine learning on GPUs by becoming the first convolutional neural network to win ImageNet in September 2012.

 

[thesmokingman]

Every time I see these "other companies are just going to invert their own cheap AI chips and then NVIDIA will die" posts I LOL at the inability to understand basic facts. Guess what, designing your own silicon is difficult and guess what, those companies are going to be headbutting the exact same wall of lack of foundry capacity that NVIDIA is.

What is actually happening is NVIDIA is telling these companies "we can't get capacity but if you're willing to buy some of Apple's allocation at TSMC for <stupid price>, we'll happily make those GPUs for you" and these companies are going "HOLY SHIT THAT'S EXPENSIVE" and NVIDIA is going "yeah, now you see our problem". Then these companies shit out press releases subtly threatening NVIDIA, who's sitting back going "why are we getting blamed when Apple is the one taking all the capacity"?

I'm more and more surprised every day that NVIDIA is still making consumer GPUs. Switch all your production to server GPUs, let the consumer market sit it out a few generations, it's not like we won't be waiting a couple of generations down the line. Especially given how AMD is barely competitive and Intel isn't in any way shape or form.


You mean the Dojo that also uses 10,000 NVIDIA GPUs?

No one said NV is gonna die. They do however have a captive market with no real competitors till Q4 when AMD releases their accelerators. However even that is coming at a time when the biggest compute consumers MSFT, Google, Tesla have started to plan out their next five years (Tesla already on year 3 and ramping) and it has them pivoting away from NV. That's just the reality of gouging the fuck out of the market, shrugs.

 

[noel_fs]

the future is compute on memory

 

[ymdhis]

Just buy a kilo potatos for 1.17£, make your own "chips" and call it a day

okay, but then you have to factor in the cost of oil, salt, a deep fryer, and the cost of electricity and manual labor. It comes out significantly more expensive.

Personally I prefer popcorn. It's cheaper, takes longer to eat, and fits better when you are watching movies.

 

[A&P211]

I love Nvidia, I bought their stock over 9months ago, has not disappoint.

 

[AusWolf]

I love Nvidia, I bought their stock over 9months ago, has not disappoint.

I wish I did too. As soon as others start manufacturing AI chips, the bubble will pop.

 

[A&P211]

I wish I did too. As soon as others start manufacturing AI chips, the bubble will pop.

Then I go back to VOO, and brk.b.

 

[ARF]

okay, but then you have to factor in the cost of oil, salt, a deep fryer, and the cost of electricity and manual labor. It comes out significantly more expensive.

Have you calculated or just blind guessing?
All of the costs are distributed in much larger quantities, and there is no way that rubbish 150 g chips pack would cost more than 2£.
Electricity cost is towards 0, salt is towards 0.

1.17£ is 1 kg potatos. Then, 150 g is only 0.17£ raw material.

 

[stimpy88]

A 150 g pack of potato chips costs way over £2 now in most shops now, which is outright ridiculous, so I guess I'd better make my own as well.

But if they were £10 for a pack, you'd be an idiot not to.

 

[AusWolf]

But if they were $10 for a pack, you'd be an idiot not to.

You're already way better off making your own if you value your money more than your time. Even with a £2 pack price, you can make about 10x as much for yourself.

 

[Assimilator]

You're already way better off making your own if you value your money more than your time. Even with a £2 pack price, you can make about 10x as much for yourself.

The inverse of this is what so many fail to understand. Yes, Tesla built Dojo because they couldn't get enough NVIDIA GPUs fast enough... but it took them half a decade. That's a huge time and money sink into something that isn't your company's core competency.

The other problem is that, because nobody outside Tesla knows how Dojo works, anyone they recruit to work with it is going to need to be trained on its intricacies - which increases the cost of hiring and time for a new dev to ramp up. It also decreases the pool of talent willing to work for you, because Dojo isn't a transferable skill, so you are basically shackling yourself to Tesla if you choose to work there - and people experienced in the software development industry are simply too smart to do that. Which kinda sucks, since those are the people you most need.

Of course, maybe Tesla spent part of those 5 years perfecting a CUDA-to-Dojo transpiler and none of the above is relevant. But my experience with big companies is that they love building walled gardens - if not to trap users, then to trap developers. This is why open and shared standards are important.

 

[AusWolf]

The inverse of this is what so many fail to understand. Yes, Tesla built Dojo because they couldn't get enough NVIDIA GPUs fast enough... but it took them half a decade. That's a huge time and money sink into something that isn't your company's core competency.

The other problem is that, because nobody outside Tesla knows how Dojo works, anyone they recruit to work with it is going to need to be trained on its intricacies - which increases the cost of hiring and time for a new dev to ramp up. It also decreases the pool of talent willing to work for you, because Dojo isn't a transferable skill, so you are basically shackling yourself to Tesla if you choose to work there - and people experienced in the software development industry are simply too smart to do that. Which kinda sucks, since those are the people you most need.

Of course, maybe Tesla spent part of those 5 years perfecting a CUDA-to-Dojo transpiler and none of the above is relevant. But my experience with big companies is that they love building walled gardens - if not to trap users, then to trap developers. This is why open and shared standards are important.

Based on how much of a walled garden Tesla cars are, I can imagine how much they invested into that transpiler, if it even exists.