Wednesday, March 20th 2024

Tiny Corp. Pauses Development of AMD Radeon GPU-based Tinybox AI Cluster

George Hotz and his Tiny Corporation colleagues were pinning their hopes on AMD delivering some good news earlier this month. The development of a "TinyBox" AI compute cluster project hit some major roadblocks a couple of weeks ago—at the time, Radeon RX 7900 XTX GPU firmware was not gelling with Tiny Corp.'s setup. Hotz expressed "70% confidence" in AMD approving open-sourcing certain bits of firmware. At the time of writing this has not transpired—this week the Tiny Corp. social media account has, once again, switched to an "all guns blazing" mode. Hotz and Co. have publicly disclosed that they were dabbling with Intel Arc graphics cards, as of a few weeks ago. NVIDIA hardware is another possible route, according to freshly posted open thoughts.

Yesterday, it was confirmed that the young startup organization had paused its utilization of XFX Speedster MERC310 RX 7900 XTX graphics cards: "the driver is still very unstable, and when it crashes or hangs we have no way of debugging it. We have no way of dumping the state of a GPU. Apparently it isn't just the MES causing these issues, it's also the Command Processor (CP). After seeing how open Tenstorrent is, it's hard to deal with this. With Tenstorrent, I feel confident that if there's an issue, I can debug and fix it. With AMD, I don't." The $15,000 TinyBox system relies on "cheaper" gaming-oriented GPUs, rather than traditional enterprise solutions—this oddball approach has attracted a number of customers, but the latest announcements likely signal another delay. Yesterday's tweet continued to state: "we are exploring Intel, working on adding Level Zero support to tinygrad. We also added a $400 bounty for XMX support. We are also (sadly) exploring a 6x GeForce RTX 4090 GPU box. At least we know the software is good there. We will revisit AMD once we have an open and reproducible build process for the driver and firmware. We are willing to dive really deep into hardware to make it amazing. But without access, we can't."
Another post provided a behind-the-scenes look at Hotz's diplomatic approach: "I have spoken with AMD on multiple occasions, we have gotten through to top people, and they have been quite nice to us. I believe they want to be more open, and obviously they don't want their driver to have bugs. Unfortunately, this access and responses prolonged this decision, part of me wishes they just said it's a consumer card, you get what you pay for and we could have switched earlier. We probably tried too hard to make it work. We have an amazing team at tinygrad. Someday, we are going to make our own chips, and I figure if we can make our own chips, we better be able to make the 7900XTX software great. But we can't if we don't have access. The firmware is complex, undocumented, closed source, and signed, all struggles we wouldn't have with our own hardware. If and when the firmware is open and installable, if we aren't too far along with a different chip, we are down to put resources into writing fuzzers and rewriting whatever needs to be rewritten. The 7900XTX hardware seems great, but we aren't going to put resources into fixing a black box."
Sources: tinygrad Tweet, Tom's Hardware, Wccftech
Add your own comment

36 Comments on Tiny Corp. Pauses Development of AMD Radeon GPU-based Tinybox AI Cluster

#26
rv8000
LabRat 891History, would like a word with you (and this thread).

Raja Koduri's Ellesmere-Polaris and (especially) Fury and Vega *started* CDNA/AI-MI compute @ AMD.
To this day, Vega 10 and Vega 20 cards are some of the best 'budget' options for 'tinkering' with LLMs, etc.

He's well-within his (and his team's) rights to have thought that 3rd Generation Navi could be used for such.
Not to mention, he wasn't told that; AMD historically 'likes to see' new uses for their products, and can more-less crowd source off them.

'Consumer Hardware' started the AI/MI revolution.
Vega is retaining support largely because of how similar it is to currently-supported CDNA.

We're only 2generation off from Raja's last pre-CDNA work, Navi 1x. (Navi 12 is something strange...)
I can't blame Hotz for poking @ AMD when, previously they'd been quite accommodating towards this kind of use.
Ex: I received VII air coolers (for a MI25 mod) from a EHW'r that *still* runs quad VIIs for his work.


Quite often, I'm reminded that: Gamers =/= Enthusiasts.
You clearly have no enthusiasm for technology, beyond the FPS and the pretties on the screen...
It’s a consumer GPU, end of story. The startup owner targeted hardware that would maximize their profit, performance, and availability on a consumer “gaming” product not intended for professional work; the brand of GPU is entirely irrelevant.

All this is is poor investment/research on their end in the pursuit of making easy money on the “next big thing”.
Posted on Reply
#27
Cheeseball
Not a Potato
PatriotThey support single cards of the W7900 7900xtx xt and gre. 2 cards is in beta support. They aren't having trouble till cards 5 and 6. They are simply trying to use things in ways that are unsupported and crying about it.
They made an announcement before partnering with AMD, before qualifying a solution like any competent firm would have. If they want 8 card nodes, they should be using MI210s.
MI210 is not viable at ~$15,000 each with wholesale pricing (Dell has a deal for it though), which is the point of why he attempted to go with the RX 7900 XTX since they are affordable and will fit the $15,000 target price.
Posted on Reply
#28
Crackong
I won't be surprised when he find the same problem exists in his newly build 4090 box.

Posted on Reply
#29
SLObinger
If Tiny wants to unload thoes 7900XTX's for a deal I'll buy one or two of them. I'm just going to play games and create content. The one I already have is awesome for that. It's almost like AMD designed it just for games and content creation. The drivers needed some work at launch but work great now.
Posted on Reply
#30
Crackong
CheeseballMI210 is not viable at ~$15,000 each with wholesale pricing (Dell has a deal for it though), which is the point of why he attempted to go with the RX 7900 XTX since they are affordable and will fit the $15,000 target price.
Isn't this the whole point people are trying to say?

If he wants enterprise-class support and development resource, then pay the enterprise price.
If he wants to DIY the whole gap between consumer and enterprise, then hitting a brick wall is as expected.

I would say it is just him underestimated the cost of 'Breaching the gap between consumer and enterprise' at the first place.
Posted on Reply
#31
SirEpicWin
Man, never thought I would hear geohotz name mentioned in 2024 brings back the good old PS3 jailbreaking times, good times!
Posted on Reply
#32
ghazi
rv8000“Small startup got burned by jumping into the AI buzz market with consumer GPUs not designed for professional work in an effort to make a quick buck”

Oh no… anyways.
Glad to see this as the first post. I have no idea how anyone who works in a field like this could do anything but shake their head. They announced a vaporware product that goes out of its way to use a consumer hardware option - where an enterprise version is sold instead specifically for the purpose of supporting the application in question. The guy then throws a public fit in the media at the company they wanted to source that consumer hardware from, because it doesn't have the proper enterprise software support.

There's no shame in the idea or that he tried to make it work, but making this public fuss about the consequences of his own silly decision... I remember this guy from many years ago and that I liked him so that's all I'll say. The main selling point was "nobody else cut this corner to lower the price" and there was a real reason why.

Also, not a chance in hell Intel or Nvidia do any more to help. The markup on enterprise cards is the cornerstone of Nvidia's business strategy for crying out loud.
Posted on Reply
#33
TechLurker
CrackongIsn't this the whole point people are trying to say?

If he wants enterprise-class support and development resource, then pay the enterprise price.
If he wants to DIY the whole gap between consumer and enterprise, then hitting a brick wall is as expected.

I would say it is just him underestimated the cost of 'Breaching the gap between consumer and enterprise' at the first place.
Frankly, he's lucky that AMD even bothered to put him in contact with a high level engineer and coding staff; instead of being added to the regular consumer (or at least prosumer) queue, or worse, being told to pound sand and buy their enterprise cards.

He knows full well that NVIDIA WILL tell him to pound sand AND to cease and desist before forcing a driver update that will nerf AI usage for the workloads he's trying to run on their GPUs.

And Intel probably would only help him insofar as it benefits them, which really isn't much as Intel's current priority is AI on everything (but mostly CPUs and dedicated accelerators) and gaming on GPU. Esp. given the massive charm offensive they're doing trying to show that they're not going to abandon gaming GPUs; what with all the regular interviews with GN and others discussing the difficulties and what they're doing to catch up, on top of trying to be the fastest of the 3 GPU makers to release new driver updates for new, big-title games. Sure, there's some money in AI, but Intel would also be better served offering up dedicated enterprise solutions to it than risk unsavory headlines about selling their gaming GPUs to AI farms, much less gambling on a return from a startup.
Posted on Reply
#34
Haile Selassie
This guy screams main character syndrome.

I mean, what did he expect, to use public media to force a vendor to open our proprietary microcode for his own benefit and at expense of the said vendor?

Gee, who would have thought how this would end...
Posted on Reply
#35
Vayra86
seventyIf you don't know who Geohot is, you are very new.
He is absolutely right here, software of AMD gpu's are a mess, only fanboys would disagree.
He is giving AMD a fighting chance to compete with NVIDIA by kickstarting the grassroots enthusiasm for their chips in ML. And AMD is throwing it away.
Another reason to never buy an AMD gpu, their software team is just too incompetent.
He ain't giving shit, he's trying to get a free lunch here to kickstart his own company.

Let me help you help me, is what this is. I smell narcissism, bigtime.
Posted on Reply
#36
Redwoodz
Haile SelassieThis guy screams main character syndrome.

I mean, what did he expect, to use public media to force a vendor to open our proprietary microcode for his own benefit and at expense of the said vendor?

Gee, who would have thought how this would end...
And you must do this all in 2 weeks...lol
Posted on Reply
Add your own comment
May 18th, 2024 02:08 EDT change timezone

New Forum Posts

Popular Reviews

Controversial News Posts