Saturday, March 9th 2024
Tiny Corp. CEO Expresses "70% Confidence" in AMD Open-Sourcing Certain GPU Firmware
Lately Tiny Corp. CEO—George Hotz—has used his company's social media account to publicly criticize AMD Radeon RX 7900 XTX GPU firmware. The creator of Tinybox, a pre-orderable $15,000 AI compute cluster, has not selected "traditional" hardware for his systems—it is possible that AMD's Instinct MI300X accelerator is quite difficult to acquire, especially for a young startup operation. The decision to utilize gaming-oriented XFX-branded RDNA 3.0 GPUs instead of purpose-built CDNA 3.0 platforms—for local model training and AI inference—is certainly a peculiar one. Hotz and his colleagues have encountered roadblocks in the development of their Tinybox system—recently, public attention was drawn to an "LLVM spilling bug." AMD President/CEO/Chair, Dr. Lisa Su, swiftly stepped in and promised a "good solution." Earlier in the week, Tiny Corp. reported satisfaction with a delivery of fixes—courtesy of Team Red's software engineering department. They also disclosed that they would be discussing matters with AMD directly, regarding the possibility of open-sourcing Radeon GPU MES firmware.
Subsequently, Hotz documented his interactions with Team Red representatives—he expressed 70% confidence in AMD approving open-sourcing certain bits of firmware in a week's time: "Call went pretty well. We are gating the commitment to 6x Radeon RX 7900 XTX on a public release of a roadmap to get the firmware open source. (and obviously the MLPerf training bug being fixed). We aren't open source purists, it doesn't matter to us if the HDCP stuff is open for example. But we need the scheduler and the memory hierarchy management to be open. This is what it takes to push the performance of neural networks. The Groq 500 T/s mixtral demo should be possible on a tinybox, but it requires god tier software and deep integration with the scheduler. We also advised that the build process for amdgpu-dkms should be more open. While the driver itself is open, we haven't found it easy to rebuild and install. Easy REPL cycle is a key driver for community open source. We want the firmware to be easy to rebuild and install also." Prior to this week's co-operations, Tiny Corp. hinted that it could move on from utilizing Radeon RX 7900 XTX, in favor of Intel Alchemist graphics hardware—if AMD's decision making does not favor them, Hotz & Co. could pivot to builds including Acer Predator BiFrost Arc A770 16 GB OC cards.
Sources:
tinygrad Tweet, Phoronix, Tom's Hardware
Subsequently, Hotz documented his interactions with Team Red representatives—he expressed 70% confidence in AMD approving open-sourcing certain bits of firmware in a week's time: "Call went pretty well. We are gating the commitment to 6x Radeon RX 7900 XTX on a public release of a roadmap to get the firmware open source. (and obviously the MLPerf training bug being fixed). We aren't open source purists, it doesn't matter to us if the HDCP stuff is open for example. But we need the scheduler and the memory hierarchy management to be open. This is what it takes to push the performance of neural networks. The Groq 500 T/s mixtral demo should be possible on a tinybox, but it requires god tier software and deep integration with the scheduler. We also advised that the build process for amdgpu-dkms should be more open. While the driver itself is open, we haven't found it easy to rebuild and install. Easy REPL cycle is a key driver for community open source. We want the firmware to be easy to rebuild and install also." Prior to this week's co-operations, Tiny Corp. hinted that it could move on from utilizing Radeon RX 7900 XTX, in favor of Intel Alchemist graphics hardware—if AMD's decision making does not favor them, Hotz & Co. could pivot to builds including Acer Predator BiFrost Arc A770 16 GB OC cards.
12 Comments on Tiny Corp. CEO Expresses "70% Confidence" in AMD Open-Sourcing Certain GPU Firmware
They have nothing their competitors want from a software perspective and are broadly seen as the has been in the GPU market.
While I would never want to encounter an AI driven card shortage, AMD working with a dedicated partner to get their client side software up to spec would be a boon for them and get them in on the ground floor of AI with start-up companies who use the Tinybox and it's ilk.
But ultimately I don't think RDNA3 is a good fit for whatever that company is trying to do.
Super great update from the manufacturer, forget it. XFX in particular has never released a firmware update for a GPU in its life, although they have already released updates with a new GOP for cards produced later.
With regard to GOP, AMD is also constantly releasing updates, but as I said, none of the manufacturers implant them for the end customer. AMD would have to put the GPU manufacturers through the wringer, just like with its AGESA updates for the chipsets.
The big joke is that up to the 6000 generation of Dell and Co, I had extracted the GOP parts from their updates from their notbook updates and then integrated them into my desktop bios.
With the 7000 series, of course, this is no longer possible without mod drivers.
Lets ask Microsoft to open source Windows next.
Closed source software exists for a reason. We need both open source software and closed source.
Also Nvidia fanboys that love Android phones and hate Apple confuses me to my core.
With Apple, even to include a link saying you should register and pay through the website is too much, it needs to be hidden in a help screen in very specific ways - and that is now after losing in court, before they just couldn't mention it AT ALL.