Monday, July 3rd 2023
Inflection AI Builds Supercomputer with 22,000 NVIDIA H100 GPUs
The AI hype continues to push hardware shipments, especially for servers with GPUs that are in very high demand. Another example is the latest feat of AI startup, Inflection AI. Building foundational AI models, the Inflection AI crew has secured an order of 22,000 NVIDIA H100 GPUs and built a supercomputer. Assuming a configuration of a single Intel Xeon CPU with eight GPUs, almost 700 four-node racks should go into the supercomputer. Scaling and connecting 22,000 GPUs is easier than it is to acquire them, as NVIDIA's H100 GPUs are selling out everywhere due to the enormous demand for AI applications both on and off premises.
Getting 22,000 H100 GPUs is the biggest challenge here, and Inflection AI managed to get them by having NVIDIA as an investor in the startup. The supercomputer is estimated to cost around one billion USD and consume 31 Mega-Watts of power. The Inflection AI startup is now valued at 1.5 billion USD at the time of writing.
Source:
HardwareLuxx.de
Getting 22,000 H100 GPUs is the biggest challenge here, and Inflection AI managed to get them by having NVIDIA as an investor in the startup. The supercomputer is estimated to cost around one billion USD and consume 31 Mega-Watts of power. The Inflection AI startup is now valued at 1.5 billion USD at the time of writing.
12 Comments on Inflection AI Builds Supercomputer with 22,000 NVIDIA H100 GPUs
If something Retail like this MSI B360-F PRO can do 18x, then a specialist bespoke xeon board could easily do many more. After all Intel Xeon W-3400 Series has 112 pcie lanes and could therefore run 112 GPUs, let's call it 100.
Here there are huge amounts of data to move from and to storage, and processing also takes place on the CPUs in part. Monster computing nodes with 8 GPU accelerators and twin Xeons or Epycs aren't uncommon. One variant of the MI300 is going to have as many as 24 CPU cores in the same package as the GPU, which will enable operation without a separate Epyc - and think about how much bandwidth those CPU cores need to communicate with the GPU part.
A valuation of $1.5 Billion sounds fair because that's barely much more than the underlying hardware.
For creative use: AI seems like it will be with us with Photoshop's Generative Fill: (www.adobe.com/products/photoshop/generative-fill.html). I'm not convinced text is quite ready yet, even with GPT4. ChatGPT / GPT4 is good enough to make very annoying spambots, but the content / hallucinations / lying is just awful and makes practical use of GPT4 just unworkable in many cases.