Friday, January 19th 2024
Meta Will Acquire 350,000 H100 GPUs Worth More Than 10 Billion US Dollars
Mark Zuckerberg has shared some interesting insights about Meta's AI infrastructure buildout, which is on track to include an astonishing number of NVIDIA H100 Tensor GPUs. In the post on Instagram, Meta's CEO has noted the following: "We're currently training our next-gen model Llama 3, and we're building massive compute infrastructure to support our future roadmap, including 350k H100s by the end of this year -- and overall almost 600k H100s equivalents of compute if you include other GPUs." That means that the company will enhance its AI infrastructure with 350,000 H100 GPUs on top of the existing GPUs, which is equivalent to 250,000 H100 in terms of computing power, for a total of 600,000 H100-equivalent GPUs.
The raw number of GPUs installed comes at a steep price. With the average selling price of H100 GPU nearing 30,000 US dollars, Meta's investment will settle the company back around $10.5 billion. Other GPUs should be in the infrastructure, but most will comprise the NVIDIA Hopper family. Additionally, Meta is currently training the LLama 3 AI model, which will be much more capable than the existing LLama 2 family and will include better reasoning, coding, and math-solving capabilities. These models will be open-source. Later down the pipeline, as the artificial general intelligence (AGI) comes into play, Zuckerberg has noted that "Our long term vision is to build general intelligence, open source it responsibly, and make it widely available so everyone can benefit." So, expect to see these models in the GitHub repositories in the future.
Source:
Mark Zuckerberg (Instagram)
The raw number of GPUs installed comes at a steep price. With the average selling price of H100 GPU nearing 30,000 US dollars, Meta's investment will settle the company back around $10.5 billion. Other GPUs should be in the infrastructure, but most will comprise the NVIDIA Hopper family. Additionally, Meta is currently training the LLama 3 AI model, which will be much more capable than the existing LLama 2 family and will include better reasoning, coding, and math-solving capabilities. These models will be open-source. Later down the pipeline, as the artificial general intelligence (AGI) comes into play, Zuckerberg has noted that "Our long term vision is to build general intelligence, open source it responsibly, and make it widely available so everyone can benefit." So, expect to see these models in the GitHub repositories in the future.
53 Comments on Meta Will Acquire 350,000 H100 GPUs Worth More Than 10 Billion US Dollars
P.S.
Was interesting to learn about GPT hallucinations, that stuff is going to make more headlines when people do not check the result. Like that lawyer, for example.Well Musk already has failed us with the promised genetically engineered catgirls, this will too.
Now server farms are made of much much much less CPUs usually only in dual sockets. The rest of the compute power comes from GPUs. While both exist simultaneously in the same farm, the number of CPUs dropped by the 1000s.
For example, a 2005 server farm could have 100,000 CPUs in lets say 12,500 nodes (8 CPUs per node). Each 8 socket CPU would cost $30,000 due to the number of coherent interlinks to connect 8 sockets together. That’s $3 billion CPU revenue.
In a 2023 server, that same 12,500 node server farm only needs two CPUs per node at a much less $10,000 per CPU due to less interlinks. That’s now just $250 million in CPU revenue. The rest of the compute power is GPUs.
In this hypothetical that’s 12 times less CPU revenue!!!
Yep all we need now is AI mining boom to drive gpu/... hardware up :slap:
At 300W-350W each and up, I'd start investing in power companies lol
350W x 350000 = 122,500,000 W = 122.5MW * $65/MWHr = $7962.50/hr * 8760hrs/yr = $69,751,500 electric bill/yr (assuming $65/MWhr)
only way that will crash is if some flows start a fight over an island and that one company that makes everything suffers Damages.
those top stocks serve as a refuge, they are considered safer than government bonds for most nations.
I'd MUCH rather own a basic 8-core Ryzen Threadripper or Emerald Rapids Xeon W than this i9-13900KS, even if it's actually slower in terms of absolute compute, it's got the quad-channel memory, the PCIe expansions... but nope, this isn't even a choice, they're not available at all in the DIY channels - and are priced very much accordingly to enterprise gear instead. I do not expect this to change - stock prices are comfy, as you can see. It looks like their business strategy is on point.
Server farms currently consist primarily of HCI (highly converged infrastructure).
This is where you have companies like VMWare, HPE (HP Enterprise), Dell / EMC, Nutanix, and Cisco UCS.
A Cisco X series rack for example, can have 7 nodes (blades), each blade with 4 x 60 core Xeons (240 cores) and 16TB of RAM, for a total of 1680 cores in a single 7RU form factor.
This fits in a 12.25" high rack mount chassis.
There are bajillions of these things in server farms all over the country.
The computing industry is moving from a store and retrieve model to one of generated answers. It's a change that is happening as we speak, you can ignore it, but is is happening and it's the future.
I'm telling you what is. I've worked in a multibillion dollar company for 30 years and I think we have like 3 or 4 dozen GPUs, used in some very specific areas like transportation. We literally have thousands of normal servers.
Meanwhile Meta are laying off people still as well.
I thought this Mark with the strange surname was more of a strategist.
The race to lower operating costs will begin at Open AI, I guess
those numbers might seems a lot but I think it just average at best to a company at that magnitude and ambition (meta).
When you consider this 10B investment is worthwhile to them based on the free data each Facebook user is generating, be sure to see life changing computer based processes to emerge in the coming years.
They are also getting another 250k "equivalent"
There wasn't a price disclosure, just numbers