BaksAnd put a couple of people in jail.

The AI enthusiast community has been able to get it to run on much less complex accelerators such as the M4 SoCs of Mac minis, and gaming GPUs.

MaMooMore efficient AI was always going to come as the next most obvious advancement. Better invest wisely and not forget the fundamental disruptness of technology in the first place.

Vya DomusEveryone can see everything is heading towards cheaper to train/infer models not bigger and bigger as the narrative has been until now. DeepSeek isn't going to be what will burst this bubble but simply the unsustainability of investing more and more without enough revenue to back it up. Reminder that OpenAI is still losing billions of dollars to this day.

Chrispy_Good. OpenAI isn't open and Meta are anti-consumer in every valid definition of the term.

The situation is so bad that I don't even care that DeepSeek is Chinese! They are the lesser evil here.

mb194dcTheir model is open source, so anyone can copy their methodology, don't need to run their version of it from China. You just need training data which shouldn't be too difficult. Then if you can pay the few million for GPU training time... You've got a business similar to OpenAI for less than 5% of the cost... (possibly even for far less than that, we'll see for that...)

Which I guess is a fairly big problem for those that already sunk billions in to hardware...

phanbueyIt seems the majority of the cost is training the model. They took the model that was trained and "Distilled it" i.e. they essentially copied openai's homework that they spent so much time and money crunching and then added their own improvements on top.

Solid State BrainThat's not what they did for DeepSeek R1. That one was trained from scratch using an innovative method:
arxiv.org/abs/2501.12948

It's the the smaller models that were released together with it that were "distilled' from DeepSeek R1 outputs.

phanbuey+1

It seems the majority of the cost is training the model. They took the model that was trained and "Distilled it" i.e. they essentially copied openai's homework that they spent so much time and money crunching and then added their own improvements on top.

It's like if Luis Vitton made a handbag and they took that design, improved it and then manufactured it for 1/20th of the cost...

But essentially this wouldn't exist if it was not for the already trained models out there, so there will be a fight about IP in the AI space coming soon, especially all those people that sunk money into open AI I imagine they're a bit salty about this.

mb194dcTheir model is open source, so anyone can copy their methodology, don't need to run their version of it from China. You just need training data which shouldn't be too difficult. Then if you can pay the few million for GPU training time... You've got a business similar to OpenAI for less than 5% of the cost... (possibly even for far less than that, we'll see...)

Which I guess is a fairly big problem for those that already sunk billions in to hardware...

phanbueyThey have clips of it saying it's chat gpt tho... So the accusation will be there.

Solid State BrainThat's because the training data included 600k samples of reasoning outputs from DeepSeek-R1-Zero (the model trained via so-called Reinforcement Learning), plus 200k samples non-reasoning outputs from a variety of sources, including presumably publicly available data originally obtained from GPT4 and likely not cleaned well enough. Putting that aside, GPT refusals or GPT-style text have now also contaminated the web as well, so it could have picked it from that too.

mb194dcHere we go ? You don't need all that hardware to create a good model, evidently.

Deepseek trained its DeepSeek-V3 Mixture-of-Experts (MoE) language model with 671 billion parameters using a cluster containing 2,048 Nvidia H800 GPUs in just two months, which means 2.8 million GPU hours, according to its paper. For comparison, it took Meta 11 times more compute power (30.8 million GPU hours) to train its Llama 3 with 405 billion parameters using a cluster containing 16,384 H100 GPUs over the course of 54 days.

Solid State BrainWorth pointing out that the models that people are running on M4, Mac Minis and gaming GPUs have very little to do with the actually capable one DeepSeek is operating on its website. That is a completely different, much larger model requiring at least 700GB of VRAM.

What the market is concerned about is that such a capable model could be trained with ~5M USD worth of compute (excluding GPU costs). That doesn't mean though that putting more compute on it won't improve the results...