igormpI can see the point of your idea, but is not something that will take place at all within the next 5 years, and may take 10 years or more to become feasible. One pretty clear example of that is PCIe, with the current version 5.0 being a major bottleneck still, version 6.0 only coming to market next year, and 7.0 having its spec finished, but still way behind the likes of NVLink (PCIe 7.0 bandwidth will be somewhere between NVLink 2.0~3.0, which were Volta/Ampere links).
I believe NVLink is the fastest in-node interconnect in use in the market at the moment, and even it is still a bottleneck compared to the actual GPU memory.

pk67But even if I'm wrong it still have minor impact on validity of my conclusion - at that fundamentally changed market today 5090 with their soldered GDDR7 ram will looks like a toy. That is my point.

igormpBy then a 5090 will (hopefully) look like a toy no matter if your idea came to be or not, given enough technology advancements.

If a 5090 is still able to be competitive with the status quo 5+ years from now, something wrong happened along the way.

pk67Keep in mind Jensen and his marketing department telling us otherwise. They are trying to convince mainstream users ( and their investors as well ) cos the Moore law is dead the progress must slow down substantially and everything they are offering us must be extraordinary expensive.
But it is totally false picture.
The similiar picture were painted not so far ago in space industry - access to orbit must be expensive. But Musk show us otherwise.

edit
There are more factors than pure Moore law which keeping progress at fast pace now like arms race, US -China rivalry, etc
So goverments trying to stimulate their high-tech to stimulate their expansion plans and pace of progress as well.
Marketing departments trying to fool us in every possible way but we should be aware - what today looks like a bargain it wont be after a year or two so we should be more carefull which way we are spending our money cos future bargains coming to us (despite mainstream media outlets are mostly silent )- like decoupled memories - so we should be a bit more patient.

HankierosemanSomebody needs to make a card to run Samsung's LS57CG952... MONITOR @ 7680x2160, 240 Hz and DP 2.1. No?

igormpYou don't use toy hardware for such requirements tho. No one is trying to fine tune the actual large models in their basements, that's why the large H100 deployments are a thing.

3090s are still plently in use (heck, I have 2 myself), and A100s are still widely used 4 years after their launch.

There's no decoupled solution that provides the same bandwidth that soldered memory does, which is of utmost importance for something like LLM, which are really bandwidth-bound.


Mind providing any lead on such kind of offering? Current interconnects are the major bottlenecks in all clustered systems. Just saying "optical interface" doesn't mean much, since the current solutions are ate least one order of magnitude behind our soldered interfaces.


Something like a 5090 would fit in this. It's considered an entry level accelerator for all purposes. The term "gpu-poor" is a good example of that.

I can see the point of your idea, but is not something that will take place at all within the next 5 years, and may take 10 years or more to become feasible. One pretty clear example of that is PCIe, with the current version 5.0 being a major bottleneck still, version 6.0 only coming to market next year, and 7.0 having its spec finished, but still way behind the likes of NVLink (PCIe 7.0 bandwidth will be somewhere between NVLink 2.0~3.0, which were Volta/Ampere links).
I believe NVLink is the fastest in-node interconnect in use in the market at the moment, and even it is still a bottleneck compared to the actual GPU memory.