Will we ever have customizable graphics cards like motherboards?

[tugrul_LordOfDrinks]

Some people don't want to give $ to unused VRAM. They just want more compute power for some reason.

Some people want faster VRAM, not more VRAM probably because of mining.

Server programs are even ok with asymmetric setups like 2GB fast memory + 4GB slow memory, only throughput is important, not latency nor stutter.

GPGPU programmers want to use 2 GPUs on same board because motherboards with multiple bridges are expensive.

Image processing requires 16x lanes of pcie v5.0 even on a low-end gpu. Moving data through pcie is generally a bottleneck.

Gamers want balanced setups like RTX4090 GPU + 1500 GB/s bandwidth & at least 20 GB memory.

Any gpu + any vram + any sensible vrm combo.

Why not have customization option to let gamers make their own ratios? My favourite would be 64 bit memory bus + 10k cuda pipelines because I only care about compute power and big cache and sometimes gaming but not highest setting.

 

[Macro Device]

User error rates will just explode. This will make warranty service basically impossible to handle provided how many idiots there is. Imagine an average Joe from some unknown village from Alabama cooking up his Frankenstein using a set of five mutually incompatible components and then whining on the support forum that they make shit equipment. Not to mention that allowing at least half users a screwdriver is already an ambitious play.

'sides, this will make the profitability per se much less achievable. With users constantly thinking what they actually wanna make instead of just buying the corp gets fairly limited revenue. Plus, developing drivers and other software will become a massive discomfort. "Do we optimise around average or do we optimise around median? Is there a reason to implement this since a lot of dudes use X but the dudes using Y might get upset?" and so on.

All in all, we have to accept the fact we have a limited list of possible purchases. The best shot is bringing the competition back so the perf/price ratio stops going down.

 

[Vincero]

Once upon a time there was upgradable graphics cards, but a large part of that was due to the fact that the memory system was essentially not much different to that used on every other DRAM implementation.

The very high frequency makes having a removable connection unreliable. The higher memory bus width means needing a lot of connections. This would drive up the cost of the baseline cards so, in scenarios where the card isn't upgraded (ever), the purchase price is higher with no real benefit to the customer.

 

[AusWolf]

Here you go:

This is an S3 ViRGE DX with swappable VRAM chips from the mid '90s.

It's not a feasible option anymore for reasons stated above. Not to mention, modern VRAM chips need cooling, which you can't do if they're swappable.

 

[tugrul_LordOfDrinks]

Isn't plug-n-play type of vram modules possible? I mean, we have DDR5 RAM for AM5 mobo, we just plug to their slot. Is it bad for performance actually?

 

[Macro Device]

GDDR7 is at least three times faster than the fastest DDR5. Not to mention latency fiesta going from it being slotted in instead of direct soldering. Go figure.

 

[AusWolf]

Isn't plug-n-play type of vram modules possible? I mean, we have DDR5 RAM for AM5 mobo, we just plug to their slot. Is it bad for performance actually?

DDR5 is not the same as GDDR6/7. System RAM is slow enough not to need cooling, or to be fine with a basic heat spreader. Modern VRAM is usually cooled by the GPU's main cooling array.

I might be wrong with my reasoning, I just find this to be the most plausible explanation.

 

[tugrul_LordOfDrinks]

Im comfortable with latency as long as throughput is high because its a gpu, not a cpu.

I dont care if kernel function receives data 1 microsecond slower as long as it completes the work in 1 millisecond.

 

[Onasi]

Lets say Im comfortable with latency as long as throughout is high because its a gpu, not a cpu.

It doesn’t matter what YOU are comfortable with. NV/AMD/Intel needs their SKUs to be balanced for most applications for the widest array of users and they want to have a limited amount of GPU chip versions and models to save costs, AIBs want the card design to be as simple as they can feasibly manage to have SOME margins and most users want plug and play. What you are asking for is, essentially, a bespoke solution for a nonexistent problem that will skyrocket the pricing even more. That isn’t happening. The logistics just won’t allow for it, even leaving aside the technical aspects. It’s like going up to buy a Porsche 911 and asking the manufacturer to put in a Chevy LS V8 in your specific car instead of a flat turbo six because you just feel like it.

 

[tugrul_LordOfDrinks]

Yes its like Dominic Toretto making a car specific for a drag race event.

 

[BSim500]

Isn't plug-n-play type of vram modules possible? I mean, we have DDR5 RAM for AM5 mobo, we just plug to their slot. Is it bad for performance actually?

You used to be able to do that with some Matrox Millenium GPU's. Just slot a "SODIMM" like module into the white socket at the end. As people said above though, GDDR6, etc, is quite a bit faster now vs then, VRAM now needs cooling, we didn't have 2.5x slot thick coolers back then, etc.

 

[leonavis]

Once upon a time there was upgradable graphics cards, but a large part of that was due to the fact that the memory system was essentially not much different to that used on every other DRAM implementation.

The very high frequency makes having a removable connection unreliable. The higher memory bus width means needing a lot of connections. This would drive up the cost of the baseline cards so, in scenarios where the card isn't upgraded (ever), the purchase price is higher with no real benefit to the customer.

Well of course.

However... I mean one could just solder a graphics card themselves with the right tools. I realize that such tools are very expensive, but I work in the field (electronics). I think the chips themselves are buyable via China. Would only have to create a usable PCB, the Pcie-slots are standardized, as is mostly everything else... but the drivers could be a problem. On Linux with an AMD chip, a solvable one, but, as the whole project, not easily solveable, this would take so much work.

I don't know. Could be fun as a project. Then again, so much work.

 

[ty_ger]

You would end up with a slower, more expensive, worse product that almost no one would want.
Plus, for NVIDIA at least, buying the memory at bulk discount and making it a forced bundle that they sell with the core to the AIB at markup pricing is part of their profit strategy. They wouldn't want to give that up.

 

[Vincero]

Well of course.

However... I mean one could just solder a graphics card themselves with the right tools. I realize that such tools are very expensive, but I work in the field (electronics). I think the chips themselves are buyable via China. Would only have to create a usable PCB, the Pcie-slots are standardized, as is mostly everything else... but the drivers could be a problem. On Linux with an AMD chip, a solvable one, but, as the whole project, not easily solveable, this would take so much work.

I don't know. Could be fun as a project. Then again, so much work.

People have done this, boosting older GPUs with more RAM. Nvidia essentially did this themselves when they added the RTX 2060 12GB to the SKUs during the pandemic availability issues.

Drivers aren't so much the problem as getting the VBIOS to work on the card - they are usually tuned for the RAM characteristics and timings of the components used on board (something else that excludes having mix-&-match removable parts) and their performance characteristics at different voltages, clock rates, etc. - you can solder all the higher density RAM modules to the card that you want but without the right programming on the card it'll at best work as before or more likely not work.

 

[silentbogo]

1) GDDR memory has wa-a-a-ay faster per-pin throughput than DDR. For modern chips GDDR5 and newer it's simply impossible to put it on a module and guarantee stable operations. And even if it's possible - it'll be very-very expensive.
2) Now in addition to matching/compatible memory modules you need to create a standard akin to SPD which all manufacturers must agree on. We all know how well that usually goes.
3) Now imagine you've made something similar to a desktop 64-bit memory module with more or less reasonable size, footprint and price. If you want to populate a decent GPU and take advantage of, let's say, 384-bit bus width, you'll need 6!!! slots. Even if you make 128-bit modules(like upcoming CAMM), it's still 3[!] modules. GPUs will look like christmas trees or humpty-dumpties.
4) Cooling? Even if our hypothetical prototype will be restricted to single-sided modules, you still need to cool it from both sides. On lower-end GPUs it's done by the "heavy" PCB itself, and on high-power cards you have backplates and front plates, and all kinds of other tricks to help move heat around and away from VRAM.
5) even without considerations for cooling and accomodating "newly-invented" memory modules, your GPUs will become thicc AF. I'm already terrified of modern triple- and quadruple-slot GPUs that weigh more than my microwave oven, and with all the extras needed for slotted/socketed card it'll be HUMONGOUS
6) I'm still not convinced by use cases you listed. For mining - you are always limited by VRAM on the low side, but not on the high-side, so "modernizing" GPUs is only necessary when they run out of compute, and not memory.
Same for GPGPU compute - multi-GPU is still a thing, and if you need more memory and little GPU performance(for example a GPU-powered database or something like that), then you can always go for older Tesla and Quadro cards with larger framebuffer, and you can always go for multi-GPU since in those particular cases PCIe bandwidth won't be an issue at all. I'm talking about explicit multi-GPU, not the proprietary stuff like SLI/NVLink/CF.
And for gaming - VRAM size is overrated, and if anything, I would rather stick with NOT increasing VRAM size and force AAA game devs to re-learn video game optimization. Some modern games manage to look worse than 7-8 years ago, require nearly double VRAM for minimum requirements, and still manage to work like total s&^t. And modern features like nanite and other object streaming techniques only make things worse. This year I've played some UE5 games and while they looked marginally better than older titles, the actual gameplay experience was by far the worst. Heck, Crytek had it figured out more than a decade ago. idTech had it figured out even before the Bethesda buyout. And even older versions of UE had some impressive huge open worlds without all of the modern problems they themselves created.
P.S. Sorry for the rant, but the final answer to OP: that would be a bad idea.

Moving data through pcie is generally a bottleneck.

No, only in certain applications where you need to copy lots of data from RAM to VRAM frequently. It does not happen as frequently as you think.
Not in games, not in cryptomining(especially not in crypto). Only in some GPU compute applications, and even then it's moslty developers fault(bad memory management) or conceptual miss(e.g. they are trying to GPU-accelerate something that does not need GPU-acceleration).

 

[Bomby569]

I don't think cooling would be an issue at all, there is enough cooling on modern gpus. Design the thing so the user would just have to slot the vram and remove some plastic so the already existing thermal pad do it's job, just like with ssd's. It would not be hard to design the cooling solution with the slotted vram in mind.

The slotted vram would be, but there could be the usual soldered vram and an extra expandable slot, it would have a slower speed, all the vram would have to work at that slower speed, but it would probably be better than the alternative: e-waste a perfectly good card, or have it underperform on lower end systems, what will probably be the destiny of many cards that are in use today

It would drive costs up but i don't think that would ever be an issue and it would be negligible

Having space for a vram slot could be an issue, especially with the flow through models, those have literally no space for it, but i guess just do a normal size card

Performance issues for idiots, could easily be just disclaimed, if you add memory it will be slower, so no one would buy it, immediately slot a vram module and complain it doesn't perform as advertised.

I have no idea about the physical connection part, but i doubt it isn't some they couldn't resolve. If you can do it for a cpu.

I think it could be done. The biggest obstacle would be more sales for vram modules but less sales for nvidia/amd/partners. That's a big no right there.

 

[silentbogo]

there is enough cooling on modern gpus.

On modern GPUs your entire card is part of your cooling system. Put it on the module, and you take at least half of it away.

The biggest obstacle would be more sales for vram modules but less sales for nvidia/amd/partners.

That's a big roadblock for sure.

It would drive costs up but i don't think that would ever be an issue and it would be negligible

That's definitely going to be noticeable. First off, PCB routing is a nightmare as-is, and if you add CAMM connectors or something else of this sort into equation - it will increase both R&D and actual PCB manufacturing prices significantly. And if current GPU prices and upcoming "predictions" aren't indicative enough, you won't see an adequately-priced GPU at all. The low-to-mid-end won't even exist with this approach.


If anything, I would rather see both NV and AMD go for unified memory at some point, so we can have a moderately-powerful dGPU (or iGPU?) that can share framebuffer with main memory dynamically, like consoles. Or something like long-forgotten Subor Z+, which had a fixed split, and used GDDR5 for the entire pool.

 

[_JP_]

Yes its like Dominic Toretto making a car specific for a drag race event.

In that metaphor, the graphics card is already the drag racer, while the CPU is your average commuter.

 

[freeagent]

I would say outside of high speed interconnects performing better with solder, bios issues.

Edit:

Also...

If you can think it, its probably been done already.

 

[AusWolf]

On modern GPUs your entire card is part of your cooling system. Put it on the module, and you take at least half of it away.

That's a big roadblock for sure.

That's definitely going to be noticeable. First off, PCB routing is a nightmare as-is, and if you add CAMM connectors or something else of this sort into equation - it will increase both R&D and actual PCB manufacturing prices significantly. And if current GPU prices and upcoming "predictions" aren't indicative enough, you won't see an adequately-priced GPU at all. The low-to-mid-end won't even exist with this approach.

Those are valid arguments from a design and manufacturing perspective.

On the user's perspective, most GPUs are usually equipped with just as much VRAM as they need. Low-end cards come with 8 GB. Increase image settings beyond that usage, and your GPU isn't strong enough to deliver more frames. The only thing you could increase indefinitely with more VRAM is texture resolution. Everything else needs a more powerful GPU, too.

 

[freeagent]

I wouldn't use off the shelf parts like for PC, these would have to be like pc parts, but for GPU, so a whole new standard would have to be created. These components would be tiny, with tiny sockets and ram would have to be something we haven't seen before. It would be like a motherboard, so an SoC type system would have to be created, and then UEFI to drive it.

This would be easier to do once we cant really progress too much further due to some kind of quantum problem.

 

[tugrul_LordOfDrinks]

What about custom gpu chips? Can Nvidia produce pure-cuda gpus without any tensor/rt/graphics?

 

[freeagent]

What about custom gpu chips? Can Nvidia produce pure-cuda gpus without any tensor/rt/graphics?

You are forgetting 1 key thing..

Why sell a socketable GPU for XXX dollars, when you can sell the whole thing for XXXX.xx dollars?

I am sure that part plays a big role.

 

[AusWolf]

What about custom gpu chips? Can Nvidia produce pure-cuda gpus without any tensor/rt/graphics?

I guess they could, like they did an RT and tensor-free version of Turing, called GTX 16. I'm not sure who would buy it, though.

 

[tugrul_LordOfDrinks]

It's about usability for a general purpose cause. For example a super computer being made for anything, not for unused RT, etc. RT core is not usable in CUDA afaik.