Dear AMD, NVIDIA, INTEL and others, we need cheap (192-bit to 384-bit), high VRAM, consumer, GPUs to locally self-host/inference AI/LLMs

[lexluthermiester]

i think we're past that, unless you have a very high end card, there is no point in turning RT on

Nonsense. The resolution used plays a factor, but even a 2070 can still do a reasonable amount of RTRT at 1440p. Newer cards have no issues unless we're talking about 4k.

 

[Bomby569]

Nonsense. The resolution used plays a factor, but even a 2070 can still do a reasonable amount of RTRT at 1440p. Newer cards have no issues unless we're talking about 4k.

what's this than

 

[Vayra86]

Nonsense. The resolution used plays a factor, but even a 2070 can still do a reasonable amount of RTRT at 1440p. Newer cards have no issues unless we're talking about 4k.

A 2070.... yeah. Dream on. The end result of that is like you're back in 2009. 720p isn't really a current resolution anymore

 

[lexluthermiester]

what's this than

That is one game. Nice cherry pick. Of course you kinda just proved my point for me.

A 2070.... yeah. Dream on.

Nope, in reality.

The end result of that is like you're back in 2009. 720p isn't really a current resolution anymore

I've recently demonstrated a 2060Super(which is very close to a 2070 by the specs.) to a customer and RTRT was on in CyberPunk2077 at 1080p. At 1440p, some settings had to be turned down but the performance was still acceptable.

 

[Bomby569]

That is one game. Nice cherry pick. Of course you kinda just proved my point for me.

that's not one game, all "you must have RT games" will look like that. Indiana Jones who could use RT settings?

 

[lexluthermiester]

that's not one game, all "you must have RT games" will look like that.

Are you kidding? Do you think you're talking to a moron?

Indiana Jones who could use RT settings?

Indiana Jones & The Great Circle runs fine on a 2060. Sure, you have to turn some settings down, but it runs at perfectly playable framerates. At 2070 can do better.. along with everything else that is newer/better.

 

[Bomby569]

Are you kidding? Do you think you're talking to a moron?

Indiana Jones & The Great Circle runs fine on a 2060. Sure, you have to turn some settings down, but it runs at perfectly playable framerates. At 2070 can do better..

we're talking about RT specifically, not sure what you are replying to.

i try not to judge people

 

[lexluthermiester]

i try not to judge people

Fair enough.

we're talking about RT specifically, not sure what you are replying to.

Yuppers. Indiana Jone is an RTRT only game. It won't run unless you've got it on your card.

 

[Rover4444]

Yes and there are some of us that are trying to help them and everyone else parroting this idea that it is not going to happen. And it does NOT take a genius to figure that out.

Either spend the money for the professional compute cards or live with the reduced speed of the consumer cards. Those are the choices. There are no others.


That's ok, no worries.

What, you're the invisible hand of the free market now?

Give us cheap VRAM or we won't buy new cards. There's plenty of current and upcoming alternatives available right now if you actually bothered to look, from Apple M2s and Strix Halo to repurposed mining rigs and networked machines. I am NOT obligated to buy crumbs for the price of a cake and say "thank you" for the privilege of winning a GPU lottery.

Ask for more and you'll get more, simple as.

 

[mechtech]

Fair enough.


Yuppers. Indiana Jone is an RTRT only game. It won't run unless you've got it on your card.

Interesting, I did not know this. Well another thing to look out for........see if RT is mandatory and see if it can be disabled.

 

[lexluthermiester]

What, you're the invisible hand of the free market now?

No. I'm someone with a ton experience and have been in this industry for 4 decades.

Give us cheap VRAM or we won't buy new cards.

And the short answer is: Have fun with that. IF you want the high end spec, you PAY the high end price. This is a dynamic that has been true ALWAYS. It has never not been true.

Ask for more, pay for more and you'll get more, simple as that.

Fixed that for you. Thus.

Interesting, I did not know this. Well another thing to look out for........see if RT is mandatory and see if it can be disabled.

Yeah, there are a few games going that direction. Players will need a card with RTRT features to play them.

 

[mechtech]

Yeah, there are a few games going that direction. Players with need a card with RTRT features to play them.

Well that's a turn off...............

 

[lexluthermiester]

Well that's a turn off...............

For some it is. I personally love the effect. I think it's the nature direction for games and tech to be going.

 

[Vayra86]

That is one game. Nice cherry pick. Of course you kinda just proved my point for me.


Nope, in reality.

I've recently demonstrated a 2060Super(which is very close to a 2070 by the specs.) to a customer and RTRT was on in CyberPunk2077 at 1080p. At 1440p, some settings had to be turned down but the performance was still acceptable.

Sure thing, and you'll still be using some DLSS blurfest upscaling method to get there at 1080p. So like I said, great, but it looks like you're gaming in 2009. I'm sure some nitwit said WOW AWESOME in your store, that's really indicative of nothing and you know this. If the bar can be dropped that low for you to like something, again, great for you, but I'll pass. Additionally, we both know the lower RT settings (almost everything below Psycho) barely changes a thing in the image but sure you can say you've got the RT option enabled. Even so, the performance hit is still a DLSS scenario more often than not on the class of cards in 2060-2070 range. Which means you're literally rendering at 720p or some order below 1080p.

I mean sure I played Cyberpunk on a GTX 1080 but I'm not going to sell the fairy tale its a great experience, because you had to use upscaling to get playable frames and even then it wasn't exactly, you know, beautiful to look at. So enabling RT and then having to scale down settings left and right to get that playable on 1080p with a lower render res to boot is... yeah. The 'mental asylum is that way-territory', if you ask me. But sure, its a great way to make your point I guess. You CAN indeed run RT on a 2070. I can run PT too on a 7900XT. Don't ask how it runs but hey... if I turn everything down to low...

What, you're the invisible hand of the free market now?

Give us cheap VRAM or we won't buy new cards. There's plenty of current and upcoming alternatives available right now if you actually bothered to look, from Apple M2s and Strix Halo to repurposed mining rigs and networked machines. I am NOT obligated to buy crumbs for the price of a cake and say "thank you" for the privilege of winning a GPU lottery.

Ask for more and you'll get more, simple as.

Lol yeah like you ask for more and you get a 2,5k 5090 to replace your 1,5k 4090.

Is that the more you mean? Or is the more being asked in this topic a straight up joke? Because you already GOT more. You're still saying you want more, and now its a class of card nobody in their right mind would make. And its already been explained to you but... ok you won't buy new cards. You can get in line with the other half of the world that isn't, and it hasn't stopped Nvidia reaching a 3 trillion market cap anyway. You think companies are looking for your business. You are mistaken.

 

[lexluthermiester]

Yeah, we're off topic and I'm not going to take this any further.

 

[Rover4444]

No. I'm someone with a ton experience and have been in this industry for 4 decades.

And the short answer is: Have fun with that. IF you want the high end spec, you PAY the high end price. This is a dynamic that has been true ALWAYS. It has never not been true.

Fixed that for you. Thus.

Okay? Do you want cards with cheap VRAM or not? Just say you're running defense for corporations so people can stay constrained with what models they can run.

I don't have to buy anything, so I won't. If I really wanted to though, I'd just get two 22GB 2080 Tis for $400 each.

Lol yeah like you ask for more and you get a 2,5k 5090 to replace your 1,5k 4090.

Is that the more you mean? Or is the more being asked in this topic a straight up joke?

No, it's not the more I mean. I thought it was very obvious what I mean. Did you not read the subject?

My idea of more would be a 24GB 4070 for $700-$800 as a true 3090 replacement. There's been rumors of 24GB B580s and 32GB 9070XTs, so there is plenty of very obvious market demand. Even before the AI bubble production users have been crying out for more VRAM, and now it's gotten so bad that even gamers are taking notice. Things really don't have to be this way.

16GB cards are already possible at $330. 24GB at $500 and 32GB at $700 isn't crazy, especially if you're using a B580-tier core. People even bought the 4060 Ti 16GB after NVIDIA priced it $100 over the 8GB version.

 

[Vayra86]

Okay? Do you want cards with cheap VRAM or not? Just say you're running defense for corporations so people can stay constrained with what models they can run.

I don't have to buy anything, so I won't. If I really wanted to though, I'd just get two 22GB 2080 Tis for $400 each.


No, it's not the more I mean. I thought it was very obvious what I mean. Did you not read the subject?

My idea of more would be a 24GB 4070 for $700-$800 as a true 3090 replacement. There's been rumors of 24GB B580s and 32GB 9070XTs, so there is plenty of very obvious market demand. Even before the AI bubble production users have been crying out for more VRAM, and now it's gotten so bad that even gamers are taking notice. Things really don't have to be this way.

16GB cards are already possible at $330. 24GB at $500 and 32GB at $700 isn't crazy, especially if you're using a B580-tier core. People even bought the 4060 Ti 16GB after NVIDIA priced it $100 over the 8GB version.

I'm not sure if you're trolling or just really that dense at this point. Mate. Your plea has been heard. It ain't happening. But again, feel free to write a 4 page essay to Nvidia on why they should. If you're trying to convince us first, you've already failed, as 90% of the people on TPU who know two things about GPUs have explained, with argumentation, to you at this point.

The best you will get is scraps and leftovers indeed like a 250gb/s 4060ti with 16GB. I think that's as close as you'll get; its cost effective because they just plastered more VRAM chips around the die, and that's that. Its NOT about what GBs you might desire for price point x or y, its about what's easy to bake because it was already being made anyway. Nvidia is already making poor man's AI Geforces as you pointed out. You say it isn't crazy, but I think it is, to sacrifice highly capable x70s on this crap, when the market is already short on GPUs to begin with. Every x70 with 24GB, is one less with 12GB, and functionally pointless as a gaming card which it is targeted at.

Moreover, its something the vast majority buying GPUs do not want for the same reasons crypto was a shitstorm for gamers. Which is further compounded by the fact there are indeed 24-32GB GPUs already available.

 

[Rover4444]

I'm not sure if you're trolling or just really that dense at this point. Mate. Your plea has been heard. It ain't happening. But again, feel free to write a 4 page essay to Nvidia on why they should. If you're trying to convince us first, you've already failed, as 90% of the people on TPU who know two things about GPUs have explained, with argumentation, to you at this point.

Disagree. And I don't like you strawmanning like a 5090 is what I'm asking for.

Whether it does or doesn't happen isn't my concern and I don't really care about convincing people who advocate against their own interests. I'll keep on asking until it happens, otherwise I'll do something else.

Effectively being told "shut up and buy expensive GPU" makes me think anyone who makes that argument is a bigger shill than I am a troll. Sorry.

 

[Vayra86]

Disagree. And I don't like you strawmanning like a 5090 is what I'm asking for.

Whether it does or doesn't happen isn't my concern and I don't really care about convincing people who advocate against their own interests. I'll keep on asking until it happens, otherwise I'll do something else.

Effectively being told "shut up and buy expensive GPU" makes me think anyone who makes that argument is a bigger shill than I am a troll. Sorry.

I don't think we disagree, you just don't like it. You can advocate all you want, we're just here to explain how that stance is problematic. You're also barking up the wrong tree here, as is this topic. Shilling isn't a thing at all though, not sure where you got that from.

 

[Rover4444]

The best you will get is scraps and leftovers indeed like a 250gb/s 4060ti with 16GB. I think that's as close as you'll get; its cost effective because they just plastered more VRAM chips around the die, and that's that.

A770 is cheaper, faster, and doesn't do clamshell. Even the B580 is pretty good if you can get it at MSRP.

Moreover, its something the vast majority buying GPUs do not want for the same reasons crypto was a shitstorm for gamers. Which is further compounded by the fact there are indeed 24-32GB GPUs already available.

It's not the same. VRAM is more than available, and gamers are now more than aware about VRAM nowadays, and that something like an 8GB GPU for >$300 is something they do not want. Adding more VRAM to a card isn't bottlenecking TSMC's 4nm production, and HBM is used instead of GDDR for datacenter AI cards so there's not even a bottleneck there, either. And that's not even getting into the fact that the prospect of generating unlimited free(!) money has a MUCH greater pull than generating unlimited AI waifus in the first place. I just don't think it's the same.

I don't even think market segmentation would be a good excuse, at least for AMD and Intel. What, 24/32GB of GDDR6 is suddenly gonna compete with 128/192GB of HBM3 now? Come on. And who's even buying the Radeon Pro cards?

I don't think we disagree, you just don't like it. You can advocate all you want, we're just here to explain how that stance is problematic. You're also barking up the wrong tree here, as is this topic. Shilling isn't a thing at all though, not sure where you got that from.

I can do both. And I disagree because there's already shipping manifests of 24GB B580 cards. I'd be very surprised if it comes out anywhere close to used 3090 prices.

 

[Vayra86]

A770 is cheaper, faster, and doesn't do clamshell. Even the B580 is pretty good if you can get it at MSRP.


It's not the same. VRAM is more than available, and gamers are now more than aware about VRAM nowadays, and that something like an 8GB GPU for >$300 is something they do not want. Adding more VRAM to a card isn't bottlenecking TSMC's 4nm production, and HBM is used instead of GDDR for datacenter AI cards so there's not even a bottleneck there, either. And that's not even getting into the fact that the prospect of generating unlimited free(!) money has a MUCH greater pull than generating unlimited AI waifus in the first place. I just don't think it's the same.

I don't even think market segmentation would be a good excuse, at least for AMD and Intel. What, 24/32GB of GDDR6 is suddenly gonna compete with 128/192GB of HBM3 now? Come on. And who's even buying the Radeon Pro cards?


I can do both. And I disagree because there's already shipping manifests of 24GB B580 cards. I'd be very surprised if it comes out anywhere close to used 3090 prices.

Time will tell how silly this market will get over time, I guess.

Here you go..

Despite Frank Azor's Dismissal, Whispers of a 32 GB Radeon RX 9070 XTX Resurface

Recent rumors hinted at a 32 GB variant of the Radeon RX 9070 XT being in the works, which were quickly dismissed as false information by AMD's Frank Azor. However, reliable sources seem to point at the contrary, stating that a 32 GB variant of the RX 9070 XT, likely dubbed the RX 9070 XTX, is...

www.techpowerup.com

 

[Dr. Dro]

Indiana Jones & The Great Circle runs fine on a 2060. Sure, you have to turn some settings down, but it runs at perfectly playable framerates. At 2070 can do better.. along with everything else that is newer/better.

Ahem.

How dare you ask owners of such luxurious RTX hardware to do something as egregious as *gasp* lower their settings!? Proof this gamer has a ultra high end PC is that he has the most expensive OLED TV in the market. Even I have only the previous year's model. Checkmate, liberal.

(this post is pure mockery and sarcasm, please don't take it seriously)

I'm not sure if you're trolling or just really that dense at this point. Mate. Your plea has been heard. It ain't happening. But again, feel free to write a 4 page essay to Nvidia on why they should. If you're trying to convince us first, you've already failed, as 90% of the people on TPU who know two things about GPUs have explained, with argumentation, to you at this point.

Agree, but I think that eventually given the rising the demands of LLMs, they'll develop a solution that is dedicated to training and inference. IMHO, Project DIGITS is probably a "prototype" of sorts, remember the CMP crypto mining processor series? It'll likely be something like that, but not rushed out and derived from gaming cards as those used to be: probably a dedicated GPGPU or FPGA processor that lacks a display engine and pretty much every other area useful for most computing tasks, but tailored specifically for inferencing, kind of something like this AMD/Xilinx card, which is a bit older now:

VCK5000 Versal Development Card

The AMD VCK5000 Versal development card is built on the AMD 7nm Versal™ adaptive SoC architecture and is designed for (AI) Engine development with Vitis end-to-end flow and AI Inference development with partner solutions. For full Vivado™ flow and device customization, please contact sales.

www.xilinx.com

That's how I see the future of specialized AI. If and only if this ends up trickling down to local consumer installations, we'd buy AI processing cards like GPUs for our desktops... but given anyone can open a browser and bring up Grok or Gemini to get their work done, I personally don't see that happening any time soon.

 

[Vayra86]

Ahem.

View attachment 385035

How dare you ask owners of such luxurious RTX hardware to do something as egregious as *gasp* lower their settings!? Proof this gamer has a ultra high end PC is that he has the most expensive OLED TV in the market. Even I have only the previous year's model. Checkmate, liberal.

(this post is pure mockery and sarcasm, please don't take it seriously)



Agree, but I think that eventually given the rising the demands of LLMs, they'll develop a solution that is dedicated to training and inference. IMHO, Project DIGITS is probably a "prototype" of sorts, remember the CMP crypto mining processor series? It'll likely be something like that, but not rushed out and derived from gaming cards as those used to be: probably a dedicated GPGPU or FPGA processor that lacks a display engine and pretty much every other area useful for most computing tasks, but tailored specifically for inferencing, kind of something like this AMD/Xilinx card, which is a bit older now:

VCK5000 Versal Development Card

The AMD VCK5000 Versal development card is built on the AMD 7nm Versal™ adaptive SoC architecture and is designed for (AI) Engine development with Vitis end-to-end flow and AI Inference development with partner solutions. For full Vivado™ flow and device customization, please contact sales.

www.xilinx.com

That's how I see the future of specialized AI. If and only if this ends up trickling down to local consumer installations, we'd buy AI processing cards like GPUs for our desktops... but given anyone can open a browser and bring up Grok or Gemini to get their work done, I personally don't see that happening any time soon.

Sure I just don't think its something Nvidia necessarily needs to be making. Or AMD. Or Intel.

Crypto went the way of ASICs too

 

[Dr. Dro]

Sure I just don't think its something Nvidia necessarily needs to be making. Or AMD. Or Intel.

Crypto went the way of ASICs too

True, although, I think their expertise is relevant and they might want a slice of that pie, especially Nv

 

[lexluthermiester]

That, right there, is freaken hilarious!! Ultra High End my hairy bum..