NVIDIA RTX Voice: Real-World Testing & Performance

[midnightoil]

This is great for people playing games with friends, but to my ear it sounds pretty nasty from a content creation or home studio perspective.

It's being way over-hyped.

 

[Lindatje]

It's being way over-hyped.

It’s Nvidia, of course it’s over-hyped.
Most people are blind to the name ‘Nvidia’....

 

[SetsunaFZero]

It’s Nvidia, of course it’s over-hyped.
Most people are blind to the name ‘Nvidia’....

indeed. Since the last Discord update, krisp.ai is now a part of discord. krisp can also be run as an standalone app but is costs some $/mo.
There are also some other noise suppressing software's available

But you have 6GB, so what's the problem?

I was playing CoD Warzone yesterday and noticed some shuttering and checked task-manger if any app was messing with my resources again and saw that RTX Voice was using 13% and ~500Mb system-ram not VRAM.
Looks like the GPU load depends on what app we are running. I will test this later after my job.

 

[Chrispy_]

I don't really care or need sound reduction according to people on my Teams, Discord, TS - even on my RTX card I'd rather not add 30W+ to the power consumption and definitely don't want it to reduce my framerates.

However, the tech does seem to work really well and I'd like to say that it was a really cool review, thanks Inle.

 

[TheoneandonlyMrK]

Well that's just failed logic. Of course it doesn't need tensor cores.
It doesn't even need a GPU. Frankly, everything can be calculated on a CPU - it'll just take longer.
In fact, everything can be calculated on a piece of paper - it'll just take longer.

It's CUDA code underneath. You can't "mod" it. You'll have to rewrite it.
Of course it is possible to write a program giving identical results just on CPUs.
But Nvidia makes GPUs and they made a program that runs on their product, as an additional feature. Are you able to accept this or not?

Have you tried it?.

Do you have proof that some use it on non Rtx with issues that are not just issues with the software itself as you claim to have seen?

And it seems Zonedymo got it right to me ,you don't need Rtx , despite Nvidia proclaiming this an RTX exclusive feature you don't need Rtx at all.

In fact vayra86 provided proof(FACTS not opinions) that tensor cores offered no improvement in performance, efficiency or quality to at least a small degree.
But substantially more via evidence than the opinion based argument your relying on.

 

[hurakura]

What is the point of all this? Where is that magic you talk about?

Every Sound Card has this capability and with decent mic it works very well.

nVidia is constantly reinventing the wheel

 

[Lionheart]

In B4 some hacker team find a way to exploit this and capture all your communications

Nvidia's already doing that.

 

[notb]

indeed. Since the last Discord update, krisp.ai is now a part of discord. krisp can also be run as an standalone app but is costs some $/mo.

Krisp is free up to 2h/week. $3.33/month for unlimited.
Also, if I understand correctly, it's training the model on users' calls. Can someone confirm?

How is the CPU usage?

edit:
I've listened to their examples. You can hear voice processing - Nvidia definitely does this better. And I assume they've chosen the best stuff they had...

There are also some other noise suppressing software's available

Of course there are. Nvidia didn't invent audio processing.
But RTX Voice is currently a free addition to their GPUs. We'll see if it's still free after beta.

What is the point of all this? Where is that magic you talk about?

Every Sound Card has this capability and with decent mic it works very well.

Hardware can filter some frequencies or use multiple mics to eliminate sound coming from far away etc.

RTX Voice is built around machine learning. There's some model. It was trained to tell apart voice and noise. It processes the signal.

Actually, I really like the examples from Krisp's website.

World’s #1 Noise Cancelling App and AI Meeting Assistant

Krisp cancels background noise, records, transcribes, and summarizes meetings and calls.

krisp.ai

There's a section "See Krisp in Action" with a switch.
That said, there's a very pronounced voice processing. RTX Voice works a better.

Keep in mind this is trained to leave the main (loudest, clearest - assuming: nearest) voice. So it may remove some of the sound that you'd like to keep, like music or clapping.
Also, since it removes voices in the background, it may also mute someone who actually is in the meeting - just sits further from the mic.

 

[midnightoil]

This reminds me a bit of all the silly hype around GoXLR ... which is 'clever', but the sound quality is really poor.

 

[notb]

This is great for people playing games with friends, but to my ear it sounds pretty nasty from a content creation or home studio perspective.

It's being way over-hyped.

Oh, the hateful term strikes again.
Creating what content exactly?

Obviously, you wouldn't want to use this if your goal is to record high-quality voice: reading, commentary, singing especially.

Everyone who works remotely and has voice meetings has surely heard someone speaking in the background, crying children, blenders, car engines, toilet flushes and farts.
Obviously, it also removes the random noise coming from cheap mics.

I was playing CoD Warzone yesterday and noticed some shuttering and checked task-manger if any app was messing with my resources again and saw that RTX Voice was using 13% and ~500Mb system-ram not VRAM.

Oh, fine. I've never used voice communication in a game, so I just don't think about this scenario.

Apps just got big and they tend to keep a lot of stuff in RAM "just in case". And reserve some more - also "just in case".
My MS Teams client uses between 500 and 600MB outside of meetings ("idle"). And it's running all the time.
When I open a meeting with chat, voice and video, it takes maybe 50MB more.

Well, it's RAM. It's there to be used.

 

[midnightoil]

Oh, the hateful term strikes again.
Creating what content exactly?

Obviously, you wouldn't want to use this if your goal is to record high-quality voice: reading, commentary, singing especially.

Everyone who works remotely and has voice meetings has surely heard someone speaking in the background, crying children, blenders, car engines, toilet flushes and farts.
Obviously, it also removes the random noise coming from cheap mics.

It's not much better than stuff already available, and the vast majority of people in those scenarios are not going to have a compatible card or any need for it whatsoever. So that would mean acquiring it would make it vastly expensive dead weight.

It's good for gamers. That's it.

Almost anyone else ... it's either unsuitable or better solutions exist.

 

[sutyi]

Just out of curiosity... could guys check if the "modded" version can be used with a secondary card (GT1030 (?)) to offload RTX Voice workload from your main GPU?

@Inle @W1zzard

 

[notb]

It's good for gamers. That's it.

Actually, I think it's the other way round.
Gamers like to spend on their gear. I'm sure those streaming or doing a lot of voice communication will gladly spend a fortune on expensive mics with noise cancelling etc.
It's not like we didn't have guys like that in RTX Voice threads.

Most gamers probably use headphones. They mostly connect from home, so the background is not bad to start with.
I can't think of a group that would benefit less.

Actually, this is most useful for people doing casual or work-related voice communication. Most of them will use the laptop mic. Some may have headphones, but mostly cheap stuff - bundled to their phone or something like that. Some will work from busy places, be it office or on the road. You often have someone joining from a phone, preferably while driving or shopping...

Sure, this officially only works on RTX GPUs, so not exactly what you'll find in typical "casual" or "work" laptops.
If Nvidia opens it to all recent chips, we have a whole different story: a lot of laptops come with an MX or better. And maybe starting from next generation all chips will have tensor cores?

And let's be honest... there's a very good chance Intel or Microsoft will launch something similar now.
Honestly, with their computing budget and number of employees, they probably could make something similar in a week. But it's a process. They have to have a vote for project name, get a budget approval, write some cute Confluence pages, run tests, make a presentation for the Board.
2 months, maybe more.

Almost anyone else ... it's either unsuitable or better solutions exist.

Hmm... You say "better", so you probably haven't listened to recordings in the review. Do it.

Well, there are other apps, but they're either paid or limited. Companies somehow have to earn. It's a complex, time-consuming process and you have to collect *a lot* of data.
Honestly, I've never heard anything that comes close to Nvidia's. It's really, really good.

 

[Fergutor]

Amazing.
What about latency (I mean the processing time it takes)? Sure it's minimal and most of the times wouldn't matter, but in FPS games it adds

 

[W1zzard]

secondary card

I haven't checked, but I haven't seen any options that let you select which card to run on. Obviously this is possible with CUDA, maybe feedback the idea back to NV

 

[notb]

Amazing.
What about latency (I mean the processing time it takes)? Sure it's minimal and most of the times wouldn't matter, but in FPS games it adds

You're worried about voice communication latency or game latency? 2 separate beings.

 

[Vayra86]

It’s Nvidia, of course it’s over-hyped.
Most people are blind to the name ‘Nvidia’....

You're not wrong I think.... but its also the buzz word AI and doing things over tensor cores that was the draw. That is also what got people questioning it, and researching and modding it to work elsewhere.

That was also my key point here... its not like I use voice a lot on my desktop PC and its also not like the suppression already in use wasn't sufficient I think that goes for almost everyone. But still, what is nice about this is its unique feature of running over the GPU and being just software you can use for anything else.

And let's be honest... there's a very good chance Intel or Microsoft will launch something similar now.

Oh yes this is just dying to get added to a new Windows version, Intel chipset software and AMD driver.

Its funny how Nvidia has this trickle down effect on everyone else every once in a while

 

[notb]

Have you tried it?.

Have I tried what? Calculating on paper? Quite a lot.

Do you have proof that some use it on non Rtx with issues that are not just issues with the software itself as you claim to have seen?

I don't even have a proof that they really use it. I didn't go to their homes to check.
It's Internet. People say all kinds of stuff. But I see a lot of comments suggesting that.

And it seems Zonedymo got it right to me ,you don't need Rtx , despite Nvidia proclaiming this an RTX exclusive feature you don't need Rtx at all.

I said he's absolutely right. You don't need GPUs for matrix multiplication.
There are multiple reasons why Nvidia would say it's RTX-only.
2 most obvious:
- because they think it won't perform well on non-RTX,
- because they promote their flagship product line.
Both are valid. It's their product. Nvidia makes quite a lot of software that only runs on their GPUs (or some of their GPUs).

It's almost like that Intel MKL discussion some time ago. How dare they?!

In fact vayra86 provided proof(FACTS not opinions) that tensor cores offered no improvement in performance, efficiency or quality to at least a small degree.
But substantially more via evidence than the opinion based argument your relying on.

@Vayra86 said he run the app on his 1080 and it seems to work similarly well. That's a proof?

Well, I haven't used my PC since the weekend and probably won't today.
But here's an idea:
We'll all use a common audio source. @Inle 's "RTX Voice Off" files would be perfect - if he gives us permission (and ideally the high quality original recording ).
Everyone runs using a more or less identical workflow on an Nvidia card at hand. We compare the results (and the GPU load).
Interested?

 

[holyprof]

If I had a RTX card (don't want to install modded drivers/programs on my PC) I would definitely pit RTX Voice against Audacity or some other sound editor's noise reduction filters.

Can please someone do a test: record yourself speaking without any filtering. This will assure the tests are using the same base sound.
Then play the recorded sound on high quality speaker(s) and record it again twice: one using RTX Voice, another without it. Next, apply noise reduction filter in Audacity to the second recording and compare results.
I've recorded sound in a server room, the signal to noise ratio was like 20 dB - that's really bad (those of you that have ever been in a server room know what I'm talking about). Audacity's noise reduction filter does magic with removing the fan and hard drive noise (server room fans sound like a vacuum cleaner, more like 50 high RPM small vacuum cleaners with different noise pitch). If RTX Voice is at least as good as Audacity's noise reduction, but works in realtime, it's really a great piece of software.
I would even pay for it if Nvidia wishes to release it as a standalone offline program that only works on Geforce cards, keeping the realtime RTX Voice for RTX card owners. Or as a plugin for sound editor programs.

So take note Nvidia, keep your marketing happy (RTX Voice sill only works on RTX cards) and release a non-RTX paid version that run on CPU or CUDA GPU but not in realtime.

 

[toilet pepper]

Can someone try turning this on while streaming an RTX game like Control and see how much of a performance hit there is?

 

[holyprof]

Can someone try turning this on while streaming an RTX game like Control and see how much of a performance hit there is?

I have an idea, I can sell my kidney and buy two RTX cards - a 2080Ti for my main PC and a 2060 for my old PC. Then use an HDMI capture card so I can stream and use RTX Voice on the streaming PC and keep my framerates without drops on the main PC .

 

[sutyi]

Obviously this is possible with CUDA, maybe feedback the idea back to NV

Considering the performance implications and how they are branding it RTX Voice and trying to limit the usage to RTX cards... yeah I don't see that happening.

I just thought that it has a configuration console by default where you can select what GPU it does use, considering there might be RTX SLi users out there or just people with multiple RTX GPUs in their system for whatever reasons.

UPDATE:

 

[Houd.ini]

So it's like an inverted karaoke accelerator?

 

[Vayra86]

Have you tried it?.

Do you have proof that some use it on non Rtx with issues that are not just issues with the software itself as you claim to have seen?

And it seems Zonedymo got it right to me ,you don't need Rtx , despite Nvidia proclaiming this an RTX exclusive feature you don't need Rtx at all.

In fact vayra86 provided proof(FACTS not opinions) that tensor cores offered no improvement in performance, efficiency or quality to at least a small degree.
But substantially more via evidence than the opinion based argument your relying on.

No no, its only proof if we ALL do it, according to him Totally worth doing as well... lol maybe in week 25 of lockdown?. Gotta love the selective reading comprehension ey.

I think its clear to everyone what this app does even if it does 'ask' for Tensor first before falling back on CUDA. There are no secrets in audio... And as usual, lots of BS / snake oil involved. That is the logical assumption when you consider ANY audio solution, really. The only interesting part of RTX voice is how it works with the data itself along the way as the suppression is effective. But Tensor is a complete non issue and I think that is clear enough... So yes, it might save a few % perf in game... but if you needed that, you wouldn't run RTX voice over your Turing card either, because it STILL takes performance away.

 

[notb]

No no, its only proof if we ALL do it, according to him

Still not a proof. But at least adds some statistical significance.

And yes, I proposed an experiment which would probably provide some interesting results and smear some scientific method on this discussion.
But maybe it's too much for this community.
I apologize for the idea. Let's go back to the usual "Nvidia is evil" points.