Tuesday, August 22nd 2023
NVIDIA Paves the Way for Natural Speech Conversations with Game NPCs
Imagine you're in a vast RPG fill with hundreds, if not thousands, of interactive NPCs (non-playable characters). All current RPGs conduct your interactions with them over a bunch of pre-defined statement selections, where you choose among a bunch of text-based options on the screen, which elicits a certain response from the NPC. This feels very unnatural and railroaded, but NVIDIA plans to change this. With ACE (character engine) and NeMo SteerLM (a natural language model), NVIDIA wants to make voice based interactions with NPCs possible. This is a very necessary stepping stone toward the near-future, where NPCs will be backed by large GPTs letting you have lengthy conversations with them.
The way this works is, the player gives an NPC a natural language voice input. A speech-to-text engine and LLM process the voice input, and generate a natural language response. Omniverse Audio2Face is leveraged to create the NPC's response in real time. Announcing this Gamescom, NVIDIA's new NeMo SteerLLM adds life to the part of ACE that processes the natural voice input, and based on the kind of personality traits the game developer gives an NPC, generates responses with varying degree of creativity, humor, and toxicity among other attributes.
The way this works is, the player gives an NPC a natural language voice input. A speech-to-text engine and LLM process the voice input, and generate a natural language response. Omniverse Audio2Face is leveraged to create the NPC's response in real time. Announcing this Gamescom, NVIDIA's new NeMo SteerLLM adds life to the part of ACE that processes the natural voice input, and based on the kind of personality traits the game developer gives an NPC, generates responses with varying degree of creativity, humor, and toxicity among other attributes.
26 Comments on NVIDIA Paves the Way for Natural Speech Conversations with Game NPCs
"In video games and entertainment systems, a motion controller is a type of game controller that uses accelerometers or other sensors to track motion and provide input. "
en.wikipedia.org/wiki/Motion_controller
If you follow the link of track motion, guess what there isn't a single mention of phones either. The only mention of gyros in general is that of inertial systems which is not used in a single motion controller system for the reasons mentioned in my last post (drift, inaccuracy). You seem to be the only person in the world who thinks phones are suitable for motion controls, all to try and foist your own bizarre hate of motion controls and your corresponding invented narrative. I really think it's silly because, in a hypothetical scenario where phones are motion controllers (they aren't), the level of capability provided by phones vs even the Wii and especially VR headsets is worlds apart. You are in essence saying that in such a hypothetical scenario that no one wants motion controls because the cherry picked worst example of them is underused / not used at all. It's absolutely ridiculous, you've just exchanged your argument from one logical fallacy to another, in this case cherry-picking. As I've pointed out multiple times already, the capability of what phones can do and what motion controllers can do are vastly different in regards to motion tracking.