Tuesday, March 19th 2024
NVIDIA Digital Human Technologies Bring AI Characters to Life
NVIDIA announced today that leading AI application developers across a wide range of industries are using NVIDIA digital human technologies to create lifelike avatars for commercial applications and dynamic game characters. The results are on display at GTC, the global AI conference held this week in San Jose, Calif., and can be seen in technology demonstrations from Hippocratic AI, Inworld AI, UneeQ and more.
NVIDIA Avatar Cloud Engine (ACE) for speech and animation, NVIDIA NeMo for language, and NVIDIA RTX for ray-traced rendering are the building blocks that enable developers to create digital humans capable of AI-powered natural language interactions, making conversations more realistic and engaging.
"NVIDIA offers developers a world-class set of AI-powered technologies for digital human creation," said John Spitzer, vice president of developer and performance technologies at NVIDIA. "These technologies may power the complex animations and conversational speech required to make digital interactions feel real."
World-Class Digital Human Technologies
The digital human technologies suite includes language, speech, animation and graphics powered by AI:
To showcase the new capabilities of its digital human technologies, NVIDIA worked across industries with leading developers, such as Hippocratic AI, Inworld AI and UneeQ, on a series of new demonstrations.
Hippocratic AI has created a safety-focused, LLM-powered, task-specific Healthcare Agent. The agent calls patients on the phone, follows up on care coordination tasks, delivers preoperative instructions, performs post-discharge management and much more. For GTC, NVIDIA collaborated with Hippocratic AI to extend its solution to use NVIDIA ACE microservices, NVIDIA Audio2Face along with NVIDIA Animation graph and NVIDIA Omniverse Streamer Client to show the potential of a generative AI healthcare agent avatar.
"Our digital assistants provide helpful, timely and accurate information to patients worldwide," said Munjal Shah, cofounder and CEO of Hippocratic AI. "NVIDIA ACE technologies bring them to life with cutting-edge visuals and realistic animations that help better connect to patients."
UneeQ is an autonomous digital human platform specialized in creating AI-powered avatars for customer service and interactive applications. Its digital humans represent brands online, communicating with customers in real time to give them confidence in their purchases. UneeQ integrated the NVIDIA Audio2Face microservice into its platform and combined it with Synanim ML to create highly realistic avatars for a better customer experience and engagement.
"UneeQ combines NVIDIA animation AI with our own Synanim ML synthetic animation technology to deliver real-time digital human interactions that are emotionally responsive and deliver dynamic experiences powered by conversational AI," said Danny Tomsett, founder and CEO of UneeQ.
Bringing Dynamic Non-Playable Characters to Games
NVIDIA ACE is a suite of technologies designed to bring game characters to life. Covert Protocol is a new technology demonstration, created by Inworld AI in partnership with NVIDIA, that pushes the boundary of what character interactions in games can be. Inworld's AI engine has integrated NVIDIA Riva for accurate speech-to-text and NVIDIA Audio2Face to deliver lifelike facial performances.
Inworld's AI engine takes a multimodal approach to the performance of non-playable characters (NPCs), bringing together cognition, perception and behavior systems for an immersive narrative with stunning RTX-rendered characters set in a beautifully crafted environment.
"The combination of NVIDIA ACE microservices and the Inworld Engine enables developers to create digital characters that can drive dynamic narratives, opening new possibilities for how gamers can decipher, deduce and play," said Kylan Gibbs, CEO of Inworld AI.
Game publishers worldwide are evaluating how NVIDIA ACE can improve the gaming experience.
Developers Across Healthcare, Gaming, Financial Services, Media & Entertainment and Retail Embrace ACE
Top game and digital human developers are pioneering ways ACE and generative AI technologies can be used to transform interactions between players and NPCs in games and applications.
Developers and platforms embracing ACE include Convai, Cyber Agent, Data Monsters, Deloitte, Hippocratic AI, IGOODI, Inworld AI, Media.Monks, miHoYo, NetEase Games, Perfect World, Openstream, OurPalm, Quantiphi, Rakuten Securities, Slalom, SoftServe, Tencent, Top Health Tech, Ubisoft, UneeQ and Unions Avatars.
More information on NVIDIA ACE is available at https://developer.nvidia.com/ace. Platform developers can incorporate the full suite of digital human technologies or individual microservices into their product offerings.
Developers can start their journey on NVIDIA ACE by applying for the early access program to get in-development AI models. To explore available models, developers can evaluate and access NVIDIA NIM, a set of easy-to-use microservices designed to accelerate the deployment of generative AI, for Riva and Audio2Face on ai.nvidia.com today.
Source:
NVIDIA
NVIDIA Avatar Cloud Engine (ACE) for speech and animation, NVIDIA NeMo for language, and NVIDIA RTX for ray-traced rendering are the building blocks that enable developers to create digital humans capable of AI-powered natural language interactions, making conversations more realistic and engaging.
World-Class Digital Human Technologies
The digital human technologies suite includes language, speech, animation and graphics powered by AI:
- NVIDIA ACE—technologies that help developers bring digital humans to life with facial animation powered by NVIDIA Audio2Face and speech powered by NVIDIA Riva automatic speech recognition (ASR) and text-to-speech (TTS). ACE microservices are flexible in allowing models to run across cloud and PC depending on the local GPU capabilities to help ensure the user receives the best experience.
- NVIDIA NeMo—an end-to-end platform that enables developers to deliver enterprise-ready generative AI models with precise data curation, cutting-edge customization, retrieval-augmented generation and accelerated performance.
- NVIDIA RTX—a collection of rendering technologies, such as RTX Global Illumination (RTXGI) and DLSS 3.5, that enable real-time path tracing in games and applications.
To showcase the new capabilities of its digital human technologies, NVIDIA worked across industries with leading developers, such as Hippocratic AI, Inworld AI and UneeQ, on a series of new demonstrations.
Hippocratic AI has created a safety-focused, LLM-powered, task-specific Healthcare Agent. The agent calls patients on the phone, follows up on care coordination tasks, delivers preoperative instructions, performs post-discharge management and much more. For GTC, NVIDIA collaborated with Hippocratic AI to extend its solution to use NVIDIA ACE microservices, NVIDIA Audio2Face along with NVIDIA Animation graph and NVIDIA Omniverse Streamer Client to show the potential of a generative AI healthcare agent avatar.
"Our digital assistants provide helpful, timely and accurate information to patients worldwide," said Munjal Shah, cofounder and CEO of Hippocratic AI. "NVIDIA ACE technologies bring them to life with cutting-edge visuals and realistic animations that help better connect to patients."
UneeQ is an autonomous digital human platform specialized in creating AI-powered avatars for customer service and interactive applications. Its digital humans represent brands online, communicating with customers in real time to give them confidence in their purchases. UneeQ integrated the NVIDIA Audio2Face microservice into its platform and combined it with Synanim ML to create highly realistic avatars for a better customer experience and engagement.
"UneeQ combines NVIDIA animation AI with our own Synanim ML synthetic animation technology to deliver real-time digital human interactions that are emotionally responsive and deliver dynamic experiences powered by conversational AI," said Danny Tomsett, founder and CEO of UneeQ.
Bringing Dynamic Non-Playable Characters to Games
NVIDIA ACE is a suite of technologies designed to bring game characters to life. Covert Protocol is a new technology demonstration, created by Inworld AI in partnership with NVIDIA, that pushes the boundary of what character interactions in games can be. Inworld's AI engine has integrated NVIDIA Riva for accurate speech-to-text and NVIDIA Audio2Face to deliver lifelike facial performances.
Inworld's AI engine takes a multimodal approach to the performance of non-playable characters (NPCs), bringing together cognition, perception and behavior systems for an immersive narrative with stunning RTX-rendered characters set in a beautifully crafted environment.
"The combination of NVIDIA ACE microservices and the Inworld Engine enables developers to create digital characters that can drive dynamic narratives, opening new possibilities for how gamers can decipher, deduce and play," said Kylan Gibbs, CEO of Inworld AI.
Game publishers worldwide are evaluating how NVIDIA ACE can improve the gaming experience.
Developers Across Healthcare, Gaming, Financial Services, Media & Entertainment and Retail Embrace ACE
Top game and digital human developers are pioneering ways ACE and generative AI technologies can be used to transform interactions between players and NPCs in games and applications.
Developers and platforms embracing ACE include Convai, Cyber Agent, Data Monsters, Deloitte, Hippocratic AI, IGOODI, Inworld AI, Media.Monks, miHoYo, NetEase Games, Perfect World, Openstream, OurPalm, Quantiphi, Rakuten Securities, Slalom, SoftServe, Tencent, Top Health Tech, Ubisoft, UneeQ and Unions Avatars.
More information on NVIDIA ACE is available at https://developer.nvidia.com/ace. Platform developers can incorporate the full suite of digital human technologies or individual microservices into their product offerings.
Developers can start their journey on NVIDIA ACE by applying for the early access program to get in-development AI models. To explore available models, developers can evaluate and access NVIDIA NIM, a set of easy-to-use microservices designed to accelerate the deployment of generative AI, for Riva and Audio2Face on ai.nvidia.com today.
3 Comments on NVIDIA Digital Human Technologies Bring AI Characters to Life
Also I already touched on AI being used to make NPC interactions more realistic recently in regard to Multi-Core CPU's and threading. The GPU's will probably do a lot of the grunt work, but CPU threading will still be pretty important if you involve lots of AI with lots of different interaction behavioral algorithm patterns. I'm sure there will be cases where both GPU or CPU is a bit more important from a latency standpoint to keep frames reasonable yet entail very broad game scene dynamics and interactions.
It's cool though I like it I had mentioned AI behavior of zombies to the developer of DeadPoly about a year ago and get a bunch of different scripts that can be hybridized in a procedural manner. Like honestly overly predictable set patterns is a bit dated for games. There needs to be a good bit more variance to the behavior though controllable to reasonable point by a weighting system.
UO had a interesting eco system game environment, but was very simplistic, but imagine something like that meets Everquest with variable AI pathing and spawns that can be single spawns or groups and with variable behavior on how they fight and aggro. At that point combining all of it together you'd end up with something rather spectacular I believe. Mixing and combining and sort of hybridizing it a bit like Grim Dawn with classes, but more for procedural aspects of different parts of the AI design to blend and combine. You could even weight each part a bit differently. You could even then rationalize it into proximity area zone presets that follow their own biome AI behaviors.