Monday, May 8th 2017
NVIDIA Releases VRWorks Audio and 360 Video SDKs at GTC
Further planting its roots on the VR SDK and development field, NVIDIA has just announced availability of two more SDK packages, for their VRWorks Audio and 360 Video suites. Now a part of NVIDIA's VRWorks suite of VR solutions, the VRWorks Audio SDK provides real-time ray tracing of audio in virtual environments, and is supported in Epic's Unreal Engine 4 (here's hoping this solution, or other solutions similar to it, address the problem of today's game audio.) The VRWorks 360 Video SDK, on the other hand, may be less interesting for graphics enthusiasts, in that it addresses the complex challenge of real-time video stitching.
Traditional VR audio ( and gaming audio, for that matter) provide an accurate 3D position of the audio source within a virtual environment. However, as it is handled today, sound is processed with little regard to anything else but the location of the source. With VRWorks Audio, NVIDIA brings to the table considerations for the dimensions and material properties of the physical environment, helping to create a truly immersive environment by modeling sound propagation phenomena such as reflection, refraction and diffraction. This is to be done in real time, at a GPU level. This work leverages NVIDIA's OptiX ray-tracing technology, which allows VRWorks Audio to trace the path of sound in real time, delivering physically accurate audio that reflects the size, shape and material properties of the virtual environment.
NVIDIA says that this is, at least for now, the only hardware-accelerated and path-traced audio solution that creates a complete acoustic image of the environment in real time without requiring any "pre-baked" knowledge of the scene. As the scene is loaded by the application, the acoustic model is built and updated on the fly, with audio effect filters being generated and applied on the sound source waveforms. The software release consists of a set of C-APIs for integration into any engine or application, and an integration for Epic's Unreal Engine 4 that's now available on GitHub.
As to its VRWorks 360 Video SDK, it enables real-time 4K 360 video capture, stitching and streaming. Regarding this feature, Kinson Loo, CEO of Z CAM, said that "The fact that NVIDIA manages to stitch 4K 360 stereoscopic video in real time, making livestreaming possible, changes the production pipeline and enables entirely new use cases in VR."
Source:
NVIDIA Blogs
Traditional VR audio ( and gaming audio, for that matter) provide an accurate 3D position of the audio source within a virtual environment. However, as it is handled today, sound is processed with little regard to anything else but the location of the source. With VRWorks Audio, NVIDIA brings to the table considerations for the dimensions and material properties of the physical environment, helping to create a truly immersive environment by modeling sound propagation phenomena such as reflection, refraction and diffraction. This is to be done in real time, at a GPU level. This work leverages NVIDIA's OptiX ray-tracing technology, which allows VRWorks Audio to trace the path of sound in real time, delivering physically accurate audio that reflects the size, shape and material properties of the virtual environment.
As to its VRWorks 360 Video SDK, it enables real-time 4K 360 video capture, stitching and streaming. Regarding this feature, Kinson Loo, CEO of Z CAM, said that "The fact that NVIDIA manages to stitch 4K 360 stereoscopic video in real time, making livestreaming possible, changes the production pipeline and enables entirely new use cases in VR."
14 Comments on NVIDIA Releases VRWorks Audio and 360 Video SDKs at GTC
have you tried any headset? it's not clunky if it's light & fits well, if backpack computers are becoming a thing thanks to the performance we get per watt by now
did old failed VR look good? did it have positional tracking? no, old VR sucked
let me show you what i tried a few days ago
it was a project for museums where you can walk through an upscaled CT scanned archaeological object, an object that will be damaged if it was physically opened, but turning it into a 3d render means we can examine it without any worries
therefore it's naive & offensive to think an entire platform, a user interface, a set of concepts has anything to do with a flood of pointless pew pew games on steam, as if this exact conversation didnt happen over & over again with smartphones, social networks, 2d games, movies, or books
so if you have an actual idea of what VR should be, start making it, unreal & unity are free... or tell us the idea so someone else can make it
My 5 years old Creative card is more than up for the task.
We've had audio tracing with environmental effects even back then and after sacking DirectSound 3D and all extensions for it, fast forwarding 2 decades and we're again at point ZERO, reinventing everything we already had ages ago, AGAIN. Sometimes I just don't understand the industry. We already had jaw dropping audio with breathtaking 3D positioning in games and all of it got scraped with dumb Microsoft screwing it all with garbage software audio layer. Which hasn't improved or changed since Vista.
Creative is imo still the king for audio, as much as they can even do on top of a "clueless" audio API. The surround softens the transitions better between audio channels which makes audio tracking better, but they still can't do it as well as if the game actually told the audio API "hey, the sound is coming from this location exactly, project that into speakers/headphones".
People have no clue how far audio in games has regressed since sacking of hardwre accelerated API's...
"considerations for the dimensions and material properties of the physical environment, helping to create a truly immersive environment by modeling sound propagation phenomena such as reflection, refraction and diffraction" as someone who has produced music, these material effects are too demanding to have been simulated in realtime over 15 years ago (but yes there are shortcuts that could be done instead, games are all about shortcuts & i do like sounding good enough rather than perfectly accurate... riddick dark athena & killzone 2/3 come to mind as having nice sounding fake reflection or occlusion on last gen consoles, the same way an environment map is a fake reflection that looks fine in most situations)