Website powered by

Unity - NVidia GameWorks VXGI Custom Integration

This project came out of a few different needs and wound up solving a lot of problems for us in the long run. Our spaces are all dynamic and user generated. This pretty much means that outside of a directional light, there is nothing we can bank on for light mapping or baking prior to runtime. RTX was also not around at the time, and we still have hardware support limitations for it on our enterprise boxes out in the field. However, for our VR applications on big desktop computers, we needed something to push the visual quality forward, beyond our more limited platforms (mobile, web etc.).

So we found NVidia GameWorks VXGI library. This checked all of our boxes: realtime, dynamic, global illumination, and could be run on a laptop (albeit a nice one) if need be. The integration process was admittedly arduous and required a lot of tinkering and modification to our core shaders to get it to work, but Unity's low level rendering plugin API was our main path to success. We wrote the entire plugin in C++ using the libraries provided, and we were able to get it stood up.

The video above shows it in action (not hooked up to an HMD, though it could be) in the classic Sponza demo scene. The video has a camera moving around while I toggle through the different debug views to show how the end result is assembled. raw albedo color and normals are necessary for compositing the end result. Those are first two views seen.

The third is the end result of the process called cone tracing. Cone tracing is the stage where light is actually propogated through out the scene from direct lighting sources. and NVidia's plugin spits back the screen space texture you see in the video for use in your final composite.

There are however two stages prior two cone tracing that need to be completed: opacity, and emittance voxelization. Essentially these processes re-interpret the scene in a uniform voxel grid (like minecraft), and that data is stored in 3D RenderTextures. Opacity voxelization simply gathers objects, it makes the determination of whether something even gets voxelized to be included in the next stages of calculations, so named because it uses a material's opacity to make the determination. Emittance voxelization then runs our standard lighting calculations to compute the direct lighting that's effecting every voxel.

From there conetracing is performed to output the final render texture mentioned above. It produces wonderful results and has, thus far, been a tremendous value-add to the render pipeline in our VR platform.