Realtime Rendering for AR/VR


In today’s blog we’ll be diving into a simple but effective Physically Based Rendering (PBR) pipeline for Unity, with an emphasis on Virtual Reality (VR) and Augmented Reality (AR) applications. For starters PBR pipelines have, until recently, been exclusive to PC and console experiences. But with recent hardware updates and the use of Unity 2018 and other tools, we can now create highly optimized, physically based scenes in Unity while maintaining that ever so important 60 FPS (Frames Per Second).

Also, whether you are new to PBR or not, I highly suggest taking a look at Allegorithmic’s Substance Academy, it should help you get a better understanding of Physically Based Rendering. 

Transforming 3D Assets

Let’s start with the 3d assets. As mentioned, the rule of thumb is to hit a minimum of 60 FPS or the experiences’ performance begins to suffer. If you’re unfamiliar with the requirement of frames per second, it’s essentially a measurement for a device’s display performance. It’s critical in VR because lower frame rates can make users nauseous. In order to create a 60+ FPS experience, you must model your “in game” assets within a reasonable vertex/poly count. This is highly dependent on the complexity of your scene, but a good rule of thumb is to shoot for JUST above mobile quality for any AR/VR experience. The scene complexity is determined by the number of polygons (or how the cool kids say “polys”) are included in the model(s). There are no exact numbers that qualify an asset as “mobile” quality. Assume that the less polys you can get away with, the better. If you have details that can be baked into a normal map, DO IT!

This is where PBR tools like Allegorithmic’s Substance Painter can make a huge difference in terms of quality and authoring time. You can quickly bake your high poly information (normals, AO, curvature, etc.) down into low poly models using Substance Painter. Regardless of your 3D package though, I suggest baking a material ID map (unique colors for each material) so they can be imported into Substance Painter for easier authoring. Substance Painter allows you to paint, bake, and export your final PBR textures into Unity in a highly efficient manner. There is also a texture export preset called “Unity 5” that is built specifically for the Standard Shader in Unity and will ensure that your materials are a 1 to 1 from Substance Painter. For more information on Allegorithmic’s Substance Painter, Substance Designer, and a bunch of helpful tutorials.


Once your final art content is in Unity, the next step will be scene optimization. Assuming you have a complex scene, you can drastically reduce draw calls by using (in my opinion) the most valuable plugin in the entire asset store, Mesh Baker.

This plugin will allow you to combine your meshes and create texture atlases which can reduce your draw calls by up to 90%. This, combined with baking lighting can help take your AR/VR experiences to the next level in terms of quality and performance. Overall this process allows for more efficient rendering, in turn creating high quality assets in a shorter time frame.