Yay! Welcome to part 2 of our render pass adventure! In this part of the so-called tutorial, we are going to focus on using render passes for the layering of objects. But before we begin, let us do a quick recap of what we went through in Part 1.
Scene Render Pass patch outputs only 2D texture, even when the rendered objects are 3D
Output texture can be manipulated with any 2D shader effects
With this in mind, let us move on to the main event!
Layering With Spark AR
One of the most common problems faced by new creators (including me when I just started) is Facemesh being covered by a background plane/rectangle when the face moves too far back. The easiest solution is of course to uncheck the depth test for your Facemesh material under the advanced render option. However, let's look at how render passes can be used to solve this problem too.
There are 2 inputs on a Scene Render Pass patch, 1) the background texture and 2) the object to render. In part 1, the background textures used are all camera textures but really we can use any texture we want; including texture from a Scene Render Pass patch. Also, background texture will always remain as background and will never intersect with the object rendered.
Hence, we can make use of how Scene Render Pass works to make sure our Facemesh will never intersect with any background by doing the setup below.
First, the background rectangle object is rendered as a texture before it is connected to the background input of the 2nd Scene Render Pass. The Facemesh is then the 2nd object to be rendered using the 2nd Scene Render pass, which is then output as a texture to Device Screen Output.
Result: the white Facemesh will always appear in front of the blue rectangle no matter how far back the Facemesh is. I know I know, this isn't anything exciting or fun. So let's increase the number of objects in the scene.
Consider the following scene we need to setup:
A rotating 3D cube in front of the face
Glasses on a face with face and head occluder
Person Segmentation
2 3D spheres moving in circles behind the person
In terms of layering, the layer from the back to front are 4, 3, 2, 1.
In a setup without render pass, this will happen. The moving cube and sphere will overlap with the occluder and glasses, covering partially. The segmentation is also unable to cover the 3D spheres. There is a non-render pass solution available but it is not quite as straightforward as using render pass, especially when you decide to make changes in the layering. So here's the setup for render pass to achieve what we want:
As you can see, the setup is simple. All you need to do is to follow the layering sequence you need, for our case it will be 4, 3, 2, 1. The objects you want to appear on the top layer will always be the last to be rendered out and objects at the bottom-most layer will be the first to be rendered. And here's the output:
If you compare what you see on the viewport (left) with the simulator, you can tell the differences before and after using render passes.
To recap,
Background texture for a Scene Render Pass patch will always remain as the background behind the rendered object
The objects you want to appear on the top layer will always be the last to be rendered out and objects at the bottom-most layer will be the first to be rendered out
If you have made it this far, congrats! Whenever you are ready, we will move on to part 3 of this tutorial to talk about scaling Facemesh and Background in proportional to Face tracking position with render pass (what a long name, I know. )
Again, you can download the working file for this tutorial HERE.
Comments