Today we are coming back to the post-process material we had, and I wanted to have a look at masking. Right now the outline is applied to everything, including the skybox, and although there is a manual way to mask these things, I believe automating it would be best.
First of all, we would have to enable the custom depth stencil. This will allow us to select specific objects and make them render on a different layer. We will be able to get this information then and use it to mask our post-processing.
After this, we can get it into our post-process material with the scene texture -> custom stencil.
To get the information, we can just go to the mesh and tick the Render CustomDepth Pass checkbox and set our value to a number. As we want a true or false - these can range from 0 to 255 - I just selected 1.
The problem with this approach is that it would apply to the whole element, not taking into consideration any occlusion.
To solve this, I found this clever solution online. Using a custom code node, instead of the Scene Texture stencil, BYC writes this code:
First, we make check for the Scene Textures. After that, we gather the Screen Space Data from the viewport with the Screen Position UVs, and then subtract to the custom depth that we got the depth data, ceiling and clamping this afterwards. These are made to get integers rounded up and set between 0 and 1. With this information, we multiply by an alpha that we set, to lower the values.
Now we can see that the depth is taken into consideration when calculating the values of the custom stencil, and allows for a much better look.
As this kind of sharp lines doesn't suit very well the environment, there is an opacity value that Lerps between the normal screen values and the post process values depending on the range that we set. This way we can have a more subtle look to it.
This is how it looked before any post process. It is a very subtle change, but I believe it adds to it, and I got to learn a lot from this experience.