#deferred lighting

[gamedesignblogrole]

Deferred Lighting

[pixlmelon]

In the last couple of days I've been working to improve the lighting engine. I've added support for multiple lights as well as a small trick that renders shadows onto vertical surfaces. All the lighting calculations are now based on the game's internal resolution (the sprites resolution). The processed frame is then rendered with a transformation matrix that scales it up. This behaviour allows me to get that pixelated low-res look I am after, and should also speed up the lighting engine calculations. It's also a way to achieve resolution independence. I've also upgraded the player sprite with a (flat for now) normalmap and a heightmap, so it's showing correctly now. One thing that bothers me is the artifact that shows during the walking animation, which wasn't present before I started programming the lighting engine and was probably introduced somewhere along the way.

[fdastero-blog]

parallax mapping. Note: I'm using no magnification filtering in this shot because there is the seam issue with texture atlas and mip maping which confuses the hardware. See here: http://www.gamedev.net/topic/602143-texture-atlas-seam-issue/

Also fixed specularity issue. In this shot there is the environment (low freq. SH sky light) lighting from the sky and also specular only contribution from a directional light representing sun/moon. filtering is none.

[fdastero-blog]

It turns out that I made a REALLY SILLY mistake. Like I mentioned in a previous post, I need to compute a matrix which transforms a normal in tangent space into view space on a per-pixel basis. The general case is a bit involved however if you assume that your mesh is cube, then this becomes easy because you assume that ddy(viewPos) aligns with the tangent vector T. (You know because T and B vectors in tangent space are supposed to align with V and U). I was correct, the TBN matrix I derived was indeed correct. EXCEPT, I forgot I'm trying to transform from tangent-space to view-space, which means I need the inverse of the TBN matrix. I left that step out. So to make it work you just transform your tangent normal by the transpose of of the derived TBN. Or simply do a left multiply assuming row vector matrix multiplication (the opposite rotation).

Okay, now I have something that I know works. It's time to go back to what I had before. It's not much code to change. Basically I need to go back to use instanced lights. But this time, I did some refactoring so it doesn't matter whether it's instanced or not. I can also compare instanced and non-instanced code path now.

[fdastero-blog]

still something wrong

I don't understand what is wrong with diffuse lighting, where half of the sphere (or something like that) is dark when viewed from certain angle. When viewed head on there doesn't seem to be a problem. The problem most likely has to do with either the way I'm render the light volume, or one of my vectors is wrong. I don't see why it would be the vectors because specularity seems to be correct. By mentally judging where specular ought to appear I'm think it is right. For example, when the view is aligned with the half vector that's when specularity should be the strongest, and it appears this is the case.

I also got rid of the problem of seeing the shape of the light volume. Yes, I wasn't modulating the light "power" to take into account the light power of the entire scene. I think I got it now.

[fdastero-blog]

does this look wrong to you?

[fdastero-blog]

deferred lighting in viewspace

Last week I was working on doing deferred lighting. I'm at the point now where I can write the light buffer generating shader. The input into the light buffer shader is the G buffer (mini), the G buffer is generated by outputing the normal R,G,B and linear depth A into a float16 buffer. The format for this buffer is not final since I'll test different formats later on.

Anways, in order to write the light buffer shader, we need to reconstruct a position from the G buffer--meaning we need to figure out which space we're going to do our lighting computation in, viewspace or worldpsace. I've decided to do it in view space for now and still leaving open the possibility of worlspace light computation later on.

There are several ways to reconstruct the position of a pixel (texel?) in viewspace. One can work with the screenspace position and transform it back into view space, by doing some similar triangle thing with the view frustum.

http://jcoluna.wordpress.com/2011/01/27/reconstructing-view-space-position-from-depth/

Or you can do things in viewspace, you can read up on this here:

http://mynameismjp.wordpress.com/2010/09/05/position-from-depth-3/

...

Actually, I didn't spend too much time reading those links. I'm just going to try my own thing here and if it doesn't work then I will read those links. But it should be pretty simple if you work in viewspace (actually, just from looking at the diagram in the 2nd link, this is what I'm doing). For some arbitrary 3d light volume, the idea is to transform vertices into viewspace and pass it to the vertex interpolator. The interpolated positions in the pixel shader are the positions of the light volume in viewspace. Now if we have the far clip plane plus linearized depth of the scene, then to reconstruct the view space position of the G-buffer given the view space position of the light volume, the G buffer position is:

(X', Y', Z * farClip). Where X', Y' are from light volume viewspace position and Z is the linear depth, farClip the farClip distance. Why does it work? Z the linear depth is proportional to far clip. Also because we're trying to compute light parameters of the position corresponding to the depth in G buffer, i.e. we need to "light" the G buffer. Thus there is a correspondence between the light volume and the G buffer, there is a relationship between G buffer and light volume. It could be distance, inclusion, etc, etc.

[fdastero-blog]

light pre-pass practical implementation notes

Looked at how to do this in Ogre some more. 

There were some issues I wasn't sure about like whether the Z buffer were shared between RenderToTextures. 

This thread on Z buffer sharing seems to answer that:

http://www.ogre3d.org/forums/viewtopic.php?f=4&t=53534

So here is the current implementation plan for light pre-pass:

-Setup RTT.

The RTT I will need are the mini "G" buffer. They will be a single RGBA FP16 texture storing both the normal and depth as FP16s. If depth precision is not enough then perhaps try MRT to two textures: RGBFP16 and R_FP32. 

-Deferred lighting stages

In my custom update method which I talked about a few post back. I will need to do a custom call sequence. I think I want to do all the rendering first, followed by update render queue calls. If I do it this way it will speed up the render because of the fire and forget nature of making calls to the GPU. (I forgot which is the order though, I'm sure drawing then calling update render queue is what I want.)

1) Render To Texture the mini "G" buffer.

Set the scheme to MiniG on the viewport of the RTT. (I'm not going to worry about materials that doesn't support this since everything I'm doing is custom).

2) Render the light buffer.

I'm not sure exactly how I want to do this yet. I guess for start I will just use the issue RenderSystem operations to render the light geometry. I can check the deferred shading example for some hints. 

What I really want is use the instancing pipeline for this. My lighting system is custom anyway so this is the most efficient for my case. What I plan is to keep light buffers the same way I have voxel buffers. Then using instancing, I translate each light volume in the vertex shading by using translation and scaling to correspond to the position and shape of the light volume. 

3) Another geometry pass.

Do another pass on the main render window, setting scheme to mainpass, and also sample from light buffer.