Lighting Effects at the End of the Tunnel
Previously on Stare Down the Abyss: the hero of the story attempted to implement a pretty basic deferred lighting system using basic pixel shaders and ended up hitting a wall of solid obsidian. I simply couldn't make it work, no matter what I tried.
I figure that if I spend enough time working on something, I can grasp a concept pretty well. It's usually just a matter of time before I get it - but this was a situation that really broke my expectations. After a solid week of trying to get this stuff working and failing; I was pretty much no closer to understanding how it all worked than I was at the start.
I put it mostly down to a single problem: I could not do any debugging
Waiting... waiting... waiting... (forever)
For some context: I was getting some really weird results with my shaders. Textures cycling all over the place, pixels rendering at random positions with random colours; even situations where parameters just weren't resolving in the shader at all. I'm not one to exaggerate, but I'm pretty sure I hit just about every single possible problem ever, and every single tool I tried to use just didn't work at all. Here is a list of everything I tried:
VS2012 Graphics Debugging Tools - stuck waiting for application to draw a DirectX frame (note that when I was preparing images for this blog, this actually worked - albeit on a completely different machine to the one I develop on - what the fuck mate?)
nVidia Nsight - stuck waiting for application to draw a frame. I had a feeling that this was probably because Mono has to compile shaders to the lowest common denominator - dx9 - where Nsight will only recognise dx10/11 code. That's just a feeling, though; I have no evidence to back any of it up
PIX - Abyss executable crashes on load - every time. This was basically what all forums everywhere said to use to debug dx9 shader code. I know there are some issues with running XNA 4 applications in PIX, but I used the workaround and the crashing continued. It was a very frustrating experience.
FX Composer - another nVidia tool that ended up having some stability issues on my computer. Not really sure what is going on there; might have to spend some time rebuilding my machine and try it again.
nVidia Shader Debugger - Had the same issues as PIX; would just crash on startup. At this point I wanted to throw my desktop through a window; so it's probably better that I just moved on. Although... I could do with a new machine...
The final thing I could think of to actually get any sort of help was to fall back on some oldschool techniques:
alert("this is the worst shit");
For those that don't instantly understand that and feel like vomiting: whenever I had some condition that I wanted to check, or some variable I wanted to see, I would basically return a colour value that represented a hit; like a shade of red or green. Much like filling all of your javascript with alerts so you can figure out why your code doesn't work for the millionth time: this is just an awful, inefficient, gross form of debugging - but it was all I had. No wonder it was so difficult to figure this stuff out. Actually, the more I think about it, the better I feel about it - this stuff is really fucking obtuse.
Typically, though - this wasn't the only problem I faced. I mentioned earlier that shader parameters weren't resolving. That was almost as frustrating as the general debugging issues I had, but at least I could figure out how to workaround it.
I'm going to caveat this with: this may not be factually correct, it's more my general understanding, so don't hate if I don't get it right but basically, in game programming, you have a rendering pipeline, where you do a bunch of passes to get a finished scene. With deferred lighting techniques, you would render your basic scene first to a buffer, do a lighting pass and then render the final result as a combination of the two. This is pretty basic stuff, and can be explained elsewhere much more succinctly than I could (my English Lit teacher noted my lack of brevity on... just about every assessment that I ever did).
Anyway, when you use XNA/Monogame, you use a class called RenderTarget2D to do render passes for a particular scene. You can then apply specific effects using shaders and what-not to these objects and create a nice, complex, effect-rich final scene. It's pretty cool how all this stuff works, but there are issues when you try to pass a RenderTarget2D through to a shader as a Texture2D parameter.
The problem here is that RenderTarget2D objects inherit from Texture2D in XNA/Monogame, but when you try to use one as a shader parameter, it just doesn't resolve - for some reason; so you have to do some gross rubbish and take a huge performance hit to get it to work:
You have to pull out the color data (ie: pixels) from the texture buffer, put it in a struct and then use it to set the data of a different texture. You're basically duplicating a texture here, which is weird because a RenderTarget2D IS A TEXTURE. Why this doesn't work is absolutely beyond me.
The weird thing here is that if you use a SpriteBatch (a basic XNA class that batches and renders sprites - super useful) and apply an effect to that, the texture you batch resolves perfectly fine - I suspect this is a weird bug that just never got fixed at some point; that, or I'm doing it wrong. Both are equally likely.
Long story short: I ended up ditching doing shader-based deferred lighting completely and opted instead for the tried-and-true method of using a semi-transparent light texture and made it additively blend to the scene. The result is actually really nice, and it took me about an hour to do after I finally decided that I'd had enough of feeling stupid.
To finish this wall of text off, I should probably say what lessons I learned out of all of this stuff. I don't think I learned anything new, more that it reaffirmed that pragmatism should always win out in these situations. Just get things done and worry about how they look/perform later - it's really the best advice I could ever give anyone. Finishing is always better than spinning your wheels.
theartofmadeline
snowghoul
baratheonpilled
pixelskylines