normal mapping without precomputed normal
Update 4:
For cubes my method will definitely work. The partial derivative of P with respect to x will always be aligned to T, because we're working with a cube here. So we that can be use for T. THen we can just cross product N and T to produce B.
For the general case we can't assume that the dFx is aligned to T, so my method cannot work. A solution to that can be found in that link I posted earlier to gamedev.net, which comes from ShaderX5, is a the general solution for normal mapping without precomputed normal (Note: it's an optimized solution of computing the inverse matrix to derive T and B).
....
Update3:
actually the per triangle encoding thing won't work. it's really stupid. I was going to store this in the color channel per vertex. That won't work because this vertex could belong to the triangle that faces either direction....
....
Update2:
I'm not so sure about dx being the tangent. dx is change of the interpolate value with respect by to screen x. This may not be aligned with one of the edges? Need to to think about this
Update: On second thought, I will just modify the c++ code to write a code to denote the normal direction and pass that as a per vertex color. I already pass material code in the x component of color per vertex anyway. This way I only do this for cubes. For normal character mesh the T and B vectors are pre-computed per vertex as usual.
....
In my engine for cubes I do away with storing per vertex normal in order to save up on bandwidth and instead compute the normal in a pixel shader using dx,dy. Problem is normally for tangent space based normal mapping one pre-compute the tangent and bitangent vectors per vertex. That's a luxury I don't have.
So how to solve? One way I could do is store a code per triangle because I'm working with cubes and there are only 5 possible normal directions for those. This involves modifying the current SurfaceMesh to take an code per triangle. I figure I could do this by using an vertex index to store the code, thus each triangle consists of 4 indices. When constructing the triangle skip every 4th index. But I'm lazy, I don't want to modify c++ code right now.
So the only option left is to compute the tangent basis per pixel. How to do this? What is a tangent basis matrix anyway? It's a matrix such that Af = { f(e1), f(e2), f(e3)} where f is a linear map which maps the standard basis into some other vector. So f(e3) maps e3 into the normal. So at each pixel we can find dx, dy, of the pixel. Then compute the normal N. Using Gram-Schmidt process we can find the bitangent, assuming that dx is the tangent. I believe we can assume dx is the tangent because it's the partial derivative with respect to screen space x along a interpolated surface (this gives a tangent in view space).
I found a post here also doing this I don't understand how he is computing the tangent basis skimming it. (http://www.gamedev.net/topic/539061-solved-normal-mapping-without-precomputed-tangents/)
Does it work? I think it ought to work because I said a matrix is Af. So if f(e1), f(e2), f(e3) forms an orthonormal basis, then f(e3) maps e3 into the normal, and f(e1), f(e2) maps the respective standard basis into S and T in tangent space.