The effect I'll be describing is our light-emitting logo whose white color chromatically disperses (its composite red, green, and blue wavelengths separate), while sitting above a partially-reflective surface:
If you have a WebGL-enabled, non-mobile browser, you can view the effect live here.
Like many computationally-expensive effects that must be made real-time, this one accomplished primarily by cheating. Rather than constructing a physically-accurate scene where we simulate the bouncing light rays of a blurred reflection, we will instead create a much simpler scene composed of a few effects, the output of which looks close enough to the real thing.
This tutorial assumes some basic knowledge of javascript and WebGL.
We need to know if the client's browser can support WebGL before we can render anything. I use a helper function, refactored and altered from http://get.webgl.org/, though one could just as easily use Modernizr to feature-detect WebGL:
function getWebGL(canvas) { var experimental = false; var gl = null; try { gl = canvas[0].getContext("webgl"); } catch (x) { gl = null; } if (gl == null) { try { gl = canvas[0].getContext("experimental-webgl"); experimental = true; } catch (x) { gl = null; } } return gl; }
This takes a jQuery canvas element which exists on the page as:
<canvas id="logo-canvas" width="256"; height="256"></canvas>
And is fetched from the page with:
var canvas = $("#logo-canvas");
If the client supports WebGL, getWebGL() will return an OpenGL rendering context, otherwise null. This can then be used to determine if we should continue to create a WebGL visualization, or gracefully degrade to a fallback logo.
Shaders compose the primary rendering processes in OpenGL. They come in two flavors: vertex shaders and fragment shaders. Both are required* to create a shader program — a GPU executable that takes vertices, textures, and other data as input, passes it through its vertex shader and then fragment shader, to produce an image as its output.
The vertex shader and fragment shader in a shader program are linked together in a special way such that the output of the vertex shader flows into the fragment shader as interpolated values:
For our uses, we will not be sending rendering-specific vertex data to our shader programs (like 3d models). The bulk of our rendering work will live in the fragment shader half of the shader programs, so we will use a single vertex shader for all of our shader programs. This shader will serve simply to provide us with accurate texture coordinates to each invocation of the fragment shaders:
/* * a single vertex data attribute is sent to us from our javascript code: * the vertex's xy position */ attribute vec2 pos; /* * the texture coordinate sent to our shader program. because `texcoord` will * be generated by the location of the vertex itself, our fragment shader will * be aware of it's own location in texture coordinate space */ varying mediump vec2 texcoord; void main() { gl_Position = vec4(pos, 0, 1); /* * the vertices we're sending in as `pos` are in clip space ([-1, 1], with * the center of the viewport as 0,0), so it's an easy conversion to * texture space [0.0, 1.0] */ texcoord = (pos + vec2(1)) / vec2(2.0); }
The vertex data fed to every shader program is:
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([ -1.0, -1.0, 1.0, -1.0, -1.0, 1.0, -1.0, 1.0, 1.0, -1.0, 1.0, 1.0 ]), gl.STATIC_DRAW);
The idea here is we send 4, 2-component vertices into the vertex shader — one for each xy corner of a quadrilateral in clip space. This vertex shader then turns the position of each vertex into its corresponding texture coordinate (texcoord), which is then fed to the fragment shader. The result is that in the fragment shader, we'll know the exact texture coordinate of the pixel we're operating on.
Our pipeline relies heavily on a mechanism known as render-to-texture. This is a process where the output of one shader program is stored as a temporary image (a texture) to be fed as the input to another shader program. It is analogous to a function which returns an array as the input to another function. These rendering processes can be chained together to produce a post-processing pipeline. The end of the chain is usually displayed to the user. Many modern games use pipelines like this to create advanced effects, like depth of field, SSAO, and cartoon shading.
In Python pseudo-code, here is what we're going to do at a very high level:
logo_tex = open_texture("logo.png") separated_logo_tex = chromatic_separation_shader(logo_tex) blurred_logo_tex = variable_blur_shader(separated_logo_tex) display_on_cavas(blurred_logo_text)
To first know how the effect should look, we can use Blender to produce how the effect would look with physically-accurate reflections. This means:
Design the pre-color-separated logo
Place it above a reflective surface and cause it to emit light
Position the camera to capture the logo and reflective surface
Setting up the Blender scene and materials is outside of the scope of this post, but the end result is a simple scene:
When rendered, the scene produces the following image, which is used as both a fallback logo (in the case of WebGL being unavailable), and as a reference image when creating the shaders. Since this image is physically accurate, we must make our WebGL shaders come as close as possible to replicating it:
Now that we know what the end result should look like, we can start creating it. We start by performing a chromatic dispersion, or color separation. Essentially, the red, green, and blue color channels of a white logo need to be separated by given amount, specified by a value that we pass in.
At first glance, this seems like a simple operation: for each pixel in the input image, use the fragment shader to take its red, green, and blue channels, and write them to other pixel locations, according to how you want the channels separated.
One major gotcha in shader development is thinking of effects as "scatter" operations, or in other words, operations where a pixel is processed, and the result is written to other pixel locations. Blurs are a typical example of a conceptually "scattering" operation: each pixel is distributed to other pixels over a radial distance according to the amount of the blur.
Unfortunately, modern GPU hardware cannot perform scatter operations. For each pixel a fragment shader operates on, the shader must return the new value of that one pixel — it can't write to other pixel positions. This limitation requires that we think of scatter operations instead as "gather" operations. Instead of spreading our pixel onto new pixels, we will say that our new pixel's value will be the result of spreading the surrounding pixels onto it. The end result of a gather is the same as a scatter, so this is what we will do for chromatic dispersion, and later for our blurs.
Fig. 1 shows the color separation as a scatter operation. The highlighted white pixel's color channels are scattered by -2 pixels (negative, to the left). This means the white pixel must add its blue value to the pixel at x-2, its green value to the pixel at x-4, and it's red value to the pixel at x-6.
Fig. 2 shows the color separation as a gather operation. The yellow pixel gathered its color channels at +2 pixels spacing (positive, to the right). This means it gathered its red from the pixel at x+6, its green from the pixel at x+4, and it's blue from the pixel at x+2. Because green and red had values of 1.0, while blue had a value of 0.0, the resulting color is yellow.
Fig. 3 shows the end result of applying the gather operation to every pixel in the input image.
Now that we understand how a scatter-as-gather operation works, we will use it in our chromatic dispersion fragment shader below:
/* * chromatic dispersion fragment shader */ precision mediump float; /* * input from our vertex shader. each pixel that we process with our fragment * shader will know its own x,y location */ varying vec2 texcoord; /* * our logo texture */ uniform sampler2D tex; /* * the size of our logo texture. basic metadata about our texture is unavailble * in a shader unless we pass it in explicitly, like this */ uniform vec2 tex_size; /* * the amount, in pixels, that each color channel should separate from eachother */ uniform float spacing; void main() { /* * here we compute, in texture units, how far `spacing`, in pixels, * actually is. dividing spacing by the width of our logo texture puts us * in the range [0.0, 1.0] */ float separation = spacing / tex_size.x; /* * because our logo texture is shorter than it is wide, and yet both x * and y coordinate systems range from [0.0, 1.0], we compute the ratio so * we know how much to adjust the y-coordinate lookup */ float ratio = tex_size.x / tex_size.y; float offset = -2.0 * tex_size.y / tex_size.x; /* * this is where we're going to sample the logo texture from. notice that * we're adjusting the y-axis. this is because the texture that we're * rendering to is twice as tall as our logo texture */ vec2 sample_coord = vec2(texcoord.x, texcoord.y * ratio + offset); /* * perform our individual color channel lookups, using separation and * a direction multiplier */ float red_sample = texture2D(tex, vec2(sample_coord.x + separation * -1.0, sample_coord.y)).r; float green_sample = texture2D(tex, vec2(sample_coord.x + separation * 0.0, sample_coord.y)).g; float blue_sample = texture2D(tex, vec2(sample_coord.x + separation * 1.0, sample_coord.y)).b; /* * now that we have each color that this pixel should be composed of, we * add them together setting this pixel's color components to the values * we retrieved */ gl_FragColor = vec4(red_sample, green_sample, blue_sample, 1.0); }
Creating the blurred reflection
The blur happens in two passes:
The reason for two passes instead of one is that two passes results in a reduction in the number of required texture samples. Two passes gives us O(2n) time complexity for each pixel, while one pass is O(n^2) complexity. This allows us to do larger blurs for much cheaper, since performing texture lookups is expensive, and we're doing fewer of them.
The first pass will be vertical. This means that for each pixel we're operating on, we will only look up and down to gather the blur samples. The second pass will be horizontal, which means that we'll only look left and right to gather the blur samples.
The blur itself is performed by sampling a texture at specific locations along a gaussian curve, then weighting the sample by an amount according to its position on the curve. So starting from a central pixel (the pixel being gathered), and walking outwards to the left and to the right (in the case of a horizontal blur), sampling your texture as you go, you are accumulating these samples into the final pixel value:
The thickness of the arrows represents the weight of that sample, according to its location on the gaussian curve.
One thing to keep in mind is that that the sum of the weights should always equal 1.0, otherwise the resulting blurred image will be brighter (if the sum is > 1.0) or darker (if the sum is < 1.0) than the original input image.
Below, we've translated this logic into the vertical blur shader. It has some additional logic to mirror the logo across the x-axis at a virtual "ground" line, so that instead of the blur happening in-place, the blurred pixels are drawn below this ground line and flipped vertically.
precision mediump float; varying mediump vec2 texcoord; uniform sampler2D tex; uniform vec2 pixel_size; /* * ground represents the location of the mirroring line, in texture space * (so from 0.0 to 1.0) */ uniform float ground; /* * this is the height of the reflection in texture space. we use this to * determine how much we should be blurring, since we want the reflection to * blur more, the further away it is */ uniform float ref_height; /* * these arrays represent 16 fixed points along a gaussian curve, but picked * in such a way as to minimize the number of texture lookup required by * relying on the GPU hardware's bilinear interpolation */ uniform float weights[16]; uniform float offsets[16]; uniform vec2 sample_axis; /* * a helper function that gives us the texture location by only passing in the */ vec2 sample_coord(float offset) { return sample_axis * offset * pixel_size; } void main() { const float logo_height = 0.12; vec2 adj_texcoord = texcoord; adj_texcoord.y = 1.0 - adj_texcoord.y + ground - logo_height; /* * `gradient` and `inv_gradient` will serve as an amount by which to blur * our current texel. the idea here is that we're blurring more, the * further away from the "ground" we are */ float gradient = mix(0.0, 1.0, ((texcoord.y - (ground - ref_height + logo_height)) / ref_height)); float inv_gradient = max(1.0 - pow(gradient, 2.0), 0.0); vec4 color; // center texel color = texture2D(tex, adj_texcoord) * weights[0]; /* * sample the textures */ int j=1; for (int i=1; i<16; i++) { color += texture2D(tex, adj_texcoord + sample_coord(offsets[i] + float(j)) * inv_gradient) * weights[i]; color += texture2D(tex, adj_texcoord - sample_coord(offsets[i] + float(j)) * inv_gradient) * weights[i]; j += 2; } gl_FragColor += color; gl_FragColor.a = 1.0; }
The second blur pass, the horizontal blur pass, is much the same as the first blur pass, except no mirroring is performed.
Connecting the different stages
To tie it all together, we need to set up the rendering pipeline we described earlier.
Let's take a look at our javascript render function. Each pass is performed by first binding a framebuffer object, defining or attaching the various shader inputs, then executing the shader program by calling drawScreen():
function render() { /* * pass 1, color separation. the width of the separation is controlled * by logoAnimator.value */ gl.bindFramebuffer(gl.FRAMEBUFFER, colorSeparationFBO); gl.useProgram(colorSepProgram); gl.activeTexture(gl.TEXTURE0 + 0); gl.bindTexture(gl.TEXTURE_2D, logo); gl.uniform1i(gl.getUniformLocation(colorSepProgram, "tex"), 0); gl.uniform2fv(gl.getUniformLocation(colorSepProgram, "tex_size"), [1024.0, 512.0]); gl.uniform1f(gl.getUniformLocation(colorSepProgram, "spacing"), -logoAnimator.value); drawScreen(); /* * pass 2, vertical blur, using colorSeparationFBO as our input */ gl.bindFramebuffer(gl.FRAMEBUFFER, blurStage1FBO); gl.useProgram(gaussianBlurProgram); gl.activeTexture(gl.TEXTURE0 + 0); gl.bindTexture(gl.TEXTURE_2D, colorSeparationFBO.colorAttachment); gl.uniform1i(gl.getUniformLocation(gaussianBlurProgram, "tex"), 0); gl.uniform1f(gl.getUniformLocation(gaussianBlurProgram, "ground"), groundLocation); gl.uniform1f(gl.getUniformLocation(gaussianBlurProgram, "ref_height"), reflectionHeight); gl.uniform2fv(gl.getUniformLocation(gaussianBlurProgram, "pixel_size"), [1.0/CANVAS_SIZE[0], 1.0/CANVAS_SIZE[1]]); gl.uniform1fv(gl.getUniformLocation(gaussianBlurProgram, "weights"), [ 0.0335575010461,0.0665417731923,0.0642977802085,0.0604475739708, 0.055289549608,0.0492026511107,0.0426005515661,0.0358858466478, 0.0294111777665,0.0234521609327,0.0181942655819,0.0137330623107, 0.0100851283165,0.00720570737276,0.00500902207902,0.00186499881285 ]); gl.uniform1fv(gl.getUniformLocation(gaussianBlurProgram, "offsets"), [ 0.0,0.497422733884,0.493986615759,0.490551065619,0.487116407772, 0.483682966191,0.480251064389,0.476821025299,0.473393171151, 0.469967823353,0.466545302369,0.463125927601,0.459710017271, 0.456297888301,0.4528898562,0.0 ]); gl.uniform2fv(gl.getUniformLocation(gaussianBlurProgram, "sample_axis"), [0, 1]); drawScreen(); /* * pass 3, horizontal blur, using blurStage1FBO as our input. notice * that we're UNBINDING the current framebuffer, because the output of * this stage is not being saved to a texture, but rather being dumped * to the GL context on the screen. */ gl.bindFramebuffer(gl.FRAMEBUFFER, null); gl.useProgram(gaussianBlurHorizontalProgram); gl.activeTexture(gl.TEXTURE0 + 0); gl.bindTexture(gl.TEXTURE_2D, blurStage1FBO.colorAttachment); gl.uniform1i(gl.getUniformLocation(gaussianBlurHorizontalProgram, "tex"), 0); gl.activeTexture(gl.TEXTURE0 + 1); gl.bindTexture(gl.TEXTURE_2D, colorSeparationFBO.colorAttachment); gl.uniform1i(gl.getUniformLocation(gaussianBlurHorizontalProgram, "logo"), 1); gl.uniform1f(gl.getUniformLocation(gaussianBlurHorizontalProgram, "ground"), groundLocation); gl.uniform1f(gl.getUniformLocation(gaussianBlurHorizontalProgram, "ref_height"), reflectionHeight); gl.uniform1f(gl.getUniformLocation(gaussianBlurHorizontalProgram, "reflection_dim"), reflectionDim); gl.uniform2fv(gl.getUniformLocation(gaussianBlurHorizontalProgram, "pixel_size"), [1.0/CANVAS_SIZE[0], 1.0/CANVAS_SIZE[1]]); gl.uniform1fv(gl.getUniformLocation(gaussianBlurHorizontalProgram, "weights"), [ 0.0335575010461,0.0665417731923,0.0642977802085,0.0604475739708, 0.055289549608,0.0492026511107,0.0426005515661,0.0358858466478, 0.0294111777665,0.0234521609327,0.0181942655819,0.0137330623107, 0.0100851283165,0.00720570737276,0.00500902207902,0.00186499881285 ]); gl.uniform1fv(gl.getUniformLocation(gaussianBlurHorizontalProgram, "offsets"), [ 0.0,0.497422733884,0.493986615759,0.490551065619,0.487116407772, 0.483682966191,0.480251064389,0.476821025299,0.473393171151, 0.469967823353,0.466545302369,0.463125927601,0.459710017271, 0.456297888301,0.4528898562,0.0 ]); gl.uniform2fv(gl.getUniformLocation(gaussianBlurHorizontalProgram, "sample_axis"), [1, 0]); drawScreen(); }
In this way, 3 moderlately-complicated effects are performed as a pipelined operation.
That's it! Some things I did not cover:
Setting up the rendering loop
Smoothly interpolating animation values
Setting up the basic WebGL objects (FBOs, textures, shader programs, etc)
If you're interested in these topics, examine the javascript source.
Your donations directly help us create new innovative performances. We use all donations to help pay our dancers, to rent rehearsal and performance space, to licence media used in our shows, to rent projectors, and to buy new equipment.
If you value the performances we create, if you see potential in our art form, or if you just want to express your appreciation for a recent performance, please use the donate buttons below.
Thank you for helping us create!
Form Constant Dance NFP is a 501(c)(3) non-profit organization. Your donations are tax-deductable. Please keep the PayPal or Coinbase email confirmation of your donation for your records if you wish to deduct your donation.
