KNB 221 CG Toolkit Assignment 2 - Milestone 2A
In order to create a composited shot in Unreal, I first had to download both Unreal Engine 4.17 and the example file for the composure plug-in through the Epic Games marketplace. After following the instructions as per the online documentation, I was able to create the desired effect.
I then began testing the camera tracking effects with After Effect, shooting a test plate. The import process into Unreal proved difficult, as I had to first download a script that allowed me to export the camera data into a Maya file, and export again from Maya as an FBX for Unreal. However, I was then faced with the problem of the camera being out of perspective with my 3D object. At this stage, I decided to simply play it by ear and continually adjust the object until it was in close enough perspective.
Even after all that struggle, I was still faced with the problem of the background footage having distorted colours from the original file, an issue I have yet to solve.
I decided to then tackling the rigging phase. Apply FK rigging to my model proved easy enough, however issues arose once I began attempting to IK rig the multiple-jointed legs, resulting in a lot of breakage.
I put that aside for the moment and focused on blocking animation. I envisioned my little robot scurrying around in the backyard ground, following it with my camera. The ground would shake, and the little robot would get scared and run off. I would then look up to see a giant robot walking towards the house, looming over me as it stares with curiosity. I would use the blue sky at a means of keying in my giant robot behind other houses.
I blocked out the animation as best I could, knowing this would very likely have to be altered once I had camera data to work with. Going back into rigging, I found an outline tutorial on how to rig double jointed limbs using two IK rigs compiled together. The result still had some issues regarding independent ankle rotation and breakage if it was stretched too far, but turned out an overall improved result. I may or may not try to refine my leg rigs depending on how well they can function once I flesh out my animation.
Next up, I sort to shooting my background footage. I tried my best to create markers for where my robot would run and follow them in time, as well as using an onset stand-in for height and shadow reference.
I also photographed my surrounding environment with the hope of being able to create a panoramic reflection reference at some point.
However, difficulties rose again as the footage would not properly camera track. I decided to try again on a different day, with more shadows on the ground to help with tracking (as well as giving me more lighting conditions to experiment with). After Effects was finally able to track the ground footage, but could not keep up with the sky footage due to the leaves blowing in the wind causing too much interference.
After some experimenting, my solution to this issue would be to instead use a still frame and apply an artificial camera shake, before stitching the two videos together. My process was once again halted when I tried to import the camera data into Maya, where due to the warped camera lens and off perspective, I could not properly adjust my blocking to follow the camera.
This wouldn’t matter, as the blocking animation itself refused to import into Unreal.
My solution at this moment is to block the animation once again in Unreal using the camera data, and later reimport the blocking back into Maya to use as reference, hoping this method will work. This again involved me playing perspective by ear, however I applied a plane to help locate the position of the ground.
In After Effect, I attempted to use a hue/saturation effect to fix the colouring, which worked to an extent with fixing reds and blues, but now has issues processing greens.
While I was there, I decided to make an attempt at my giant robot composition, using my original footage and a red dot as reference.
For a first attempt, the result turned out remarkably well, and so I stitched it together along with the ground footage, using a directional blur to mask the transition.
The result is a close approximation to how my final composition will play out. Obviously the transition will need to be made smoother, the camera shake will required more work to look natural, and the colouring will eventually need to be corrected (hopefully within Unreal). But as of now I am incredibly proud of how the composition as turned out so far, in spite of my multiple setbacks.
My next step down the pipeline is to attempt my import method into Maya, and refine my robot animation.
