Animation: 3D Toolkit blog - John Everitt

[3dtoolkitje]

KNB 221 CG Toolkit Assignment 2 - VFX Composition

Final Composition:

Effects Breakdown:

One of my biggest hurdles throughout this project was working and animating within a slanted perspective in both Unreal and Maya, due to the imported camera track data .I was presented a solution to this issue, however at that stage it had become too time consuming to correct after all the work that had been done. Had I spent the time setting up the camera in Unreal correctly, this would have made for a far less problematic and frustrating animation process.

Setting up camera tracking markers before shooting my footage would have helped greatly with analysing the data in After Effects, especially for the pan up shot. I also realise now that while effective enough for my project, using the sky as a blue screen runs into many problems down the line with keying.

Making sure my character rigs would import correctly into Unreal before going into detail with my animation would have saved from a lot of revision work and repeated tasks. Unwrapping the robot UVs before skin weighting would have also made applying materials a far easier and tidier process.

Lighting is something I wish I had taken more time to truly figure out when compositing a character moving through different lighting conditions. I made the mistake of forgetting to properly bake my lighting until the final week, bringing up its own set of issues. I was adamant about creating my own cube texture for reflective surfaces, and even started by creating an (albeit rough) panoramic view of my environment.

However, creating a cubemap in Unreal turned out a far more complicated process than I realised and I decided to scrap the idea, using one of the built-in maps instead (just as effective in the end). While not essential, I do wish I made the effort to put this asset together far earlier in the pipeline.

[3dtoolkitje]

KNB222 Virtual Environments Progress 4

From Thursday Week 10 to Thursday Week 11

Textures:

Left Building 1

Left Building 3 (main structure and windows)

Right Building 1 (all levels)

Roof lamps and Lamp bulbs

Ground/Gutter (retextured)

Left Building 2 (retextured)

Progress Sheet:

https://docs.google.com/spreadsheets/d/1qQm7uybqzzA7rCj2ENTONFeMYw2nriXJHOFqqvjLQ2c/edit#gid=1472238718

[3dtoolkitje]

KNB 221 CG Toolkit Assignment 2 - Progress Update 3

Covering what I have done over the past two weeks (I missed last week’s submission), I began with fleshing out my robot animation. My method to bring the blocking animation from Unreal into Maya managed to work, and I was able to fix the camera in Maya by adjusting horizontal film aperture, giving me an eyeline to work with. Animating on a slanted view did prove difficult, especially with the leg controls. When it came to the robot’s run, I decided instead for a small bird-like hopping motion, which turned out a lot smoother

However, troubles rose once again when the rig failed to import into Unreal. This turned out to be the result of both incorrect joint setup, and an unskinweighted mesh, leading to me having to fix my rig and animating the character all over again. I tried to use the original animation as much as possible for reference, and in the new setup I detached the feet controls from the main character root, allowing me to perform a much more natural run than I could with the previous setup.

But once again, the rig fails to import into unreal, and after going down the line attempting to fix all the errors, this is unfortunately still the case. I instead decided to focus on unwrapping my UVs and texturing the robot, however the UV map appears to distort when scrubbing along the timeline. This may be an issue involving keyframed UVs that I hope to be able to fix later.

Experimenting with lighting in Unreal proved its own set of problem, particularly with working on that slanted view. After some time, I decided to see if importing the camera through a skeletal mesh would make the process easier, allowing me to move the camera path in Unreal, along with the ground level.

However, the process of figuring out perspective through this camera turned out to be an even greater hassle. I finally decided to go back to my original method, and after playing around with it a little longer, I was able to find a way to cheat the shadows without being 100% accurate.

I am now hoping to find a method in order to project shadows onto a plane while making the plane invisible.

As far as my week to week progress, I find I’m spending more time on the animation/composition process than I had hoped, putting me behind on both materials, and completing the final Large Robot shot.

My schedule for the next few weeks is as follows:

 Week 11: Refine lighting, Create materials, Import Small Robot animation into Unreal, Refine Large Robot animation and import into Unreal.

Week 12: Refine AfterEffects composition for final shot, investigate and apply particle effects in Unreal (fire and smoke)

Week 13: Refine elements and composition, sound mix (time permitting)

My critical tasks over the next few week, if short on time, will be importing animation, correcting lighting, and fixing the AfterEffects composition.

[3dtoolkitje]

KNB222 Virtual Environments Progress 4

Thursday Week 9 to Thursday Week 10

Textures (all self-modeled assets and some of Garry’s):

Right Buildings 2 (plus pipes and windows)

Right Building 3 (plus structures)

Left Building 2 (plus windows and doors)

Ground (plus guard rails, bridge, garbage bags, garbage cans, large pipes, stairs)

Aircon (R1)

Sunshades (plus support columns)

Satellite

Video (does not include some textures above:

Progress Sheet:

https://docs.google.com/spreadsheets/d/1qQm7uybqzzA7rCj2ENTONFeMYw2nriXJHOFqqvjLQ2c/edit#gid=1472238718

[3dtoolkitje]

KNB 221 CG Toolkit Assignment 2 - Milestone 2A

In order to create a composited shot in Unreal, I first had to download both Unreal Engine 4.17 and the example file for the composure plug-in through the Epic Games marketplace. After following the instructions as per the online documentation, I was able to create the desired effect.

I then began testing the camera tracking effects with After Effect, shooting a test plate. The import process into Unreal proved difficult, as I had to first download a script that allowed me to export the camera data into a Maya file, and export again from Maya as an FBX for Unreal. However, I was then faced with the problem of the camera being out of perspective with my 3D object. At this stage, I decided to simply play it by ear and continually adjust the object until it was in close enough perspective.

Even after all that struggle, I was still faced with the problem of the background footage having distorted colours from the original file, an issue I have yet to solve.

I decided to then tackling the rigging phase. Apply FK rigging to my model proved easy enough, however issues arose once I began attempting to IK rig the multiple-jointed legs, resulting in a lot of breakage.

I put that aside for the moment and focused on blocking animation. I envisioned my little robot scurrying around in the backyard ground, following it with my camera. The ground would shake, and the little robot would get scared and run off. I would then look up to see a giant robot walking towards the house, looming over me as it stares with curiosity. I would use the blue sky at a means of keying in my giant robot behind other houses.

I blocked out the animation as best I could, knowing this would very likely have to be altered once I had camera data to work with. Going back into rigging, I found an outline tutorial on how to rig double jointed limbs using two IK rigs compiled together. The result still had some issues regarding independent ankle rotation and breakage if it was stretched too far, but turned out an overall improved result. I may or may not try to refine my leg rigs depending on how well they can function once I flesh out my animation.

Next up, I sort to shooting my background footage. I tried my best to create markers for where my robot would run and follow them in time, as well as using an onset stand-in for height and shadow reference.

I also photographed my surrounding environment with the hope of being able to create a panoramic reflection reference at some point.

However, difficulties rose again as the footage would not properly camera track. I decided to try again on a different day, with more shadows on the ground to help with tracking (as well as giving me more lighting conditions to experiment with). After Effects was finally able to track the ground footage, but could not keep up with the sky footage due to the leaves blowing in the wind causing too much interference.

After some experimenting, my solution to this issue would be to instead use a still frame and apply an artificial camera shake, before stitching the two videos together. My process was once again halted when I tried to import the camera data into Maya, where due to the warped camera lens and off perspective, I could not properly adjust my blocking to follow the camera.

This wouldn’t matter, as the blocking animation itself refused to import into Unreal.

My solution at this moment is to block the animation once again in Unreal using the camera data, and later reimport the blocking back into Maya to use as reference, hoping this method will work. This again involved me playing perspective by ear, however I applied a plane to help locate the position of the ground.

In After Effect, I attempted to use a hue/saturation effect to fix the colouring, which worked to an extent with fixing reds and blues, but now has issues processing greens.

While I was there, I decided to make an attempt at my giant robot composition, using my original footage and a red dot as reference.

For a first attempt, the result turned out remarkably well, and so I stitched it together along with the ground footage, using a directional blur to mask the transition.

The result is a close approximation to how my final composition will play out. Obviously the transition will need to be made smoother, the camera shake will required more work to look natural, and the colouring will eventually need to be corrected (hopefully within Unreal). But as of now I am incredibly proud of how the composition as turned out so far, in spite of my multiple setbacks.

My next step down the pipeline is to attempt my import method into Maya, and refine my robot animation.

[3dtoolkitje]

KNB 222 Virtual Environments - Progress 3

From Thursday Week 8 to Thursday Week 9:

2nd Right Building (pipe 3) 

https://sketchfab.com/models/7b57bbc999f8423d8a78233986e8eadd

2nd Right Building (Top Level) 

https://sketchfab.com/models/ca83d6e1d176454984ebdc702520ef04

Garbage Bags 

https://sketchfab.com/models/3afbea6bc4d948fe907463d0b88acf73

Ground Pipe 

https://sketchfab.com/models/1f6ae01870c84bb88b09e9e372240594

Satellite 

https://sketchfab.com/models/85832f8c7dc045c584d0ee037d89fbfa

Ground Guard

https://sketchfab.com/models/7b24116b2a294334ac4102f0ed412b74

Garbage Can

https://sketchfab.com/models/2268e8bbd0154c70953ce03696f50660

Ground Bridge 

https://sketchfab.com/models/152f3f26acda4dc0bc91d0bc7f9d5407

Stair Guard

https://sketchfab.com/models/4b4be56f84884888b71ead645977aad9

‘Forest of the Flies’ (updated greybox Week 9, includes Thomas Stig and Garry Wu’s models)

https://sketchfab.com/models/d9410416cd6041df988b3bf1979db95b

Progress Sheet:

https://docs.google.com/spreadsheets/d/1qQm7uybqzzA7rCj2ENTONFeMYw2nriXJHOFqqvjLQ2c/edit#gid=1472238718

[3dtoolkitje]

KNB 222 Virtual Environments - Progress 2

From Thursday Week 7 to Thursday Week 8:

2nd Left Building (main)

https://sketchfab.com/models/8eae6029a4084845a3f1e8ace74ed6f7

2nd Left Building (angle)

https://sketchfab.com/models/e57c81affbd341c99ff023527c413160

2nd Right Building (main)

https://sketchfab.com/models/bab8f1b0772c416ca572e02f523a6537

2nd Right Building (pipe 1)

https://sketchfab.com/models/665907befaea477892d0b813035f013e

2nd Right Building (pipe 2)

https://sketchfab.com/models/27ae33222b8e4fe2b65412fa2ec33442

'Forest of the Flies' (updated greybox Week 8, includes Thomas Stig’s models)

https://sketchfab.com/models/a114453af23c4f54b51cbcae660b687c

Progress Sheet:

https://docs.google.com/spreadsheets/d/1qQm7uybqzzA7rCj2ENTONFeMYw2nriXJHOFqqvjLQ2c/edit#gid=1472238718

[3dtoolkitje]

KNB 222 Virtual Environments - Progress 1

Completed Ground/Gutter Mesh:

https://sketchfab.com/models/3d7783f723d147b6adf07837d19f8c8c

UV issues:

Started Stair Mesh:

Environment Progress (week 7)

https://sketchfab.com/models/106bfed26ae04dca8632064c83888f20

Group Spreadsheet (includes week 8 sprint plan):

https://docs.google.com/spreadsheets/d/1qQm7uybqzzA7rCj2ENTONFeMYw2nriXJHOFqqvjLQ2c/edit#gid=1472238718

[3dtoolkitje]

KNB 221 CG Toolkit Assignment 2 - Progress Update 1

I have officially chosen the Visual Effects stream, with the brief of compositing a rendered element from Unreal with live action footage. In specifics, I will model, rig and animate a virtual character in Maya, based on the provided concept art below.

I will then composite this animation, through Unreal, onto a live-action plate filmed by myself, complete with lighting and additional cinematic post-production effects (i.e. colour correction, lens effects).

Examples:

While composition effects will be by primary focus, I also wish to put some time into rigging and animating my character. Because of this, the level of detail in my character model (mesh and materials) may end up comparatively lacking, depending on the amount of time I have to refine these elements after completing effects work.

 At my current stage, I have completed a basic model for my robot character to test composition effects and later begin rigging.

I have also shot test footage to experiment with the camera tracking functionality in After Effects, which I am currently attempting to apply in Unreal.

Pipeline Schedule:

Week 7: Test model with test footage and basic lighting in Unreal, Begin rigging character.

Week 8:  Complete rigging character, Animate Model (blocking).

Week 9: Shoot final footage, Basic composition w/ basic lighting in Unreal.

Semester Break: Refine Animation, Refine composition.

Week 10:  Create Materials.

Week 11: Refine Lighting, Investigate Post-Production Effects.

Week 12: Apply Post-Production Effects, Refine Elements.

Week 13: Refine Elements.

[3dtoolkitje]

KNB 221 CG Toolkit Assignment 1 - Final Submission

Wrap-up Reflection:

The past few weeks of activities have helped me greatly to understand areas of the animation pipeline I had previously remained distant from. This mainly included cinematics and rendering in Unreal, and its use for composing elements in After Effects I feel will become a huge benefit for me further along in this course.

The post-production volume in Unreal was also an incredibly handy tool I was glad to discover, and I wish I had more time to experiment with the various functions and settings. Working with blueprints is another area I find myself less intimidated by, though the full scale of their use and functionality may still take some time to become properly familiar with.

While I still find character animation to be my strong suite, blending said animations in Unreal still proves to be a complicated process for me, though I hope to become more familiar with it in the future. Using the A.R.T. Character Creation Tool in Maya has made me more keenly interested in the rigging process than I had been before, however I am uncertain what time I’ll have within this unit to go more in depth into that area. Lighting in Unreal is also another aspect, while I definitely have a broader knowledge of now, I feel I still need a deeper and more confident handle on.

Assignment 2 stream:

For the upcoming assessment, I have chosen to take on the Visual Effects Development stream. This decision was difficult, as character animation I have found to be my area of strength and confidence, as well as the role I want to take on within 3rd year projects. However, while I have already spent quite some time with 3D Animation, visual effects work is another aspect that I have always been highly curious about, yet have never had the opportunity to fully experience or experiment with.

That being said, while I aim to make visual effects my number 1 focus during Assignment 2, I wish to then still make 3D animation my second-most priority. My chosen task for this pathway will be to composite a shot from Unreal Engine with live-action footage. This aspect of visual effects work has always been an interest of mine, and after experimenting with similar compositional effects within the first milestone, I want to look into expanding my skills and knowledge in this field and perhaps even consider incorporating it into future projects.

My prior experience and knowledge working in Film and TV related units should prove helpful going forward with this task, as well as improving skills in areas I felt weak at or dismissive of (i.e. onset lighting, cinematography).

Inspirational Artist/Studio:

My main source of inspiration is filmmaker Neill Blomkamp, along with his production house Oats Studios. His unique approach to compositing visual effects into documentary-esque, cinema verite-style short films within his earlier carrier lead him to direct his feature film debut District 9, which has since become a major influence and inspiration for myself as an aspiring filmmaker and animator. Blomkamp often takes advantage of limited resources and non-cinematic footage/techniques to believably composite fantastical elements into a grounded environment.

His recent venture in partnership with Steam is his own in-house production company Oats Studios, dedicated to releasing short film online, while sharing animation assets with the audience, employing an open-source, collaborative, viewer/filmmaker production system.

https://oatsstudios.com/studio/

Job Listing Examples:

http://www.ilm.com/ilm-careers/

http://www.cuttingedge.com.au/jobs/

Looking into some career requirements via studio job listings, I see that an understanding of simulation effects plays an important role in working within the VFX field. This includes particle and fluid mechanics, used to visualise and simulate effects such as fire, smoke, water, dust, clouds, explosions, or other virtual elements that require chaotic (randomised) systems.

An understanding of lighting and its impact on both real and virtual materials (i.e. fur, metal) play an essential part when applying compositional effects in order to believably integrate two or more separate components into the same environment. A comprehensive understanding of chroma keying and blue-screen extractions is one listed skill I have already begun to explore in After Effects, while skills such as camera tracking play an obvious role when compositing onto a moving canvas.

Starting Resources:

An understanding of the compositional software NUKE is also a common listed requirement found on both ILM’s and Cutting Egde’s job listings, and this will be one of the first areas I will explore in preparation for the upcoming task.

https://www.thefoundry.co.uk/products/non-commercial/nuke-non-commercial/

I have also found a tutorial video detailing the camera tracking functions available in After Effects in order to produce a 3D virtual camera from a live action reference.

Another resource I found details the overall production pipeline to creating believably integrated 3D effects, however it is the areas on capturing light and environment referencing I feel are most useful to me here, as well as how camera lens and focus will affect the shot.

http://www.creativebloq.com/3d/how-seamlessly-integrate-cg-and-live-action-51411697

That last aspect may prove difficult as I am not confident in the quality of my current camera, and may need to look into borrowing or hiring out equipment more appropriate for professional production.

[3dtoolkitje]

KNB 221 CG Toolkit Assignment 1 - Milestone 3

Basic Lighting and Bringing it Together

I wanted to take this opportunity to go back and experiment with the A.R.T. character creation tools. In my initial attempt at altering the mesh, I made the mistake of reshaping the geometry without moving the joints; the result below.

I took note of this, and after a difficult process, I managed to create a functioning rig of a kind of spider/monster-like, lanky humanoid creature.

I spent some time trying to perfect my new animation; I wanted to have the creature creep towards the screen, stepping over a title effect that would be added in After Effects. When animating walks, I’m used to first creating a stationary walk cycle, which makes blocking easier but causes problems when attempting to then have the character walk over a distance. This time, rather than moving the rig from the root, I moved it from the hip while adjusting the feet, which turned out to be far more time consuming but resulted in a far cleaner walk.

Importing the animation into Unreal, I wanted the cinematic to reflect the unsettling, horror-style tone of the animation, projecting dark, hellish colours from low angle lights. My recent experience in Virtual Environments gave me a welcome head start in using Unreal’s light controls and functionality. To add to the tone, I also applied a last minute dolly forward and tilt to the camera before exporting the final image and depth map renders.

Final Image render (lighting 1):

Depth Map:

My previous handling of depth maps in After Effects made adding text a far easier process. I did however find that my process for creating the effect had been far more complicated than needed, having initially applied the luma matte to the final image render rather than the text, which forced me to add a second final image render on the bottom layer to cover the blank background.

Old Layering:

New Layering:

Applying the text proved difficult after the camera movement was added, at first attempting to track the image by hand. I instead when back into Unreal and rendered out a reference version, including a blank plane used to track the size and rotation of my text.

While this helped greatly, the result still comes off a little jumpy, and I feel I need to look into more precise tracking methods/effects.

Lighting 1 (with text):

For my second lighting setup, I wanted to apply a scary, flickering light. I initially found a method to create such an effect in the material nodes window. However, while the effect was working in said window, for some reason it would not apply in the viewport.

After searching for alternate methods, I found one using Graph Editor within a blueprint containing a spotlight, and after activating the Simulate function, the light effect did finally apply to the scene.

Lighting 2 (with text):

I find myself a lot more confident now in using A.R.T. to create new meshes for the rig, as well as curious about the rig creation process in general.  The task has also helped me greatly in become more familiar with the lighting process in Unreal, especially involving more sophisticated effects. If I had time, I would have also seen how I could apply the lighting conditions from Unreal onto the text in After Effects (i.e. visibility and shadows).

 Character creation – 2 hours

Animation – 3 hours

Unreal lighting – 2.5 hours

After Effects comp – 1.5 hours

Reflection – 2 hours

[3dtoolkitje]

KNB 221 CG Toolkit Assignment 1 - Milestone 2

Character Implementation and Basic Animation

I began by messing around with the body proportion controls on the A.R.T. model. I was hoping to create a more extreme, possibly non-human design, but I found the slider controls to be somewhat limited and I didn’t want to risk breaking the skeleton through other means. I settled for a slender female model as seen below.

I consider 3d animation a strong suite of mine, so I decided to have a bit more fun with my idle animation, giving her a highly energetic wave. I playblasted the blocking and early animation passes as seen below. (Side note: the hand waving gesture was created with IK controls, and I consider now FK would have come across a lot smoother.)

After some difficultly attempting to use the A.R.T. window, I finally managed to export the animation and bring it into Unreal.

I noticed when I did, a weight control was automatically added, which created a hassle as it kept dragging down my animation. It is an interesting tool however, and I wish to find a constructive way to use it later on.

Another tool I found was the play rate function, which gave me the idea to go back to maya and export individual sections of the animation; her noticing, her pause, and finally her wave. In Unreal, I was then able to stretch or compress the pause section to be as long as I wanted (rendering any play rate below 0.7 must be compressed or Unreal crashes on my laptop).

I then chose to experiment with animation blending, seeing if I could seamlessly blend the ‘notice’ and ‘wave’ portions using Unreal. This turned out to be an extremely difficult process, attempting to use the AnimMontage tool with no success, and spending quite some time trying to figure out Blend Space.

Eventually, I managed to create pose assets for the end and beginning of the two animations, and after a process that was very likely far more difficult and time consuming than it needed to be, I was able to create an animation sequence that bridged the two poses using the curve controls.

But just when I thought I finally worked it all out, of course, I encountered another setback where the track would appear motionless in Sequencer. My last minute solution for this was to record the blending animation in the Animation Sequence window, which for some reason did work in Sequencer, and cut it down to fit with the other two sections.

Attempting to find an easier method for this process is obviously a goal of mine in the future, and well as having a better understand of the Blend Space tool. Otherwise, the final result did turn out very well and I hope to find a genuine use for it in later projects.

Setup and Modeling: 1 hour

Animation: 2 hours

Importing/Exporting videos from Unreal: 1 hour

Blending Animation: 4 hours

Reflection: 2 hours

[3dtoolkitje]

KNB 221 CG Toolkit Assignment 1 - Milestone 1

Cinematics and Rendering

My first goal in this process was to animate a moving actor in Unreal. However, I discovered the BSP geometry used to build the greybox environment were not visible in the camera viewport once animated. 

Regular shapes did show up fine, so I went ahead a made a simple bouncing ball animation. I found the transform tools in Unreal easy enough to use, however when an object is rotated, the axis for the move and size controls change position as well, making animation with multiple controls a hassle.

I added a post-production volume, stretching out over the camera’s dolly path, and played around with a few settings (bloom, vignette, fade and lens flare effects). But when it came to exporting, I ran into the problem of the video not rendering from my selected camera, the result below.

After some messing around with the export options, I eventually found that a Camera Cut track had to be added with the corrected camera applied, which finally resulted in my desired output.

But as evident from above, the surface of the sphere failed to register for reasons I was not able to solve. After experimenting with various materials, I eventually had to resort to applying an emissive colour to the ball in order for it to be visible.

Final Image render:

Depth Map render:

Along with the final image render, I also outputted a depth map to use in After Effects. I wanted to experiment with keying effects, using a depth mask to insert the footage into the scene. The first attempt at filming myself was plagued with bad lighting conditions, however it did give me the opportunity to experiment with After Effects’ keylight function, applying that knowledge and settings to the improved footage.

Attempting to create a depth mask proved to be a difficult process, but after some assistance and tinkering with the white-black level controls, I was able to place my footage in the position I wanted. This did however require me to first take a still frame of my depth map into Photoshop in order to register the background as white.

Some of the cutting in the image above is a bit obvious, and unfortunately becomes somewhat noticeable  when the keyed footage is applied. A solution to this if I was to attempt a similar effect again would be to place a plane shape in front of the background when exporting the depth map, blocking the swirled image.

Otherwise, the final output turned out remarkably well, and I would enjoy the chance to refine my skills with these effects in the future.

After Effects Comp: