"The Dead End" Interactive Sonic Environment in FMOD Sandbox using FMOD Designer- BreakDown
Project “The Dead End” was my Sonic Structures course project for MSc Sound Design programme (2009-2010) at the University of Edinburgh. It is based on a short fictional story called The Sound Sweep written by J.G.Ballard, a british author well known for his weird/science fiction novels and short stories. The objective of the project was to take an extract from The Sound Sweep and sonically recreate the chosen scene/environment described in the story using FMOD Sound System. The brief for the project was provided by Dr. Martin Parker (Programme Director, MSc Sound Design at the University of Edinburgh) to the whole cohort pursuing the masters in sound design programme. After reading the story, which was open ended, I felt the need of coming to a conclusion with my own interpretation of the ending for the story and recreate that extended end as a dark, nightmarish and melancholic sonic environment with linear passage in a non-linear set-up. FMOD Designer and FMOD Sandbox were used to create the interactive sonic environment.
Read more on the BreakDown and detailed explanation.
The Dead End (Extended end of The Sound Sweep) - Hrishikesh Dani
The doom like howling of the wind was sweeping over the dunes. Mangon drove his sound truck deliriously towards the stockade. Infuriated with a mixture of guilt and anger he bangs into the rusty gateway. The stockade was bustling with the pandemonium of the dead remains of the sounds which were dumped into it. The uproar of trains, the clamor of cats and dogs, the cacophony of sonic music which Mangon grew up on, the tumult of city traffic, the crying of choking baby, the harsh words of Madame Gioconda and LeGrande which were left on the walls of the studio, the agitation of electric static of transformers, all these sounds sprung out of their graves to haunt Mangon, as if they plotted a conspiracy among themselves.
Mangon drove his sound truck past the disturbing commotion of sound ghouls and found himself near the shore. A dispassionate female voice announced you have reached the dead end at the point where the stockade ended. Mangon leapt out of his sound truck. He felt a strange sense of fear. He started running around the stockade. He ran past through the chaos of sounds that reflected his past and the present. He tried to run away from each of those ominous sounds but kept returning back to the dead end. Panic attack struck into his juvenile mind and he started to run like a crazy child. His panting grew louder as the cacophony started to fade out behind him. He looked back at the stockade and felt despair and lonely, he felt that life has been harsh on him and there was no point pushing the rock uphill like Sysphus. He turned around with his back to the stockade. A gunshot sound joined the haunting sounds of the stockade as Mangon’s body hit the ground. The doom like howling of the wind carried on sweeping over the dunes.
How “The Dead End” was reached?
To be honest, it was more of venturing into the unknown and experimenting unusual tricks than knowing what can be done with the usual process. FMOD module was introduced in Sonic Structures course around 3-4 weeks before the semester end in December 2009. This left the whole batch with hardly any time to understand the enigma called FMOD. With absolutely zero information available on the internet regarding how to get your head working around with FMOD Designer and FMOD Sandbox, it was difficult to even get started with the project. Thanks to Nick Koumentakis (part-time lecturer) for some basic tutorials on how to get started designing sounds in FMOD Designer and how to place a sound emitter in FMOD sandbox’s virtual space (the tutorial which I missed :(, because of a workshop I was part of), we all were at level 1 of understanding how things can be achieved as far as the project submission was concerned. For me, I was still completely clueless regarding FMOD sandbox, thanks to one of my colleagues, he explained me how to put the sound emitter into FMOD Sandbox, just 5 days before submission deadline.
But more I started getting my head working around with FMOD, more I felt at home. Tackling with its erratic, sudden crash behavior was quite annoying to all of us, but I was smitten by the scope of interactivity and variations that could be achieved using FMOD, a very powerful interactive tool indeed. And that’s when I decided to push the envelope to the point I would go insane with it. Just a night before the submission date, everything fell in its right place. I hope the following video helps you understand, things that can be achieved using FMOD Sound System and it will be interesting to see how people will use the whole bunch of various audio interactive/game tools to create a powerful game and interactive audio madness.
Detailed Explanation of the Sound Events and Sound Definitions in FMOD Designer
Each unique sound emitter inside FMOD Sandbox has a corresponding audio event in the FMOD Designer project. Real world sounds can be modeled by creating audio events which control the in-game behavior of the sounds. The sound files are put into sound definitions which act as the containers for real sounds. These sound definitions can be placed inside the event and the behavior of these sound definitions can be manipulated using various parameters, DSP effects and randomized behavior. FMODTV YouTube channel has recently put up some excellent tutorial videos for the beginners to get started with FMOD Sound System. I suggest you go through those videos for better understanding on how to get started.
The sound events used in “The Dead End” project are explained in detail below. Also, there is a video capture of these events and the sound definitions along with the property settings for each audio event as well as the sound definition. Please note that the events described here, were designed specifically for creating an interactive sonic environment or an audio only “level” in FMOD Sandbox. It should not be considered as a standard approach towards designing the game audio events in general. But nevertheless, I hope it will help you understand how FMOD can be a powerful audio interactive tool. For any queries about the project, contact me on hrishikesh_dani[at]yahoo[dot]com
EVENTS
Tracker : I used trackers to create a line map inside the Sandbox. With the use of proper placement and setting up their trajectory paths, I was able to have a road map of the audio level I was intending to create. This served as a backbone to create the whole sonic space and also a reference for the positioning of other sound events/emitters. Tracker is essentially an empty event or a dead emitter with no sound definition or behavior attached to it.
Footsteps and Panting : I clubbed footsteps and panting as a single event. Footsteps event was one tricky event which took some time for me to figure out. After trying several methods and failing again and again, I reached this unconventional approach. The primary parameter for this event was Distance parameter, so this event was essentially a distance –based… sounds insensible for a First person perspective sound but it worked. Here each footstep is one sound definition for a fixed range of distance from the emitter position. So I had to place 80 sound definitions in one layer… yes 80 and the version which I was using that time (December 2009) did not have Undo…FML :-S … so had to be very cautious. But I was extremely happy the trick worked inside the Sandbox and it completely modeled real world footsteps without having the sound attached to any object as it would be done in an ideal game footstep audio implementation. I layered panting sounds along with the footsteps at specific distances. The panting sound was designed to create uneasiness and tension, in order to achieve that I used reverse processing on some breath sounds and created a set of 7 samples of panting inside a single sound definition which were volume, pitch and sample randomized to create variations.
The Dead End sonic environment is more of a nightmarish unreal world set up where the sounds are ghosts and do not have any physical source, so some sounds are unreal sounds (in the short story, these are residues of sounds dumped into a wasteland sonic space where sounds linger around in the human audible frequency spectrum… please read The Sound Sweep to get the clearer picture). To create this sense, some sounds are designed in such a way that they create a sense of hearing sounds coming from an unsteady source which could be in a state of suspension or ethereal. In order to get the unreal volume attenuation as the character moves away from the sound emitter, occlusion DSP effect was extensively used in most of the audio events.
Voices : The real voices were recorded in the high ceiling staircase of my school building. It had this natural eerie reverberation which suited the dead end environment. Luckily two of my colleagues were around in the lab that late night and I was able to record their voices. The other voice is a computer generated female voice announcing the trespassers that they have reached the dead end. The audio events for voices are simple events.
Tuned Percussion: Tuned percussion is my favourite and first event which I created in FMOD. It consists of three sounds which I recorded from my metallophone (used in gamelan music) which I got from India. This audio event showcases the power of pitch randomization property for sound definitions in FMOD. The event is looped and generates a serene composition which can be considered quite spiritual/meditational because of the metallophone sound. It helped in creating a good contrast to the other tensed algorithmic midi music in the environment.
Algorithmic Midi Music : This event could have been much better with the use of clever interactive music trick but at that time I guess the idea did not click to me. I composed this algorithmic midi piece using Glitch Sequencer which is a free cellular automaton based midi sequencer. It is a fairly simple event.
Wind : Wind event is the one of the few real world sound events in The Dead End that sets the eerie atmosphere in the whole environment. It is dynamic in its position as it goes around the sonic space in a circular trajectory giving a sense of real eerie deserted place. The event uses real-time low pass filter automation on the sound definition. The sound is looped and is edited meticulously to create the seamless loop.
Sea : Sea sound event is another real world sound event which is positioned at the dead end point in the sonic space. The event consists of two layers of sound definitions which create a more dynamic sea sound by overlapping each other at random times.
Truck Ignition/Run : This is a head-relative (similar to first person perspective) event which consists of two sound definitions out of which one is a start or ignition sound and the other is truck engine in motion sound. The ignition sound is oneshot sound whereas the truck run sound is a seamless loop. When the sound emitter is triggered in Sandbox, the truck ignition happens only once and after that the truck run goes on until the emitter is stopped. This cheap trick was used because it is practically impossible to have a dynamic engine audio event inside the Sandbox and for the story, the sound truck sound was essential. The trick served its purpose but it is definitely not how it is done in a real game audio implementation :P
Vehicles : The vehicles audio event consists of several traffic sounds of Bombay (now Mumbai), I had to use these as honking is such a rare form of communication in Edinburgh :P, I couldn’t get any useful traffic sounds here. This event could have been much better if I knew how to create Doppler effect. But dynamic positioning and trajectory of multiple emitter of this event created a chaotic and ghostly traffic pandemonium.
Trains : In the short story, aviation/plane sounds are mentioned. Ideally I wanted those sounds but could not record any sounds at the Edinburgh Airport, so had to change it into train/traction sounds which surprisingly did good to the sonic environment. Again some sounds are of trains from Bombay… how I love chaos sounds of Bombay :).
Electric Static : I tried two different approaches for electric static events. One event with looped cycle parameter consisted of two layers with one shot sound definitions. The electric static was sparsely crackled in these sounds. The other event was single layers with dense static and looped sound definitions, this event was very dense and breaking in nature which enhanced the chaotic nature of the event. Both these events were used in multiple emitters around a turning point of the road map, it helped in creating a sense of urgency and tension while turning and navigating on the “road”.
Animals : Cats and Dogs audio events were created using multiple layer and multi-cycle loop parameters approach. These are unreal sounds and contribute more towards the nightmarish nature of the environment and blend along with other sounds to create an unstable and tensed atmosphere.
Please check out the video grab of the placement of the sound emitters inside sonic space of the Sandbox. Also check out the Property Sheet of each emitter on the right hand side to get details about the positions and trajectory.
That’s all about The Dead End Interactive Sonic Environment in FMOD project. I hope it helps you understand FMOD better and encourages you to freak out with your ideas and the power of audio interactivity.
PS : A big thanks to Damian “lostchocolatelab” Kastbauer and EngineAudio.com for tweeting the dead end youtube video in November 2010, the tweet resulted in making several people curious and emailing me about explaining how the project was achieved. And that’s why this blog post came into existence. Never in my wildest dreams I had thought that anyone would be curious about this project of mine, I guess I was totally unaware :) .
Also please check out the following Augmented Reality project video (sorry for the bad quality) of a group project that followed in the second semester’s Digital Media Studio Project course. A 17th century sonic environment of the horrible history of the Royal Mile (in Edinburgh)was created using FMOD Sound System and was presented real-time to the tourists as they walked on the Royal Mile with radio headphones on.
http://fmodproject.wordpress.com/