MOSI'S SOUND COOKBLOG @mosiblog - Tumblr Blog

#9 A GENERAL OBSERVATION ON SOUND DESIGN FOR NON LINEAR MEDIA USING FMOD

I downloaded Fmod Studio a few weeks ago, to start explore a software commonly used to work with sound design for games and more broadly non-linear media... Fmod can be implemented in Unity, a real-time 3D development platform lets artists, designers and developers work together to create amazing immersive and interactive experiences.

With Fmod you can :

Design sound for your game using an intuitive DAW-like interface, with no programming knowledge necessary.

Design adaptive audio that dynamically reacts to everything that happens in your game.

Design sound for any game engine, and for any major platform.

Mix and balance your game live, as it runs.

Profile and record your game's audio output, then make changes and audition them without having to re-record.

FMOD Studio is an application that allows you to create adaptive audio content for games. Content created in FMOD Studio can be built as bank files, which can then be loaded in the FMOD Engine.

The FMOD Engine can be integrated into your game, allowing your game to play adaptive audio. The FMOD Studio API allows the FMOD Engine to load and play bank files built in FMOD Studio, while the FMOD Core API allows the FMOD Engine to create audio content without using FMOD Studio and provides more advanced features. The FMOD Studio API and the FMOD Core API are together called the FMOD API.

#8 PRESENT & FUTURE PROJECTS

In these times of chaos, where a confusing and unknown future awaits us, my artistic practice finds itself deeply inspired by obscure and dark aesthetics and resonates with my feelings towards these sinister and unwelcoming days to come.

I am currently working on a 7 minute horror-inspired soundscape project, where I have blended together field recordings, sampled audio clips and synthesized drones, all modelled through different granular synthesis tools such as vst's like Scope by Glitchmachines in Ableton and Endjinn, a granular synthesis module designed to function with BEAP modules inside Max Msp.

The soundscape could work well in a horror-based video game, since it features a multitude of sinister and creepy sounds such as daunting voices and screaming children, intense heartbeat and evolving breaths, resonant cymbals and metal doors molded together with drones, modelled pads, ambience field recordings and distant lullabies in a constantly evolving sonic environment where various elements travel through the spectrum as the piece moves on.

The main character of the sonic landscape and its starting point is what sounds to me like a synthetic flying insect like some sort of cyber-mosquito flying up and down around the soundscape. The original sound is the recording of a squeezed sponge spilling water in a bowl, which I processed with a heavy dose of Ableton warp, stretching the clip and modulating it with a granular synth vst called Portal, so I could regulate and modify the grain size, density, speed and position throughout the track.

In the near future I would like to deeply explore different techniques and procedures in order to design sound and special effects for horror-inspired games and movies, I find creepy and sinister sounds very intriguing to work since they can alter our perception of space by triggering our sensations causing emotions like fear, suspense and anxiety.

In general, people don't pay that much attention to sound when they're gaming, but when it comes to horror, it seems to make a significant difference.

The sonic context in this genre is very important, since it can guide the player through the gameplay by setting expectations, defying them , triggering feelings and delivering scares!

Check out my sinister soundscape below and listen to it with some good headphones or with a surround sound system to experience it at its maximum potential.

#7 THOUGHTS AND INSPIRATIONS

As a creative individual, I always feel the urge to work on something new, whether it's starting an unusual sound design project, a musical composition or going around the city recording soundscapes, objects or people.

I am constantly inspired by movement. When I see motion, I tend to imagine it as sounds. For instance, if I'm wandering through the chaotic streets of Berlin and spy a rat running by, I imagine hearing the sound it makes (isolated and in a different context, without all the surrounding city noises) and then go back home and start working on designing imaginary rat footsteps with whatever type of audio tool I stumble across. The end result of these experiments is usually rather unrealistic and synthetic, but then, how do rat footsteps sound anyway?

As conscious human beings, we already know what to expect to hear when we perceive movement in our daily lives…. However, it's by taking sounds from nature and ordinary life as we know them and replacing them with unexpected ones that we can create something new and walk the spectator through the unknown.

Let's take footsteps as an example, one of the most essential and basic sound in movies and a well-known (and pretty straightforward) technique in the world of foley...I first took a short video clip of footsteps on a wooden pavement and replaced the sound with a recording of footsteps on a parquet floor (like a foley artist would do). Then I duplicated the video and with the aid of Operator(probably Ableton's most well-known stock instrument) I synthesized a simple sine wave sound and played in sync with the muted video clip. The final product doesn't sound very realistic, but replacing "real" and "organic" sounds with synthesized ones can be a fun technique.

GIFs are another great source of inspiration for my creative practice.

Sometimes I feel like I can hear them, and GIFting sound to them is a fun practice for both amateur and professional sound designers.

Let's take for example the GIF on top of this post, it's around 3 seconds long but you can already find a lot of movement in it: the green creature "melting" and the camera angle zooming out. I duplicated the GIF 2 times and designed some sound for it layering different audio clips.

For the first segment I used the sound of an engine modelled with lots of reverb and granular synthesis (using the vst Portal) while for the second clip I reversed a field recording and layered it with some creepy violin sound effects that feel as if they are moving closer as the camera angle zooms out (I used the plugin proximity to simulate their distance in space) and added a whooosh as soon as the GIF fades to black.

Check out the final product here below.

#6 THE DEGRADED_ EXPLORATION_ LISTENING_SESSION

The Degraded_Exploration_Listening_Session is the most recent musical project i've worked on. It will be released in a month or so on https://linearecords.net but, meanwhile, you can find part of it on youtube.

After months of exploration and field recordings in various abandoned locations in Berlin (Bärenquell Brauerei, Kinderkrankenhaus Weißensee, Spreepark Berlin among others) , I decided to blend everything together and work on creating a sonic voyage through different atmospheres and contexts. There is no defined logic underling the piece as it's always moving through different sonic environments, from mysterious melodies and distorted drums in the beginning to conversations about life in Berlin and language differences between Italian and Spanish. Various field recordings cope and fight with the mysterious and shady lullabies representing a dark and confusing future awaiting us.

This project focused on exploring the magic conflict between industrialism and nature through sonic experimentation. The main purpose was to recreate the specific sensations and emotions felt in "dead" degraded spaces, where many atrocities occurred in the past and where time seems to have stopped for years, decades or even centuries, taking a break from this globalized, fast-paced and energetic world we live in.

2020 has been a weird year for me (and for everyone I suppose) and with this musical piece I wanted to evoke these feelings with the aid of bizarre and obscure recordings such as bowed cymbals rich in resonance, squeaky doors together with longer recordings taken in abandoned locations and "mysterious" sounding melodies.

The Degraded_Exploration_Listening_Session broadcasted live with a listening session on 9th of June from an abandoned location in East Berlin.

For the video, I decided to bring my computer, a mini launchpad and a pair of headphones in an abandoned location in East Berlin, the idea being that one should listen to the semi-finished Ableton project in a context that symbolized what the piece was about.

The hardest challenge was probably filming myself wandering across this degraded space with my mac book in my hands. Having somebody else filming me would have been a lot easier, but I liked the idea of a "point of view" with non-perfect shaky camera shots that could fit well with the ambience of the music and video, giving them an even more "realistic" vibe.

In addition, wandering alone around this sinister space with my laptop didn't feel like the safest idea, as I was afraid of dangerous encounters that would beat me up and rob my computer(a bit of paranoia is always present, but it can bring out loads of inspiration and creative ideas). In the end, I stumbled across a few people (not dangerous at all!) who were just as curious as I was [one of them can be seen in the video] we had a little chat, and I described my project and shared the link to this blog, to further expand this network.

#5 PERFORMING AT THE AKOUSMATIK BUSH

In mid July my friends Wei and Sid invited me to play at their be-weekly event: The Akousmatik Bush.

The Akousmatik Bush is an open air, experimental improvisation event series in Berlin that took place be-weekly throughout the whole summer. The main character and center of attraction of this experience is a thorny blackberry bush, an invisible vibrating source breathing sounds that have no visually identifiable cause through a 4-speaker sound system.

But what does Acousmatic mean?

"Acousmatic sound is sound one hears without seeing their originating cause - a invisible sound source."*

In the Akousmatik Bush environment, the performers were placed in a room (Wei's bedroom as a matter of fact) behind the bush, so that the audience couldn't see who was performing, giving a touch of mystery to these interesting gatherings that took place in an unusual location: a student housing center called EBA51 in the Berlin Plänterwald district.

The architecture of this location is very unique - the students' houses are basically fitted containers covered externally by a rusty metal sheet that when receiving the sun’s reflection turns into a beautiful bright brown and orange, like in the photo below.

Photos from the Akousmatik's Bush Facebook page : link here

Hanging around the bush and getting to see how a d.i.y event idea like this could grow and eventually form a network of people was very intriguing. It also gave me the chance to meet a lot of new faces, exchange ideas and help contribute to this wicked project. The fact that this event took place in a student dormitory and observing how the inmates that lived there interacted and approached the weird-sounding bush was very interesting too, because most of the performers played extremely experimental and noisy sounds that most people normally wouldn't understand and appreciate.

My performance at the Bush was a mixture of field recordings from the debugged_database project, distorted drones and lullabies, and lots of spoken word recordings of people I had met in the summer just saying random stuff, basically stream of consciousness narrations...

I had around one week to prepare a 40-minute live set from scratch, so instead of a 100% improvised performance I decided to present my degraded_exploration_listening_session in a live context by playing sinister melodies with Hive and Diva (2 synth plug-ins I often implement in my compositions ) and a selection of recordings blended together with binaural effects, plenty of distorted feedback loops and heavy doses of panning to give even more space and ambiance to my composition allowing it to move right and left through the 4speaker sound system of the Akousmatik Bush.

If I had the chance to play again in the bush I would probably implement more hardware tools such as midi controllers in order to assign more parameters and modulate my plug-in synths even more, but listening to the recording of my live set, I must say that I am satisfied with what I played.

Here I am, performing and turning knobs in Wei's room behind the bush

The recording of my liveset will be uploaded on youtube in the next weeks, together with other snippets of performances from past Akousmatik Bush events.

Being part of this project was a delightful and amusing experience and I hope there will be more in the future. The bush closed the season last weekend with some beautiful performances and installations by Andre Badim, NNVVKK, Katharina Bévand, Rrill Bell and Vito Willems.

*Chion, Michel Audio-Vision http://filmsound.org/chion/acous.htm

#4 MY BIGGEST INFLUENCES

As a music addict and sound chef, I've always been in search of music that sounded completely new and like nothing ever heard before. One of my biggest influences is Amon Tobin, a Brazilian musician and composer, well-known for his methodology in sound design and music production. His music has been used in many major motion pictures including The Italian Job and 21. Tobin has scored soundtracks for several independent films, such as the 2006 Hungarian film Taxidermia and has also composed impressive musical scores for videogames such as Tom Clancy’s Splinter Cell: Chaos Theory and Infamous.

Amon Tobin's first albums (Bricolage, Supermodified, Permutation) were sample-based records, in which he used various snippets of classical, jazz and rock songs and melted them with futuristic drum 'n' bass and hip hop influenced rhythms.

In 2007, Amon Tobin released Foley Room, an album created entirely using techniques from the foley world of sound effects in the movie industry.He himself recorded all of the samples used for production using an omnidirectional microphone . Tobin recorded from various sources including motorbikes, tigers, insects, and water falling from a tap and then modulated the recordings creating avant-garde sounds.

Tobin's most interesting album from a sound design perspective is probably ISAM (an acronym for "Invented Sound Applied to Music").

"ISAM was an experiment in synthesizing field recordings and transforming them into new physically playable instruments to create unique sound palettes." *

ISAM is a masterpiece, I believe one of the most interesting albums ever created. Tobin is able to take field recordings and mutate them into something never heard before using complex granular synthesis and spectral morphing techniques with Kyma. This extremely powerful sound creation and manipulation environment is comparable to softwares such as Reaktor and Max Msp with which Tobin guides sounds through a sonic metamorphosis, like the manipulation of his own voice to gender-modify it or the modelling of a squeaky bench into an ensemble of alive-sounding wood.

Amon Tobin isn't only a regular "electronic music producer", he is a multi-talented sound designer that utilizes various techniques, many previously used in the world of film Ben Burt for example (the sound designer behind Star Wars).

His strong influence on my creative practice lies in the way in which he has blended together and explored throughout his career different techniques and approaches to music production using different contexts of sound design and music. Rather than being stuck in the same bubble for years, he has reached out for novel sounds, taking creative risks to craft mind bending sounds, like in the case of ISAM, something that can be categorized as music obviously, but also as science. Amon’s music has changed significantly in the last 20 years, his discography is continuously evolving and moving direction based on the techniques and tools he has mastered throughout the years.

In my artistic practice I would like to be a bit like Amon, meaning that I want to cook sounds and music blending together many different techniques and approaches to music from different sound contexts of sound and I want to release music based on the audio tools and methods I am exploring at that current time without caring too much about the "music hype of the moment" and what the mainstream music consumers "want" to listen.

I listen to new music every day, always discovering and shifting through genres, new artists and sound designers that may inspire my daily artistic and creative practice. However, artists like Amon Tobin will always be in my playlists and have a big role in my sound and approach to electronic music.

*from Amon Tobin's Biography on https://ninjatune.net/artist/amon-tobin

#3 MY APPROACH WITH FIELD RECORDINGS

Field recordings are at the core of my creative practice: during the first months of the Covid-19 pandemic I spent a great deal of time exploring abandoned locations in Berlin and recording their sound. I documented my project progress in this other blog I created (link to https://debuggedatabase.carrd.co/) .

Every space has its own unique sound, and when it comes to large halls and warehouses, natural reverb has an important role in defining the sound. It’s with the use of reverb (natural or added in post-production) that we can unleash illusions of space in the listener.

In this entry I will give you some tips for good field recordings and how to create illusions of space in the listener by modulating recordings with reverb, panning and special binaural plug-ins

Some people think that you need very expensive gear to have good quality recordings, personally I don’t really agree with that, in fact all my recordings have been carried out with a very cheap field recorder (zoom h5n) and still sound great! Of course, having a professional field recorder would be ideal but not everyone has tons of money to spend on gear right?

Here are some tips for having some good field recordings:

Use a windscreen solution on your recorder, it can also just be a DIY solution (sometimes I end up using socks or any old cloth).

Set your levels carefully to avoid having unexpected strong peaks, I usually let the levels peak at around -12dB.

Record in a high quality format! I use 24 bit WAV 48 kHz, some people prefer using even higher sample rates like 96kHz, do some testing, listen to the recordings carefully and choose what the right sample rate is for you.

Listen to the signal coming through the device with Headphones (Monitoring), this is very important to properly understand if something is going wrong or if the levels are too high in order to avoid unexpected surprises in post production.

Be spontaneous!! If something seems and sounds interesting, don’t hesitate to record it!

On top: Field Recording by Hollow Creature , Ink on illustration board

In spring 2020 I decided to explore and do some field recordings in the astonishing abandoned Bärenquell brewery in the Schoneweide district of Berlin.

During the last 25 years, degradation and dust have overwhelmed the ruins of what once was the home to one of the most popular beer factories in Berlin. However, I recently discovered that the space will host  Revier Südost, the new venue opened by the owners of Griessmuehle, a well known club in Berlin whose doors had to close in it’s older location.

The amount of natural reverb and echo in the indoor sections of the brewery is incredible, and my recordings here are all very particular and have a “sanctuary” feeling, I processed them through different tools to create movement and play with perception.

To give even more illusion of space to the listener, I duplicated the audio file in 4 parts, left 2 channels in the center and panned one all the way right and the other one on the left. I then added an auto pan (Ableton stock audio effect) and a very intriguing free-download plug in by Sennheiser called Ambeo Orbit, a binaural panner plugin, designed to facilitate mixing immersive binaural content*. I then used another vst called Proximity to manipulate the “depth” and simulate distance in the 4 clips, and the waves J37 tape machine to bring some analog simulated warmth.

The final piece is a deep listening experience rich in movement and space; I close my eyes and listen to it on some good headphones or on a surround sound system. This makes my mind travel far and evokes some of the emotions I felt while exploring this magical location.

#2 LAYERING SAMPLES & RECIPE #1

In this entry I will go through some basic knowledge about layering and then I will explain the process I put into place one warm August afternoon while recreating the ambience sound for a spaceship scene that I stumbled across on Youtube.

Lets start from the basics… what is layering?

Layering is basically combining elements to either enhance an existing sound or to create a new one. It’s probably one of the most used techniques in the world of sound design.

Sound designers carefully select sounds and modulate them with the use of equalizers, compressors and various tools in order to fill up all the spectrum’s frequencies and to get a final “full body” and powerful sound.

This technique has been used very often in the context of sound for movies, a good example is the sound for the roaring velociraptor in Jurassic Park, where sound designer Gary Rydstrom layered the recording of an underwater dolphin making a strong high pitched scream, coupled with the deep resonant roar of a walrus.

Gary Rydstrom's Velociraptor Sound

My Approach To Layering Audio Clips

Layering various audio clips together and using small details from each one of them together to create a final complex sound is one of those things where there is no right or wrong, just freedom to use whatever tool to manipulate audio as you please. This can be found everywhere in music, where different instruments alternate each other to generate what we would normally call “a song”, or in nature, where in a forest for instance, different animals emit different sounds and all together deliver the sound of the environment.

In this entry I will focus on layering from a more “atonal” and perhaps experimental point of view, where rhythms and harmonies aren’t relevant and where the only important thing is how sounds melt and dissolve into each other while following movement.

To train myself and develop my skills as an amateur sound chef, one warm August afternoon I decided to download a short video of a spaceship and tried to recreate its sound by layering different audio clips (some of my own field recordings and some material off the Internet).

I stumbled across this 36 second video of a 3d modelled spaceship, the original sound isn’t very complex, just some very low frequency drones modelled with reverb: my aim was to create a bit more movement by additionally crafting the sound of the spaceship engine with some beeps and blops when the camera angle would change.

To generate the sound of an engine, I took a noisy loop, added some saturation to make it sound stronger and modulated it through Portal, a granular synthesis plug-in which I utilized to divide the audio clip in tiny grains, creating a rhythm with them. Check out the video below to hear the difference between the audio clip with and without the granular processing.

The other sonic elements in the project are mostly field recordings I took of indoor ambiences and noises emitted by electronic devices, most of them filtered heavily and then processed through the Soundmagic Spectral plug-in pack by Micheal Norris, a series of plug-ins that implement real-time spectral processing of sound.

The clips melt into each other thanks to the heavy use of fade in and outs, one of the most useful and essential tools in layering, allowing to gradually increase or decrease the level of an audio signal.

#1 THE AUTHOR

Hi! I’m Simone aka Mosi and welcome to my sound cookblog.

In this entry I will spend some words to talk a little bit about myself, my creative background and my artistic ideas.

When it comes to sound design and specifically crafting special effects, I like to see myself as a sound chef.

I feel like layering audio is a bit like cooking something without following a recipe, improvising by putting together different ingredients and spices to create a totally new dish… and as in the context of cooking and coming up with new dishes, there is no magic formula to layering, its all about experimentation and trial and error.

Sound designers choose sounds and modulate them in order to fill up all the spectrum’s frequencies and to have a final “full body” and powerful sound. I like to compare this phenomenon to the art of mixing the right amount of different flavors in order to come up with unique tasting dishes, and very often opposites attract each other: like mixing salty and sweet ingredients or blending low frequencies and very high pitched sounds… but I will talk about layering in more detail on my next entry.

A few words about me: I was born in Milan 21 years ago and thanks to my father’s taste in music, I had the good fortune to listen and get lost in electronic sounds from a very early age… Artists like Aphex Twin, Air, Amon Tobin, Squarepusher and many more were always on my playlists.

My first real approach with the world of audio was when I visited my aunt Marta in her studio in Rome 10 years ago, she has been working as a sound editor for over 20 years and has collaborated with various well-known Italian directors. Staring at the enormous mixing desks and getting to see how sound designers and editors work, gave me the benefit of a conscious perspective on the process and all the hard work behind sound in movies.

In 2018, after my high-school years spent exploring and getting to know the world of Digital Audio Workstations and music technology, I moved to Berlin to start an academic path in Electronic Music production and performance; once there, I realized that I wanted to focus on the world of sound design, exploring the full potential of sound and understanding why it works the way it does.

In 2020, during the early stages of the covid_19 pandemic, I worked on a personal project called the “Debugged_Database”, where I collected various field recordings from abandoned locations around Berlin and mixed them together in a composition called the Degraded_Exploration_Listening_Session. I will talk about this in more detail in the next entries but in the meantime you can find it here : https://www.youtube.com/watch?v=9ebFlWKGuks&t=1s

These days, with the world struggling with the threat of a global crisis and with an undefined future ahead of us, I stay put, engaged and alert in my creative path as an amateur sound chef, exploring and capturing the soundscapes of new spaces and intriguing locations around Berlin and learning new and interesting ways of approaching and challenging the art of sound design.

Other than this blog, I am currently working on a horror-inspired soundscape project, where I have been layering together a number of sinister recordings, modulated with the use of granular synthesis and reverb to create illusions of space and perception and hopefully will delight listeners with some chilling sounds.

Check out my other posts to learn about my creative process and ideas, my tools, strategies and challenges during the development of my upcoming projects.

Trending Blogs

Recently Viewed Blogs

MOSI'S SOUND COOKBLOG