My Journeys into Sound

Music Tech. Degree Diary

[mattcollierblog-blog]

Reading

Here’s an eclectic mix of reading related to this project. Two books I’m reading in relation to Music as Therapy are “Applied Musicology” written by Adam Ockelford, and a collection of research papers brought together under the heading of “Handbook of Neurologic Music Therapy” edited by Michael H. Thaut, Volker Hoemberg. Here’s an interesting article presented in the International Journal of Technology, Knowledge and Society on the subject of Sonic Art and performers with disabilities written by Frank Lyons in 2006. The magazine “Sound on sound” ran two informative articles in 2012 on the subject of Music Technology & Special Needs. Music Hackspace and Human Instruments are websites that offer a wealth of information surrounding the adaption of technology to musical expression. The Steim organisation, based in Holland, is active in the field of live performance using unique bespoke electronic instruments.

[mattcollierblog-blog]

Music participation frameworks

The intention that underpins the design of this system is to provide an environment for anybody regardless of age, physical or mental capacity to express themselves musically. To this end I have sourced material that provides frameworks designed to evaluate and progress musical understanding in a variety of settings. This publication, from 2011, offers an extensive evidence review of Engagement with Technology in Special Educational & Disabled Music Settings. In 2016, two organisations whose primary function is to facilitate musical learning and expression, Youth Music and Drake Music, produced “A quality framework for use in music-making sessions working with young people in SEN/D settings.” Another source of valuable information is produced by the Training and Development Agency for Schools, titled “Including pupils with SEN and/or disabilities in primary music.” Much of the research that these publications are based upon has been compiled at Roehampton University Institute of Education. Here they have developed a framework of musical development borne from the Sounds of Intent research project begun in 2002. All of these organisations act as hubs for further research and resources surrounding the application of music in educational settings.

[mattcollierblog-blog]

Visit Original Link

One feature of the system is the ability to trigger backing tracks and sound layers via the Ipad. There are thirty tracks in total, and these could be loaded with any sounds, tailored to whatever the particular application of the system. At present, I have decided upon a mix of ambient environmental sounds, rhythms, drones, bass lines and generative tracks. I have field recorded all of the ambient sounds, the rhythms are a mix of off the shelf samples and my own constructions, the generative tracks were built using Patter and Pure Data, with the drones and bass lines to be recorded. Here are a few samples.

[mattcollierblog-blog]

System testing

Development and initial testing of the system has been carried out in my home studio, but proper commissioning, testing and debugging needed to be done in a real world setting. I have used the large hall in university for sixteen days to trial, test and refine the operation of the system. The system has proven to be portable, reliable and relatively quick to set up, however a lot of development has been required in terms of function and operation, and hearing the system fully functioning led to some changes in effects choices and settings. I’ve invited a number of friends along to try using the system, and their feedback has proven important in helping to refine the functionality and clarify the purpose. Through evaluating how users interact with the system I have begun to develop an introductory procedure to familiarize users with the controls and features. Consideration with regard to what instruments and noise makers to utilise is ongoing; I have a variety of midi and acoustic instruments, plus there is a whole world of ‘non-musical’ sounds that could be used.

[mattcollierblog-blog]

Choosing Effects

I’m using Ableton Live as the sound processor. A main feature of the system is the ability to manipulate both live and sampled sounds. With this in mind I set about exploring what effect plugins would be suitable for this application. Firstly I have to decide how the effects will be controlled. There are four joysticks; two have five buttons available to switch effects on & off, 2 have eleven buttons. Each stick also has a finger trigger, and a horizon slider, the Z movement. The X and Y movements of the sticks will control mapped parameters in any chosen effect device. One button on each stick is already designated for turning the quad panner on or off, and I have mapped the trigger press in Puredata so that it acts as a master reset to turn all effects off. I have also mapped the Z slider to perform the role of effect send. One issue I encountered was how to pan both live sounds and looped sounds independently; my solution was to employ another, fifth, joystick to spatially control all loops and backing tracks.

The next step was to audition a whole host of effect devices in Ableton. What I consider to be essential would be an equaliser, delay and reverb. The equaliser is an insert and one has to be placed on all audio input and Looper channels. Reverb wise, I liked using the Amazing Noises Outerspaces device, as it has a huge variety of sound shaping controls and responded well to parameter mapping. After playing around with not only different delays, but also how the delay is adjusted, I settled on a relatively simple device. Other devices I’ve settled on include speed shifter, comb filters, spectral delay, resonators, space echoes, Slow motion LFO drive and grain filters. The auditioning of the devices involved studying all the possible parameter adjustments, and deciding what changes to map. The devices settled on have been chosen as they show an obvious effect on sound and the adjusting of parameters via joystick movements are tangible. As any combination of effects can be chosen, these devices have been used as they work well in tandem with each other. Each input group has it’s own effects rack located in a return track, it’s my intention to build multiple effects racks in each return track, any of which can be chosen via the Ipad.

[mattcollierblog-blog]

System hardware

It’s time to talk about hardware. Hardware choices have been dictated by the need for the system to be portable, durable, reliable, plus safe and easy to use, all within the context of it having open access for users.

Wireless microphones were high on the shopping list; I managed to find a b-stock Chord Quad system, with two handhelds and two headsets. It’s not studio quality sound but they’re rugged, cheap and work well. Other mics employed are a pair of mini omnidirectionals, equipped with Primo EM172 capsules, which have an excellent Signal to Noise ratio and sensitivity. These are mounted on mini mic stands to sit on the playing table. I’ve also got five cheap contact mics running through a Behringer MX400 mini mixer, plus a better performing contact mic, and a coil pick up. This variety of microphones allows for generating and capturing sounds from a wide range of sources.

Speaker wise, I needed four small active units. I spent a lot of time researching this, and kept coming up against the budget versus performance paradigm. Eventually I settled on Alto TX8 as they offered the best performance versus budget compromise. The packs I bought included stands, bags and leads.

The computer driving the system is a MacBook Pro with a 2.5 GHz Intel Core i7 processor with 16gb of RAM. As there is potentially a lot of USB traffic going on I needed an excellent performing USB hub. After much research I splashed out on a 10-port hub made by Orico. The Midi devices attaching are an Akai LPD8, an Akai LPK25 keyboard, a Korg Nanopad 2, and a Medeli Cubeat drumpad. Other sound making devices are an Xbox 360 wireless controller, and Nintendo Wii controller. I don’t envisage that all these controllers will be used simultaneously, but the system is designed to be modular and offer multiple means of creating and controlling sounds. I now have five second-hand joysticks, variously made by Logic3, Logitech and Saitek. The Looper handheld controllers are from the Sony PlayStation game Buzz! 

The wireless microphone receiver is 1U rack size, so I set about finding a rack case that could fit under a table and house the receiver, laptop, audio interfaces, USB hub and mains power strips.  I was fortunate to find a used case locally; it is the type made for mounting a mixer so has removable top and both sides, it also has an internal draw to store microphones and interfaces. With a small bit of adapting it absolutely fits the bill.

I have designed the system to be powered from one mains plug. A 10 metre mains lead feeds to the rack case, and in turn powers all speakers through custom length cable runs. The rack case has a 2” hole at one side for feeding cables through, by fitting all power leads with IEC C13 & 14 connectors I am able to route all power and audio cables through this access hole and close the side, safely hiding away all the connections. Customs cut strips of rubber matting are used to cover all cables running across the floor.

[mattcollierblog-blog]

Aggregate inputs & quad panning

I’m using two audio interfaces, but need them to arrive at Ableton simultaneously. I need to be able to choose both as one input; this requires setting up an aggregate audio device. This is relatively simple to achieve, check out Apple’s tutorial.

As my system uses a quadraphonic speaker set up, I need a quad panning device to use in Ableton. I’ve trialled quite a few, and the easiest and most effective I’ve found is Somnitecs 4.1 surround panner. It’s fully midi mappable, free, and allows the sound to be moved around live and also ‘dropped’ in any place so it remains fixed to that spot. I’m currently using 13 of these devices in my set up, and all can be used simultaneously without any issues. Exactly what I need.

[mattcollierblog-blog]

Latency issue & new direction

My ambition is to build a system that features six microphone inputs and six midi device inputs, with the ability to sample and loop any of these inputs via hand held game controllers. The output sounds are to be played through four speakers, with effects and spatial manipulation of live and sampled sounds achieved by using joysticks. I have developed the system to the point where Pure Data patches receive all microphone inputs, Ableton Live handles all midi inputs, backing tracks are played in Ableton and triggered via Ipad. Testing of the system has revealed a worrying amount of analogue sound latency, due to the process of inputting through Pure Data, then outputting to Jack OS X and then across to Ableton for processing. All the triggering and channel switching works fine, but latency of this order in a live situation is unacceptable. A new method is needed.

I need to restructure and slim line the process so all microphone inputs arrive directly at Ableton, and hopefully eliminate processing latency. As a key feature is the ability to sample and loop sounds, I explored how this can be achieved in Ableton. It turns out Ableton has a device called Looper. On the face of it this device appears to fulfil my needs; it will loop on the fly and it’s midi mappable. However there are a number of ways to configure and control the device, I have explored the options, and through studying an excellent tutorial I have arrived at a signal flow that works without latency and incorporates all the features I require.

This diagram details the signal flow for one group input, the system will have four groups, twelve inputs in total. I have begun midi control mapping, and everything appears stable at this stage.

[mattcollierblog-blog]

Generative Audio

The system I am designing is not only about creating sounds, but also about listening and responding. An audio element I’d like to include in my interactive system is Generative audio, as this type of sound provides an evolving sonic layer that can be responded to. In 2007 Dave Burraston & Ernest Edmonds published an historical and technical review of Cellular Automata in Generative Electronic Music and Sonic Art, which provides an excellent overview of the subject in a variety of contexts. I have researched generative video game audio produced by Leonard J. Paul, generative music apps designed by Brian Eno and Peter Chilvers, Pure Data patches constructed by Martin Brinkmann, and have also explored Algorithmic Composition. I have adapted some Pure Data patches produced by Martin Brinkmann, and have integrated them into my system. I have also experimented with generative music plugins that are available for Ableton Live including nwdlbots and Patter. I found it easier to achieve interesting results using Patter, and have programmed and employed some generative templates using this into my system.

[mattcollierblog-blog]

Ipad and Bluetooth

My ambition for the interactive system I am developing is to do away with the need to interface with a computer screen. However, I still need control over elements such as audio input on/off, channel volume and clip launching. An Ipad would provide an ideal solution. There are a number of apps that allow parameters in Ableton to be controlled via Ipad; the one I opted to use was Touch OSC by Hexler. I constructed a bespoke interface that lets me control Ableton and Pure Data via midi message mapping.

However one drawback is the need to join a wireless network to get the laptop and Ipad talking. What happens if there is no wireless or it is unpredictable? I explored the possibility of using Bluetooth instead of Wi-Fi. Midimittr by developer Matthias Frick offers a superbly functional and free solution. Now I can connect the Ipad to Ableton and Puredata over Bluetooth, seamlessly sending low latency Midi Note and Control Change messages.

[mattcollierblog-blog]

Getting Pure Data talking to Ableton

So far Pure Data has been used for all sound manipulations, now I would like to use Ableton Live to carry out this function as it has all manner of device plugins to experiment with. Pure Data can send audio signals to Ableton by using system extensions that allow different applications to connect via the system soundcard.  Soundflower and Jack Audio are the most commonly used servers for this purpose. After using both I have opted to use Jack OS X, as it is the most stable and easiest to use in my experience.

The audio input hardware I am using is a Motu 4pre and an Art USB dual pre, allowing up to 6 analogue inputs combined. By utilising Jack OS X these 6 audio signals are input to Pure Data, and up to 14 audio signals can be simultaneously routed out to Ableton.

[mattcollierblog-blog]

Visit Original Link

Wireless Xbox controller synth experiment

[mattcollierblog-blog]

Contact mics

Contact microphones offer an affordable addition to acoustic microphones in picking up, amplifying and manipulating sounds. On the face of it contact mics offer many benefits to capturing sounds; they rely on Piezo pickups to sense audio vibrations in solid objects, and turn this structural sound into usable voltage changes. Which in short means no background noise in recording and no feedback in live situations. So what are the drawbacks? Piezo transducers require a high impedance input to prevent loss of frequencies below around 200Hz, using some form of tailored preamp will transform the sound.

A great amount of contact mic information can be read here

[mattcollierblog-blog]

Hand Synth

Thought I’d have a go at turning one of these....

into a synth controller. All the info and software needed to get my Mac and controller talking was found here https://github.com/360Controller/360Controller

I then created this HID patch in PureData 

to change the controller data into midi and CC messages. Dirty monophonic noise is the result courtesy of Ableton Live and Amazing Noises Pulsor  max4Live device.

[mattcollierblog-blog]

HID resources

Here are some excellent resources for understanding and using Human interface Devices:

 http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.122.8517&rep=rep1&type=pdf

 https://at.or.at/hans/misc/itp/thesis/Hans-Christoph%20Steiner%20-%20ITP%20Thesis%20-%20Fall%202004.pdf

 https://at.or.at/hans/writing/BuildingYourOwnInstrumentWithPd/

[mattcollierblog-blog]

Twisting the samples

So now I’ve got a bunch of samples to play with; it’s time to call upon the joysticks to twist and reshape the sounds. Puredata will not only handle the HID inputs but will also carry out the sound manipulations. Here is a simple patch designed so the XYZ joystick movements send out Control Change messages, and the five button presses will send note on/off midi messages.

Next I need to send meaningful messages to the sound samples I’m using; this patch sends the button toggle and CC joystick messages directly to the controller patch where the sound manipulation takes place.

The patch below records, plays and manipulates the sounds. The joystick movements change the delay, EQ and flanger parameters in real time.

One issue I had to overcome was how to select which channel a sample was recorded to. This next patch is designed so that the last channel that is switched off using the Buzz controller buttons becomes the next channel that is recorded onto.

So now I’ve got sixteen channels that can record, play and stop sound samples, all defined by four handheld controllers. The sound clips can have single or multiple manipulations applied simultaneously in real time, all controlled by the buttons and movements of the joysticks.

[mattcollierblog-blog]

The hassles and successes of HID exploration.

On the face of it the task I’ve set myself appears straightforward; use these devices as sound controllers for multiuser noise interaction. Some are dedicated to this service already, some require coaxing into this purpose. This path to appropriation has been winding one.

The programming language Pure Data offers flexibility in converting Human Interface Device movement and button presses into useful messages to use in sound manipulation. However during research and development I have discovered a number of idiosyncrasies inherent in this chain of commands; different releases of PD behave differently in their reaction to HID handling, and this issue is further compounded by differing behaviour of pre-scripted HID patches. Symptoms included irretrievably losing connection with USB game controllers, not being able to connect to discovered devices, freezing and general instability. Through extensive trial and error I’ve arrived at a stable solution; PD version 0.43.4 in combination with HID 0.7 developed by Hans-Christoph Steiner. This combination allows connection of four game joysticks complete with all (up to 12) buttons triggering, four Sony ‘Buzz’ controllers also adds to the collection of HID. I’m now able to send XY motion as Control Change messages, and all button presses can either toggle or trigger Midi note or CC messages. Once this stability has been arrived at, focus can shift to the creative sound potential of these devices.

Original trials involved using PD as the generator of analogue sound samples captured from microphones. This involved creating wav. clips as opposed to array based sound files which require a predefined time scale; wav clips can be of any length.

Here’s a patch I developed that can be switched to accept one of three audio adc~ inputs and samples can be made from any input.

Firstly assign a slot for a sample by pressing one of the four coloured buttons and by next holding down the big red button on the Buzz controller a recording of any portion of the incoming sound is made, recording stops when the button is released. By repeatedly pressing the relevant coloured slot button the sample either plays back or stops. New recordings can be made at will by repeating this sequence.

By employing all four controllers up to sixteen samples can be generated and played with simultaneously, no freezing, slowdown or wayward behaviour. Success.