PRTCL

Digital Media Collective

[prtcl-collective-blog]

Montage

[prtcl-collective-blog]

Interacting with the Ableton with the Leap Motion Controller Jack Cleary on the hands and Tarek El Goraicy on the cam

[prtcl-collective-blog]

Notes on Interacting with sound and visuals with Gesture based technologies -The Leap Motion and Beta Glover Software by Jack Cleary

PRTCLs interactive audio/visual installation Cog.Flux is the first project to use gesture detecting technology such as The Leap Motion Controller and their relative softwares, that can communicate MIDI/OSC messages to other programs. This new territory had a baggage of issues (latency and gesture recognition being the main two). With these limitation in mind, we had to create an audio/visual experience that is reactive to appropriate gestures. Below, the process of constructing (conceptually and practically) an interactive audio/visual compositions via gestures will be explored and discussed.

Interacting with Digital Sound through Gesturing

As mentioned before latency and gesture recognition are two issues that effect the interaction between user and computer. Firstly I will discuss latency; Hardware sends data messages to computer software which reads the messages. The time in which this takes is called latency. Depending on the size of the message and its complexity, the software will process the data and use the data for a specific purpose. An example: The Leap Motion Controller (Hardware) sends co-ordinates of the hand constantly to a software (Geco or Glover), which can map the roll of the wrist to a audio parameter like volume. With this example in place, if one imagines this, but with multiple and various mappings, changing many parameters, it should be expected that one would run into some processing problem. In short; the more messages the software has to decode the more processing power need, if not the software will output the messages much slower. 
As one may theorise, latency may weaken the experience between the audience and the controller/computer/sound/visuals. This can be resolved with faster processing, but when resources are limited the solution lies in being resourceful of the computers power. And by this I mean: maximise the use of simple gestures and be reserved in mapping the many possible gestures.

I shall now discuss the issues surrounding the recognition of hand gestures: The data read by the software is not always accurate, there are many discrepancies between what gesture is made and what gesture the software recognises. Many factors effect the recognition of the gesture such as:

The software and the controllers relationship

The complexity of the gestures

The sensors used to detect the movement

When using Imogen Heaps Mi.Mu Gloves (gloves lined with tension resisters to sense the tension created by the flex of the fingers and an accelerometer for measuring speed of hand movement) and the software design specially for the gloves (Glover), it was clear that the software recognised the gestures/hand postures from the gloves much clearer than the Leap Motion did with the same software, which is to be expected. Additional, the Glover software has many different gestures/hand postures that can be recognised. The Geco software on the other hand (which was designed specially for the Leap) can only read two postures (open and closed hand) and recognises the much clear than the glover software, again this is to be expected. The information to extract from this example should be that the Leap Motion is very the early stages of it’s capability and is not flexible for expressive audio, especially when compared to the Mi-Mu Gloves in a similar application. 

When thinking of how these technologies could be implemented in media the issues I outlined seemed to creep in a limit any greater use, but thats to say there isn’t hope. The future of gesture technology can only improve with time, David Gear wrote in the journal Computer vol.36  “Methods, which are designed to cope with some degree of uncertainty, more accurately predict the likelihood that a motion is the intended gesture, despite challenges created by such factors as lighting and background.” p. 21 . this means that, the computer algorithms are becoming better at distinguishing between an un-intentional motion of a hand and a purposeful gesture. The precision at which the computer can interpret the intentions of the user, will greatly improve the users experience.

Conclusion

The use of gesture technology in that arts and in media should be further developed to appoint at which the computer, hardware and software can easily distinguish between the gestures. until then; the use of gesture technology will be limited to a few gestures. 
The Mi.Mu gloves are evidence of the power of gesture technology. They also hold a key idea that will make the tech common place, this being: the sensors located and attached to the human body. Although, for the tech to become common place, a compound of Computer Science, Media, Fashion, Art, Music (and many more) will have to take shape and form. With the mixed disciplines working together, an interactive world can be created, a world where the technology is at our finger tips and universally present.

1. Geer, D. 'Will Gesture Recognition Technology Point the Way?', Computer, vol. 37/no. 10, (2004), pp 20-23

2. Kanga, Zubin. 'Gesture-Technology Interactions in Contemporary Music', Contemporary Music Review, vol. 35/no. 4-5, (2016), pp. 375-378.

[prtcl-collective-blog]

Audio manipulation and Spatialisation. Louis Sterling

Taking on a generative approach towards a sonic performance can be limiting in the premise of working with softwares and the loudspeaker. When working in stereo within a given space, it is undoubtably important to consider the acoustic confinements of the room in question; be it direction or positioning of the speakers, to the reflections of materials surrounding the intended work or curation. For PRTCL’s interactive work, it is important for us to maintain an organic engagement between the technology and the user or audience, as an interaction between the electronic soundscapes and physical gestural engagement should - in itself - be a fluent process. Technological complexes have indeed been the centre-point of our practise, however we have faced the issues of troubleshooting and software-juggling, which (at this point in time) has proved to be challenging. 

Throughout todays session, we have creatively programmed several .adg files encapsulating several audio components : Specialisation, granular/looping manipulations, delays, reverbs and equalisation. These are to be triggered through external MIDI messages sent via. Mi.Mu’s Glover program to Ableton Live, which is the main gestural component to be used throughout our performance. The live set itself is comprised of a multi-channel setup within Ableton Lives session view, which is highly flexible for arranging and drafting MIDI sequences, as well as creating unique audio effects. 

We understood that it would be important to complement the generative set with a series of effects that would not deter the listeners interest too far away from the sound, as by processing the audio too much - it could potentially disrupt the mediative and reflective embodiments edged from the Kaihōgyō practice. 

[prtcl-collective-blog]

Generative Music - by Tarek El Goraicy

What is Generative Music?

Born out of apathy towards the old and a necessity towards a non-linear narrative, Generative Music is the logical answer to the boredom a generation reached in this age of information.

It brings an element of unpredictability into the creative process of the composer and opens doors that not so long ago were not even taken into consideration; quite the contrary: one would imagine that the 18th Century Beethovens and Chopins would turn in their graves at the thought of a composition not being in the composer’s absolute control. However, with the rapid advancement of technology, Generative Music is an inevitable step in the ever-growing ladder that is the media landscape of the 21st Century.

Why Generative Music?

When one has just picked up a long awaited new record, an instant feeling of excitement and anticipation overwhelms the mind and even-more-so when the music tops one’s expectations.

But with every listen, the record loses its spark and eventually becomes just another chapter in the ever-expanding book of the music collector. Generative Music attempts to break the tangibility that fully arranged compositions have. It will never sound exactly the same again—forever suspended in time. Greek philosophers have spent their lives investigating the concept of flux—the constant flow and change of life, the eternal motion. “Into the same rivers we step and do not step, we are and are not”. (Heraclitus of Ephesus)

PRTCL Collective Live Set

With the concept of Generative Music in thought, we have just completed the first version of our live set. Each one of our members contributed to creating sounds that we have compiled in Ableton Live. Based on travel theme, we have used field recordings from various forms of transportation and turned them into textural sounds. Some ambient melodic and percussive elements have also been added to the set. With the help of Ableton Live’s follow actions, we have set up the project to always trigger a random sample once you press play with a maximum of 5 different sounds playing at once. This way, the sequence/arrangement of the composition is always randomised.

[prtcl-collective-blog]

The Sound of Kaihōgyō                Matt Were

The sound for this project should be that in which reflects the state of mediation via walking, coined by the Buddhist monks as, Kaihōgyō. With that in mind we set out making our own samples that we thought interpreted this state of mind. Through much trail and error a couple of techniques shone through to the group, that presented themselves well for the task in hand. 

The first of which was, instrument hacking, this is the idea of using instruments in a way in which they were not intended to be used. We experimented with taping a SM57 to a guitar tuned to C, then proceeded to play it with a Violin bow  the signal going into the DAW was then  processed with effects such as granular synthesis, reverb and echo, making the sound that came out, unique and what we thought would fit our criteria well. 

The second idea was to creating soundscapes of different environments that peoples travel in, we initially had the idea of recoding the entirety of a journey from Brighton to the Tate Modern, but soon realised that would be an inefficient way of working. Elaborating on the idea of soundscapes while traveling, we thought we could take a modern twist on this aged practice. Recording sounds that are constantly around you whilst traveling, that one become’s desensitised to. Blending all these sound scapes together and putting them through a granular synthesiser’s with filters and such also effecting the sound, which resulted in some strange soundscape’s which we thought worked well. 

The idea of adding binaural beats to the project was something that interested us, the connection between two frequencies being played close to each other was something that linked directly into Chakra healing practices, Tibetan Singing Bowls are the earliest example that we found of binaural beats being used for Chakra healing purposes, the frequencies that are supposedly the holders of these healing properties are called, Solfeggio Frequencies, there are six of them, and if all played in harmony it was suppose to, “infuse great spiritual blessings and transformation”. Solfego frequencies were founded in ancient Christian times, were the some of the first Christian chants were sung in this scale. We found that a clean sine wave playing at 60hz with another sine wave playing at a two frequency of set was the most suitable for use in our scenario, it invoked peace and calm, ideal for a mediative state.

[prtcl-collective-blog]

The Concept                                           Jedd Winterburn

Deciding on the intention of a piece is the most crucial task we can complete. As working with advanced technologies and dynamic installation techniques will create impressive results, a powerful conceptual approach grants the work substance that stands to achieve a greater potential and sculpt what is uncut. A group working together can either be a fertile ground to grow ideas or an abrasive arena to refine what is put forward, we found a bit of both as we began prying material out of our minds, through heavily advanced methods such as mind mapping and word association.

We settled on something we all had experienced but could not quite put a name too, which stood out as a rather interesting oddity. The context is set when travelling long distances either by foot or other means, something I am sure most people have experienced at some point if not day to day. Whilst mind becomes lost between places a transition begins in thought where one enters a reflective, meditative state, immediate consciousness evaporating while the body is occupied by the auto pilot. It is this state, this brief glimpse into higher thought that we wanted to inspire and explore. Little did we know that it had been heavily documented and highly sort.

Whilst there remains no English word for this happening, we found the earliest reference is to a Buddhist meditation practise called kaihōgyō which dates back 1000 years. Kaihōgyō directly translating to Circling the mountain, is sort of a form of pilgrimage, a path taken around Mount Hiei on a 1000 day travel often seeking enlightenment through the act of journeying which takes 7 years and a total distance almost equal to the circumference of the world. In a less official reckoning, this meditative state is recorded by many authors, creatives and what must be boundless more unrecorded reflections.

"I write not with the hand alone, My foot would write, my foot that capers, Firm, free and bold, it's marching on, Now through the fields, now through the papers." Friedrich Nietzsche.

"Me thinks that the moment my legs begin to move, my thoughts begin to flow" Henry David Thoreau.

"I travelled on foot and I travelled alone...my pleasing chimeras kept me company, and never did my heated imagination give birth to any that were more magnificent.” Jean-Jacques Rousseau.

And a more modern account;

“An author who composes while walking, on the other hand, is free from such bonds; this thought is not the slave of other volumes, not swollen with verification, nor weighted with the thought of others. It contains no explanation owed to anyone: just thought, judgement, decision. It is thought born of a movement, an impulse. In it, we can feel the body’s elasticity…the result will not be long and meticulous exegesis, but thoughts that are light and profound" Frédéric Gros.

The fact that our concept is yet to be coined by terminology promotes the notion that communication in the abstract, through emotional response, will be the most effective. Musically provoking a similar experience is a well partnered choice of medium, as sound and music are so commonly noted to acquire the features of motion and movement. A piece of music is highly compared to that of a journey, or narrative, predominantly due to the fact it is a medium based in time which evolves, grows and ages. It is a most interesting property that lends itself well to our needs. At this point we begin to collect our artistic material and look at how we can package this into an interactive installation.

[prtcl-collective-blog]

In the Beginning...

The PRTCL Collective is a group project consisting of five members. We aim to create an interactive sound installation, implementing gesture-based controls to blur the lines between audience and performer.

Our interests lie within the immersive experience, with a goal to deliver something that is unique, a rarity that grants the work a magic quality. This Tumblr has the purpose of documenting our progress exploring techniques and developing our concept for the first production by the PRTCL Collective.