GENERATIVE MUSIC

THE UPDATES BLOG

[generativemusic]

WE WORK ON PAPER TOO

You know, just in case you were wondering. You weren’t? Well that’s fine I guess...

[generativemusic]

🚨🚨Video update 🚨🚨

First, watch the video. Things have changed since the last video posted here.

Let’s start with the obvious. The sounds have been changed, and also the names of the balls, which are 4 and there is no main/most important one, like in the other video. 

Another, maybe less obvious change is the keyboard that’s at the bottom. Although it may look similar, it works in a different way. While previously, each key was assigned a fixed note, now an appropriate note is generated in realtime in cases of both bounce or movement along the keyboard. The “soprano” ball is the master ball, meaning that when the other balls bounce, or move, the notes that they generate are notes from a chord that will harmonise with the soprano. The key is D major. By the way, the term “keyboard” isn’t really the best way to call the thing at the bottom, so we are going with the term “triggers”, even though i can imagine that “keyboard” made it easier to understand. Feel free to send us suggestions.

Last but definitely not least, is that the sounds now sound more like a melody (chords are created when the balls bounce at the same time on the triggers). This is the new auto mode that (i think) we talked about in previous post(s). The auto mode is what generates a proper melody, which allows operators to fully focus on the users rather than having to split their attention between making the users play the background/base, and playing the melody over what the user is playing. The auto mode can still be switched off, so the melody can be played manually over it.

[generativemusic]

Did you miss us?

We are back!! Now, can’t just have software, can we? So we created a support for the popular buttons typically used for people with motor impairments. We made them (the blue bit in the picture below) on Tinkercad, a super simple online 3D modelling tool, and later 3D printed them out. The support is used to anchor the button to the clamp, which is much cheaper and versatile than other adjustable arms (like the MagicArm by Manfrotto). The clamp will be used to fix the sensors to the exposed tubes of a wheelchair. We used this specific example to place the buttons on the vertical tubes on the sides of the headrest, allowing for the buttons to be pushed with sideways movements of the head, which were the only useable movements that the person that we tested on could do.

[generativemusic]

2

Greetings to you all. Make yourself a cup of tea or coffee, or any drink you like. You don’t actually have to drink, but you do need to get comfortable and ready to read this update post, because round 2 of testing has been done, with a different user. The session went great and the users are finding the whole system rather simple and clear to use, which is great for us, and it means that the interaction part of the system doen’t need to change that much. We will most likely stick with buttons or capacitive buttons. In some cases, users may not necessarily be able to actually press the buttons, so other controllers, for example a breath controller would be “custom made” for them, however it would only act as a replacement to the button, and not an extra controller to add to the system.

The photo above shows the setup used in the tests. There are 3 buttons; the red one on the right, the green one on the left, and the pink and yellow one on the right. There is a speaker on the right, in red.

We found out that a scenario could be played for 15-20 minutes before the user gets a bit bored, so creating at least 3 others, can allow sessions to be longer, which will greatly benefit the user.

An important part of the project is having group sessions. What we discovered during these tests is that the operator is quite busy with making the musical melody that goes over the background created by the user. Since he or she has to also “conduct” what the user plays with the buttons, it makes it more difficult for them if they cannot fully focus on that. On an individual basis it can still be doable, and allows for more freedom in playing the melody, however, if the sessions were done in groups, an automatic mode would have to be created to allow the operators to fully focus on the users rather than the music. This is already a hard task for the operator because every user has a different kind of motor disability.

[generativemusic]

Visit Original Link

Well, good evening to you all. Or afternoon or morning, depending on where you are reading this. So there’s another update so get comfortable and here we go. We have successfully started the first testing phase of the project. We found a person who was willing to try it out, and as a first trial it went very well. It was of extreme value because it helped us identify some things that will have to be changed. You can hear the track created by the user during testing by pressing play above.

First of all, we want to encourage the users to always be focused on interacting with the music. We want to do this by muting the sound coming off of the balls that are resting on the keyboard. This should help us avoid scenarios where a user asks him or herself “Why should i keep interacting, when there’s still sound?” Obviously, it wouldn’t be a harsh stop of the sounds, but more of a gradual one, where the sounds of each ball mutes one by one.

Let’s also point out that the setup as it is is like just one track off an album. Other tracks would be created using different sounds and visuals. We call these tracks scenarios, so right now there’s one scenario which is visually represented by the bouncing balls.

[generativemusic]

Update Time 🙌

Sooo, how’ve you been? Great! Back to us now. We are now fully working on Ubuntu (Linux) now, as it works best with no glitches like we had with Windows every now and then.

The keyboard at the bottom (where the balls bounce) has been changed, and rather than a pentatonic scale, each key is now a note of a particular chord.

We plugged in a capacitive shield from Adafruit (here pictured below).

This allows us to control the wind direction just by tapping on the cable end that matches the direction you wish. Whats really cool is that you can connect or insert the cable into pretty much anything as long as it is electrically conductive. As you can see from the first picture you can plug it into quite odd things like fruit. The great advantage is that it allows us to create unique buttons that work just by tapping them, and they can also be easily placed more or less anywhere, which is great in the world of accessibility. Below is a picture of the current setup. The keyboard is there to play some sort or melody over the base sounds created by the balls. The physical keyboard is intended to be used by an operator/therapist, not a patient.

Lastly, the auto mode will be changed. In the first video posted (here) auto mode controlled all directions of the wind, so there was no input from any user. What we want, is the possibility of auto mode to be activated for just a few selected directions, like upwards and left, that would allow the user to control the wind direction to the right.

[generativemusic]

Guess who is also running on Ubuntu? That’s right, we are. 🎉

[generativemusic]

MIDI filter - Part 1

Using a past idea of a MIDI filter (Read more here, in italian)

With that, notes that are played in real time by an operator can be passed through a keyboard within the Processing sketch, which through a particular MIDI filter will be able to select the accompaniment notes (chords) to address the balls with melodic sounds: chords made up of 3 notes, intended for 3 balls.

It’s very likely that the operator on the keyboard communicates the chords to Processing, the notes that will be distributed to the 3 balls, playing with his or her left hand. The right hand can keep on playing melodic sounds accurately chosen in LMMS. What this means is that the sounds you will hear from keyboard are the notes of the melody played with the right hand, while the left hand sounds will only be heard thru the balls.

[generativemusic]

YAYY AN UPDATE!

Time for another update y’all! Right, let’s get straight to it:

Firstly a new ball has been added. It is the one labeled “pad” in the upper left corner of the photo. This means that we now have 3 balls that are combined to matching melodic sounds. An algorithm could be defined that makes the 3 “voices” intertwine around certain pre-defined patterns. A Markov Model could be an example of such algorithm. (read more about the Marvok Model here)

The hold/one shot option has been added for the manual controls, in order to avoid the prolonged use of a command: left, right, up

Something to keep in consideration is the possibility of using the Ubuntu operating system in order to have better stability and lagginess. The latter parameter is crucial for real time execution of the accompaniment by a therapist/operator.

[generativemusic]

This is the first “Nature of code” style Processing sketch. The noise ball triggers MIDI when it bounces on the floor, which is made of some kind of pentatonic keyboard. The pink “noise” ball will always create a sound of a percussive character, that will always be audible. The other blue balls generate sounds with a slower starting rates, which makes their sounds only audible when they rest on the floor. All of the balls are affected by wind, which makes them move and bounce around, creating an always evolving song. Wind is mainly changed by the user, however an auto-mode can also be turned on. 

Synth sounds are played by hand on top of the soundscape the balls create. The melody and harmonies will be automated within the source code, and some parameters will be visually layed out on an interface with buttons, sliders, and other controllers.

All synth sounds come from an LMMS software template built upon various istances of ZynAddSubFX softsynth.

[generativemusic]

The inspiration for the first sketch came from Daniel Shiffman’s book “The Nature of Code”, and to trigger MIDI events, some kind of natural environment has been decided on. People with special needs interact with the program with simple and intuitive actions, like changing wind direction. These commands can be controlled by various sensors connected to an Arduino board. Some very interesting evolving soundscapes are generated through this sketch, that can act in the background of a more traditional instrument like a keyboard or a guitar.

Video coming soon

[generativemusic]

Hello world