BetaController

Project for controlling midi with web interfaces

[betacontroller]

Phase 3: Transport on an iPhone

Pulling all these lessons together, it's time to make an interface!

The DAW I've been using partly for cost reasons, but mostly because I don't like DAW's enough to spend $$$$$ on a new one is Cakewalk Sonar X1.

In Sonar, you have the ability to set up custom mappings for midi interfaces, using a series of notes and controller messages to control core DAW functions.

I started with the basic idea that it'd be SUPER HANDY to have transport controls (play, stop, record, fast forward, rewind) for recording. I've started a recording project for my friend Andy Quitmeyer's PhD project where I had to lay down drums without a recording assistant. Having a little iPhone transport control was sooo helpful, eliminating the RECORD - RUN BEHIND THE DRUMS - PUT ON HEADPHONES - PLAY DRUMS - DROP A STICK - STAND UP - STOP THE RECORDING - UNDO THE RECORDING - AND REPEAT cycle of endless frustration.

For this design, I'm using Entypo web fonts and some simple little buttons. Here's a screen grab of the transport from my iPhone 4s:

Simple enough!

I mapped out the notes in Sonar, and keep a reference of the the transport controls and their note mappings in my Node code:

var map = { methods: { noteOff: 8, noteOn: 9, keyPressure: "A", controller: "B", programChange: "C", channelPressure: "D", pitch: "E" }, transport: { "tostart": { pitch: 63, velocity: 127, channel: 1 }, "toend": { pitch: 64, velocity: 127, channel: 1 }, "rewind": { pitch: 65, velocity: 127, channel: 1 }, "fastforward": { pitch: 66, velocity: 127, channel: 1 }, "play": { pitch: 60, velocity: 127, channel: 1 }, "stop": { pitch: 61, velocity: 127, channel: 1 }, "record": { pitch: 62, velocity: 127, channel: 1 }, "test": { pitch: 60, velocity: 127, channel: 1 } } }

By keeping this reference object, I'm able to quickly keep track of what human-language commands are mapped to what midi controls. For the transport, all commands are mapped to midi notes.

In the browser, I'm sending tiny socket messages with this human-readable code as a message

//an example button html: <button class="play">▶</button> //button event handlers $("button").on(eventstring(), function(e){ if(e.type.indexOf("start") > -1 || e.type.indexOf("down") > -1){ //shows a visual highlight when you tap the button $(this).addClass("active"); }else{ //removes the highlight $(this).removeClass("active"); //calls the send message function sendMessage("transport", $(this).attr("class")); } }) //and here's the function to send out the socket message var sendMessage = function(message, data){ socket.emit(message, data); }

Now on the server, I wanted to separate the messages for transport controls (navigating the timeline) vs. channel navigation (finding your tracks) vs. channel conducting ('playing' levels, pans, effects). I set up a socket listener exclusively for transport which uses the functions outlined in my previous post.

socket.on("transport", function(data){ var messages = noteOn(map.transport[data].pitch, map.transport[data].velocity, map.transport[data].channel); messages = messages.concat( noteOff(map.transport[data].pitch, map.transport[data].channel) ) serialSender.sendMessages(messages); });

And there you have it, instantly useful and not so terribly complex :)

[betacontroller]

Phase 2.2: Playing Middle C with Node.js

I'm using Node.js for this project, partly because I am already familiar with it, but mostly because it offers a ton of awesome tools for building realtime web apps.

To play middle C, I used a few modules I've found handy in my hacking adventures. Express.js, serialport, and socket.io are the big three.

The Midi of Middle C

Midi messages are communicated in bytes over serial. For my first attempt at meaningful communication, I wanted to play a middle C. The Note On command in midi is comprised of three bytes:

The first byte stores information about the command type (in this case, note on) and the midi channel number (1-16)

The second byte stores information about the pitch. In this case, middle C is pitch #60.

The third byte stores information about velocity- let's rock it full out at 127.

Basic Binary in Node.js

In Node.js, there is a core object called a Buffer() which is designed to handle binary transactions. JavaScript is excellent at handling normal text and strings, but falls way short when it comes to raw data. Using JavaScript on the server, the Node.js developers found they needed a proper way to handle file transfers and other such binary transfers.. hence: Buffer.

The Buffer can be initialized with a fixed number of Bytes, which I will use for this example since we know exactly how many bytes are required to describe a Note On command (3).

The trickiest part of this command is the first byte. The first 4 bits represent the command (note on), and the second 4 bits represent the channel number (0-15 in computer, 1-16 in human). Together, this equals some decimal number that is not very easy to intuit.

To construct this first byte, here's what I did.

First, I referenced the and this excellent writeup from Stanford to reference the values I'd need to construct a Note On.

Using this chunk of code:

function Dec2Bin(n){ return String("0000").substring(0, 4-n.toString(2).length)+n.toString(2) } function Hex2Bin(n){ return String("0000").substring(0, 4-parseInt(n,16).toString(2).length)+parseInt(n,16).toString(2).toString(2) }

I am able to convert values that I understand (0x9 for note on, and 0-15 in decimal for the channel number) to a single binary value of "10010000". After I've constructed the binary, I use:

function Bin2Dec(n){ return parseInt(n,2).toString(10) }

to convert the binary back to decimal. In this case, for Note On of pitch #60, at velocity of 127, the decimal value is 144.

I wrote a noteOn function to take a pitch and velocity and construct an array of decimal values, using the functions listed above.

function noteOn(pitch, velocity, channel){ var bytes = []; bytes.push( Hex2Bin(0x9)+Dec2Bin(channel-1) ); //noteOn + channel byte bytes[0] = Bin2Dec(bytes[0]); //convert the binary to decimal bytes.push( pitch ); //note byte bytes.push( velocity ); //velocity byte return bytes; }

Now that we have these 3 meaningful byte values, let's add them to a buffer and output it over the serial.

var SerialPort = require("serialport"); //import serialport //set a var for it, and our default baudrate var serialport; var serialoptions = { baudRate: 38400 } //initialize the serial port serialPort = new SerialPort.SerialPort("/dev/ttyAMA0", serialoptions); //create a placeholder function, bind it to a serial action once we have a connection open var writeMessages = function (messages, callback) { /* placeholder */ }; //this gets called when the serial port is opened up serialPort.on("open", function () { //now, redefine the function to send messages over serial writeMessages = function(messages, callback){ var message = new Buffer(messages.length); //initialize our buffer for(var i=0; i<messages.length; i++){ message[i] = messages[i]; //now we write the bytes into the buffer } serialPort.write(message, function(err, results) { if(err){ console.log(err); }else{ console.log("success!"); } });

Structurally, I have a midicontroller.js module and a serialsender.js module, which separates the concerns of parsing/constructing the midi messages, and the sending of those midi messages over serial. So a message comes in via web sockets, midicontroller creates my byte array, and then serialsender packages it up as a binary and sends it out over AMA0.

Once I've cleaned this guy up a little bit and abstracted everything properly, I will post my foundation code on my GitHub.

[betacontroller]

Phase 2.1: Talking Sense over Serial

Following a series of copy-paste hacks scraped from various websites and forums, I was left frustrated and confused. Everything seemed to be set up properly, from baud rate through Node.js buffer code through the wiring.. yet I was still receiving nonsense messages into my audio interface.

After several days and rewrites, I was on the verge of giving up.. which is when most excellent breakthroughs happen. Here are the steps I took to effectively communicate Midi messages over the UART serial on the Raspberry Pi. 

This is a revised workflow from my previous post regarding Raspberry Pi setup, but has an important addition that made all the difference.

First, I edited /boot/cmdline.txt on the Pi:

sudo nano /boot/cmdline.txt

replace contents with:

dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 bcm2708.uart_clock=3000000 elevator=deadline rootwait

This enables the AMA0 serial port on the Pi to be used by Node. The important change here vs. my previous attempt is initializing the uart_clock to 3000000. I honestly have no idea what this is for, but thanks to this guy and his writeup for giving me something to try!

Second, I added some lines to the end of /boot/config.txt :

sudo nano /boot/config.txt

Add lines to end: 

# change uart clock to 2441406 for midi 31250 baud rate init_uart_clock=2441406 init_uart_baud=38400

This slows down the Pi serial baud rate to approximately 31250 if a serialport is opened at 38400 baud rate (one of the standard baud rates supported). Note that only after initializing the UART clock, this was successful.

Third, I modified write access to the AMA0 port to allow Node.js to use it

chown pi /dev/ttyAMA0

chmod a+rw /dev/ttyAMA0

Next up, talking midi with Node.js!

[betacontroller]

Phase 2: Talking Nonsense over Serial

This has been an involved process, and I still have not solved the core of the problem, but I've learned some stuff. Here goes.

I've set up a tiny Node.js app to test Midi output via a web interface. The front-end is a few buttons that send socket messages via Socket.io to the Node app.

Upon receiving these socket messages, the Node app will ideally translate the intended result as defined in client-side code to meaningful Midi messages out over serial.

Here are the issues I've run into:

I can get midi messages sent out from the Pi, but the messages received and logged by my MOTU audio interface are NOT what I intend. 

I do not understand binary code, or converting binary, hex, and decimal numbers in a really meaningful way. I can definitely hack things together, but hacking with intent is the next big step.

I do not understand the low-level C++ code used in the Node serialport module enough to tell if the protocols being used are correct.

The process I'm using encodes a NoteOn message into the values specified by the Midi standard. For a Note On event, there are 3 bytes sent. The first byte specifies the command (note on) and the channel # (1-16), the second byte specifies the pitch (0-127) and the third byte specifies velocity (again 0-127).

I am writing these three values into a Node buffer and writing them out over serial. 

What I see in Midi Ox (on my PC, screen cap below) are three separate midi messages, none of which is a note on, and all of which have the wrong channel number. 

There is obviously some extra data getting inserted somewhere between Node and the serial port, I will continue to investigate.

[betacontroller]

Phase 1: Configure the Pi for Midi

I chose the Raspberry Pi as my primary communication mechanism for a couple reasons. First, I am already familiar with the device having used it on a couple hack projects this year. 

Second, after attempting to use the Pi with an Arduino to relay web events over serial and output Midi messages from the Arduino, I discovered a translation issue with differing baud rates between the devices. At this point, I chose to pair down my attempts and use only the Raspberry Pi.

Setting up the Pi to send midi messages is a little involved, but not over complicated. I referenced an excellent post by Silicon Stuff to hack the serial baud rate close to 31250 (Midi spec).

First, I hooked the Midi cable up to the Pi. 

Pin 1 (5v) goes through a 220 ohm resistor to Pin 5 of the Midi cable.

Pin 5 (Ground) goes to the ground Pin 2 of the Midi cable.

Pin 7 (TX - serial transmit) goes to Pin 4 of the Midi cable.

Second, I edited /boot/cmdline.txt on the Pi:

sudo nano /boot/cmdline.txt

replace contents with:

dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait

This enables the AMA0 serial port on the Pi to be used by Node.

Third, I added some lines to the end of /boot/config.txt :

sudo nano /boot/config.txt

Add lines to end: 

# change uart clock to 2441406 for midi 31250 baud rate init_uart_clock=2441406 init_uart_baud=38400

This slows down the Pi serial baud rate to approximately 31250 if a serialport is opened at 38400 baud rate (one of the standard baud rates supported).

Fourth, I modified write access to the AMA0 port to allow Node.js to use it

chown pi /dev/ttyAMA0

chmod a+rw /dev/ttyAMA0

Now, the Pi is able to communicate at the speed that Midi requires.

[betacontroller]

Project Overview

My goal for this project is to create an open-source platform that allows ad-hoc web interfaces to communicate with a Digital Audio Workstation via Midi.

The tech I’m using is:

- Raspberry Pi with Wheezy Distro

- Node.js, Socket.io, Express.js

- HTML5, JavaScript, CSS

- Midi Cables

With the base platform, it will be possible to create instruments and midi controllers using devices I already own- including (but not limited to) iPhones, iPads, Laptops, Leap Motion, etc. 

All devices will interface directly with the Raspberry Pi via a broadcasted Wifi network and send midi messages out via serial (using Midi cables for v1).

I will update with source code, problems and progress as I go!