Creative Developer Notes by M.Romeo

This blog contains notes about working creatively with tech equipment and software for interactive installations. I work professionally in this field and documenting and distributing these types of notes is very helpful to myself and others in my field. I do not come from a computer science background. I learned how to work with computers through creating and performing music live as well as producing personal art projects. This blog is meant to help people who have a strong interest in making creative projects with tech. I describe things to the best of my knowledge and as simple as I can.

[nycinteractiveinstallation]

Github Submodules

When adding code to a git repo you might be referencing some addons that you’re keep locally. In my case I'm working with addons specifically made to work with openframeworks.

Git users are constantly updating their projects with different versions and branches so it can be hard to remember exactly which branch you were using if you go back to try an build your code at a later date.

Submodules are very useful for this purpose. You can basically add a reference to other git repos within your own git repo. The code won't actually live within your repo but it's the reference for the git repo will be there for you to iniate later in time.

The following steps show how to do so:

Create Submodules:

Go to directory of_v0.9.8_osx_release/apps/myNewapp

git submodule add https://github.com/kylemcdonald/ofxCv.git addons/ofxCv

Now you should see a folder you just created with the code from the repo inside!

Install Submodules from fresh Clone:

Go to directory you want clone git clone myReposCloneAddress

Go to the base directory of the repo myReposCloneAddress

Now while in the base directory:

git submodule init

git submodule update

You should now have all your addon folders filled with the code from the repos you made into submodules in the previous steps.

[nycinteractiveinstallation]

Messaging between Python and C++ using ZeroMQ

I was recently introduced to ZeroMQ it is a distributed messaging system that allows developers to connect code in any language. In this example we will connect a simple Python program with a simple C++ program. We will be doing this in MacOS. We'll start by creating the 'client' application in python by doing the following:

Create client app in Python

Open the 'Terminal' app and type

* pip install pyzmq 2. Now navigate to your 'Desktop' and type * touch client.py 3. Now open 'client.py' in a text editor or in my case PyCharm IDE 4. Type the following code into the document

import zmq context = zmq.Context() socket = context.socket(zmq.REQ) port = "5555" socket.connect ("tcp://localhost:%s" % port) for i in range (1,10): socket.send ("saying hello from python") message = socket.recv() print "Received reply from server:", message

Now save this document and back to 'Terminal' navigate to your 'Desktop' (where you made the client.py file) and run the code by typing:

* python client.py

Create server app in C++

In my case I followed a great tutorial here but I will explain the same in the following steps.

Open up 'Terminal' and type (note: you will need Homebrew) brew install --universal zeromq

Now open up Xcode and create a command line program.

In Xcode update your 'Header Search Paths' with /usr/local/include

In Xcode update your 'Library Search Paths' with /usr/local/lib

In Xcode update you 'Other Linker Flags' with -lzmq

In your xcode project change the 'main.cpp' to 'hwswerver.cpp' and type the following code into that file:

#include #include #include int main() { // Prepare our context and socket zmq::context_t context(1); zmq::socket_t socket(context, ZMQ_REP); socket.bind("tcp://*:5555"); // forever loop while (true) { zmq::message_t request; // Wait for next request from client socket.recv(&request); std::string replyMessage = std::string(static_cast(request.data()), request.size()); // std::string replyMessage = std::string((request.data())., request.size()); // Print out received message std::cout << "Received from client: " + replyMessage << std::endl; // See the gradual sending/replying from client sleep(1); // Send reply back to client std::string msgToClient("greeting from C++"); zmq::message_t reply(msgToClient.size()); memcpy((void *) reply.data(), (msgToClient.c_str()), msgToClient.size()); socket.send(reply); } return 0; }

Build and Run your C++ program and you should see the following messages in your terminal window that is running your 'client.py'

* Received reply from server: greeting from C++ 9. Note: if you're not able to build make sure to revisit steps 4,5,6 10. Note: if you don't receive messages in your client.py app make sure you have the correct PORT number 5555 between both programs

[nycinteractiveinstallation]

Automating Mouse Clicks and Key Presses

Using a handy command-line application called Cliclick you can automate keypresses and clicks. This is a useful tool for permanent interactive installations. For example, you could use it to click off dialog boxes on startup, or click on specific windows that need to be brought to the front most application, etc. Here is a quick demo to get started.

For Mac users you'll need to install Homebrew.

After you have installed this tool you can quickly and easily download and install other handy command-line applications in this case Cliclick.

Open up Terminal, type the following brew install cliclick

You should be able to run the command cliclick -h and you'll be given a list of all the commands you can use with this application.

Now while still in terminal navigate to an appropriate directory in my case the desktop will do. Run the command touch myNewCliclickApp

Run the command chmod u+x myNewClickApp to make that file you just created an executable.

Open up and text editor in my case Sublime Text and open the file you just created and write the following in that document:

``` #!/bin/sh

echo "Cliclick is clicking and writing on your page!"

cliclick c:640,320;

cliclick t:"Hello I'm your computer writing by myself in your text editor"

cliclick kp:enter

cliclick t:"Nothing weird about this at all"

``` and now 'Save'

IMPORTANT NOTE: Before running this script open up a blank 'Sublime Text' text document and maximize it's window. What this script will do when you run it is first click on the center of your page i.e. my screen resolution is 1280 X 640 / 2 = 640 X 320

Now in terminal you can run the command ./myNewCliclickApp and it will run the script you just wrote.

It should work something like this

[nycinteractiveinstallation]

Read JSON files Unity

using UnityEngine; using System.Collections; using System.IO;

public class ReadJson : MonoBehaviour {

private LitJson.JsonData itemData; private string jsonString; void Start () { jsonString = File.ReadAllText (Application.dataPath + "/JSON_Example.json"); itemData = new LitJson.JsonData (); itemData = LitJson.JsonMapper.ToObject (jsonString); //Debug.Log (itemData ["Weapons"] [1] ["name"]); Debug.Log("name: " + itemData["name"]); Debug.Log("price: " + itemData["price"]); Debug.Log("tags: " + itemData["tags"]); } // Update is called once per frame void Update () { }

}

[nycinteractiveinstallation]

Multi-Remote Application Open

In this entry I’m going to describe how to remotely open an application for multiple computers on the same local network. In order to do this you will need to utilize a few different tools called SSH Pass and SSH Sudo. This is specifically for Mac OSX. To do this for windows it is a much more difficult approach using Powershell (as far as I know).

Install SSH Pass: This is required in order to utilize SSH Sudo which is the main script you will need in order for this to work. So go ahead and open up terminal and type the following:

(note: You will need Homebrew for Mac OSX)

brew install https://raw.githubusercontent.com/kadwanev/bigboybrew/master/Library/Formula/sshpass.rb

Download SSH Sudo: This is the main tool that will allow us to send a command, enter our password once, then have that command automate without having to enter the password again on multiple computers on the same network. Note: For each computer you want to run a command remotely on, they will require the same username and password for this script to execute. I will go through that setup later in this entry.

You'll first need to download the sshsudo script.

Now, place this somewhere accessible. Example:

~/Desktop/ShellScripts/sshsudo

Now that you have downloaded the file you need to make the script executable by running the following command in terminal. chmod +x ~/Desktop/ShellScripts/sshsudo

3.Setup Computers: As I said previously, we will only be using Apple computers for this so that is what I will be describing.

Go to System Preferences >> Users and Accounts.

Unlock to make changes using your admin credentials.

Click on the + sign to add a new user.

It is very important that all your computers have the same username and password, also they should all have admin access, which is just a check box when creating a new user.

Now that you have created a new user open System Preferences >> Network and write down your IP address. You'll need this to access the computer later.

Make sure every computer you set up is connected to the same Wifi or LAN network.

Place an application or file on your ~/Desktop/SomeFileToTest.app

4.Run Terminal Commands across Multiple Computers: Now that you have installed SSHPass, SSHSudo, and set up all your computers on the same network with the same usernames and passwords it's time to run a command across all of them with ease. Do the following:

Create a file within the directory of your sshsudo file. Open up terminal and do the following: cd ~/Desktop/ShellScripts/ touch myAppRunner chmod u+x myAppRunner

Now you should have a file in the directory that you can run. If you double click on myAppRunner nothing will happen because we need to give it some code to work with. Open myAppRunner in a text editor like Sublime Text or Xcode.

Enter the following code in myAppRunner and click save. #!/bin/sh IP[0]=192.168.1.1 # Computer 1 IP[1]=192.168.1.2 # Computer 2 IP[2]=192.168.1.3 # Computer 3 ~/Desktop/ShellScripts/sshsudo -u myUsername ${IP[0]},${IP[1]},${IP[2]} open ./Desktop/SomeFileToTest.app

Now that you have entered and saved the previous code to your myAppRunner file you can go into terminal within the same directory as myAppRunner and type the following:

open myAppRunner

You will be prompted to enter the password on the first computer, but on each consecutive one sshsudo will automatically take care of it and run the command open ./Desktop/SomeFileToTest.app *note: another great way to test is change that last command to say "My computer speaks"

[nycinteractiveinstallation]

Automated Shell Scripts using Launchd

Using Terminal commands to navigate and control your computer can be very useful. Running those commands together in a shell script can be even more useful, but executing it manually in my opinion seems to defeat the purpose. In my case I'm running OSX 10.11 which includes Launchd. It loads on boot and allows you to specify programs to execute, in my case a simple shell script.

There are inifinite practical applications for using this technique, in my experience I've found it useful for:

Getting diagnostics from a remote computer.

Making certain applications run on boot for easy installation startup.

Checking status of pjlink or other hardware.

In the following steps I'll discuss how to create a simple shell script and a launchd plist that will automate that shell script based on a specified time. I won't however get too in detail about the actual terminal commands, but if you're a terminal beginner I highly recommend running through the Code Academy tutorials.

Create Shell Script

Open up Terminal and enter the following command to create a new file.

touch testingShell

Now enter the following command to open a built-in Terminal text-editor.

nano testingShell

You now should be viewing the nano text editor. This is where you will write all the lines of commands you want to execute in one single shell script file. Here is a simple example that when executed will write the date and your ip address into a text file. Write the following and once finished hit Ctrl + X to save.

#!/bin/sh

date >> ~/Desktop/testOutput.txt

ifconfig en0 >> ~/Desktop/testOutput.txt

Now that you've created and saved the shell script which contains some terminal commands, you are ready to run the following command that will give that file the ability to execute.

chmod u+x testingShell

Finally, run this command to execute your shell script. Once you do you will notice that the testingShell script will create a text file called testOutput.txt (contained in the same directory as your shell script) with the date and time you ran that script, as well as the ip address of your computer.

./testingShell

Create Launchd Plist

Just like we did with the shell script above, we now want to create a launchd plist which will give us the ability to execute our shell script routinely at a specified date and time. Go ahead and run this first command.

touch ~/Library/LaunchAgents/com.yourusername.testLaunch.plist

You will notice that the last command created a file with the extension .plist in the LaunchAgents directory of your computer. It is crucial that you keep your plist in that folder for the computer to know to execute it. You will also notice that the file you just created has no content so run the next command.

nano ~/Library/LaunchAgents/com.yourusername.testLaunch.plist

Once again we will use nano editor to insert content into our plist. I would recommend using another text editor like Sublime Text when doing this in the future, but for our purposes nano will do just fine. Now copy and paste this linked text into you com.yourusername.testLaunch.plist file. Once again, when finished hit Ctrl + X to save your file modification.

Lastly, run the next three lines consecutively everytime you modify and save your plist file, otherwise it will fail to update.

launchctl unload com.yourusername.testLaunch.plist

launchctl load com.yourusername.testLaunch.plist

launchctl start com.yourusername.testLaunch.plist

If all goes well you should see that at the specified time in the plist a text file with the date and time, as well as your computer's ip address will pop-up on your desktop! If you run into issues a helpful command is launchctl list Your plist should be listed and have a PID number. Usually a status means something went wrong but it will give you a number to search out what may be the problem.

[nycinteractiveinstallation]

Remote SSH Using LogMeIn Hamachi

Recently, I had been having trouble with my automated projector control (PJLink) code that I have built-in to an openFrameworks application running a permanent installation at the office of the company I work for. Just to roughly describe this installation, the software had to take incoming serial data from a physical sensor, run some generative sound code to headphones, as well as run real-time generative graphics, all while I needed to make sure the projector stayed automated with PJLink and connected via an ethernet connection.

With so many variables things tend to go wrong for a while until you can debug the issues over time. In this case it was pretty low pressure since it is sitting in our office so I've had a bit of time to work on it day-to-day, but that doesn't mean it shouldn't work flawlessy.

In this case I decided to use this installation as a good test case scenario for long-term installs. I setup some tools to make working remotely quick and easy. First off, I set up a live camera feed of the installation so I could quickly view it in a web browser locally, which I've described in a previous entry. But this wouldn't be useful unless I was able to be outside of the LAN to view this camera feed. In order to do so I would have to setup LogMeIn Hamachi. This service enables you to get a safe outside connection to the network running your installation. In my case it enabled me to do the following:

View a RaspberryPi2 Camera Feed of an installation outside my LAN.

Obtain SSH control over an installation outside my LAN.

How to set this up:

In my case my company has a LogMeIn Pro account so I started from there. If you don't want to sign up have a look at this tutorial but I can't speak from experience whether this works well.

Once you've successfully logged in go to Networks >> My Networks.

Next click on the Add Client button towards the top. If you are on the computer you want to be able to SSH into then choose Install LogMeIn Hamachi on this computer which will download the 'client software'. Once downloaded you will need to run through the installation process. If you want to remotely deploy the 'client software' click on that option which will send a safe link out to an email address and someone on the other end will need to go to that link and download software on their end. Note: you will need to do this on the computer you want to remote from as well.

Now that you've added a client computer (stay on the LogMeIn site) you'll want to add that client to a network so now click on Add Network button towards the top. Enter a Network Name, Network Description, and Network Type in my case I used a Mesh setup so all computers can see each other.

Now that you have created a Network You should see it appear within the My Networks tab from the previous screen. Click on the new network with the name you just created and you should see and edit tab to the far right of it's name. Within the edit screen of your newly created network there is a Network ID number at the top left of the screen. You will need this number in the next step.

For this next step open up the LogMeIn client software you just downloaded and installed on the computer you wish to have connection to. If you aren't prompted upon opening this application to 'Join a Network' then go to the top bar of the LogMeIn Hamachi program Network >> Join an Existing Network. Now enter the Network ID which we found in the previous step and Network Password which you entered when you created the network. Hit the Continue button. Now on the next window set Join Request Permissions, set a Network Password and set your Subsciption.

Once you have joined the network you will see all the other clients on that are online with a green status light. Right-click on any of the Networks and you'll have the option to Ping the client. In my case I'm on Mac OSX so my Terminal application opens and automatically starts pinging the receiving computer.

Once you've made sure that you have a connection you can now take that IP address you just were pinging and start a new terminal window and write in the following line:

ssh username@ipYoujustPinged

Enter a password if there is one. If there isn't just hit 'Enter'. Now you should be logged into the client computer. To make sure, double check that your command line now lists the remote computers username and not the computer that you're physically on.

[nycinteractiveinstallation]

Raspberry Pi 2 Cam Live-Stream for Installations

The Raspberry Pi is a very affordable computer that I am still figuring out useful ways to integrate into my professional work. Just as an example, it could sit inconspicuously in the corner at an install and be sending out OSC messages to multiple computers.

Recently I utilized the RPi Camera Board to be able to watch an installation from afar. Of course you could just set up a screen sharing client on a computer that is running your installation, but sometimes you may just want to take a quick glance just to make sure it's still running, see if any dialog boxes popped up, or even something as silly as a cursor is visible onscreen.

For this kind of setup there is a very useful git repo called RPi Cam Web Interface that allows you to do just this. This app let's you go straight to a url to view streaming video from you RPi camera. There is a really quick setup guide on the Elinux wiki.

Once I got the RPi Cam Web Interface streaming live and double checked that I could view it at a local IP in my web browser, it was time to figure out how to view the camera feed outside the network. This I will discuss in my next post but for now let me just run down a broad list of steps it takes to get a camera feed live stream setup on a RaspberryPi2.

How to set this up:

First off, you need to purchase an RPi2, RPi Cam Board, and don't forget an SD Card which will store the RPi operating system.

Next you'll want to choose and image an operating system. In my case I used Raspbian Wheezy. Here is a great setup guide.

Setting up the Camera Board is pretty straightforward but here is a great tutorial. Once you have the camera connected turn on your RPi2 and type the following line into the terminal:

raspistill -o cam.jpg

If all goes well you will end up taking a still shot with your camera!

Now that you have your RPi2 setup and taking pictures it's time to install the RPi Cam Web Interface (here is a great setup guide) so that you can view it in a web browser at a local IP address, meaning you can only view the camera feed if you're on the same network as the Pi. To learn how to view it outside the network see my post about LogMeIn Hamachi.

[nycinteractiveinstallation]

PJLink for Installation Projector Control

PJLink is a unified standard for operating and controlling data projectors. PJLink enables central control of projectors manufactured by different vendors and projectors can be operated by a controller.

There are a few different ways I have been able to utilize it for interactive installations that had permanent or longterm timescales.

ofxPJControl created by NullBoundry allows PJLink control to be inserted directly into the build of your openFrameworks build.

Node-PJLink is another tool that can be written directly into a node application which runs in javascript.

Lastly there is a Python PJLink app that allows the user to send commands to PJLink via Terminal on Mac OSX.