IGME 480 - Julia Hotaling @jgh6364 - Tumblr Blog

Going Outside Simulator ™

I didn’t really know what to do for this project, since the Arduino bits and pieces I had were sort of random and I can’t make phone apps without a Mac...in comes Processing which we were learning how to use in another class this semester.

For this project, I used Processing’s Serial library in Java to allow my arduino to control a canvas and audio. Here’s what Processing was doing in response to the above video:

https://www.screencast.com/t/jA1Dg2WfPfz

The potentiometer alters the hue of the background color and the tint applied to the other images on the screen (clouds and trees - I drew these myself). The colormode of the Processing sketch was set to HSB to allow the potentiometer to alter the color shade and the CDS cell to alter the brightness.

The CDS cell alters the brightness of the color set by the potentiometer. When I started I wasn’t thinking and put the CDS in front of the Neopixel which then kept making it strobe due to the lights on the Neopixel changing. It was moved behind the Neopixel for the final iteration.

The mic amp uses the volume level of the enviromemt to set the sound of the audio file being played by Processing’s sound library (sound free use from here: https://freesound.org/people/lextrack/sounds/272417/) The values of the potentiometer, CDS cell, and mic amp are mapped to values 0-255 in the Arduino code. While I ended up not being able to use serial.write, the 0-255 is still more convenient for using in Processing. In Processing itself, the mic amps values had to be mapped to 0.0-1.0 for the Sound library’s amp() function. Originally I was going to have the potentiometer cycle through different photo filters on Processing, but for some reason Processing lacks the ability to remove filters after applying them so I switched to changing the hue. 90% of the problems with this project were due to Processing being weird.

Processing will send the Arduino an int (0, 1, or 2) when one of the canvas variables changes meaningfully. The CDS cell and mic especially tend to have 1-2 differences in the numbers it sent, so I tried to catch then when they had differences of greater than 3-5 for the three inputs. The 0 represents the mic has changed volume, the 1 means the hue has changed via the potentiometer, and the 2 means the brightness has changed via the CDS cell. These ints were then sent back to the Arduino, where they light up the Neopixel green (mic), blue (potentiometer), or yellow (CDS) to give feedback on what is being changed. This was especially helpful because I can’t open the serial monitor to ensure what’s being sent back while the Processing sketch is running.

Coding stuff:

Processing’s loop runs every 60 seconds, which is slower than the Arduino sending information. This caused backlog and made it so the canvas was responding with a large delay. Delaying the Arduino sending data and using port.clear(); in Processing made it so the lag was eliminated and it responded in real-time without backlog

The mic amp’s effect on the audio is very subtle - my screen recording has terrible audio quality but it is noticeable if you pull the mic out and then plug it back in - my room’s noise had the Processing audio set to around half volume (it defaults to full volume with no input)

Sending three different variables from the Arduino didn’t really work super well and needed multiple fixes

I sent the variables all in a string seperated by “,” using Serial.println() and then threw an extra line break in there

In Processing, I had the port be read until it detected a line break - this ensured it only got three variables at a time instead of an endless stream of data

The string was then split at the commas, saved to an array, and cast to integers and floats when needed. The [0] index of the array was the mic, [1] was the potentiometer, and [2] was the CDS cell

Bonus: I had to duct tape my phone to a tripod to get the simultaneous videos (when I picked the Arduino up I was blowing into the mic)

#IGME470

The screen capture has bad quality audio, sorry.

The goal of this project was to use Processing’s sound FFT object to analyze and visualize audio. To deviate from a basic histogram, I used an altered sign wave with ellipses to create a visualization of the audio waveform. The ellipses fade out over time if they are a lower amplitude (< .05) , with only the higher amplitude dots (> .05) retaining full opacity. Below each dot, although a little hard to see in the video, is text which registers that specific dot’s max amplitude it has reached.

The music I picked out is from a movie soundtrack, and due to it’s more slow and calm tone I chose a very basic and clean visualization style. Here is a more energetic song (Marty Robbins “Big Iron”) instead of the slower one:

https://www.screencast.com/t/uGMDzFmBQ

#igme480

Air Music Box

Using a proximity censor, a potentiometer, and a speaker, I created something that will play notes based on how far your hand is from the censor. You can also change the delay between notes using the potentiometer, which will speed up or slow down your music as wanted.

Originally, I intended to have audio input come in through the microphone and have the speaking candence be mimicked by the speaker in 8-bit notes. However, I quickly found out without other equipment the mic was only really good for detecting the existence of noise and the volume, so I ditched it.

Then, the next iteration had 5 CDS Cells to create a sort of “air piano”, but they weren’t as sensitive as I liked and only would create 1-3 different notes, so I switched to the proximity censor. I would have liked to have been able to use multiple proximity censors to allow for a piano-type experience, but I only currently have one with me. The potentiometer was then added as I liked the sound of both more spaced-out notes and the weird electronic noise of them all being played rapidly, and now it can be changed while playing.

In the future, I would want to add more distance censors (maybe with different note ranges) to allow for a more engaging experience in creating music.

#IGME470

Only code pictures this time because my webcam only works on my laptop which currently does not have enough storage space to download processing

Redone using a video instead of a capture

For this assignment, I took the example that found the brightest pixel in the video capture and modified it to also find the darkest pixel. First, I found the brightness of of very first pixel in the video capture. Then, it’s compared to each subsequent pixel until one is darker. The darker one and it’s x,y coordinates are saved, and this process continues until the darkest pixel is found. A green circle is then drawn on this pixel to denote it’s location, the orange circle shows where the brightest pixel is.

#igme480

The goal of this assignment was to create an AR scene utilizing a terrain with two distinct sets of autonomous flockers. For mine, I used lizards and frogs as my flockers. Clicking on the menu buttons for each one will send the other group of flockers offstage, while the chosen ones come onscreen. Also, by clicking on a sphere in the plane, a butterfly will spawn. The butterfly is unique from the flockers as it uses wandering behavior to circle the sphere, while the lizards and frogs use seeking behavior to pursue a target around the ground plane stage.

As the flockers approach their target, it will randomize a new location within the terrain, making it so they are constantly in motion. The overall theme here is tropical creatures. The transparency of the terrain was mostly to help users still be able to see their surrondings on cameras due to the terrain size.

I have gifs of the butterfly and the flockers, but unfortunately they’re too large to upload to tumblr

#igme480

Design Inspiration:

Noticing the CDS Cell’s output was falling into the general range of RGB color values gave me the inspiration for the color changing based on brightness. For the second one, I was thinking more of the streetlights that turn on based on dimness to have the pixels respond to different levels of brightness

How It Works:

The first mode of my project takes the output value from the CDS Cell and uses it to manipulate the color saturation of the Neopixels. I noticed the CDS Cell’s output in the serial monitor was within a 0-255 range generally, which made it perfect for manipulating RGB values. There are a few checks in the code to ensure it doesn’t go above 255, but the value of the CDS Cell’s output is plugged into a RGB value of ( - value -, 255, 255) which means the brightness of the room impacts the intensity of the red color of the Neopixels. A slight delay is used for the “wipe” effect.

The pushbutton, when held down, will completely shut off the Neopixels. Based on the CDS Cell’s output value, 1-3 of the Neopixels will light up. The first pixel will light up green if the room is relatively bright. The first and second will light up green and yellow respectively if the room is a bit dimmer, and any darker will light up all three green > yellow > red. If the room is incredibly bright, the Neopixels will not light up at all. There is a slight delay just to make it appear smoother, but the three pixels will respond in real time to changing brightness.

Challenges:

I originally wanted to use the CDS Cell’s output values to control the brightness level of the pixels, but the pixels responded weirdly to it and I was scared of damaging them, so I switched to the traffic light idea

Figuring out how to set up the schematic for the A0 wire, resistor, and Cell leg that all overlap each other was difficult

I jacked up the brightness of the Neopixels so much at one point it was legitmately hurting my eyes to work on this

Future Improvements:

I would like to include something that allows you to change the color of the first mode. I chose to hardcode in red, but it would be cool to maybe hook up an app or colorwheel that would allow the user to choose a color for that state

Maybe more animation on the light states, there is a slight “wipe” effect on both but it would be fun to look into different ways of animating the lights past that

#igme470

For this assignment a UI was added to the ground plane that tells a user if the ground plane can’t be found, if the lighting or features are insufficent, and other information about the AR scene.

When the ground plane is found, a green indicator appears over it, and when clicked it’s replaced by the capsule. Hitting ‘Reset Plane Finder’ sets it back to searching for the ground plane.

#igme480

IGME 480 - Custom AR Target Interaction

The goal of this assignment was to add a custom image target in Unity using Vuforia target image databases. This shell image I created was the only target image I uploaded that was 5 stars according to Vuforias rules for good target images. I put a little shark model and some fish over it after downloading and importing the Vuforia target image database.

Resources

https://pixabay.com/vectors/shell-scallop-beach-sand-ocean-47214/

https://www.cgtrader.com/free-3d-models/animals/fish/fish-3d-6a34c6e0-dff2-4375-9257-469577d423cd

https://www.cgtrader.com/items/252849/download-page

IGME 480: AR Illustrations

#igme480

Trending Blogs

Recently Viewed Blogs

IGME 480 - Julia Hotaling