ARCHIVED. @chaiexe18-a - Tumblr Blog

LET’S TALK: EXHIBITION SPACE & EXPERIENCE

There are several things to consider when thinking about how this experience will exist in a physical space and interact with a user.

window size — currently, the window size for the starfield is 600px by 600px. This has worked fine for testing on-screen, but I need to find the optimal dimensions the starfield window when shown by the projector (I might have to make the window bigger). If I have to change the window size, I may also need to change the number of total stars in the array.

projector — I have to make sure the placement of the projector won’t be behind the person interacting with the exhibit, or the projection image will be interfered with.

maximum / minimum distance — stepping too close to the sensor makes the pplPosition variable go to 0, and when that happens, the program needs to be restarted. I also need to calculate a maximum distance so that the pplPosition variable won’t go over 1000 (the value I’m subtracting the pplPosition variable from to invert the z axis).I could mark the minimum and he maximum distance so people know the boundaries for the exhibit for a good experience.

multiple body interference — I’m concerned that the starfield will fluctuate weirdly if people walk in and out of the exhibit space when they’re just in passing because of how the sensor detects multiple presences. I might have to define a space that people can enter when interacting with the exhibit or avoid when not interacting with the exhibit. This way, the experience will run as planned.

#cfcc.#002.#exhibition.#notes.

INVERTING Z AXIS

My original intention was for the stars to idle when a user was far from Kinect device, and when they stepped closer, the stars would move faster. Currently, because of how the z axis is set up in reference to the Kinect device + how I am using the z axis in Processing, the stars are moving faster the further you are away from the Kinect device, and idling when you are close to it.

I don’t mind this too much, as my main intention was to introduce an interactive aspect to the starfield code, but I believe that the experience could be improved if I could invert the z axis (this would give me an output that followed my original intention).

On the web, there doesn’t seem to be any other people who are using Processing and are looking to invert the z axis. I looked briefly into the reverse() function for Processing, but that seemed to only work for arrays with a series of information i.e. ‘String name[] = { "OH", "NY", "MA", "CA"};’.

I might settle for the output that I have now, considering that it is already at a satisfactory level, unless I am able to find a way to invert the z axis in time for the hand-in.

EDIT: Tim helped me out with the maths! To invert the z axis, I can subtract pplPosition from 1000 to get a larger number from a lesser pplPosition value (i.e. 1000 - 700 = 300) or a lesser number from a larger pplPosition value (i.e. 1000 - 200 = 800). Now I have to test the maximum distance so the pplPosition value won’t go over 1000 — otherwise, I will get a negative value. I could mark the minimum and maximum distance from the sensor during the exhibit.

#cfcc.#002.#notes.

COMBINING THE CODE

Now that I have the code that executes the starfield (starfield base) and the code that retrieves the depth data (interactivity base), I can combine the two. My intention is to give the starfield base an interactive quality, so users are able to alter the starfield by their distance placement in regards to the Kinect device.

The first problem I ran into was trying to use ‘_s.position.z’ in substitute for mouseX in the line of code that controlled the ‘speed’ of the stars. When simply substituting mouseX with _s.position.z, Processing told me that _s variable did not exist. After some trial and error, I figured out that anything with an ‘_s variable’ had to be placed in a function that belonged to the Skeleton Data i.e. ‘void drawPosition(SkeletonData _s) {’.

However, it wasn’t as simple as assigning the void draw() function with ‘SkeletonData _s’. That rendered the rest of the draw function null. Instead, I decided to create a global variable that would hold the _s.position.z value. This was a guess, but it worked!

I assigned the _s.position.z value to a global variable called ‘pplPosition’, and replaced mouseX with pplPosition. I then divided the pplPosition integer value by 45 so that the value was in the 0-600 range.

Here is the combined code with the necessary documentation (comments) to explain parts of the code.

#cfcc.#002.#development.#notes.

THINNING DOWN KINECT 4 WINDOWS EXAMPLE CODE

In preparation for combining the two codes together, I went through trial-and-error to see which parts of the code I needed to sense + calculate the distance of a person with the Kinect device, and which parts of the code I could delete.

Most of the code that I deleted had to do with displaying the RGB/depth/etcetera images in the Processing window, or drawing the lines used for displaying the skeleton data.

This is a link to the Kinect4Win Example code, except I commented out the parts I did not need for the Kinect sensor to send the depth data to the computer.

#cfcc.#002.#interactivity base.#notes.

PROCESS TO CONNECT & USE KINECT

I am using a 64-bit operating system, Windows 10 home OS, connected to the Kinect for Windows sensor device. The Processing Library I am using is Bryan Chung’s Kinect for Processing Library.

01. Install Kinect for Windows SDK v1.8, Kinect for Windows Runtime v1.8, and the Kinect for Windows Developer Toolkit v1.8; you can download the SDK here, the Runtime here, and the toolkit here. Windows installation instructions warn against having your Kinect device connected while downloading these packages. Make sure your Kinect device is disconnected from your computer before installing. Complete the installation process and close all the windows.

02. Connect your Kinect for Windows device by the USB port and the power socket. Check that the green light between the cameras is blinking. That will tell you the device is connected to the computer and to power.

03. Open Device Manager. ‘Kinect for Windows’ should expand to a camera element (and possibly an audio element). If there is a yellow warning sign next to the Kinect for Windows icon, you may have the wrong SDK installed (see previous post for details).

04. Open Processing and import Bryan Chung’s Kinect for Processing Library (Sketch > Import Library > Add Library > Filter by ‘Kinect’ > Install ‘Kinect4WinSDK’). If the installation seems to be stuck on loading while the library is available, close and open Processing before checking if the library is installed fully in the ‘Import Library’ tab.

05. In Processing, open ‘File’ > ‘Examples’ > expand ‘Contributed Libraries’ > expand ‘Kinect4WinSDK’ > open ‘Kinect4WinExample’. Run this program. The 4 types of images the Kinect collects information should execute (skeleton, depth, RGB, infrared). You may need to activate the skeleton + depth view by standing a slight distance away from it and make some movement.

#cfcc.#002.#kinect notes.

PROBLEMS WITH KINECT

Note: this post is friendlier on dash view because of the bullet-point format. Click here to view on the dash.

As per the instructions on the Windows site, I downloaded and ran the v2 SDK and Kinect Studio. However, my Kinect for Windows device failed to connect with my computer, as the Device Manager showed a yellow warning sign alongside the Kinect for Windows icon.

This was solved by uninstalling the v2 SDK + accompany v2 applications, and installing the v1.8 SDK + accompanying v1.8 toolkit. The Device Manager then registered the Kinect for Windows device as working.

Initially, I was also following the tutorial Daniel Shiffman provided, as well as the open source Kinect for Windows library that he recommended in said tutorial. However, I got the error message ‘initDevice() does not exist’, among other things.

This was solved by instead using Bryan Chung’s Kinect4Win library. By running the provided example program alongside the v.18 SDK, the Kinect sensors were able to provide me with all four views when executed (skeleton + depth + RGB, mask).

Oddly, after a successful connection and execution of the four-view feedback provided by the Kinect for Windows device, it ceased to work again when I got back home and tried running it. I had not altered anything since the successful attempt. When I ran the Kinect4Win example program, it gave me the error message: “(Init) Initialize sensor failed 0” — the 4-view window was also blank (no RGB / depth / infrared / skeleton view).

I uninstalled all the Kinect for Windows programs (v1.8 + v2, SDK, Runtime, Kinect Studio, Toolkit, etc) and removed the Kinect libraries from Processing (Bryan Chung’s Kinect4Win, Daniel Shiffman + Thomas Sanchez Lengeling’s Open Kinect for Processing). Then I reinstalled the v1.8 SDK and Runtime, as well as Bryan Chung’s Kinect4Win library in Processing.

In Device Manager, I also enabled the hidden devices (View > ‘Show hidden devices’) to double-check that the other elements (Audio Array, Security Control, etc) was available - just not connected.

I then rebooted my computer and ran the Kinect4Win example code. This sometimes gave me the RGB and depth, and sometimes it did not. Neither the skeleton + depth view showed. I also noticed the infrared camera on the Kinect device itself would flare with infrared light for a few seconds at the start before turning off. Note: for this attempt, the Kinect device is plugged directly into the wall socket.

After disconnecting the device and reconnecting again (I had to move to another workspace), it reverted back to only being connected to the Kinect for Windows Camera. Note: for this attempt, I am using an extension cord rather than plugging Kinect device directly into the wall socket - unsure if that is a variable?

Rebooted computer, disconnected + reconnected device, ran the example code despite only the Kinect camera showing. Started working anyways???? Don’t know how Kinect works but I’m too afraid to ask.

Skeleton and mask views weren’t showing up for some reason

Moved at a distance in an attempted to activate the sensor. It worked! Skeleton and mask views became live.

Depth data would revert back to 0 when I got to close to the Kinect device, and would not register my depth even if I moved back to a distance that it detected before.

Rebooted the Processing program and made movement at the acceptable distance to activate the sensor. Did not step back too close.

#002.#cfcc.#kinect notes.

ENTOURAGE — click for better view

✧ virtual reality experience — possibly integrated with some gameplay or as an immersive experience to showcase a space program, the user is able to experience the output with virtual reality goggles. ✧ brochure — to advertise or inform on an astral-theme exhibit or event, the output can be used as a background texture for a brochure. It would be easier than individually doing each star and trail, and look much more consistent. ✧ digital bus stop billboard — to advertise about an astral-theme exhibit or event, the output can be used in a digital advertisement that is hosted in the billboard of a bus-stop. ✧ gallery projection exhibit — as an art exhibit, the output can also be projected onto a gallery wall, with interactive qualities where users can alter the form of the exhibit by their touch or presence.

#cfcc.#002.#entourage.

STARFIELD 02 — VERSION 4 (pastebin code)

I altered the color of the star trails as well as the trail length to complement the background image and create a more immersive experience. I also added a background image, and I tried looking up tutorials to blur/change the opacity with mouseX, but decided it was probably something I could save for later if I had the time.

#cfcc.#002.#starfield base.

NOTES: NEXT STEPS!

look into the aspects i want to alter from the starfield base code to achieve a personalized code + maximize the experience for the user

apply kinect code and device to introduce the user interactivity

live test with a projector & kinect device

entourage images

#notes.#002.#cfcc.

STARFIELD 02 — VERSION 2 & 3

Retrying the second tutorial from scratch, I managed to get a successful outcome while understanding most of the coding. My next iteration of this starfield was a little more personalized to what I wanted the experience to be like: I changed the colour of the star trails as well as the star size + number.

I tried fiddling around with the star trails (”float px = map(x / pz, 0, 1, 0, width);” / “float py = map(y / pz, 0, 1, 0, height);”), but in the end, the result wasn’t desirable, as it made the light trails permanently long (rather than go from non-existent to long), or detached from the ellipse/star itself.

#cfcc.#002.#starfield base.

STARFIELD 02 — VERSION 01 (pastebin code)

With a different tutorial, I tried to aim for the look of my initial idea which had the stars ‘bursting’ outwards from the center of the window. I learnt here that you’re able to create your own functions that can execute within the draw() and setup() functions themselves. Some of the maths in the tutorial was confusing but at this point I don’t think its crucial for me to understand every single detail.

I must’ve stepped wrong somewhere during the tutorial because the array of dots refreshed again and again instead of being static like the tutorial instructed. When I continued the dots all filtered into a line (right picture). Overall, this version was a failure, but it was a refreshing outcome from error messages that I didn’t understand + unresponsive windows (no output). I’ll try again in my next iteration.

#cfcc.#starfield base.#002.

STARFIELD 01 — VERSION 01 (pastebin code)

Before I started with the interactivity aspect of my experience, I wanted to start with making the starfield first. It would be a more cohesive workflow if I left the interactivity for later (as it required learning about Kinect and how to navigate a whole new minefield) and I just started in Processing first.

The first version of my starfield was unsuccessful but still helpful. Overall, it was unsuccessful because the effect of the right-to-left starfield wasn’t what I was aiming for. My initial idea was for the stars to come from the center and burst ‘outwards’, leaving light trails.

However, the tutorial I followed for this starfield taught me a few interesting lines of codes that I could use in a later iteration. One of these aspects is the having the ‘closer’ stars brighter (true white), and the ‘further’ stars less bright (a lesser white). It also facilitated a higher probability for generating stars that were closer to create a more realistic effect.

#cfcc.#002.#starfield base.

FOR LOOP EXPERIMENTS — 012.

By adding an opacity element to the fill of a shape, I can also create a gradient effect as each ellipse stacks on top of each other, the density of the gradient increasing with each innermost ellipse.

#cfcc.#002.#fl experiments.

FOR LOOP EXPERIMENTS — 011.

‘For loops’ are also able to control the size of a shape, i.e. an ellipse. By placing the line of code inside the ‘for loop’ and using the ‘i’ variable to represent the shape of the ellipse, I am able to gradually increase the size of each generated ellipse, as well as controlling its coordinates.

#002.#cfcc.#fl experiments.

FOR LOOP EXPERIMENTS — 005/006/007/010.

Continuing the ‘for loop’ experimentation, I moved from lines to ellipses. This was extremely similar to the experiments that used lines, except that the direction of the ellipses was now also explicitly controlled by the ‘for loop’. I could control the y-axis of each generated ellipse so that there is a line of ellipses going downwards, or control the x-axis alongside so the line of ellipses is diagonal.

#cfcc.#002.#fl experiments.

FOR LOOP EXPERIMENTS — 001/002/003/004/009.

In class, we were introduced to ‘for loops’ in Processing. ‘For loops’ function in a format of for([initial;test;update]), where you set the initial value, what the ‘for loop’ tests, and what it updates the output to.

This was tested with the basic knowledge we learnt in the last unit with line stroke, weight, and colour. I used a for loop to create the succession of multiple lines instead of typing out a line of code for each line. This also ensured that the spaces between the lines were consistent, as it might not be if I were to input the coordinates for the individual lines.

‘For loops’ can also control the RGB element — the colour of the lines. This meant I could gradually change the colour with each generated line, creating a ‘gradient’ effect.

NOTE: The gradient in the example was an evenly distributed share of blue on the left and pink on the right, but Tim said that there was a mathematical bit in the code I would have to figure out to get the exact 50/50 output. I didn’t bother with the exactness.

#002.#cfcc.#fl experiments.

RESEARCH

Ku-Flex’s ‘Quantum Space’

Hosted by the M’ARS gallery in Moscow, the ‘Quantum Space’ project is a light projection art installation that reacts to the presence of gallery-goers. It was created by Igor Tatarnikov, also known as Sodazot. By using projectors and motion-tracking cameras, the interactive display reacts to the presence and movement of people, projecting an output in the form of a beautiful light projection on the gallery wall.

The exhibit itself is based on real particle physics. Tatarnikov’s aim was to connect the audience with the universe, explaining as: “When you come into Quantum Space, you break down into quantums of light and you communicate with universe."

I thought this was a good example to research because the output is the same as I am aiming for — a light projection. My output will also rely on the presence and movement of the audience, reacting with the data the sensors collect.

#002.#research.#cfcc.

Trending Blogs

Recently Viewed Blogs

ARCHIVED.