Being Responsive @beingresponsive-blog-blog - Tumblr Blog

The source

The functionality of my little color recognizing bear is done entirely in Processing. It is based on the examples found from the book Learning Processing by Daniel Shiffman.

Here's a link to the source.

Bear in mind, that it is not functional just as it is, but needs to have the appropriate sound and font files to go along as well. Get them all from the link below.

> Download the program folder and all the related files.

Video demonstration of the color loving panda.

While I was testing the color recognition and trying to perfect it, it soon came evident that some colors were recognized more easier than others. Colors such as pink or green tend to be everywhere, and the poor capture quality definitely does not help. Many times the video feed has simply copious amounts of color filled noise due to the poor light conditions of the location. This results in poor accuracy especially when trying to recognize the greens or pink.

I was already able to witness significant difference between the integrated camera of my MacBook, and the camera sewed into the little bear, which is an old Creative Ultra NX. The Creative webcam clearly yielded less accurate results.

This screen capture is from the MacBook webcam video feed under good and consistent light conditions. Red color turned out to be one of the easier colors to detect as well as a strong blue.

The problems of color recognition

As I stated earlier that even though getting started with color tracking and recognition was relatively easy, having it accurate is completely something else. Much of the problems I have stated below have to do with the poor camera quality and the auto-correction software which goes with it.

Light conditions.

The webcam makes excessive amount of color correction while trying to keep the white balance within reasonable levels. Now if there are mixed lighting conditions - for example a "warm" indoor light mixing with "cold" sun light coming through windows - the white balance algorithm of the webcam doesn't know what to do subsequently confusing my color recognition as well. Unfortunately many times there's even no need for a mixed lighting conditions, yet the camera yields inconsistent color values.

While a good camera could help to limit the problem, to really overcome it, the defined color values should be dynamic in relation to the white balance in question, and the light source should be well controlled.

Tolerance level.

Even though something is blue, not all blue is the same even when looked under consistent white balance conditions. To overcome this we need an appropriate tolerance level which states how close to the defined blue color the color has to be in order to be recognized as a blue color. The higher the value, the less accurate it is, while the lower the value, the lower chance there are to recognize the intended color at all.

It came apparent that it would be ideal to have a dynamically changing tolerance value. The idea is that if no color match is found, it is safe to assume that the tolerance level should be higher; if a matching color is found, it is safe to gradually lower the value until there's a moment when we don't find colors anymore.

Color dominance.

The problem of detecting the dominant color from the video feed is also something which needs to be considered. Not everyone is wearing a plain colored shirt. There could be several colors, which the program may or may not recognize, and from the available colors it should decide which one should it pick, or should it pick several colors.

One solution to this problem could be that the program simply "records" all the color picks it's able to find within a time period of a second or two, and only after that it decides on the color while checking which color was found the most. The larger the sample, the more accurate it would be, yet the longer it is "calculating" and waiting, the more likely it is that the situation has passed already.

Interference.

After we have solved all the above problems, there are of course the interference of the background and surroundings. The program should recognize the "object" - a person - of which it is supposed to recognize the color from. This problem could be overcome in a way that as the camera is ideally still, it would only find the color of the pixels which have changed within the video feed effectively limiting the possibility of recognizing the color of a background wall or other unrelated objects.

Let's see if I have the resources and time to tackle this problem. At this point it's all mostly just fine tuning and perfecting the recognition.

“Oh, by my tinkly toes, do I love colors!”

I haven’t given him a name yet, but certainly his character should deserve one. At the moment he is able to give six different reactions on the colors he is able to recognize. He also gives alternative comments if he hasn’t seen any recognizable colors in a while.

At the moment he is able to recognize the colors: yellow, blue, pink, red, purple, and green.

Listen reactions for the blue color:

"Oh, I'm just so happy when I see such a pretty blue!"

"That blue simply blows my little teddy mind!"

"Oh, doesn't that blue look lovely!"

"Oh, I haven't seen such a lovely blue in a long time."

"Where did you find that blue? It looks so good!"

"Uuuu, I simply love that blue!"

On top of the color reaction there are five tiny stories which play randomly after the initial reaction.

"It reminds me about little miss Ella Cuddly-bear..."

"That color is so this season..."

"Have I told you about my friend already..."

"This color is my favorite by far..."

"Oh, you have to show that color to my friend..."

Additionally there are also two alternative sounds. The intro and none-found reaction, which is being triggered if no color is found after a certain time interval.

Intro

None-found reaction

He has Creative NX Ultra webcam sewed inside. The camera had to be slightly disassembled in order to fit in.

The teddy is finally ready and hooked up to start commenting on color. He is from Jenners for only £3,99.

How would a fashion conscious teddy sound like?

Now, I chose to make it as a teddy instead of a mirror - which was the initial plan - because a teddy can act as a character, which the user can relate to and furthermore form a relationship with. Mirror in other hand doesn’t have these qualities, though having a teddy with a mirror could be even better. The user could visually recognize the color, which the teddy is referring to.

While having a teddy commenting on the colors the user is wearing could be somewhat of a humorous thing already, but in order to enhance it, the voice and the phrases should be well thought out. The voice and the chosen phrases should emphasize the small size of the character and the uncommon context. The phrases could escalade into a small stories. For example the teddy could say: “I haven’t seen such a lovely yellow since I met my old friend, little miss Ella Cuddlebear. She used to wear exactly that yellow the very first time we met back in…” and so on.

Already the fact of having the option to use small stories along with the color recognition brings more depth into the entire concept; in comparison of simply having a mirror stating the colors for you.

At the moment my little computer vision program is able to recognize multiple colors from the video feed. Here it recognizes the blue color of a light blue shirt. Quite quickly it came apparent, that even though it is fairly easy to recognize a certain color, there are several aspects making it difficult to have the recognition accurate; what seems to be relatively straightforward ends up being quite a bit more complicated as we are dealing with uncontrolled environments and objects.

The core color recognition algorithm is based on a color tracking example found from the book Learning Processing by Daniel Shiffman. The example gives a really good starting ground for my particular concept.

Computer vision to recognize colour

The artifact which I will produce for the Responsive Environments final coursework will be an item, which interacts in relation to the colours the user might have on.

Initially the item was intended to be a mirror which would respond what colours it sees on the user. As people tend to look themselves through mirror while putting a shirt on, or dressing up otherwise; having a discussion with the item which reflects your colours anyhow could have been fun and even make a sense a bit.

Alternatively the item could be just as well "a person", a teddybear with it's unique voice for example. Mirror talking doesn't sound as fun (=cute) as a teddybear giving you fashion advices. The form of the item isn't so important in the functional sense, but it does matter in relation to how people react with the item, and ultimately what kind of relationship do they form with it.

The book "Making Things Talk" (by O'Reilly 2007) has a great example on how to recognize colour using Processing language, or "computer vision". While the use of pure Processing for the functionality may give you a very powerful and straightforward platform - your computer, it does have it's limitations, as the poor teddy would be relying on such a large piece of equipment in order to function. There are other options as well, such as using only a colour sensor with arduino.

Because of the additional costs involved while purchasing sensors etc, I'm planning to use only processing to create a demo of the working system for this particular project.

One of the most important aspects in the interactive shirt project would be the motor which pulls the sleeves up. The motor should be small and light enough to be virtually unnoticed while attached to the collar area of a shirt.

I found a very promising micro motors by Faulhaber (http://www.micromo.com/datasheets/DC%20motors/0615_S_MIN.pdf) but after a few phone calls it turned out that the motors really were designed for the ultimate precision and accuracy used in scientific applications and medicine - thus the price for a single motor would have been 50£ and up.

I was able to find other promising options though. Small but not too expensive. For example this hobby micro motor system: http://www.hobbyking.com/hobbyking/store/uh_viewItem.asp?idProduct=10000 or this 2 gram "brushless outrunner": http://www.hobbyking.com/hobbyking/store/uh_viewItem.asp?idProduct=7230

The torque of these motors is somewhat of a questionable issue though. In order to pull the sleeves up, the motor should be relatively powerful, but without a real life tests, I believe there is no way to say. It might be that there should be two motors instead of just one simply because of the lack of torque.

This is an illustration of an interactive shirt concept. The idea is to have the sleeves automatically roll up depending on the outside temperature. The system would use a single or multiple micro motors depending on the torque output and the drag generated while pulling up the sleeves; all while still maintaining light weight and small size. The motor(s) would pull strings which subsequently would pull up the sleeves. An Arduino Lillypad board would be used to control the sensors and motors.

This project would be an additional piece to the uni work, made in collaboration with a tailor based in Helsinki, Finland. I'd make the interactivity, and functionality of the piece, and he would design the implementation of it for a suitable shirt.

This very eye catching shop window installation goes to the same category as one of my earlier post about the conspiracy interactive shop window. Very innovative way to create interactivity around a nostalgic topic - something which todays electronic/virtual world so often lack of.

Stairs Responding

Stairs Responding concept is part of the first coursework of the Responsive Environments module. The initial idea came from the Louise Vuitton plasma video stairs. Instead of just making the stairs pretty to walk on, I wanted make the interactivity useful for the user as well. This is an A1 size poster of the concept and will be put on display at the location where the project hypothetically could be built on - the Merchiston campus of Edinburgh Napier University, that is.

Stairs Responding poster

The Merchiston campus spans across multiple floors, and hundreds of students attend classes daily on the upper storeys. They can choose either to use the stairs or the lifts. However, the lifts clearly lack the capacity to cope with the number of people. Many students therefore either willingly choose to use the stairs or are forced to do so as queuing for the lifts is often impractical. The journey up to the top floor is a long one; couldn’t we make it more enjoyable, interesting and even useful?

The project relies on an assumption that every student is carrying a bluetooth enabled mobile device along with a student card. The bluetooth technology provides location information and additional data such as the individual’s name. The RFID rich student card is also paired with the bluetooth located user providing even more information about them. This can be anything from their module choices and lecture timetable to any previously recorded information regarding their stair use.

The interaction between the stairs and the user is visualized on the stair itself. Each stair is covered with a custom flat screen display. The moment a student steps on the stairs the system starts to fetch the user related information. The stairs then display the compiled information underfoot in the form of different colored and sized circles which follow the user as they progress.

Each circle displays it’s own piece of information including how many calories the user is consuming in their current trip, or a total of all their stair journeys so far. Additional information can be anything; how many steps still to climb until their destination is reached, or the latest news headlines from bbc.co.uk. We could have a circle which traces the user’s average pace, floating behind or in front of their current position. The possibilities are virtualy limitless and can be adjusted for a specific season or theme.

The stairs at the Merchiston campus.

The stairs 3D mock up.

"A Day Made of Glass" by Corning.

This is a demo video about the variety of uses an interactive surface glass could provide. Quite amazing future vision I have to say. I found it on Rene Prang's blog.

Control Arduino board with As3

Arduino board is meant to be programmed with the Processing language, but could you use other methods instead? How about ActionScript 3? Personally I have a bit more experience with As3 than Processing so maybe it's not too far-fetched idea in the first place.

I stumbled on Jean Philippe's blog where he has short instructions how to get started with Arduino with As3. It could be worth the try.

I saw this very funny piece, "My Little Piece of Privacy" on Arezoo's blog.

So simple, yet so brilliant. It's basically quite the same - with a little bit different implementation - as the interactive shop window few posts back.

#interactive installation

Trending Blogs

Recently Viewed Blogs

Being Responsive