Sherr(i)ng BCT

First year, Bachelor of Creative Technologies, AUT

[sherringbct-blog]

#Ctec503Signal - Final Entry - Sherry Ng - The Look of Sound.

Over the course of this semester I have learnt a lot about team work and the creative process. 

From the Systems Experienced brief I learnt that the creative process is cycle and that feedback and engaging with our intended audience (peers and tutors) is always useful in order to gain an outside perspective and outlook on work. This was evident as soon as we had started sharing our initial concepts and idea to tutors and peers, especially after the elevator pitch where we were directed to examples of what had been done before. We did this continuously went through the cycle of iterating and presenting/seeking feedback throughout our final project and multiple times during the finalisation of our setup to ensure our installation would be accessible to users. 

The Tangible Data Conversations brief highlighted the importance of starting by doing, my team and I carried this on from the beginning by experimenting before we had a base idea or theme that we wanted to follow. This allowed us to understand how different materials behaved and reacted with one another and allowing the material to “talk back” to us. 

Blog Index

[All posts written individually unless stated otherwise]

Group Formation

Initial Experiments

Sound Printing

More Experimenting

Elevator Pitch

Inspiration and Research

Development and Setup

Light Experiments

Physical Computing Inspiration

New Setup

Final Setup

Message

[sherringbct-blog]

Signal: Message

We have designed our installation to be a medium that creates universally personal experiences for users. Things like songs and art pieces are universal but the way they are interpreted by individuals are different depending on their background, beliefs, morals, connections and overall individuality.

Our installation involves three senses, the launchpad and feedback from the buttons involve touch, the laser sight and songs from the speaker audio. We wanted to keep the visualisation of the songs and sounds as raw as possible so that the visualisation produced is a direct representation of the vibrations from the speaker (without just using an oscilloscope), this allows users to interpret the visualisation and songs in their own way unlike existing music visualisers like music videos and media players like Winamp which are already an artist(s) interpretations of the music.

We want our installation to show that disadvantages and disabilities do not mean that life and its beauties are no longer enjoyable. We have taken inspiration from musicians who have overcome their disabilities. We want to show that there is always a way for people to enjoy experiences like how Beethoven found a way around the loss of his hearing (by feeling the vibrations of music through a pencil between his teeth and the floor) to continue composing and enjoying music. Our installation changes the medium through with music is typically presented which leads to a change in the outcome, it combines senses to show how music can be enjoyed in a myriad of ways that will spark personal attraction and enjoyment in users.

[sherringbct-blog]

Signal: Final Setup

A slight change in our setup for assessment, after setting up the laser and the speaker and testing the live demonstration of our project we realised that this would leave a small and awkward amount of space for the tv to show the trailer of our installation. This led us to removing this part of our installation as we all agreed and that the video would distract from the live demo (or vice versa) and that we would be able to show/explain the process through a series of photos.

For assessment we have decided to use our Twall as our poster for submission (as there were no size restrictions). This is a mock-up I did of our wall layout.

The title of our project is “The Look of Sound”, this is going to be written in wire and attached in a pop out effect around the photos to symbolise the laser in our project and to show how our installation can only be seen from certain “angles”, from experimenting we have learnt that our live demo is most effective in the dark, this is why we have chosen to showcase long exposure photographs of the demonstration that have been taken in optimal conditions. Our demonstration also looks best to the naked eye as smart phones are usually set to record at 60 frames per second which ends up making the light trail of the laser disconnect (This happens faster than the human eye can pick up so we see it as one continuous loop). It is also difficult sometimes to get automatic cameras to focus on the laser as we have noticed in the past from peers wanting to record our demonstrations to post on social media accounts.

The photos behind the wire title will be in the configuration of our launchpad and each photo will show the laser art piece that is produced when that button is pressed. The rectangle on the right side will be our technical assessment instructions and the squares will be photos of our set up and process.

*UPDATE*

Due to technical difficulties and time constraints our launchpad will not be functioning for assessment but we are working to get it done for Showcase along with our neon wire title.

[sherringbct-blog]

Signal: New Setup

Photos below are taken from our intended setup with one speaker.

For assessment I am currently worried about the camera and photos because the photos I took that are shown above had to be taken in a pitch-black environment so that the background wouldn’t show up.

We spoke to Ricardo about this and explained how the rest of our installation would be set up, he suggested that we should present our final artefact / installation as more of a showcase rather than a performance like it currently is, this is because in previous years performances have gone wrong on the day of assessment and showcase and a lot of time has been allocated to perfecting the performance rather than the actual core element.

After discussing this idea of having a showcase rather than a performance with the team members that were not here for the talk we decided to redesign our setup and this is our outcome.

Instead of having four speakers running we have decided to just have one with a small demonstration of the laser and mirror on one speaker, this would be hooked up to the launchpad so that we could keep the interactive aspect of our installation. As shown on the left wall there would be a collage of all the art pieces we created which would be representations of different frequencies. On the TV would show a trailer of our installation showing the process of making the art pieces behind the scenes.

This is an experiment we did to test what frequencies we would use for the best effect in our live demonstration,

[sherringbct-blog]

Signal: Physical Computing Inspiration.

Today in physical computing Stefan was explain to us how to use an oscilloscope and he also showed us a video where an oscilloscope was used to produce visualised music which was inspired by different frequencies. This is different to our project as our art pieces are visual representations of frequencies whereas in the video shown below, the oscilloscope shows visualisations inspired by the frequencies. They have taken some aspects from the sound like the volume and resonant frequencies but some of the things like the globe and orbits are unlikely to be directly related to the frequencies produced.

Stefan also shows us how we could an Arduino to trigger the shutter button on a DSLR camera when there is a certain amount of light detected or after an amount of time. This could be useful for our final presentation as it would mean the camera could operate automatically and we would not have to be there to explain it (which would be useful during assessment).

Reference

Fenderson (2016), Planets. Retrieved from https://www.youtube.com/watch?v=XziuEdpVUe0

[sherringbct-blog]

Signal: Light Experiments

After discussing our setup and the parts of the experiments we wanted to use I felt a little confused and overwhelmed by how we were going to incorporate and the number of elements we had to our project. This led us to experimenting with light art and long exposure photography as it would be cleaner and more controlled compared to using wet media. I had previous experience with doing light photography, so I decided to do a couple of tests at home.

These were done with an LED rod and an exposure time of approximately 20 seconds. This is an example of what we are aiming for in our project, but these were not in any way inspired by frequency and sound which is what we want our project to be. We are going to test this with the speaker and laser setup in studio tomorrow.

[sherringbct-blog]

Signal: Development and Setup

This is a statement that I have written about our project that our group has decided to use as our brief: 

 “This project is an interactive installation that makes music and sound a multisensorial experience by incorporating visual elements that are inspired by songs created by the user.”

Initially when we grouped one of the key things we all agreed on was that we wanted our final project to be was interactive. This is because we believe this creates a universally personal experience that is unique to each user which has been our overall theme. For our installation we are thinking of making a launch pad for user input that would have the different frequencies associated with it that would work with the speakers and lasers to create an art piece.

Below is a rough sketch up I did of our launch pad.

At first, we wanted each button on our launch pad to be assigned to an art piece that we have done with the different media (i.e. ink, acrylic paint, nail polish). When users pressed a certain combination of buttons the art pieces would overlap to create a unique piece that would be theirs. After speaking to Donna about our setup during studio, she reminded us that a lot of the feedback from the class after our elevator pitch was that our project “looked cool”. It is likely that this comment referred to the real time aspect of our project was interesting as you could see the direct connection between the frequency generator, speaker and the liquid inside the speaker. So she suggested that this was an important element that we should include.

This is the current setup we’re working towards.

Each row on the launchpad will be assigned to a speaker and each button will trigger a frequency, the speakers would be setup with a mirror that would reflect a laser onto a blank wall. Users would be able to pick different combinations of the frequencies to create a unique representation of it which would be captured by a camera taking a long exposure image (this element will be dicussed further in the laser and light experiments blog post). We are thinking of having a printer hooked up to the camera so that users would be able to have a physical copy of what they have created. In our setup there is also a forth speaker that would be filled with fluorescent fluid that would be lit up by a blacklight. This would provide a live demonstration of the patterns produced by the combined frequencies. The use of four speakers going off at the same time also might be a bit overpowering but hopefully the frequencies can be played quiet whilst also influencing the mirrors and lasers.

The process of discussing this within our group and with lecturers and peers has shown me that it is important to get outside input on your work, similar to play testing during the Systems experienced brief. We did this multiple times over the course of this week whilst we were trying to finalise our setup and it was important to us to get feedback because our project was interactive so we wanted it to be accessible to users. Discussing it within our group meant that we were able to narrow down and keep what we felt was a significant but talking to people outside of the group allowed us to get a different outlook on what aspects of our project were interesting and more important than others and it was good to get a balance of both views.

[sherringbct-blog]

Signal: Inspiration/ Research

Beethoven

The composer Beethoven started losing his hearing in his late 20s however this didn’t stop him from composing music. Although with difficulty Beethoven overcame his disability, he had been a composer for most of his life so he was able to imagine his music in his head. One of the techniques he used was biting on a pencil and letting the tip of it touch the soundboard allowing him to feel the vibrations through his teeth.

He changed the way he perceived something that was auditory to something kinesthetic resulting in some of his best pieces. The technique he used is similar to Mandy Harvey who was a contestant on X Factor in 2017. Mandy is a deaf musician who performs barefoot as it allows her to feel the sound of the vibrations on the stage floor through her feet.

Our project aims to provide a multisensorial experience for something that is typically monosensorial by playing with the visualisation of sound.

Michel Gagne

Michel Gagne is an award winning animator who worked on the Pixar Animated Film Ratatouille. Gagne is a Synesthete which is someone whos senses often merge together to provide an overall experience involving multiple sense. There are many forms of Synesthesia and Gagne uses Animation as way to express his experience. I like the way that he uses a myriad of different patterns, shapes and textures in a way that they complement one another along with the sounds of different instruments. The use of repetition is very effective and the short bursts of colours and animations attracts attention.

At the current stage of our project we are looking at doing something similar to this but allowing the patterns and colours to be naturally formed by the frequencies of the sounds played through the speaker. Through our chosen technique we would be “printing” snapshots of frequencies to change the way the sound is initially perceived. We have also noticed through experimentation that there is also a kinesthetic element to our current medium. We have been using a subwoofer to generate vibrations of frequencies which can be felt through the water in the bowl. Ricardo wondered whether we could make the experience involve all five senses which made the beats in the music suddenly remind me of the feeling of drinking bubble tea and having the pearls go through your straw producing a “beat”. It would be interesting to see whether we could incorporate taste into our installation.

References

Gagne (2013) Retrieved from https://www.youtube.com/watch?v=sao3NAapOAI&t=110s

How did Beethoven Compose if he was Completely Deaf? Retrieved from: https://www.classicfm.com/composers/beethoven/guides/deaf-hearing-loss-composing/

[sherringbct-blog]

Signal: Elevator Pitch

Today in studio Donna organised for every group to do an elevator pitch. We had 30 minutes to organise how we were going to present and pitch our idea to the class in three minutes and the class would be able to give us feedback and feedforward. My group and I were initially worried about doing the pitch because since grouping we had done a lot of experimentation with different materials and were yet to refine our message, purpose and overall outcome of the project. However, this was a good opportunity that forced us to find a direction (or even just the first stage of one).

For our pitch we decided to do a live presentation of the experiments we had been doing so far with the speaker and coloured Oobleck. We felt that this was more effective than explaining it and the thought behind it as it would be a visual experience which is what we wanted our final artefact to be. We accompanied this demonstration with an explanation on how we wanted our installation/artefact to change the way that sound is perceived by providing a visual element to a universal constant. We also talked about some of the research we had done on Marshal McLuhan and his well-known “The message is the medium” quote. This is significant to us because we want our final outcome to be a different medium through which we can convey a new message.

From the feedback and feedforward notes that were given from the class, many of them mentioned that our demonstration “looked cool” but they struggled to understand how our final outcome would be presented and also what our overall message was. This showed us that it was important that our final outcome had to have some part of the live demonstration incorporated into it as it was an attractive element. Currently we are still unsure about what message we want to portray is and that is something we need to discuss further.

One note pointed us in the direction of similar things that had been done before with science and music called Cymatics. I recognised a couple of the experiments from studying physics in High School, things like Chaldni Plate and Ruben’s Tube which are affected by frequency and the one inside the speaker drum which was similar to the experiment we demonstrated. Some of effects of these experiments are not visible to the naked eye and so videography techniques like Slow motion and the use of strobe lights were used to enhance the effects.

This has inspired us to look into other mediums of visualising sound, we want to experiment with long exposure photography along with lights, lasers and glow in the dark / blacklight materials.  

References

Stanford (2014) Cymatics. Retrieved from: https://www.youtube.com/watch?v=Q3oItpVa9fs

[sherringbct-blog]

Signal: More Experimenting

Yesterday we tested all our materials so far (Acrylic paint, oil paint, food colouring, ink, nail polish and matte nail polish) and printed frequencies onto white paper. We also tried a method where instead of dipping the paper into the water we put the paper above and then blasted the frequency for a short amount of time, so the contents of the cone would splatter on the underside of the paper. Unfortunately, this didn’t turn out as planned because the paint and water was too runny to hold its place and the paper was turned over. This led us to a print transfer technique where we held glad wrap above the paint, blasted the speaker and transferred the wet into onto black and white paper. This technique worked the best with acrylic paint on white paper. The ink on black paper looked cool initially however they lost their colour as they dried and ended up white which left us wondering how they would show up under black light.

We have also been experimenting with thick Oobleck and observing how it reacts on the cone of a moving speaker. We are thinking about using light photography and long exposure to capture it moving around.

During experimentation we have been using some fluorescent coloured paints as we wanted to see how they would show up under black light which we are planning on testing next week. We are thinking about taking this idea further by using highlighter ink, glow in the dark paint, Vitamin B2 (which has glow in the dark properties) and the liquid inside of glow sticks.

[sherringbct-blog]

Signal: Sound Printing

We decided to test this out the nail polish experiment the next day after getting supplies. To “Print” the frequencies we played them through the speaker and bowl, added colour to the water and allowed them to mix naturally. Then we dipped pieces of paper into the bowl and allowed them to dry. We did this with food colouring, acrylic paint and dish washing liquid and matte nail polish.

The acrylic paint and food colouring reacted in a similar way and produced an ombre of colours similar to the effect of water colours. However, when we left these two tests to dry, the corner of one of the acrylic paint tests was accidently touching the food colouring test creating the fractal tree like semicircle which was interesting. We wanted to experiment with oil paint as it wouldn’t mix with the water and therefore have a different effect compared to the water soluble acrylic paint.

We decided to add dishwashing liquid to the acrylic paint and water to see if it would produce bubbles from the movement of the water, this is something that I did in kindergarten where we blew bubbles into coloured soapy water and pressed a piece of paper onto the overflow of bubbles to make art. Unfortunately, the water was not soapy enough and the bubbles produced by the speaker were too fine to hold their shape on the paper. This led us to testing with ink to see if it would hold the colour better and we had also seen examples of ink bubble art on pinterest.

The matte nail polish didn’t react as expected in the water and didn’t form a thin film on top, we think this might have been because we used matte instead of regular nail polish so it may have had different properties. We added regular nail polish to the list of things we needed to test.

[sherringbct-blog]

Signal: Initial Experiments

Over the next couple of weeks, we are planning to experiment with the different types of materials we can use to express sound in a visual form. This Idea was originally inspired by videos we had seen previously (discussed further in research blog).

We started with a little speaker which was connected to an Arduino where we could change the frequency and the amount of time it was played for. In this speaker we put a mixture of corn flour and water (Oobleck) to create a thick non-Newtonian fluid and food colouring so that we could easily see how it was affected by the frequency. When light was shone on the dark liquid the different frequencies produced different patterns as seen below. We found these formations to be very interesting and wanted to be able to capture them frozen in time, so we decided to use the colour that was used in the Oobleck to in a way print the frequencies.

The next day we put a bowl on top of the cone of a subwoofer and filled it with water to recreate what we had done previously. We added a bit of cornflower to the water so that it would have a white base but still have the same effect as water. While the frequency played through the bowl and water we added drops of food colouring to see how it would naturally mix with the corn flour water. I really liked the way the colours mixed in time with the music and how it was a visual representation of the song or frequency that was played.

[sherringbct-blog]

Signal: Group Formation

Going into final project for studio 2 I was interested in creating a final outcome/artefact that was multisensorial and interactive. I wanted to experiment with colour and light. I like the way light photography (Long exposure photography) shows the progression of time through an image and I wanted to try and incorporate with the the mixing of RGB light. I was also looking into smart textiles and the possibilities around that.

These base ideas and interests Helped me in forming my group. I am currently in a group with Gayle, Megan, Vahid and Zane Ali. We initially grouped because we all had a mutual interest in using biometrics and smart textiles. We talked about the possibility of creating something that would aid the user or show them something about themselves i.e. visually representing their heart beat or making art out of their fingerprints.

During the early days of group formation, we did an activity which was instigated by Charles where we had 10 minutes to individually write down every single idea we had or were interested by for our final project and put it all in the middle to discuss afterwards. I found this activity really helpful because it allowed everyone’s voice to be heard equally and we were able to focus and refine each idea and narrow down what we all wanted to work towards. The time pressure forced us to write down everything little thing that came to mind because it would be better than not writing anything, this allowed us to get out of the habit of not sharing “bad ideas”.

From this activity came up with the base theme of “Universally personal”. the way we explained this was the example of music and the way it is universally heard but each person experiences it differently and has their own memories, feelings and emotions tied to a song. It is the idea of subjectivity where the quality and value of something is based on or influenced by personal feelings, tastes or opinions. From this base idea we discussed having our final artefact be the medium of expression the same way a book is an author’s medium or how body language medium for expressing emotion.

From here going into next week we want to physically experiment with low fidelity models of our existing ideas so we can narrow down what works and what doesn’t in order to finalise our brief.

[sherringbct-blog]

Physical Computing Assignment 2

[sherringbct-blog]

Conceptual Designs

During the middle of week two I was struggling to extend my low fidelity models and decide on an object. This lead me to forming a group with Jake (because we had both been experimenting with spaghetti) and Amy (because we had similar statistics, I had been working with 1 in 3 and she was working with 1 in 5). We decided to use Amy’s data set which was on workplace bullying in New Zealand where 17.8% of New Zealanders surveyed in 2013 reported experiencing workplace bullying in the last six months (https://www.stuff.co.nz/business/better-business/83618177/New-Zealand-has-worlds-second-highest-rate-of-workplace-bullying)

Soon after grouping it was challenging for us to settle on an object that we could use map data on as we didn’t really have firsthand experience of workplace bullying or working in similar workplaces ourselves. To overcome this, we started with looking at a stereotypical office and office supplies, things like stationary and chairs which lead us to business wear and ties.

We thought about using a chair to represent our data. This could be done by removing 17.8% of either the seat or the back rest of a chair making it uncomfortable for the user. We could also change the length of each leg to make the chair unstable. The chair could be used to represent a workplace which looks stable from the outside however when users interact with the chair and sit on it they will feel the inequality due to the difference in the height of each leg.

We also decided to experiment with neckties as they are a common accessory of office wear and associated with uniform. The Romans used neckwear as part of their uniform as a symbol of belonging to a particular group and it is said that a reason why ties are worn today is for the purpose of hiding buttons on a shirt.

In order to come up with more ideas the three of us split up to each make a model that mapped the data on a paper necktie. I wanted to play with the idea the tie hiding something as its intended purpose.

The concept behind my tie was the one we decided to use a our final object with 17.8% of the back covered in pins. I want to extend the idea of the hidden data with covering 17.8% of the back of the tie in coarse sandpaper. Over time and use this would distress the tie and show how working in an unhealthy workplace will effect an employees performance and presentability.

[sherringbct-blog]

Data Physicalisation

Above is a data sculpture that represents the national statistics of water usage in the United States over 50 years from 1950 to 2000. The large cannon in the middle represents how much water is used and each of the three sections represents a specific category of used water.

Before looking at what data this sculpture showed, I thought it looked like a cascading waterfall due to ridges and the convexed shape of the table legs. An aerial perspective of this would show a map of the Colorado River, while I think this is an effective sculpture I don’t think it would fit its intended purpose of being a table as it would be unbalanced due to the lack of support on the outer edges of the table.

This sculpture demonstrates the use of a combination of data by showing the percentage of water usage in different categories as well as being a three-dimensional map.

This image shows a large scale data sculpture that visualises live wind patterns. Aluminium panels are hinged to move freely in relation to its surroundings creating ripple like patterns.

I like that this installation provides a multisensorial (sight and sound) visualisation of an otherwise invisible natural weather pattern which creates a conversation between man mad objects and nature. The randomness of this installation allows it to constantly change.

Above is one of seven interactive art installations in a series called “The Long Run”. These represents the cost of healthcare in the UK for seven different age groups which is shown through the time taken for a marble fall in each sculpture.

I like that this sculpture is interactive and uses time as one of its mediums. This provides a personal experience for the audience as if they are the marble going through the different bottlenecks and expenses that are involved with healthcare. Making it a marble run allows for a wider audience, younger children can interact with this by watching the marble go through each run. As they move up to the older age ranges they will notice that their marble will take longer to reach the bottom due to the increase in the cost of healthcare. For some the concept of time will be more understandable compared to the concept of money.

This is an example of a project that required many iterations and user testing as they had to match the time taken for he marble to go through each run with the statistics.

[sherringbct-blog]

Tangible Data Conversations Week 1

This week in studio we have been making low fidelity models out of cheap materials that could be used to represent data. We used materials like Spaghetti, wool, chalk, pegs, rubber bands, straws and popsicle sticks.

From prototyping and experimenting with these materials I have learnt that the most important thing for any project is to not waste time and start by doing. If we were given the brief with no plan as to what we were meant to be doing over the next few weeks, it is very likely that I would’ve spent this entire week looking for the perfect data set and statistics. Initially I was apprehensive about spending an entire week making models, but it has taught me to interact with the materials and learn from how they act in different states and with different materials.

Towards the end of the week a couple of friends and I started playing with the bag of spaghetti. We started off with playing with sticks of spaghetti and Jake decided to cook some. We then progressed to redrying the cooked spaghetti, freezing it and microwaving it to compare all the different outcomes with one another. After talking to Charles about our experiments he said it reminded him about the study done in 2015 where scientists tried to uncook an egg. The difference in the spaghetti states could be used in the future to show how everyone essentially “starts” the same but will have different physical appearances depending on their past and or their baggage.

I found the talk that Ricardo gave about the study on inequality in America to be very interesting, specifically the representation of the expected data compared to reality. After this was shown it started conversations between peers after seeing a visual representation of the stats. This highlighted the importance of the intended audience and how they react to what is presented to them. This was also seen when Stefan showed us his project he did with the bee hives. The bees reacted and interacted with the different models that they were presented with and remodeled the to suit their needs. It will be important to get into the habit of engaging with the intended audience throughout the course of this brief and for future projects.