Neil Harbisson was born with achromatopsia, a rare form of colorblindness that causes him to see the world in grayscale. But since 2004, he’s experienced color as sound, first through a set of headphones, and shortly thereafter through subtle sonic vibrations transmitted through his skull.
Harbisson’s electronic third eye takes advantage of the fact that both colors and sounds can be represented as frequencies. The tip of Harbisson’s antenna registers light frequency, then transmits that information to a chip implanted in his skull. The chip transposes that light frequency down 39 octaves into the audible range and then vibrates, creating a color-specific tone inside the artist’s head. Harbisson says his current implant registers 360 separate color-tones, plus ones outside the range of normal human vision. Together, these notes make up a soundtrack inside his head that changes with every movement of his antenna. [sciencefriday]
How Hearing Colors Affects Music [Interview taken from Munsell]
Munsell: You were a musician, right?
NH: Yes, this actually destroyed all my musical training, because hearing color is much more complex than hearing music.
I used to play the piano, which had twelve notes, and color has 360 notes in an octave, so to me the piano now is just too simple. There isn’t enough in an octave.
Munsell: So does that mean the entire visible spectrum of colors lies within an octave? That’s a lot of colors!
NH: A bit less than that, yeah.
Munsell: So there’s no repeat?
NH: No repeat, so when you scale down light frequency to sound frequency, then all the visual colors fit within less than one octave. So it’s a bit less than an octave.
Munsell: Do you think it would have been possible when they created the software to map colors to sounds in a way that would harmonize musically when the colors harmonize visually?
NH: Yeah, but I didn’t want to perceive color like this. I wanted to perceive color as pure as possible so I wanted to perceive the frequency of the light and then we could hear red we would hear a note between F and F sharp, which is for most people, this is already annoying because you can’t play it with a piano, because it’s between notes, so it’s a microtone. But that’s the actually frequency of red is between those.
Munsell: So it’s much more granular than the notes on a scale?
NH: Yes. So it’s when you listen to the frequencies of color, you need to, basically, forget about music. Because music only has 12 colors, it’s just twelve notes that keep repeating themselves. So it’s the same 12 notes in different octaves.
Munsell: Did that change your perception of music? If you hear music that is a little off key, are you more sensitive to it because you can better differentiate notes?
NH: Yes, color perception has given me a perception of sound that I didn’t have before. So thanks to color, now I can detect and I have perfect pitch within this octave which I didn’t have before, and also, it’s changed the way I perceive untuned music. I actually like hearing music that is untuned because it gives colors that are not usually heard in well tuned music.
Munsell: So it didn’t make you more sensitive, it made you appreciate it!
NH: Yeah, exactly. So, for example, like, schools, children’s schools, when they play [out of tune] it’s perfect! It’s so much more colorful than a perfectly tuned orchestra. That orchestra will usually play the same 12 colors whereas the school will play maybe 60 more colors.
When you hear color it’s extremely microtonal. In my case I have 360 notes in an octave, so, there are many more chances of creating chords that don’t sound good.
Meet a Completely Colorblind Man Who Uses Special Tech to ‘Hear’ Colors
Produced as a part of The Connected Series, Hearing Colors, is a short film that explores the life of Neil Harbisson, a man who was born with achromatopsia that leaves 1 in 30,000 completely colorblind. Through an antenna-like object implanted into the back of his head, Harbisson is able to gain a comprehension of the colors around him by hearing distinct sounds.
