(This paper takes its title from Dr Oliver Sacks’ Book: The Island of the colorblind , Vintage Press, 1998)

The project I have created exists in outside the traditional domain of computer culture of physical installation. I have created a new sensation, a cyborgian extension of the human perception system residing in the brain of on student. Neil Harbisson.

I first met Neil at Dartington College of Arts whilst I was giving a talk on practical cyborg techniques and applications. Neil was especially interested in my earlier MyBorg project work that allowed me to extend my sensory system with 16 additional digital nerves including 4 light sensors on my back that allowed me to “see behind me” or to follow the cliché more closely “have eyes in the back of my head”. Neil became very excited about this idea.

He explained to me that he had a rare condition of achromatopsia (a rare hereditary vision disorder which affects 1 person in 33,000). One of the effects to achromatopsia is monocharmatism, the inability to perceive colour. To him the world was black and white. He explained to me how, in his paintings he had only ever used black and white paints, and when shopping for clothes he would only where black, white or grey. “Why should I wear something that I cannot see?” He explained. Ironically, because everything in his world was black and white, he never went shopping alone, requiring a friend to point out the black jeans as opposed to the blue jeans.

“I never used colours to paint because I feel completely distant to them.
Colours create a mysterious reaction to people that I still don’t quite understand.”

Neil had become aware of the existence of colours since his childhood, but he had been completely unable to distinguish red from green from blue. Even names of colours where useless even purely as labels to identify one shade from another. He described colours to me as “being an energy that I can’t see because it moves too quickly. I’ve imagined colours as fast moving energies.” Neil became curious as to the possibilities of a cyborg-like extension of his sensory system. A new input based prosthesis.

Several months beforehand I attended a lecture by Professor Kevin Warrick from Reading University, who described a project that enabled him to approximate proximity to moving objects whilst blindfolded. By using ultrasound sensors connected to an implant on his arm, he was able to feel a tingling sensation in is arm whenever something moved towards him. However, he noted that his brain quickly adapted away from providing a “tingly feeling” in his arm to being “the feeling that something is coming towards” him. In short, his project demonstrated that the brain can quickly re-map existing parts of the nervous system to new senses.

I decided that using Neil’s existing senses as a host for new artificial senses would be an effective approach. I gave Neil the difficult decision of deciding how he would like to perceive colour, since he could never “see” colour in the traditional way, I was curious as to how he would like to receive colour signals through his existing senses. Neil chose to use sound, as he felt that it would give him a good approximation of colour as he has very good pitch perception as he is a keen musician. I was confident that shifting colour into sound would be an appropriate and effective way of re-mapping Neil’s brain, as the natural occurrence of synesthesia seems to suggest that the visual and auditory senses can in some case become overlapping.

The case of how to convert colours into sounds was a difficult one. At first I experimented with the idea of playing different musical elements for different colours, for example Drums for red, Violins for Blue, piano for green and so on. This abstract approach however would only “label” colours with sound. Much like having a voice that whispered in your ear “red, red, red, green, green, green” for everything you looked at. After much consideration and collaboration with Eduardo Miranda it became apparent that I would have to create an audio experience that, like the light spectrum, would transcend labels. I used a physical model of transposing light into sound. After all, both light and sound are waves. Although light waves are far too high to hear, it is possible to mathematically transpose them down until they sit within the audible wavelength.

Clearly the lowest colour in the spectrum (Dark red) becomes the lowest note in the scale. I created colour to sound conversion software that would dynamically scale the colours from a miniature camera (A “spy” type camera) into audible frequencies. The audio output was not limited to the scale above. Instead of having one note per colour I wanted Neil to be able to hear subtle differences in colour, just as the human eye can distinguish between many different kinds of blue, I wanted Neil to be able to do the same.

I created software in collaboration that takes an average colour sampling from a selected area. This average RGB value (additive Red Green Blue, the signal most commonly used by computers) was then instantly converted into HSB (Hue, Saturation and Brightness). The software focuses purely on Hue. There are 360 different hues, one for each degree on the colour wheel. Each hue was assigned an audible frequency. This approach allows us to disregard brighter and darker variations (due to lighting conditions) and also to disregard colour saturation (The camera may over or under saturate colours depending on the environment) and instead gives us pure Hue perception.

Neil was able to run the software on a small laptop that he mounted in a backpack. The laptop was modified to run the software even when the lid was closed, allowing him to comfortably wear it in “sleep” mode. This meant that the battery time lasted for long enough periods for him to go through the whole school day without a re-charge.

Surprisingly, within 15 minutes of Neil using the system he was able to instantly recognise similarities and differences between hues, something he had never previously been able to do. Conclusively, this project exists not in the software, or domain of so called “virtual” reality, but in the reality of Neil’s perception of the world, unveiling, quite literally, an invisible architecture of energy.

The culmination of this project is the following statement:
“Neil Harbisson’s favourite colour is Red.”

If you have enjoyed this, why not check out these other interesting things?