Aisen Caro Chacin: Echolocation Headphones: Seeing Space with Sound

Abstract

Echolocation Headphones is a project that studies new applications for parametric sound technologies. This study emphasizes on augmentation of the auditory sense by enhancing our current ability of processing omnidirectional sound by providing a focal point to audition, similar to a visual focal point. Currently, human echolocation is being explored by the blind, who have reached an increased understanding of sound and spatial relationships. In other species echolocation is facilitated by different evolutionary traits that differ from the current human senses. These headphones provide the opportunity for focal audition similar to a focal point in vision, depicting a more detailed spatial image of the parameters of the space surrounding the subject.

The Echolocation Headphones are a new application for parametric sound that aids auditory spatial location. This study emphasizes the augmentation of the auditory sense by enhancing our current ability to process sound through providing a focal point to audition, similar to a visual focal point. Currently, human echolocation is being explored by some members of the blind community who have reached an increased understanding of sound and spatial relationships. This tool was designed with both blind and sighted individuals in mind. This technology might be beneficial for some kinds of blindness because it could provide a focused acoustic beam for spatial mapping, unlike omnidirectional mouth or cane clicks. It is also a training apparatus that allows sighted people to learn how to navigate space through sound. The Echolocation Headphones’ sensory-mixing functionality is to translate auditory processes to visuospatial skills.

Fig. 1: Echolocation Headphones. User: Galina Rybatsky. Photo by Vivian Xu.

Fig. 1: Echolocation Headphones. User: Galina Rybatsky. Photo by Vivian Xu.

Precedents

The perception of allocentric space is a visuospatial skill that includes navigation, depth, distance, mental imagery, and construction. These functions happen in the parietal cortex at the highest level of visual cognitive processes (Pullin 2009). An important function of the parietal lobe is to integrate sensory information, such as the manipulation of objects (Blakemore and Frith 2005). It includes the somatosensory cortex and the dorsal stream of the visual cortex. The dorsal stream is referred to as the “where” or the visuospatial processing, and the ventral stream as the “how”, such as the vision for action or imagination (Goodale and Milner 1992; Mishkin and Ungerleider 1982). Acoustics can also inform the visuospatial processes, because of the sound waves’ sensed timed arrival, processed by parietal cortex. Accounting for the speed at which sound reaches your ears can result in realistic effects generated in sound playback.

Professor Edgar Choueiri from Princeton University founded the 3D3A Lab to study fundamental aspects of spatial hearing in humans (3D3A, 2010). The lab has developed a technique to filter sound playback with a three dimensional effect, making it possible to simulate a fly circling your head using normal speakers – without headphones. He explains that our brains only require a few cues for locating the source of sound. The first being the time that it takes for sound to reach one ear versus the other, and the difference between the two time intervals. The second cue is the level differential of the sound arriving at the two ears. In order to achieve this same principle with two frontal left and right speakers, the sound must be recorded with two microphones equally spaced to the position of the ears on a human head. Dr. Choueiri has developed filtering techniques for 3D-audio playback that cancel the crosstalk between two speakers. This prevents the cues from the left speaker from arriving at the right ear and vice versa. These filters are effective in providing the brain with the right information for processing spatial audition from an artificial playback environment with no headphones required (Wood 2010).

This same principle of audio spatial attention used to recreate 3D-audio playback simulations can be applied for echolocation. Daniel Kish (Fig. 2) is the president of the World Access for the Blind (2010), an organization that helps individuals echolocate; their motto being “Our vision is Sound.” He describes himself as the real batman and has been echolocating since he was a year old (PopTech 2011). Kish has mastered echolocation through the interpretation of mouth clicks to gain understanding of his spatial surroundings. In some blind individuals, the brain has appropriated functions of the visual cortex to have a heightened sensitivity for sound in spatial processing. The dissemination of this technique has helped hundreds of blind people to regain “freedom”, as he describes it (PopTech 2011). People who are using this technique can navigate space flawlessly: ride bikes, skateboards, and ice skate, and have the ability to locate buildings hundreds of yards away with a single loud clap. The clicking is a language, that asks the environment “where are you?” with mouth sounds, cane tapings, card clips on bicycle wheels (World Access 2010). These clicking sound waves sent to the environment return imprints of their physical encounters, as if taking a sonic mold of space.

Fig. 2: Daniel Kish: Echolocation Methods for Spatial Interpretation. Image from: worldaccessfortheblind.org

Fig. 2: Daniel Kish: Echolocation Methods for Spatial Interpretation. Image from: worldaccessfortheblind.org

The Topophone (Fig. 3) was invented by David Porter Heap in the United States in 1880. Its purpose was to amplify the sensation of particular sounds. The wearable parabolas he created could be tuned to the pitch of the sound to be detected. The army later adopted his device to serve as a defense mechanism to detect enemy combatants (Porter 1897). This device became particularly useful once the airplane was invented, as it aided the location of airborne bombers. During the Second World War in the 1940s, the Allies created radar technology giving them the benefit and great advantage of spotting enemy planes over hundreds of miles. Radar utilizes the same principle of detecting the imprint of reflected sound by measuring the resonance of a semiconductor crystal, or a rectifier (PBS 1999). Without radar technology other WWII combatants had to rely on their own sense of hearing with auditory extensions, such as the topophone and acoustic mirrors.

Fig. 3: Topophone (left), Patent No. US590062, WWII Military Bomber Detector.

Fig. 3: Topophone (left), Patent No. US590062, WWII Military Bomber Detector.

Experimentation and functionality

The first instance of this idea surged while I was experimenting with a parametric speaker in an open backyard of a mountainous area in Los Angeles. I held the speaker to my ear, closed my eyes, and began to scan the environment. The imprint of my spatial surroundings reflected in the returning sound was perceivably accurate by at least 200 feet. This idea of holding the speaker to beam sound away from my ear was inspired by my then recently acquired knowledge of Daniel Kish’s experience with echolocation. It seemed to me that achieving his level of sound mapping mastery could only be acquired based on rigorous practice. People who are not used to perceiving spatial information through sound would normally have a hard time differentiating the surfaces of their surroundings 200 feet away based on sound alone, but by utilizing a parametric speaker the threshold for auditory spatial location lowers and its effect is more immediate. A parametric speaker uses a focused acoustic beam, and it does this by sending intermittent high frequency sound waves (Fig. 4). One of the waves remains at a constant ultrasonic signature and the other is equal, except it also includes the sonic frequency of the sound being played through it. For example one ultrasonic wave is 30 KHz, the other is 30+ KHz (plus the added audible sound wave). When the waves reflect from a surface they collide and their equal frequencies are subtracted to produce a differential audible sound, which is the added sound wave. This added soundwave can come from any device that plays music and has a headphone jack, such as music from an mp3 player. This is similar to the filtering that was being done by the 3D3A Lab in Princeton. Spatial behavior of sound waves can be calculated and affected programmatically by accounting for the time intervals between sonic signatures.

Fig. 4: Echolocation Headphones: Sound Interaction Diagram.

Fig. 4: Echolocation Headphones: Sound Interaction Diagram.

The parametric speaker utilized in this project (Fig. 5) was created by a Japanese company called TriState (Tri State, 2012). The availability of these speakers is limited and therefore they are very difficult to obtain. There are a few other parametric speakers such as the SoundLazer, a successfully funded Kickstarter project created by Richard Haberkern (2012). The original and more sophisticated version of parametric speakers is the Audio Spotlight by Holosonics, invented in the late 1990s by Joseph Pompei (Holosonic 2002). Naturally these speakers are of the highest quality in parametric sound and their prices reflect their sophistication. TriState offered the most economical and attainable speaker of this sort at the market at the time (Tri State 2012). The focus was directed towards the functionality of the device rather than the technical recreation of a parametric speaker.

Fig. 5: TriState Parametric Speaker. Image from: tristate.ne.jp/parame.html.

Fig. 5: TriState Parametric Speaker. Image from: tristate.ne.jp/parame.html.

Once the speaker arrived from its long journey from Japan, I began thinking about the integration design of the speaker as a wearable echolocation device. One idea involved the ability to control the rotation of speakers, similar to the dexterity and movement that animals such as dogs and cats possess on their ears. Should there be two speakers in order to inform both ears independently from each other? Thinking about a binaural hearing spun ideas of having two speakers but, while testing, it became apparent that the single speaker and the latency between the ears were crucial for the nature of echolocation – seeing through sound. It occurred to me that members of the blind community wear dark glasses. In the book Design Meets Disability, Graham Pullin (2009) describes glasses as a type of assistive technology. Appropriating the sensory function of glasses to be the placeholder for headphones seemed like the perfect opportunity for aiding audition. The priority of this interface is to place the sense of hearing at the center of the design (Fig. 6).

Fig. 6: Echolocation Headphones: Goggle Functionality.

Fig. 6: Echolocation Headphones: Goggle Functionality.

Welding goggles are designed to provide as much protection as possible from the harsh light that is produced when fusing metal. They are designed to introduce as little light as possible, which make them attractive for the acoustic-spatial training necessary for sighted subjects. The glasses are not entirely depriving the sense of sight purposefully, so that sighted users can get accustomed to mapping spatial information from sight to hearing. Another useful aspect of the inherent design of these welding goggles is the ability to open and close the obstructing panel holding the speaker. After incorporating the parametric speaker from TriState and the welding goggles, the next step was to add a 12V 1 Amp battery and charger circuit. This was the most important upgrade to the design because it allowed the user to have the freedom to explore their surroundings without being limited by the electrical connections to the wall. In order to create an enclosure for the electronics of the parametric speaker and the added battery charger circuit, I decided to incorporate concave resonator plates inspired by the Topophone. This was a great solution for integrating the electronics and the goggles, and the challenge shifted to finding the right plates. I searched for aluminium bowls and other concave surfaces that could amplify the resonating signals, and found the oddest solution, the rotary top of a trashcan. This solution is an example of the playful visualizing mechanics that occur in the ventral stream. Stripping objects from their function and adopting their form for other uses also functions with a similar process of visuospatial mapping (Fig. 7).

Fig. 7: Echolocation Headphones: Profile View.

Fig. 7: Echolocation Headphones: Profile View.

In order to add a compartment for the electronic components, I used a vacuum-forming machine. This technique would allow me to make a custom shape to fit the head plates. At first, I wanted to create a sort of helmet that kept the electronics on top of the head (Fig. 8). Through playing with the pieces of a lid and testing different forms of integration, it became clear that adding the compartment to the back of the head was a safer and more aesthetically cohesive design choice. Once the compartment shape was finalized it was vacuumed formed on to the amplifying parabola. Then, both forms were detailed with chrome trim originally used for car doors. This physical extension to the ears had a sonic amplification effect and helped the recollection of sound.

Fig. 8: Echolocation Headphones: Back View.

Fig. 8: Echolocation Headphones: Back View.

Evaluation of Potential

This case study also stimulates the mapping function of the visual cortex while aggregating spatial information; similar to the spatial processing of tactile maps on the tongue explored my PopMatrix experiment.[1] The tests displaying the feasibility of sonic visuospatial location through the Echolocation Headphones as a training device for sighted individuals are positive and more immediate than the electric pulses on the tongue. I have found a new application for the parametric speaker as a tool for echolocation that appropriates its original function by focusing the experience outward as a wearable device. Usually these speakers are targeted toward crowds, such as information sound beams used for museum or advertising displays. These headphones provide the wearer with the possibility of focal audition as they scan the surrounding environment. The differences in sound reflection inform a more detailed spatial image. This scanning method is crucial for perceiving the lateral topography of space. The constantly changing direction of the detection sound beam, the contrast in sound is what informs the spatial surrounding. The mp3 player connected to the Echolocation Headphones is loaded with a track of continuous clicks and another of white noise. I have found that noise is the most effective sound for this echolocating purpose. When demonstrating his method to an audience, Daniel Kish created a “shh” sound to show the distance of a lunch tray from his face. Noise works well because it provides a long rage of tonality at random, this eases the detection of level and speed change. Beyond the applicability of navigation, this tool is also useful for differentiating material properties such as texture. Sonic perception can also inform the somatosensory cortex about physical object based on the surface’s acoustic deflection. In one of his studies, Göran Lundborg tested the feasibility of artificial somatoception through a system of contact microphones applied to the fingers utilizing sound for tactile substitution (Lundborg et al. 1999). I conducted a series of experiments with different subject to determine that the use of the Echolocation Headphones facilitates the distinction of materials based on sound as the only informative stimuli. Subjects were able to identify distance and resonance within the first two minutes of wearing the device. I found that it is very easy for the wearer to identify metals and plastics in comparison to cloth and cardboard because of their different material porosities. The more irregular the surface the less likely it is for sound to reflect from the surface. This is why foam and cork are used to line the walls of recording studios. When experiencing the echolocation headphones, one of my test users, Joi Ito, noted a very beautiful analogy from noise to space. He said that in blindness one can finally see it all when it rains. The noise that occurs with the dripping of rain creates a detailed map of the environment. This notion of utilizing sound to navigate space is not particularly useful for sighted subjects, but this device depicts the experience of sensory substitution and it exemplifies the agility and plasticity of the brain’s perceptual pathways.

References

3D3A. 2010. “3D3A Lab at Princeton University.” Accessed May 20, 2013. http://www.princeton.edu/3D3A/.

Blakemore, Sarah Jayne and Uta Frith. 2005. The Learning Brain. Malden, MA: Blackwell Publishig.

World Access for the Blind. 2010. “Daniel Kish, Our President | World Access for the Blind.” Accessed May 20, 2013. http://www.worldaccessfortheblind.org/node/105

Goodale MA, Milner AD. 1992. “Separate visual pathways for perception and action.” Trends in Neurosciences 15 (1): 20-25.Holosonic Research Labs. 2.002. “Holosonics.” Accessed May 20, 2013. http://www.holosonics.com/

Lundborg, Göran, Birgitta Rosén, and Styrbjörn Lindberg. 1999. “Hearing as substitution for sensation: a new principle for artificial sensibility.” The Journal of Hand Surgery 24 (2): 219-224.

Mishkin, Mortimer and Leslie G. Ungerleider. 1982. “Contribution of striate inputs to the visuospatial functions of parieto-preoccipital cortex in monkeys.” Behavioural Brain Research. 6 (1): 57-77.

Mayer, Alfred M. 1880. “Patent US224199 – TOPOPHON.E – Google Patents.” Accessed May 20, 2013. http://www.google.com/patents/US224199.

Porter Heap, David. 1897. “Patent US590062 – TOPOPHON.E – Google Patents.” Last Updated 2011. http://www.google.com/patents/US590062.

PopTech. 2011. “PopTech : PopCasts : Daniel Kish: Blind vision.” Accessed May 20, 2013. http://poptech.org/popcasts/daniel_kish_blind_vision.

Pullin, Graham. 2009. Design meets disability. Cambridge, MA and London: The MIT Press.

PBS. 1999. “Radar During World War II – PBS.” ScienCentral Inc, and The American Institute of Physics. Accessed May 20, 2013. http://www.pbs.org/transistor/background1/events/radar.html.

Haberkern, Richard. 2012. “Soundlazer.” Accessed May 20, 2013 http://www.soundlazer.com/.

Tri State. 2012. “http://www.tristate.ne.jp/parame-2.htm – Topsy.” Accessed May 20, 2013. http://plus.topsy.com/www.tristate.ne.jp/parame-2.htm.

Wood, Mike. 2010. “Introduction to 3D Audio with Professor Choueiri.” Video. Accessed May 20, 2013. http://www.princeton.edu/3D3A/.

  • Footnotes
  • [1] http://www.aisencaro.com/pop.html, accessed January 20, 2015.
  • Born in Boston, MA, Aisen Caro Chacin is a regenerating composition of cells that produce a woman, a Venezuelan, a Spaniard, an U.S. American, and an animal whose patterns of migration are not based on seasons, but rather chance, chaos, and opportunity. Her curiosity led her to research the intersecting fields of art, science, and technology driven by conceptual forms of inquiry and design thinking resulting in functional prototypes. She is a professor of the MFA Design and Technology Parsons, teaching Physical and Creative Computing, The Material Spectrum Lab, and The Digital Self. Her radar is on Human Computer Interaction HCI- designing new interfaces for how we perceive the world; and discovering the limits of digital media. Featured as an inventor in Future Tech by Discovery Channel, awarded and published by PopSci, she looks forward to finding other suspended disbeliefs in her pocket. She is a creative technologist, a STEAM engine, a bucket of ideas open to merge and exchange with other buckets. http://newschool.edu, http://aisencaro.com.
  • PDF version of this article

Leave a Reply

Your email address will not be published. Required fields are marked *