This paper presents an overview of research leading to a concept for an art project using a personal smart-mobile-device with motion and sound sensors. The idea is to challenge the notion of Landscape, usually recognized as such thanks to visual perception, when the word “Landscape” is taken in its classical sense as meaning a particular framing of a piece of environment with specific topography and qualities. What would the perception of a Landscape be, if rather than being visual, it was generated by hearing sounds modulated by body movement, thus summoning up our topokinesthetic memory? How can we perceive and thus recognize a space as a Landscape? We will consider answers from environmentalists, anthropologists and neurophysiologists, before analysing some existing art works using sound and mobility.
The aim of this essay is to give an overview of different disciplines researched in my quest for a possible artwork using smart mobile devices. The idea is to explore an environment, restricted to a given space such as a “white cube” with no visual references. In such a place, removed from any clear visual meaning, we can easily focus on listening and other non visual perceptions; but the user’s hearing will be influenced through the use of topographic sensors plus a smart mobile device’s built in microphone. Might we then perceive a sound space as a whole? Might it be perceived as a Landscape? As Philippe Descola points out: “this interaction makes this place a landscape for this individual but perhaps not for his neighbour” (Descola 2014). Thus, the starting point of this research and the main object of this paper are to question the notion of landscape when perceived through movement and hearing.
First of all, I will explain how this question came up and how it leads me to take a look at ecological studies concerning sound within a landscape. Then, since perception of landscape is a cultural question, we will see what anthropology can bring us, more specifically concerning cultures where hearing dominates. Later, we will consider the neurophysiology of perception, in particular the topic of coordination between mobility and the senses, as well as the role of memory and of attention. Finally, we will briefly discuss some artworks using sound and mobility and see what we can learn from them.
Before starting, I wish to clarify that for my own research project I do not use the word “soundscape”, for two main reasons: firstly, different definitions exist but they are very specific to certain fields and they are not appropriate for my art research; secondly, in my mother tongue we do not have such a word: the word “landscape” (i.e. “paysage”) remains and is simply coupled with the word “sound” (i.e. “sonore”).
I started my research thinking of some analogies between a Landscape as a painting and a space defined by sound and mobility. How did I come to be interested in the idea of Sound Landscape? On one hand, for some years, I have regularly found myself in situations where I can no longer enjoy the pleasure of walking through some pleasant surroundings. The trouble comes from noise, mainly from so-called “anthrophonic” sounds (sounds produced by human activities, Krause 2014). On the other hand, while working on an art project about violence and voyeurism, I watched war documentary scenes where audible sounds were bucolic and gentle (mainly birds), where one might rather expect either dead silence or violent sounds as well as this same anthrophonic category. In both these non-fictional situations, real-world and documentary, I was concerned with the way in which the sound dimension can modify and question my perception: what were the environments and the situations I was perceiving and dealing with, precisely? I then started to look at Landscape paintings in a different way and realized that I had never asked myself before what the sound of Cezanne’s Mont Ste Victoire or the Van Gogh’s Les oliviers would be. What was the sound of these Landscapes for the painters? Did sound influence their perception and thus their creation? If we heard the sound how would it change our perception of these very well known art works?
It is not surprising to note that environmentalists and ecologists were among the first to become interested in the sound dimension. Indeed, the notion of “Soundscape” was first used in the 1960s by Canadian composer and environmentalist Raymond Murray Schafer and, shortly afterwards, by American bio-acoustician Bernie Krause. They had both noticed the evolution of the sound of different environments. During their research on this question, they realised how meaningful and valuable sounds can be for ecological research. Suddenly, through listening to sound, they could perceive aspects of evolution – modification perturbation – imperceptible through sight. Through their studies, they analyzed and categorised “Soundscape”. Murray Schafer’s definition identifies three main sound elements: keynote sounds (background), sound signals (foreground sounds) and soundmarks (unique to an area, like a signature). Krause (2013) talks about three groups of sources: geophony (sounds generated by non-biological natural sources such as wind), biophony (all sounds produced by non-human living creatures) and anthrophony (produced by humans). Both used sound recordings for long-term ecological studies and archived “soundscapes” as memories of places.
These concerns are all the more relevant today – even if methods might differ from country to country and from culture to culture. In 1994, W. J. T. Mitchell summed up the situation in a sentence: “Landscape is now more precious than ever – an endangered species that has to be protected from and by civilization, kept safe in museums, parks, and shrinking wilderness areas” (Mitchell 2002, 20). For him “Landscape is a medium” that can be a political, economical and social tool and marker (used by different cultures).
Yet, my concern is not with ecology. Rather, as mentioned before, I am interested in thinking about what I call “Sound Landscape”, where the word “Landscape” is taken in its classical sense of a particular framing of a piece of environment with certain topography and qualities. It coincides with the principles of landscape painting that involve, on the one hand, a “frame” through which to mark out a “point of view”, a piece of environment to focus on, and, on the other hand, the transformational process of “artialisation” (Roger 1997, 18). To be more precise, my concern is to understand what it means to perceive a “landscape” when it is defined by sound.
Thus, a landscape is perceived, so recognized as such through sight, when informed by a cultural perceptive model. The anthropologist Philippe Descola in his College de France courses specifies that: “the landscape is the result of an interaction which joins an individual and a place. This interaction forms a place as a landscape for the individual but perhaps not for his neighbour. The landscape does not exist in itself. It is a physical and cognitive mediation that objectivises a reality” (Descola 2014). He adds that we should not confuse “Sound Environment” and “Sound Landscape”, since the former refers to the direct audition of sounds in a place where we are present, and the latter requires both selective recording and a re-creation or composition – which is the artialisation dimension. But he also adds that “like a painting or a sculpture, sound recordings can provide a real experience of the subject. A sound landscape is heard – in the sense that its structure and what it refers to may emerge from the noise – only if we have already internalized the use of what the Landscape is.”
So the question remains: what is the perceptive schematic necessary for the recognition and experimentation of sound landscape? Actually, Descola mentions that Edmund Carpenter also developed the idea of soundscape in ethnology in the 1960s, while studying an Inuit group from Canada. He discovered that the Inuit world is defined through sound rather than through sight. Soundrevealspresenceswhose existence isprovenbythe echo of theiraction. More precisely, according to Carpenter, the soundscape is dynamic, always flowing, permanently creating its own dimension. A soundscape is not a surface like a Landscape (painting) but a sphere that spreads out in all directions from the listener. Similarly, Steven Feld worked for many years with the Kaluli (Bosavi) people of Papua New Guinea, whose culture is also one of hearing/listening. During 25 years, Feld learned how to listen, how to perceive a place through hearing and also how to interact with an environment through sound. He studied and made recordings of “ambient sounds that Bosavi people sing with, to, and about” (Feld 2004, 464). He showed how sound can create a place and how place can create meaning. “It’s not only their relationship to the forest but to each other. Sound raises the question of the indexicality of voice and place, to provoke you to hear sound making as place making” (Feld 2004, 465). This last remark brings up the question of territory. He specified that: “place resounds as a fused human locus of space and time. Local acoustic ecology can thus be considered as a kind of aesthetic adaptation, a naturalization of place, or, put differently, a pattern of ecological and aesthetic co-evolution” (Feld 1994, 11). He made many recordings but then worked carefully on editing them to give novices the possibility to understand this concept coming from hearing/listening culture. “When you hear the way birds overlap in the forest and you hear the way voices overlap in the forest, all of a sudden you can grasp something at a sensuous level that is considerably more abstract and difficult to convey in a written ethnography” (Feld 2004, 465).
It is noteworthy that these two non-visual, hearing/listening cultures (Inuit and Papuan) grew where the view gives hardly any information about space. The rain forest of Papua New Guinea is of such a density that sight is immediately blocked out by it, with no possibility to step back or to grasp any point of view. In contrast, the Arctic world of Canada is rather flat and uniform and without bumps or contrast to catch the eye. Thus, in both cases, listening opens up an otherwise impossible perception, giving access to dimensions of distance, time and depth, among others. Moreover, sound is the result of action, movement and vocal expression interacting with the milieu.
Sound is multi directional and requires attention; motion needs to slow down because our body itself is a vector of sound, as part of this sound space. To paraphrase the title of a book written by Michel Chion, for whom sound implies meaning and construction of a world, we become a “promeneur-écoutant” (listening-stroller) (Chion 1993). Even though we have a primarily visual education, when we walk to discover landscapes, our other senses are also solicited. We know that movement is central in our perception of the world. Alain Berthoz tells us, quoting Henri Poincaré, that: “a motionless living being would never be able to acquire the concept of space.” He specifies that: “sight and touch could not give us the sense of space without the muscular sense” (Berthoz 1997, 45). Furthermore, “topokinesthetic memory, memory of space, is memory of the movement based on body movements associated with visual or acoustic marks” (Berthoz and Mazoyer 1998). And a Landscape (even a painting) is a space that holds the promise of motion, of walking through.
Alain Berthoz speaks about the “cooperation of the different senses” while walking, thanks to the intertwined perceptive senses. Moreover, in a paragraph entitled The coherence between seeing and hearing of his book Le sens du mouvement, he specifies where precisely this perceptive coherence is built and its effectiveness:
“We found […] three sensory maps […] sharing the same neurons: a visual map […], a map of sound space […] and a map corresponding to the different parts of the body […]. These maps are unique to each species. […] The most important is that these three areas of perception share a common zone: the retinal space” (Berthoz 1997, 89).
In addition, “there is a mutual reinforcement between visual and hearing inputs for each neuron where these two modes converge” (1997, 90). Moreover, the walker has a body and a brain that roots him in reality. He is the one who, by his involvement, will give “reality” to the space. He constructs it through his motion. The coherence of the spatial representation enables the efficiency and accuracy of reaction thanks to the “predictive character of directional movements.” Thus, sounds inform us of the direction of the sound source, i.e. of the orientation of the space we are moving through.
Indeed, thanks to sight, we can anticipate the trajectory and, as pointed by Alain Berthoz, I “go where I look” (1997, 137). We could add: “I go where I hear”. Thus, “the multisensory nature of perception includes the presence of signals coming not from the senses, but from the intention of movement” (1997, 96). And the brain implements its mechanisms of anticipation of movement, especially by comparing the current sensory data to the data already encountered. What is far off becomes close, attesting Merleau‑Ponty’s hypothesis of “proximity by distance” (Merleau‑Ponty 1964, 170). Then, in turn, what is near becomes distant, in the constant ebb and flow of what surrounds us. A hierarchy of touching, of moving is established, that enables choice and marks a delay in acting; it slows down the rhythm of discovery, or rather, the construction of space. From sound to sound, like in a sound layered space, we move and perceive.
With regards to a pre-established schema of hearing/listening perception of Sound Landscape, memory must play a primal role. Henri Bergson, in his Essai sur les données immédiates de la conscience (1889), emphasized the subjective nature of perception and of awareness due to the filtering role played by memory (Bergson 1927). In Matière et mémoire (1896), he wrote that memory creates “the present perception, or rather it doubles this perception by sending back its own image […]” (Hyman 2000, 93). The painter Pierre Bonnard found herein confirmation of his research on sensations and his subjective and slow approach to a creative process based mainly on memory of feelings. While in the countryside, he took notes on site, making a large scale of brush strokes associated with feelings or sensations of any kind. For him sensations were in fact preceding and nourishing perception. He would certainly have appreciated the conclusions of Edelman and Tononi’s research that, in the year 2000, demonstrated that “every act of perception is to some degree an act of creation, and every act of memory is to some degree an act of imagination. Therefore, the biological memory is creative and non-replicative” (Edelman and Tononi 2000, 123). And more precisely, that “conscious perception and memory should be taken as two aspects of one and the same process” (2000, 204).
It should be noted that our “center of interest” seems to raise our consciousness: our perception “is reduced, in fact, to what interests us” (Bergson 1939, 38). This is what Gerald Edelman specified and confirmed when he wrote that: “attention highlights the fragility of consciousness: it focuses the mind on its subjects and abolishes or at least mitigates the surroundings. Indeed, it would seem that it is only possible to devote specific attention to some objects or to a succession of thoughts at the same time” (Edelman 1992, 224). Perception appears only as the limits of the thing and as a hierarchical division between the foreground and background of the world.
I will now describe four artworks using sound and mobility that mainly convoke our hearing and kinaesthetic sense and not necessarily visual perception. The intention is to investigate which kinds of spaces we perceive and whether some works can lead us closer to a “Sound Landscape” perception or not.
Karlheinz Stockhausen created Musik für ein Haus for musicians and “promeneurs-écoutants” in 1968. In his article Music in situ, the writer Guy Lelong recalls the innovative role of Karlheinz Stockhausen in working with movement in music. Before Musik für ein Haus, he worked on two major compositions. Gesang der Jünglinge (1956) is one of the first landmarks in spatialized electroacoustic music. The second, possibly more important, is Gruppen (1957), a new type of spatialized instrumental music. This work, which leads to an awareness of how we “perceive” the movement of sound in geometric space, “requires three orchestras in horseshoe-shape” (Stockhausen 1988, 84). Indeed, the composer
“renounced thereby the conventional layout of the concert hall and placed [the ensembles] around the audience. But above all these three orchestras were composed with similar instrumental formations so that the illusion of listening the same sound moving from one group to another could be achieved. Or, to put it another way, it is because this work is spatially indexed in the auditorium by the three orchestras that it can integrate a neglected parameter of musical composition: movement” (Lelong 1994, 33).
Later, continuing this research into movement, but also into the freedom of listening time, Stockhausen moved the audience around the musicians with Musik für ein Haus: musicians (therefore sound sources) were placed in fixed positions distributed in different rooms of a house. Thus, the listeners strolled from sound place to sound place, like museum visitors. Listening was fragmented by this polyspatial situation, being in this respect similar to the fragmented brush stroke of the painter and to the jerky mobility of sight in open space, although the different pieces of “sound” were in a certain way responding to one another. In addition, the piece incorporated the body kinaesthetically; therefore, it had an impact on the construction and the understanding of the geometric space. Somehow, listening was thus kinaesthetic: every move, every change of head orientation (so of both ears) in relation to the sound source’s position changed the sound reception.
More recently, in another manner, the visual artist Christina Kubisch has incorporated a sound dimension into her work since her first performance in 1970. She invited us to discover Le jardin magnétique (The Magnetic Garden, 2001) through movement and listening. The work refers to botanical gardens, which were originally used to study and classify nature, often from tropical countries, through collections of living plants. Although these plants are real, they somehow always appear to be fake and artificial. With Kubisch’s installation Le jardin magnétique the images “to be seen” were not as they appeared: they were not what was offered at first glance, namely artificial plants and long, thick cables. We had to get the “images” with our ears, in close contact with the false greenery, while sight was pushed into the background. With the aid of electromagnetic headphones – which also isolated us from ambient sound – we could listen to yellow and green electrical cables delimiting a “petrified and plasticized garden” – as it was called on the exhibition label. By following a path at will, walking, strolling between them, around them, toward “sculptures” of cables, we approached to hear the “out of frame” sounds they contained. The behaviour induced by this installation is the opposite to that which comes naturally. The sound-images discovered mobilize the other senses and therefore memory, which in turn leads to mobility; the body posture is guided by the “ear’s tip” – a sharp backward movement instantly accompanied with a grimace of disgust listening to a horrible flies’ buzzing! However, this installation does not lead the visitor to perceive, thus to build, a space as such – in fact there is no relation, no acoustic vibration between the different sounds that are heard. It is like a patchwork, a constellation of very vivid mini spaces evoked by sounds, with “blanks” in between them: through movement, we opened then closed a door to one world, then opened and closed another door to another completely different world, with no connection between them.
One of my artworks, Là-bas est ton ici et là-bas sera ici (2008) involved specific motion and listening actions – like Le jardin magnétique – but without portable devices. The installation was designed for a designated walk through scaffolding, while following and listening to stories of various biographies which were transformed, affected and enriched by displacements in the world, from place to place, from one culture to another. Voices moved slowly from loudspeaker to loudspeaker, carrying visitors in a slow and irregular walk. Various stories, various languages could be heard. Because of this specific set up, the experience was akin to being in an unknown place or a new culture. The visitor did not always feel at ease while searching, shifting to follow the sound scattered along the path. What was gathered from the story telling and what was experienced in other ways – such as hearing languages which were not understood and walking with caution on something unusual and which seems unstable – were parallel and complementary. The physical set up of the installation participated largely in the meaning of the work. The installation was shrouded in murmurs. Scaffolding grated under visitors’ steps. Here, rather than being a sound patchwork, visual, kinaesthetic and hearing/listening perceptions added up to build a specific world, whose colour varied with each step through the music of languages and the content of stories.
The last work we are going to look at is by Christoph Mayer, a remarkable site specific audio walk (2007) made for Gusen – a quiet, pleasant village in the middle of pastoral country-side in Austria. It was built on the former Nazi camps Gusen I and II. Some of the camp’s buildings are still present, transformed into housing. These homes where previously (or were built on the foundations of) prisoner barracks, the camp brothel, or even its gas chamber. Recorded footsteps give the rhythm for the walk and recorded vocal indications direct body movement and vision across the calm and charming village. Testimonies of survivors (former prisoners, former inhabitants from the neighbourhood of the camp, former guards and accomplices of atrocities, as well as current Gusen residents) lead to a strange and terrifying metamorphosis of reality of the village the visitor is looking at. The “promeneur-écoutant” is intensively involved in an astounding experience, at the limit of the conceivable. It is brought to them as an unimaginable clash between history and the present, between horror and everyday life. Although voluntary body movements to cross the village are guided by the vocal indications, emotions coming from the testimonies strongly affect the experience.
The aim of this presentation was to give an idea of the path taken by my current research. That is to say, to challenge the notion of landscape through movement and hearing/listening perceptions and through the use of mobile devices. We have seen that one of the main constraints is our cultural perceptive schema – for visual cultures or hearing/listening ones. But also the exploration of a space and its perception is done through the cooperation of our different senses and with the aid of topokinesthetic memory. This is why I would like to evoke Jean Rouch, the great ethnologist who said that “the only way to film is to walk with the camera, to take it where it can be most effective […] the camera becomes as alive as the men it filmed” (Rouch 1979, 71). But he also wrote, in 1973, about “tomorrow”, when “tomorrow will be the time of completely portable color video, video editing and instant replay (‘instant feedback’) […] a camera that can so totally participate that it will automatically pass into the hands of those who, until now, have always been in front of the lens” (Rouch 2003, 46).
Today is Jean Rouch’s Tomorrow. We have portable mini cameras and integrated cameras (plus microphones and loudspeakers) on mobile smart devices. However, with these mobile smart devices, we can also walk without paying attention to our surrounding, isolated in a private bubble of sound and personal messages. And yet, thanks to their sensors (among others those detecting movement and sound) we are able to record and collect data about aspects of our behaviour. The “subject” of study is now the person who holds the device, i.e. the “instant feedback” does not concern only people “in front of the lens” but the holder himself. Many aspects of our behaviour are influenced by our surroundings. Could this feedback, via sound modified by sensors, lead us to an increased perception of our environment? To be situated not in an isolated bubble but as active part of our surrounding? Could this feedback bring us a new perceptual paradigm and, why not, a new notion of Landscape?
Bergson, Henri. 1927. Essai sur les données immédiates de la conscience. Paris: PUF.
Bergson, Henri. 1939. Matière et mémoire. Paris: PUF.
Berthoz, Alain. 1997. Le sens du mouvement. Paris: Odile Jacob.
Berthoz, Alain, and Bernard Mazoyer. 1998. Sens du mouvement, mémoire du corps. Caen: CNRS. Accessed Mars 28, 2014. http://www.cnrs.fr/Cnrspresse/Archives/n355a4.htm.
Chion, Michel. 1993. Le promeneur écoutant, Essai d’acoulogie. Paris: Plume.
Descola, Philippe. 2014. Online Cours du collège de France 2012/2013/2014. Accessed November 10, 2013. http://www.college-de-france.fr/site/philippe-descola/#course.
Edelman, Gerald M., and Giulio Tononi. 2000. Comment la matière devient conscience. Paris: Odile Jacob.
Edelman, Gerald M. 1992. Biologie de la conscience. Paris: Odile Jacob.
Feld, Steven, and Donald Brenneis. 2004. “Doing anthropology in sound.” American Ethnologist, 31, 4.
Feld, Steven. 1994. “From Ethnomusicology to Echo-Muse-Ecology: Reading R. Murray Schafer in the Papua New Guinea Rainforest.” The Soundscape Newsletter, 08.
Hyman, Timothy. 2000. Bonnard. Paris: Thames & Hudson.
Krause, Bernie. 2013. “The Voice of the Natural World.” Online transcript, Accessed Mars 28, 2014. http://www.ted.com/talks/bernie_krause_the_voice_of_the_natural_world/transcript.
Lelong, Guy. 1994. “Musique in situ.” Espaces, Les Cahiers de l’Ircam, 5, 33.
Merleau-Ponty, Maurice. 1964. Le visible et l’invisible. Paris: Gallimard.
Mitchell, W. J. T. 2002. Landscape and Power. Chicago: The University Chicago Press.
Roger, Alain. 1997. Court traité du paysage. Paris: Gallimard.
Rouch, Jean. 1979. “La camera et les hommes.” Pour une anthropologie visuelle. Paris: Mouton.
Rouch, Jean. 2003. Ciné-ethnography. Edited and translated by Steven Feld. University of Minnesota Press.
Stockhausen, Karlheinz. 1988. Edited by Philippe Albéra. Contrechamps, 9, 84.
 Traditional empty gallery exhibition space is often called a “white cube.”
 Aline Veillat’s work. Accessed November 10, 2014. http://www.alineveillat.com/works/la-bas-est-ton-ici-et-la-bas-sera-ici/.
 “Instant feedback” or “audiovisual reciprocity” where the subjects in front of the lens can see and take part in the shooting, when the camera can pass into their hands to show something about themselves, about their world. The shooting becomes a collaborative action.