Joel Cahen: Strategies on Sound Based Augmented Reality Theatre

Abstract

In sound based augmented reality theatre, based in locative media, the function of sound as contextualiser of experience calls the listeners imagination to attention as an active correspondent to the city around them. This essay discusses the role of sound in shaping the personal Auditorium, offers a basic strategy for the sonification of psychogeographical affordances in an urban space and concludes with an example of how locative media can utilise these elements to dramatise a soundwalk and, by doing so, expand the realm of performer and audience relationships.

 

Header image

Introduction

In public spaces the individual encounters a multitude of sounds. Environmental sounds contain basic sonic signifiers, banal imperious announcements, passing conversations, animal sounds, machine sounds, fragments of musical notes; all mix within the spaces enveloped by architectural structures to create a complex and fluid sound field that helps shape reality.

In my recent work, Interzone Theatre, this sound field is altered entirely. The individual, wearing headphones, enters a parallel interpretation of the sound field that augments psychogeographic affects with fictitious psychoacoustic ones. Using locative media, the narrative unfolds as the individual progresses through the city; encounters with pedestrians and urban features are imbued with an alternate narrative. Here the function of sound as contextualiser of experience is to call the listeners imagination to attention as an active correspondent to the city around them. In such interventions, the public space is the “objective” reality that is being augmented by the listeners imagination, shifting attention and re-contextualising to create an experiential space.

This experiential space is site-specific and person-specific, a personified Auditorium. I describe the word Auditorium here as a psychic space drawn and constructed from a Gestalt of perception, an affect of the environmental on the personal; a spatial, cultural and experiential framework that contextualises the meaning of a state of being at that moment.

The essay begins with a description of the role of sound in shaping the personal Auditorium. It follows with a basic strategy for the sonification of psychogeographical affordances in an urban space and concludes with an example of how locative media can utilise these elements to dramatise a soundwalk and by doing so expand the realm of performer and audience relationship.

Shaping the Auditorium 

Traditionally, performance happens in an Auditorium, which as the auditory root of the word suggests, originates from Greek for “a place for listening”, for paying attention. Thus the auditorium gives context to the spectacle; it is through the framework of its seating arrangement, comfort, programme note, the cultural background and composition of its audience that the spectacle obtains context and is given meaning. In a mobile theatrical experience, considering that the spectacle is all around us, changing with our movement through it, the Auditorium is self-constructed within each listener, a psychic space that, prompted by sound, provides context for the experience.

A theatre auditorium is a segregated enclosure into which one enters wilfully with the intention to focus one’s attention on the activity presented in it, yet without losing the knowledge that this space exists among other spaces within the city and throughout a temporal continuum. This focal aspect is an active listening process that is intended to isolate sounds, actively segregating the “heard” and the “listened to”. This process can be prompted by verbal communication, for example by way of introduction: “Close your eyes now and listen to the sounds around you”; or by way of intrusion.

Fig. 1: Focal and peripheral aurality

Fig. 1: Focal and peripheral aurality

As an example of intrusion, at a conference I attended, one of the speakers was talking about the emotive impact of the soundtrack of Jaws on the viewing experience, while a live rig setup in the courtyard downstairs started blasting AC/DC’s Highway to Hell. Within the context of the lecture room auditorium, the music was a nuisance, a loud interference with what was previously framed by the expected sounds of the lecture room. By including the music to the discussion within a wider spectrum of attention, the tension created by Highway to Hell resonating from the window, augmented the somewhat dry description of the Jaws soundtrack, to create a similarly unnerving experience. The sound coming so clearly from outside drew attention from the speaker to the larger space outside the room. As a result, the Auditorium’s potential to set a context that encapsulates these external sounds as well grew and the listeners were forced to re-examine their fidelity to the original framework of the Auditorium to which they willingly committed by entering the lecture room.

Attention is continuously challenged and swept away by the fluidity of being in an environment, be it spatial, social or solitary. Attention conducts our experience, or perhaps our experience is the journey through attention. This affect of the environment on the self contributes to my description of the personified Auditorium in the introduction to this essay above.

The resizing and reshaping of the personal Auditorium is a creative act. Whether instructed by an external authoritative prompt, manipulated in an artistic creation and/or by personal choice, the act of listening is the wavering of attention between multiple focal points (LaBelle 2012). Sound in its fluidity and as a signifier of physical and cultural spaces, plays a significant role in the resizing of the Auditorium. It expands or contracts its perimeters by channelling attention to different elements that constitute it. The interplay and fluid exchanges between the focal and the periphery of aurality may be useful in giving a perspective on the effect of sound on the construction of reality, which is as wavering and fluid as are the properties of sound and its behaviour in space.

Psychogeographic Considerations in Sound Design 

The changing spatial relations between the individual and the static architecture, the movement of traffic and of other pedestrians, can invoke a fluid sense of being. In a Situationist dérive, attention is not directed, it randomly wanders according to the individual interpretations of audio or visual events that unfold throughout the walk, from the static architecture, buildings and cityscape on the one hand and the dynamic buzz of pedestrians, traffic and their respective sounds on the other. The psychic space derived from this drift of attention, which develops in the mind of the wanderer, is sparked by the friction between two contrasting elements, the static architecture and the fluid activity and sound within it. The dérive becomes a sequenced composition, a platform for imagined and projected journeys, a Gestalt forming the personal Auditorium.

In the urban architectural space, the fluidity of sound is funnelled through the streets, streaming and filling the spaces in-between brick walls, scattering into the open spaces of parks and public squares, resonating and reflecting off different surfaces, amplifying quieter sounds or dampening loud ones. Urban structures create a dynamic envelope shaping noise, the canyons created by rows of buildings can offer a compression envelope and frequency resonator, in the same way that parks and open spaces could offer a wider filter envelope.

Psychogeographical attributes of public spaces, whether they are fixed closed spaces or open spaces, are based in their geometrical, monumental, historical, cultural and dynamic attributes. These attributes evoke sonic associations and the sound designer for headphone listening may exploit these by imbuing the environment with an additional and perhaps even conflicting layer of meaning – redefining the experience of the site and augmenting it with renewed meaning. The insertion of sound design within a psychogeographical perception affords what Judith Rugg (2010, 148) describes as: “Illusory constructions of spatial meanings, the potential of parallel, normally unseen realities and the integration of the real and the virtual in presenting possibilities of perception and understanding of space.”

However, first and foremost, architectural spaces are geometrical spaces and as such have a sonic signature immediately recognisable by the perception of the particular resonances of the sound heard in them. In an attempt to break the strict mathematical relationship that geometrical space imposes on sound for the purpose of freeing up metaphorical associations that could be worked with in a fictional locative story, what follows is a table associating geometrical attributes of architectural structures with attributes of the sounds that exist or could be designed within them.

ARCHITECTURE SONIC ATTRIBUTE
Solid. static Liquid, fluid
Functional by use Informs of agency
In-betweens, alleyways Segues and silences
With boundaries Overlapping, seamless, abruptness
Vertical horizontal Omniphonic, directional
Voluminous Loudness, feelings of significance between diminutive and apotheosis
Graphic and incidental elements within the architecture and urban features like signage, bollards, bins, lamp posts A graphic score, individual notes on a score sheet, ephemeral sonic clusters of meaning

Other psychogeographic attributes of spaces such as cultural and historical contexts are conveyed by architecture design and use, places of worship, industrial areas, styles such as Art Deco, Graffiti in abandoned places, leisure centres, etc. These contexts have a sonic signature associated with them through the narrative expressed in the site’s use, either musically through its associated cultural artefacts or through the imperious sounds that accompany its function.

Open spaces such as Tiananmen Square, Tempelhof Airport, Tahrir Square and the Killing Fields are loaded with dynamic and social significance that may not be entirely obvious to an uninformed visitor, this is largely because their significance comes from activity that they hosted in the past. Sonically these may be expressed as cultural artefacts or audio documentation.

Other open public spaces such as open-air markets, playing fields, a busy junction in a city centre or a stream in a park, afford very particular sonic environments that are expected and often recognisable even without being at the site, such as the sound of birds in the park, traffic, etc.

All these attributes, like any compositional tool, are open to manipulation and play. Their sonification can add an authentic element within a designed narrative, or they can be extracted and manipulated to provide a thread in the multitude of possible stories that can be followed and developed. In the experiential environment constructed by the use of headphones in a public space, where the narrative can go in so many directions, such a thread can be a useful trajectory in the creation of a locative story, whether it is expressed through sound or visually by performance.

Locative stories draws the user into an alternate perspective where psychogeographic attributes such as particular building angles and alignments along with other urban features reveal a narrative thread connecting the imperceptible to the spectacular in an individual journey of discovery. These aim to increase attention to the surroundings and to details within them including pedestrian activity and rhythms, and creates correlations between otherwise disparate elements of the architecture, and pedestrian and cultural activity.

Interzone Theatre and Augmentation of Fictive Narrative

“[…] The nature of mixed reality and of performance is complex and hybrid, involving multiple spaces, shifting roles and extended time scales, all of which are connected in multiple ways through diverse forms of interface.” (Benford and Biannachi 2011, 7)

Working with app developers Mobile Explorer, I created the Interzone Theatre App, a platform for locative theatrical productions. These aim to reposition the user’s relationship to urban space using sound, image, performance and the creation of site specific stories that are based in the psychogeographical attributes of the locations in which they are set.

Common elements of many locative performances across the sector are the sense of play, role-playing, the delegation of tasks to an individual within an audience group or the sharing of information between participants. Interzone Theatre aims to explore the immersive theatrical element of augmented reality performance by eliminating the sense of presence of the theatrical event through minimizing audience interaction with technology (especially mobile phones) and dissolving the sense of being part of an audience by making the event available as a solo activity for the participants.

Interzone Theatre replaces the sounds of the environment with a fabricated sound space using headphones; these give access to a parallel world, and form the theatrical space, the personal Auditorium. Local information extracted from research of the locations, is processed and mixed to form a story. It is then fed back into the site through the participant’s imagination to activate a superposition of meaning that recontextualises the location and their experience.

The locative story blurs the functionality and linearity of the urban fields and “smoothes” them out (in Deleuzian terms, see Deleuze and Guattari 1987, 371), with poetry and abstraction in sound, narration, image and performance. It creates a liminal experience between the real and the imagined that differs from the traditional theatre experience in that there is physically no place to leave the Auditorium to; the audience does not inhabit the theatre, the theatre inhabits the audience and travels with them wherever they go.

One of the productions I created for the platform is The Hero With Seven Faces. It exists in three editions, Belfast, Zurich and London. Each edition has the core narrative of the seven archetypes (The Hero, Monolith, Informer, Pool/Tower, Catalyst, Inhibitor, The Fisherman) mapped onto architectural features in the city alongside a site specific inspired theme developed, after a period of research in the area identified as the site, according to the psychogeographic qualities of the space, its spatial, cultural, historical and pedestrian characteristics.

The London edition of The Hero With Seven Faces – The Panopticon, is inspired by the Panopticon shaped council estate where the tour begins, which happens to be in the vicinity of Jeremy Bentham’s (Designer of the panopticon) London home. The tour continues to a large mirror underneath CCTV cameras, to a representation of a Ziggurat, ancient place of worship, beside three tall towers to an image of the Eye of Horus (the All Seeing Eye) found etched on the ground and ends with the participant passing in the middle of a thin traffic island in between two crowded bus stops and bus lanes inspected by pedestrians on either side. “Look inside the Eye of Horus, Does the All Seeing Eye offer Protection? What quintessential identity lies behind it? Is it a reflection?.”

 

Performance is used covertly, imperceptible to pedestrians. Performers occupy various areas and compliment the narrative with slight actions even just by standing in particular places or glancing; or through direct interaction with the audience member in places such as phone booths or park benches. By inserting these sync points between the augmented narrative and the reality it refers to on the dynamic level, the potential arises for the audience member to create synchronicities between the narrative and various urban activities that have not been planned. The audience’s imagination begins to interact with the urban dynamics and buildings creating a personal experience that is a hybrid of the objective realities presented to them from the environment and the fictive narrative of the production. This interaction forms the basis of the augmented reality theatre experience.

Conclusion

The use of sound as a contextualiser of the personal Auditorium in a public space provides a platform for articulating a contemporary practice of mythmaking between individuals and their surroundings and illuminating and augmenting aspects of the urban space that may have been neglected in its daily use as a background for commuting between interiors or flattened onto the screen of a digital display in touristic voyeurism. It enables new methodologies for engagement and strategies for embedded narrative in public spaces, which can be applied in a variety of art sector contexts including museum and local history outreach.

The platform can also develop the dramaturgical scope of performance to inhabit the real and the imagined, using the omniphonic properties of sound to expand the mise en scène beyond the directional visual panorama. The technology enables implementation of gaming strategies to include participant decision-making, multiple narratives, smart props and flexible performer activation.

While the experience I describe relies heavily on technology, it is my intention, as a practitioner, that with its miniaturisation and the new horizons it opens, it will help reposition the participant’s relationship to architectural, urban space and the diversity of its inhabitants, outside the Auditorium it creates, outside the theatrical experience.

References

LaBelle, Brandon. 2012. “On Listening.” Kunstjournalen B-post. Accessed January 20, 2015. http://www.kunstjournalen.no/12_eng/brandon-labelle-on-listening.
Benford, Steve and Gabriella Biannachi. 2011. Performing Mixed Reality. Cambridge, MA-London: The MIT Press.
Deleuze, Gilles and Felix Guattari. 1987. A Thousand Plateaus. Capitalism and Schizophrenia. University of Minnesota Press.
Rugg, Judith. 2010. Exploring Site-Specific Art: Issues of Space and Internationalism. London: I.B. Tauris.

PDF version of this article

Joel Cahen is a creative producer (Newtoy). With a background in sound design and music for stage and screen productions, Joel’s interest covers exploring sound in spaces (cultural, social and physical spaces).

1 comment for “Joel Cahen: Strategies on Sound Based Augmented Reality Theatre

Leave a Reply

Your email address will not be published. Required fields are marked *