When aboarding the subject of interactive sound installation it is natural to turn to David Rokeby‘s work (1960, Tillsonburg, Ontario). If one thinks back to the line of research that was based upon the Sound Sculpture study in order to find a definition for Sound Art, knowledge of Rokeby’s Very Nervous System (1986-1990) and his theory of ‘harmonics of interaction’ is fundamental for an understanding on the relationship between sound, movement, space and technology.
The installation Very Nervous System, despite its insubstantial appearance, conveys an amazing power: in the space of an empty room the simple movement of a body generates sound/noise. As soon as an arm moves, or even a single finger, the room begins to produce sound; the human body, as if in a dance, makes its own self-generated music. The music in Rokeby’s installation is chosen and modulated by the movements of the interactors.
In the latest version of VNS, exhibited in 2004 in Invisibile at the Palazzo Delle Papesse in Siena, Rokeby used sounds and recordings sampled form several Italian cities. These soundmarks (a Murray Schafer definition) IDENTIFY the explored meanderings of the body in the empty spaces. This installation is a device that uses a video-camera which records movements that are transformed into real-time by a computer processor, thus producing the musical phrases.
Rokeby says: “While the ‘sound’ of the system and the ‘dance’ of the person within the space are of interest to me, the central aspect of the work is neither the ‘sound’ nor the ‘dance’. It is the relationship that develops between the sounding installation and the dancing person that is the core of the work” (The Harmonics of Interaction, in “Music Works 46: Sound and Movement”, 1990).
This relationship takes place through an interface, designed by the artist, which has the key role in the artwork. “The active ingredient of the work is its interface Rokeby explains that is invisible and very diffusive, occupying a large volume of space… it is vital and strongly textured through time and space. The interface becomes a zone of experience, a multidimensional encounter. The language of encounter is initially unclear, but it evolves as one explores and experiences.”
This encounter is always active inside the space of interaction, with a feedback loop affecting the behaviour of the user. In this simultaneous relationship between human and space, the user can modify his behaviour according to responses given by the machine.
“The installation watches and sings; the person listens and dances Rokeby says but the relationship that develops is not simply that of a dialogue between person and system. A developing dialogue implies a separation of the functions of perceiving and responding. But for the installation, perception and expression are virtually simultaneous. As a result the installation and participant form a feedback loop which is very difficult, very complex.”
The installation when empty stands as in a silent awareness, a fragment of a nervous system: cameras for eyes, a computerized visual cortex, a complex set of filters and translators in the software, synthesized vocal cords and speakers for mouths. It lays in wait for the first movement of a body upon which, once it enters the perceptual range of the system, it engages its synthetic awareness to begin making sound.
The sound propagates through the air, enters the body and travels through the transformations of human acoustic perception, the labyrinth of the individual’s brain, and the tangled interface (psychological and physiological) linking mind and body, to manifest itself as further movement (or lack thereof) which completes a cycle and starts a new one… But the cycles blur into one another because as soon as one moves in response to the sound of the system, the system is already modifying its response to reflect those movements and so on.
Thus, body and system are drawn into constant readjustment and a sort of oscillation begins.
“The acoustic feedback of my installations exaggerates aspects of the acoustic nature of a room, the interactive feedback loop in my installations reinforces particular aspects of the system and the person in it, producing resonances. These can be thought of as the harmonics of the interaction and are unique for each different combination of installation and individual” (The Harmonics of Interaction. Op. Cit.).
From this experience we can obtain a particular redefinition of “harmonics”, where space is part and parcel of the installation coming in contact with device and individual. Let us focus on some Sound Art works and the way sounds used can produce perception of space. Sound is modified by our body, by the noise we produce or by the resonance between the interactor and the system. The key point is the corporeal perception of sound and how sound becomes a means through which we sculpt while it is perceived in its space and volume.
I think that David Rokeby’s intuition about “harmonics of interaction” gives inspiration for further study. What is most important in his work is not the fact that space generates sound (not only?) and this is a direct consequence of our bodily movements, but it is the feedback loop. This feedback loop upon which is created the new harmony evolves from a human element which communicates with the system whilst being inside a space, therefore producing the resonance.
When Rokeby designs the programs defining the reactive behaviour of the installation, he works in order to highlight the feedback, tuning it to reinforce the hidden physiological and psychological resonances of the user’s body and mind. “The music of the Very Nervous System installations Rokeby adds is not so much in the sounds that you hear, but in the interplay of resonances that you feel as you experience the work with your body” (The Harmonics of Interaction. Op. Cit.).
This is the starting point to create an environment-interface in which the relationship between individual and system (“harmonics of interaction”) generates an added layer of meaning.
“Space becomes a volatile instrument experienced through space as well as time, responding to the movements of the user, participating to an encounter which is a musical composition” (David Rokeby in Dreams of an Instrument Maker, in “MusicWorks, n°30” 1995). When referring to several sound sculptures, for example Harry Bertoias’, Crystal Baschet by the Baschet Brothers or sound stones by Pinuccio Sciola, VS seems to have naturally evolved from these works. One could imagine VS as a kind of musical hyperinstrument where the user causes his own body to produce sound.
The extreme selectivity of the interface, which in the case of VS only comprises the perimeter of the movement, is important for the spectator’s experience of the installation. On the one hand, it is considered a mere parameter, on the other hand it acts as a more restricted range of action that the system can control. The interaction feedback is felt more directly by the user. He can clearly listen to his corporeal response, since he can act in the space only through movement.
It is possible to suppose that “harmonics of interaction” creates a new kind of spatiality which is halfway between an acoustic corporeal perception and a space of resonance with the system. Yet it is also a spatiality that is a mirror of our behaviour in the world.
In Transforming Mirrors, issued for the exhibition Invisible, Rokeby talks about interactive technology as a medium through which we communicate with ourselves. What is reflected upon is ourselves, transformed and processed. Technology reflects a recognisable image of ourselves, it provides us with a self-image, a sense of self. At the same time it transforms our image in the act of reflection, it provides us with a sense of the relation between this self and the universe.
The example the artist uses is the myth of Echo and Narcissus, told by Ovid in the Metamorphoses. Echo, forced to express her love for Narcissus by repeating back the words he used to reject her, tries to transform his words of rejection into an expression of her love. “Like Echo Rokeby says the interactive artist transforms what is given by the interactor into an expression of something other.” Such distorted reflection is the key to an encounter-feedback between ourselves and the world.
The reflection of Narcissus works differently: the perfect synchronous mirroring (Narcissus is the mythic archetype of that) produces the closed system of self-absorption in which the reflection of the self is re-absorbed. On the contrary, in the distorted reflection a stereoscopic mechanism is activated, one similar to that governs the function of human sight: the different image seen by the left and right eyes, with their two different points of view, necessitate a tension.The tension existing between these two points of view is resolved by the brain into the revelation of depth. For Rokeby, the relationship between the interactor and the transformed reflection is stereoscopic too. (Trasforming Mirrors da AA.VV., Invisibile, curated by Emanuele Quinz, exhibition catalogue, Siena, Palazzo delle Papesse Centro Arte Contemporanea, 2004).
The definition of this possible stereoscopic relationship is a useful metaphor enforcing the issue of the relationship between individual and system. Similarly, it is inevitable on an operative point of view when an interactive system is created and its interface is designed.
The idea of “harmonics of interaction” probably needs further development-examination, particularly its spatial dimension and its relationship with sound. When Rokeby describes the way people live their experience in the installation VNS, he tells how people first test the system by making the same movement in order to get the same sound. When they manage to do this they believe they have control over the system, but then they make a different movement which will produce a different sound.
At this point, I think it is important to underline the key role played by the reproduction of sound in VNS. The theoretic reference is the Refrain theory stated by Deleuze and Guattari in Thousand Plateaus.
These two philosophers define Refrain as a “strategy of appropriation of a space (or time), or more specifically a sound dimension of a territorial assemblage.” A sound defines an identifying space: like the refrain of a child to exorcise his fears, or the refrain of a bird to mark a territory. “Refrain is a territorializing mark, a signature. A signature that defines a territory” (Deleuze Gilles, Guattari Félix., Millepiani. Capitalismo e schizofrenia, Roma, ed. Cooper, 2003, p.454).
I think that the user of VNS produces this kind of “mark” and territorial definition unconsciously through a repetition of movements/sounds. Indeed, the repetition of a movement, which causes the listening of the same sound, produces the beginning of the interaction feedback loop. Such repeated action produces the beginning of territorialisation that defines a spatial limit between our body and the system.
It can be useful to remember that for Deleuze and Guattari “to the definition of sound emission as strategy of territorialisation correspond a definition of listening as strategy of individuation.” ( In AA.VV., Millesuoni. Deleuze, Guattari e la musica elettronica, curated by Roberto Paci Dalò and Emanuele Quinz, Napoli, ed. Cronopio, 2006, p.21).
The experience produced by VNS is enriched by another value. In the simultaneity of movement and sound we can see the creation of a cycle made up by territorialisation and individuation of self on one hand there is the defining of a border, and on the other hand we identify ourselves in relationship with the system.
So, the “harmonics of interaction” can be considered as a spatial element of territorialisation as well as a mirror identifying the self, in which we can find a new kind of subjectivity. Once again the ideas of Deleuze and Guattari, reported by Emanuele Quinz, prove themselves useful: “As a defining strategy, territory implies a distance, an otherness. Territorializing a space means defining its frontiers, distances building and identity, a determination in a changing and plural universe” (AA.VV., Millesuoni. Deleuze, Guattari e la musica elettronica. Op. Cit.).
The idea of “distance”, as a consequence of territorialisation and identification, can be compared to what was previously mentioned about the stereoscopic relationship between user and reflection of self inside the system. We can think of a stereoscopic distance in the individual-system relationship in which we are the right eye, the system is the left eye, and the feedback is the tension necessary to reach a sense of depth.
We obtain the “harmonics of interaction” capable of creating a new kind of spatiality in which the user (through the sound) can territorialise his own presence creating a stereoscopic distance from the feedback relationship, thus redefining a new individuation of the self.
http://homepage.mac.com/davidrokeby/vns.html
http://www.fondation-langlois.org/html/e/page.php?NumPage=2186