One of the long awaited events of the AV2006 , the performance that perfectly summarises the theme of the festival this year, will be to see the robot musicians of the project Robot Music by the Japanese Suguru Goto, held at The Sage di Gateshead . The robot orchestra will play the percussions in a brief show on Saturday 4 th and Sunday 5 th of March.
Suguru Goto is composer, inventor and multimedia artist and is considered the spokesman of a new generation of Japanese composers tied to technical experimentation in the artistic field and to the extension of the artistic potentiality that exists in the rapport Man (creator) – and robot (composer). A quick visit to its website immediately highlights the work of Suguru Goto, for the least in his main works which are SuperPolm (an example of a performance with a virtual musical instrument), the BodySuite (an instrument of sensors for the interaction between body movement, audio generated and images) and finally Robot Music presented at the AV Fest this year.
In his most recent works new technologies mix together in experimental performances; he invented what are defined as virtual musical instruments , able to create an interface for communication between human gesture and the computer, where the sounds and video images are controlled by virtual musical instruments, through a computer in real time.
 |
.
Elena Varani: Composer and multi-media artist, you’re considered the inventor of Virtual Musical Instruments, which are the interfaces between gesture of human and a computer. In your opinion can robots even exceed the capabilities of a human players?
Suguru Goto: The physical limits of human can be easily gone beyond by the capabilities of robots, such as faster, longer, and more accurate etc. However, these are not only the capabilities, and there are more in music. If I only care about these, I don’t see that these are not only for the points of music. The robots can bring other capabilities in music, which we have never experienced.
The major advantage to “RoboticMusic” is that it interactively plays an acoustic instrument with the aide of a computer. There is no problem to play complex rhythm and it easily goes beyond the limit of human performance capabilities. Therefore, it gives new potentialities in a composition for acoustic instruments. Another point is an acoustic sound. While a computer generated sound has many capabilities, an acoustic instrument has rich sonority and enormous possibilities of expression, especially from the point of view of a composer. When it is played on a stage, the vast possibilities of the acoustic aspect are obvious when compared to sound coming from speakers. Another benefit is that the audience may observe both sound and its gesture of performance.
To master one instrument is huge task for a musician, but to play together with others in an ensemble is another difficulty. Having 5 robots, one may extend the new possibilities of ensemble. For example, “RoboticMusic” allows 5 different tempos at the same time, or intricate accelerando and ralentando, but theses are exactly synchronized in music.
 |
.
There is not only an artistic advantage with “RoboticMusic”, but also a research aspect. As one works more with a robot, which works with the gestures of a musician, one can discover how a human gesture contains complex movement, although it sometimes looks fairly simple, for instance, the gesture to hit a percussion instrument. A musician knows how to play an instrument, but he may find it difficult to explain exactly how he controls each part of his muscles and bones, and how he increases and reduces speed and intensity instinctively within a very short instant.
When one hears the word, “robot”, one is perhaps reminded of an industrial robot, or maybe sometimes a robot in a science fiction movie. However, it is not the case here. This is due to the latest development of artificial intelligence and is the case of application to a hardware. This has a lot to do with the robot, which performs instruments with a human-like gesture. In particular it refers to the humanoid type of robot that contains sensors and advanced programming, which allows the robot to control itself automatically. It differs from the slave type robot in a factory, and at last we can profit from this in the field music. One may consider these robots as collaborators with humans.
 |
.
Elena Varani: You said that robots can play music we’ve never heard before. You we’re talking about the technique, but what about the emotional part that a musician can add to his performance in this robot’ execution?
Suguru Goto: Before getting in depth for this discussion, one may perhaps clarify what the emotion is, and what the emotion in music is. Why have artists been always searching a new expression and they do not belong in the style, for instance in the style of 19th century? If we don’t deal with the expressions only, we may find many aspects in music, such as feeling, perception, memory, space, time, consciousness & unconsciousness, interaction, visual & oral causalities … We, all public give a context of understanding, and are free to feel something. Music doesn’t force you to feel, “here is sad and feel it!, here is happy and you ought to enjoy it!”. It is important that every single person gives his own perception in his own, whatever he experiences.
In Paleolithic times, human beings possessed an inferior cerebrum compared to modern man (the moment in human evolution when man more closely resembled modern-day primates than his current condition). As the brain developed, human beings also experienced increased athletic ability. In particular, growth of the brain lead to a high competency in walking while maintaining a balance on 2 feet and judging distance to obstacles, etc.. It is believed that, among the different parts of the brain, a strong distinction of function can be ascertained between the left portion and the right portion; one side is thought to control emotional and intuitive activity while the other logical thinking.
If we were to ignore what present computer science could offer – assuming computer science can only function in a similar way to the right side of the brain, the logical thinking side – we would indeed be a primitive species. As stated above, computer technology can function in a similar way to the right side of the brain to a certain extent. This leads to the question, can computer technology also be used to simulate left side activity. In other words, can we use computers to act out the emotional and intuitive activities of man?
 |
.
The act of performing music is not only about the control of a complicated set of body movements; but left side and right side brain activity is a necessity. That is to say, logical thinking, emotional awareness and body motion are all needed to perform efficiently.
On the one hand, music can be seen as a logical sequence of events over time. These events occur as a result of problem solving and rely on the interaction of a set of parameters. On the other hand, music derives itself from less calculable things such as ‘good’ rhythm sense and poetic significance and expression. Ultimately, is it possible for the robot to make music as we have defined it, i.e.: to think logically; to play with emotion; to have a good sense of rhythm; to realize poetic expression; to achieve proper pitch (frequency) and delivery of sound; to have a sense of proportion through comparison? Once we discover the answer, it could become the model on which the robot is structured. Moreover, this answer could be a key to understanding the complexities of the brain.
 |
.
Elena Varani: What about the audience reaction to these human-robot’performance?
Suguru Goto: Usually it is quite good feedback. It is widely accepted by not only artists, intellectual persons, and so on, but also from children to older people. Perhaps it is not only about the robot, but it performs musical instruments. The art work can be clearly understood by anyone easily. In away, it is a communicative robot with people.
A French composer, Edgar Varèse was once asked if he want too far away. His music was thought that it was too early at that period and went beyond the understanding of people when he lived. Varèse replied, ” it is not too early, but it is just too late”. Historically, art has been always relating with a society where it exists, and has been always profiting from a culture behind. As mentioned earlier, the robot here means the application of the latest development of artificial intelligence. Robotized instruments may have a lot of possibilities with this. At this point, this “RoboticMusic” is not too early, but is not too late. This is not an abstract image, but is just practically realized technically and aesthetically.
 |
.
Elena Varani: In your performance sound and video images are controlled by Virtual Music Instrument with computers in real time, but how much is important the perfect synch in these performance?
Suguru Goto: The perfect synchronization is one thing in terms of Interaction, but it is only a technical point as well as a perceptive point. This synchronization, like a style of film, may not be applied here. I would say that it is more important to see the trigger of causality between sound and image. In my case, the Interactive Video part is intended to not express any particular meaning. It may exist in parallel with sound. This can bring another domain of interactive perception between the visual and the auditory experiences.
The relationship between gesture and interaction can be flexibly changed during the course of a composition and the perception of interactivity can be integrated into a musical context. Not only is there this abstract perceptual level between gestures and their visual/oral results, but there is also a creative approach to how the physical actions can be interpreted into the digital domain, then there is how these signals can be efficiently expanded. Therefore, the relationship between gesture and sound synthesis is a crucial theme for further exploration. For example, the relationship between composed gesture and predetermined sound synthesis can vary from section to section in terms of compositional technique. This derives further performance models, such as, not only instrumental gestures, but also visual aspects: theatrical or conceptual performance models. While the Gesture Interface physically remains the same, the function of instrumental models can change in each section according to the different sound synthesis used. In an improvisational context, free gestures can flexibly produce varying sounds according to how much the sound synthesis programming allows the original sound to alter. However, the sound synthesis has to be sufficiently prepared beforehand; the selected sound may be simply changed during a performance. Free gestures can change pitch/velocity, timbre parameters, sample presets, etc. At this point, it is more a question of controlling the level of indeterminacy.
An essential aspect of my solo performances, which I call interactive performances, is the interaction between gesture, sound and image. With the “SuperPolm,” a single body movement can control sound and images at the same time. This relationship can be clearly presented in real time, and can be an unexpected and complex one. It is a concept that could undergo considerable development inasmuch as I can play with notions such as simplicity and complexity, for instance by triggering complex textures with simple hand movements. Sound, image and gesture play an equal part in these events.
 |
.
Elena Varani: And which is the next step in your research about real/virtual in art?
Suguru Goto: The project is just to be started at the beginning. I am working with a brass instrument robot now, and will work woodwinds and string in near future. The project is intended to form a robot orchestra eventually. These technical possibilities and the aesthetic points have created further new potentialities by this system, however, this is not my only goal. While interacting with this system and myself, creating a new language and its perception are the most important goal in terms of a composition.
Furthermore, one may consider again that his own identity is not merely limited within his possessed body. While doing this “Extended Body” like “real/virtual in art”, one may perhaps ask, if such an identity really exists or not, since there is not much limit or border line to extend. Music has to progress anyway, or it goes further by itself, whether we want it to or not. Likewise, a definition of music is limitless. Perhaps coexistence with this “Extended Body” may help to develop new possibilities in the composition of music.