Use Case 4: Contemporary Music Composition


According to UNESCO, music is the most universal form of the performing arts since it can be found in every society, usually as an integral part of other performing art forms and other domains of the Intangible Cultural Heritage (ICH). Music can be found in a large variety of contexts such as classical, contemporary or popular, sacred etc. Instruments, artefacts and objects in general are closely linked with musical expressions and they are all included in the Convention’s definition of the ICH. Music that fits with the western form of notation is better protected. Nevertheless, those that don’t fit with the western notation are usually threatened. In any case, the key point for all music forms is to have access to the gestural knowledge of playing a musical instrument and the strengthening of the bond between the expert holder of the ICH, which is the composer or performer, and the learner. Motivated from this need, in recent years, researchers focused on the study of embodiment and enactive concepts. These concepts reflect the contribution of the body movement to the action/perception and the mind/environment interaction. In performing arts, and more precisely in music, body movement is semantically connected with gesture in most activities, such as performing and composing.

On the one hand, composers bring together knowledge and skills in sound colouring and organisation in terms of structure and form, as well as idiomatic gestures in musical and physical playing that lead to the organisation of the material, culminating in compositional structure. This always brings to the surface the question “how does this work?” According to various viewpoints such as those of Allen Forte, Arnold Whittall, Rosemary Killiam and Patrick McCreless, theory explains the facts. Leveraging on this, theory cannot be considered the ultimate mean for accessing musical knowledge. Therefore, theory can explain how a piece of music works but not how the composer neither the performer actually function.

On the other hand, performance is the result of the symbiosis between the musician and his/her instrument. This symbiosis takes the form of an interactional and gravitational relationship, where the musician is both a trigger and transmitter connecting:

  • perception (mediated instrumental mechanisms and physical environment)
  • knowledge (inherited music score of the composer, music theory etc)
  • gesture (semantic body movements encapsulating functional knowledge).


Consequently, the expert musical gesture can be considered as a fully embodied notion that encapsulates not only the knowledge of the performer to produce sounds and interpret music pieces, but also the knowledge of the composer communicated through the music score.

Moreover, the musical instrument is a tangible interface that can be considered as mean of musical expression and performance. Nevertheless, the learning curve of playing musical instruments requires years of training, practice, and apprenticeship before being able to perform. Furthermore, the learning of expert musical gestures is still viewed as a second-person perspective, which means as a communicative act of social interaction, rather than « my own » personal experience.  Concequently, « learning » musical gestures and « performing » music are usually perceived as separate concepts and experiences. This means that the access to knowledge is a long-term procedure since there is no quick transition from novice to expert (Figure 1).


Figure 1 :  State of the Art of learning/performing musical gestures



Τhe objectives of the i-Treasures platform for the Contemporary Music Composition use case are the following:

1.    Natural-User Interfacing the gestural expression and emotion elicitation in music

A transition from the classical perception of a musical instrument to a prototype of Intangible Musical Instrument is created. Moreover, it is provided a holistic approach on gesture capturing and mapping to sound taking into consideration the emotional status of the performer at the synthesis level. Finally, the IMI facilitates the musical skills development in terms of gestural expression and emotion elicitation in both autonomous and combined ways.

2.    Learning, performing and composing with gestures as a first-person experience

Put the user in the core of musical activities such as the performing and composing with gestures as well as appreciating music in general. Learning of expert musical gestures becomes a first-person experience since the learner has the possibility to gesturally control the sound excerpts from the expert.

3.    Augmenting the music score to facilitate the access to musical ICH

An enrichment of the standard music notation with augmented scoring that embeds gestural and emotional information. The musical knowledge of both composer and performer is provided to the learner through gestural and emotional annotations on the inherited music score about the expert performance.


Figure 2 : Big picture of the « Contemporary Music Composition » use case


The performing arts combine both the communicational (expressions, emotions, etc.) and control aspects (triggering actions, controlling continuous parameters). The performer is both a trigger and transmitter connecting perception,gesture and knowledge. A few years ago, the electronic synthesizer was a revolutionary concept of a new music instrument that was capable of producing sounds by generating electrical signals of different frequencies by pianistic gestures performed on a keyboard. Nowadays, the music production still depends on musical instruments that are based on intermediate and obtrusive mechanism (piano keyboard, violin bow etc). Many years of studies are presupposed in order to obtain a good level to (a) control these mechanisms and (b) read and comprehend a priori defined musical scores. This long learning procedure creates gaps between non-musicians and music. Additionally, even the emotional status of the performer is expressed through extremely limited effective gestures such as "keystrokes", "grand détaché" (sustained strokes) or "martelé" (sharp almost percussive strokes). Nowadays, the need of a novel intangible musical instrument, where natural gestures (effective, accompanying and figurative) performed in a real-world environment together with emotions and other stochastic parameters for controlling non-sequential music, is increasing.

The case study of the contemporary music composition aims to develop a novel intangible musical instrument. This digital music instrument will be a novel Multimodal Human-Machine Interface for the music composition where natural gestures performed in a real-world environment will be mapped to music/voice segments taking into account the emotional status of the performer. Thus, ‘everyday gestures’ performed in space or surface together with the emotional feedback of the performer, will continuously control the synthesis of music entities. Those music entities can be a) music patterns of classic composers (i.e. Beethoven, Haydn or Mozart), b) natural and instrumental sounds or c) segments of speaking/singing voice. This intangible musical instrument will not be only addressed to experienced performers, musicians, researchers or composers, but also to users without any specific music knowledge. The heritage of the classic composers can this way be available for everyone; it can be better preserved and renewed using natural body and emotional interactions.

If you are further interested in learning more about Cotemporary Music Composition, you can visit our Learning Management System

Figure 3: The world premiere of the piece "The Metamorphosis" by Leontios Hadjileontiadis. Click here to watch in youtube.