The goal of the SIEMPRE project is to develop novel research theoretical and methodological frameworks, computational models, and algorithms for the analysis of creative communication within groups of people.
Research in SIEMPRE is focused on ensemble musical performance and audience experience, an ideal test-bed for the development of models and techniques for measuring creative social interaction in an ecologically valid framework.
In particular, the focus is on exploring interpersonal interaction in (i) musician-musician, (ii) conductor-musicians, and (iii) musician-listener scenarios. The overarching strategy of the project is to study the interpersonal processes that make live performance and listening unique phenomena.
We focus on three aspects of the phenomenon - entrainment (which creates physical alignment between the individuals); emotional contagion (which creates emotional bonds between them); and co-creation, by which both performers and audience contribute to shaping the overall event. We study each at two levels - between performers, and between performer and audience.
Entrainment, emotional contagion, and co-creation will be investigated both during an explicit communication process, such as that occurring between the orchestra conductor (or the group leader) and the other musicians, and during implicit synchronisation in emotionally intense experiences (joint music performance such as in a string quartet; audience live experience).
Real-time extracted multimodal features on movement, audio, and physiological data will be used to analyse expressive qualities, e.g., starting from research on multimodal expressive gesture (Camurri et al., 2005) and quantitative emotional estimation (Knapp et al., 2009): we refer to such multi-level features as expressive Movement, Audio, Physiological (eMAP) features. eMAP features are the inputs to the theoretical and computational models that will be studied in the project, capable to explain the processes underlying interpersonal creative communication. eMAP features will be extracted from participants (musicians, listeners, conductor) using real-time, synchronized, multi-modal feature extraction techniques.
The challenges and objectives of SIEMPRE are ambitious, and require cross-fertilisation among several disciplines: human-computer interaction, computer science, neurophysiology, psychology, cognitive musicology, artistic and aesthetic theories. Research challenges address the following questions: