The MEGA System Environment
The MEGA System Environment is the main scientific/technological result and concrete product of this project. It has been used along the project in public events, including museum exhibits and artistic performances.
The MEGA System Environment includes the following
systems and components:
1. The EyesWeb
open software platform version 3.2.1, developed by DIST as a kernel.
2. A collection of software modules for analysis and synthesis of
expressive gestures, integrated or connected in the MEGA System Environment:
The EyesWeb Expressive Gesture
Processing Library (see description)
including a collection of blocks and patches
contained into three main sub-libraries (DIST):
- The EyesWeb Motion Analysis
Library: a collection of modules for real-time motion tracking and
extraction of expressive cues from human full-body motion.
- The EyesWeb Space Analysis
Library: a collection of modules for analysis of occupation of 2D (real as
well as virtual) spaces.
- The EyesWeb Trajectory
Analysis Library: a collection of modules for extraction of features from
trajectories in 2D (real as well as virtual) spaces.
– DEI Library providing a set of EyesWeb block divided into four
- Expressive Analysis with
the aim at analysing in real time the expressive content of a performance in
- Expressive Synthesis
aiming at conveying different sensitive content likes expressive intentions and
emotions by opportunely introducing systematic deviations into a MIDI score.
- Spatialization: a
library intended to provide a tool for the management of the virtual 3D space to
be simulated in generic Concert Halls.
- Generic Blocks: integrations to the EyesWeb Generic Library.DEI-KTH Expressive Director: based on the CSC-DEI Library, the Expressive Director combines two different approaches for computer modelling of expressiveness in music performance: the KTH rules and the CSC Expressiveness model. It allows an user to transform in real time the expressive intention of a musical piece.
IPEM Ear: a real-time auditory model as an acoustical front end and modules for real-time extraction of features, based on auditory model, related to loudness, roughness, pitch, onsets and rhythm, including statistical features related to time-dependent means, variances, densities. “Ear” is integrated within EyesWeb and also integrated within Pure Data (a more complete pd installation can be downloaded here). It is an attempt to model different aspects of human hearing. It consists of a set of 40 bandpass filters on a critical-band scale (cochlear filterbank). In each of the filters, there is an analysis of the loudness, roughness, virtual pitch and spectral pitch. These outputs can be visualized and processed in EyesWeb and Pure Data.
Musices version 2.5 (KTH), the KTH rule system for music performance,
summarizing 20 years of research in the field. It has been integrated into the
EyesWeb architecture for real-time control of musical expressiveness. A new
version of Director Musices (version 2.4) used for generating expressive
performances by rules was released. The new version includes the generation of
preprocessed data to be used in the Expressive Synthesis library by DEI. Several
new rules have been included and the midi input parser has been completely
rewritten. This version now works on most Windows operating systems as well as
GEM EyesWeb SoundSinanTools Library, intended to provide a set of tools for sinusoidal modelling and sound processing to the EyesWeb platform. Some of the features provided by the library are sinusoidal analysis, file generation from audio, sound resynthesis from sinusoidal analysis file, time-scale and pitch modifications. It is included in the next EyesWeb distribution.
GEM SoundSynth Library (see description)
GEM WK4Tools (see description)