InfoMus Lab
Vscope
This project regards a HARP application for tracking, gesture acquisition of human movement, and its integration with sound, music and animated human models. In a few words, a dancer can control at different levels the music output and the expression of an artificial face by means of her movement.
|
|
|
HARP/V-scope at work. The user position is the black dot in the window on the screen. |
This HARP application is composed of four main subsystems:
- human movement acquisition modules, based on the V-scope sensor system, including data pre-processing and filtering;
- human movement and gesture recognition modules, including modules based on the force field metaphor, and others capable of gesture classification by means of self-organizing neural networks;
- (cognitive) symbolic modules, for the representation and supervision of contexts in music and gesture spaces;
- system output consisting of sound, music, and computer animation (expression of an artificial face).
|
|
|
HARP/V-scope at work in a computer music concert. The dancer wears Vscope markers (one is visible
on her hand) and intervenes in real time on the sound output of the acoustic instruments. |
|
|
|
Excerpt of the symbolic KB of the HARP/V-scope system |
An excerpt of the symbolic database, including a fragment of the basic ontology for this agent, is shown in the figure above: a visual language is used to define such symbolic knowledge. Double arrows represent inheritance, single arrows are relations, which can be defined between concepts represented by ellipses. This language is derived by KL-ONE. On the left part of figure the relevant situations for this application are represented.
For further information:
Authors:
Antonio
Camurri, Giuliano Palmieri, Alberto Massari,
Fabio Arena,
Roberto Chiarvetto,
and Riccardo Rossi