InfoMus Lab

HARP : Hybrid Action Representation and Planning

 

HARP is a hybrid architecture for the representation and real-time processing of music and multimedia knowledge. It was employed in a series of multimodal environment applications developed at the InfoMus Lab, like the HARP-Vscope, the HARP-DanceWeb, and the The Theatrical-Museal Machine.

HARP was used to develop applications in the fields of music, dance, interactive arts, theatre, and entertainment. The HARP software architecture is structured in a development environment and in a run-time environment. The former supports users in the design of applications.

The run-time environment supports the real-time, multimodal interaction and can be thought as a prolongation of the human mind and senses. The system is able to represent and carry out plans in real time for manipulating knowledge according to the user's goals. HARP is grounded on a hybrid integrated agent architecture. The audio compact disk from IEEE CS that accompanied the July 1991 issue of Computer magazine includes several music examples produced with an earlier version of the system (Camurri et al 1991). A previous version of the system is also described in a paper on Computer Music Journal (MIT Press), Vol.19, No.2. The HARP environment runs under Win32 (Windows 95 and NT) and has been implemented in MS Visual C++ and Quintus Prolog. HARP agents communicate by means of Microsoft OLE Automation and a sockets library. The development of distributed applications is therefore supported.

 
HARP.gif(14524 bytes)

HARP Development environment at work in the definition of the domain applicationontology.


The user can edit/browse a HARP KB by means of the HARP development environment. The figure above shows a window of the development environment regarding the editing of symbolic components. In particular, the definition of the ontology for the domain is shown, which is the first operation the user has to do for the creation of a HARP application. In the figure, a sample fragment of an ontology for the composition domain defined in Pope's SMOKE system is introduced.

The visual language, inspired to KL-ONE, allows the user to introduce classes (ellipses) of objects she needs in her application domain (e.g., a particular composition and performance space), including properties. Large arrows represent behavioral inheritance hierarchies, whilst small arrows with boxes represent class relations or properties (roles). Different kinds of properties can be defined (temporal, aggregate or part-of, feature, use, participant, input and output signal). Their kind can be visually identified by the different filling color of the small box in the center of the arrow (see the figure above). Then, rules and modules associated to classes representing subsymbolic counterparts (actions, tasks, and servers) can be introduced to complete the development phase. Once defined the domain knowledge, the user can introduce class instances.

The source code of the subsymbolic counterparts are accessible in the development environment by clicking on the corresponding class icon to which are linked. Subsymbolic components can be of different kind, according to the nature of the object: for example, a neural network software engine capable of recognizing properties of instances of a class can be hooked to the corresponding property box (role). The value of a property can be either the result of symbolic inference (the symbolic language is a sub-set of first order predicate logics) or the result of such subsymbolic computations, a sort of "subsymbolic inference" mechanism.

The HARP context manager has the role of exploring the KB and activating in the proper sequence the subsymbolic and symbolic modules to reach a goal, recognize a situation, etc.

 

Further information:  

 

Authors:

Antonio Camurri, Carlo "Minollo" Innocenti, Alberto Massari, Riccardo Trocca

 


Back to previous page