Understanding the system

The Furhat System consists of a number of modules that send and receive events. Events can represent anything that updates the system, typically some action that the user has done which is perceived by a module in the system or an action that some module wants some other module to execute. By default, the system relays all events to all modules and it is up to each module whether to process the event or not.

Events

Each event has a name and a set of parameters. By convention, the name of events starts with one of the following types:

  • action represents things that the system should do, such as saying something, producing a gesture or turning the head.
  • sense represents things that the system perceives (such as a user saying something or approaching the system).
  • monitor represents feedback sensations that modules should produce when performing an action.

Sense events are typically created by sensor modules (speech recognizers, face trackers, etc). These are typically picked up by a Flow module that models the interaction and give rise to system behaviour. Thus, the Flow generate Action events which are then picked up by actuator modules (speech synthesis, facial animation, etc). The actuator modules send monitor events when executing actions. Such signals are essential to real-time systems, since they allow other modules to know when actions have been performed or are about to be performed, which facilitates synchronization, sequencing and interruptions of actions.

The following illustration shows how different kinds of events typically may be sent and picked up by modules:

  1. The DialogFlow module send an action.speech event which informs the Synthesizer to start speaking.
  2. The Synthesizer sends an monitor.speech.end event which informs the DialogFlow that the system is done speaking.
  3. The DialogFlow module send an action.listen event which informs the Recognizer to start listening.
  4. The Recognizer sends (several) sense.speech events which informs the DialogFlow that the user has said something.

Modules are free to invent their own events, but some events are standardized (see Events).