Personal tools
You are here: Home Projects humaine EARL Open issues and questions

Open issues and questions

This is a collection of open issues and questions that were raised by HUMAINE partners and others, and that have not yet been addressed in the current version 0.4.0 of the HUMAINE EARL specification. Comments are welcome! Add them below.

The limits of what should be annotated

It remains an open issue where to draw the line between the emotional state “itself” and external factors. Two prominent types of borderline cases have come to the fore:

  • the object of the emotion – what the emotion is about (e.g., I am afraid of the bear, angry with my sister, in love with my wife, etc.);
  • the expression of the emotion.


The object could be represented as an attribute of  type anyURI, just like the attribute xlink:href denoting the scope of the emotion (if adding an ‘object’ attribute, one could consider renaming ‘xlink:href’ to ‘scope’ for clarity).


Regarding the expression of the emotion, some aspects related to the expression are already present in the current specification despite the stated aim to only describe the emotion itself:

  • the ‘modality’ attribute only makes sense in view of the emotion’s expression in a multimodal context;
  • the ‘regulation’ set of attributes is ambiguous: ‘simulate’ clearly means an expression of an emotion that is not “really” there; ‘suppress’, ‘amplify’ or ‘attenuate’ could either mean attempts to modify the actual inner state or attempts to modify their expression.


A generic mechanism for annotating meta-data should probably be added. Information that may potentially be needed, e.g., in the context of annotating databases, includes (compiled by H. Pirker):

  • information on annotator (id/name; demographic information age, gender, nationality; level of expertise);
  • information on person/agent being described (id/name; demographic information age, gender, nationality; personality traits; physical constraints: unresticted, posture constrained, hands constrained);
  • social context (social register; constraints e.g. pressure to be expressive, neutral, pressure to formality; social setting: none, passive other, interactant, group; intended audience: kin, collegues, public);
  • overall communicative goal (e.g. to sway, to share feeling);
  • technical setting (recording context: recording style, acoustic and video quality; player technology employed);
  • spatial focus (physical focus, imagined focus, none).


One possibility would be to add an optional <info> tag that may contain arbitrary XML structures.


It is an open question how to represent different aspects of emotional life. Apart from emotions in the strong sense, Scherer proposes “attitudes”, “interpersonal stances”, “moods”, and “affective dispositions”. Cowie and O’Neill add to this list the notions of “established”, “topic-shifting” and “simmering” emotions. While these can of course be accounted for by simply using different category sets with EARL, this may not be a satisfactory solution. Indeed, for some of these states, additional information may be relevant – e.g., an interpersonal stance is by definition linked to another person. It is an open question if such information can be encoded using the existing tag sets or which types of attributes or elements would need to be added for a satisfactory representation. In addition, it may well be relevant to be able to specify which of the above aspects of emotional life is being annotated. Again, this can be done using the existing framework via the name space of the EARL dialect used; or a specialised attribute could be added in order to make the information more explicit.


Open questions regarding the current specification

A number of issues have been pointed out as being unclear.


The meaning of the probability tag is unclear. In the following example, what does the probability indicate – is the emotion probably friendliness, does it probably have medium intensity, is it probably expressed in the face or is it probably simulated?

<emotion category="joy" intensity="0.5" modality="face" simulate="0.9" probability="0.8">

A mechanism should be found to clarify the meaning of a given probability, e.g. by nesting a sub-element such as <probability modality="0.8" category="0.5"/> (which would indicate that it is probably the face in which the emotion is expressed, and that the expressed emotion may or may not be joy).


The coding of “and” vs. “or” relationships in EARL is very implicit. If a <complex-emotion> includes several emotion specifications, these are considered in an “and” relation if they contain no ‘probability’ attributes, but in an “or” relation if they contain ‘probability’ attributes. This can lead to confusion. Ways of coding uncertainty vs. parallel occurrence that are less ambiguous should be investigated, e.g. by looking at EMMA’s <one-of>, <sequence> and <group> elements.


The combination of the <emotion> and <complex-emotion> tags with text needs clarification. The current specification indicates that the <emotion> tag can contain arbitrary XML structures”, but the schema only allows for XML structures from a different namespace. It must be clarified if nesting <emotion> tags should be allowed, as in the following example:

<emotion category="disappointment">This is <emotion category="anger">such</emotion> a pity!</emotion>

The scope is tagged in three different ways: as the XML-element content (if it is text), as an attribute (if it is an external file) and as two attributes (if it is a period of time). This can lead to problems of coherence, if more than one type of specifying the scope is present.

There are no default values for the attributes. It could be desirable at least for some features such as “probability”.

Since some features are required only for some cases, it would be interesting to have a way of restricting possible combinations of features. This is not easy when all the information is at the same level (i.e., as attributes of the same element). A solution could be to annotate part of the information in elements instead of attributes so that hierarchical relations could be established. It remains to be seen in how far this would be compatible with the premise that “simple cases should look simple”.


User guide

The specified language can only be used consistently by the community if it is described in sufficient detail. For example, annotators need strict guidelines so that annotation can be reliable. It should be pointed out, for every feature of the language (e.g., appraisal labels) for which purpose they can be used (e.g., appraisal may not be necessary in driving a 3D face, but may be more important in dialog management).

Document Actions
Powered by Plone

Portal usage statistics