Domain for Experimentation

In this work Interpersonal Communication (IC) has been considered as the domain for experimentation. IC defines a communicative interaction between people, verbally or non-verbally. Non-verbal communication is instantiated through body language cues, often called non-verbal behavioural cues, and emotions are expressed in the context of social interaction between two or more individuals. These cues are transformed through the process of communication into social signals for other participants in this communication. This dimension of IC, social signals, is the focus of this thesis.

3.4.1 Motivation

Importance of the Domain. IC is regarded as a key soft skill required in the

knowledge society of the 21st century [78], and is fundamental to everyday professional and social life. In IC, emotions and non-verbal cues (i.e. social signals) play a key role. Research has shown that non-verbal communication carries most of the social meaning (about two thirds comparing with verbal communication [79], while other studies show that non-verbal cues cover 90% of the communication [80]). Body language

expresses emotions, regulates the flow of interaction and provides valuable feedback to every individual participating in IC activities.

One possible target application area is user-adaptive learning environments. Providing various perspectives on the same topic is highly beneficial for learning, and is seen as one of the challenges to the next generation of technology-enhanced learning systems [81]. More specifically one can consider informal learning environments for adults, which are growing in popularity in workplace contexts. In order to be effective, such environments should provide a range of real life examples and a variety of viewpoints [82]. We further examine this assumption and hypothesis in Chapter 5 where the potential benefit is explored in a learning context.

Relevance. Awareness and recognition of social signals is crucial in social

interactions [83], and is linked to the development of emotional intelligence [84]. Different interpretations could be possible depending on the background and experience of the “observers” and “participants” in IC activities. Hence, personalised support can be offered exploiting the diversity of viewpoints, and thus showing a variation of social signal interpretations based on authentic examples from user-generated content.

In an IC learning context, interpreting those social signals can be complicated and highly subjective. For example, in a job interview, a gesture like “waving the hands in the air” might be interpreted by one person as exaggeration and by another person as enthusiasm and willing; or a “frowning facial expression” could be a sign of boredom or intensive contemplation. These diverse interpretations, if semantically captured and processed, can provide a rich resource for personalised learning experiences to improve awareness and promote reflection.

Feasibility and significance. This will be established in greater detail in the

section below.

3.4.2 Related Work on Mining Social Signals in UGC

Social signals concern two human aspects: emotion and body language. Following we discuss related work on each aspect with respect to identifying it in UGC.

The emotional aspect is closely related to sentiment, for which related work on text mining approaches for analysis were discussed in Chapter 2. Here we list additional research work which consider more expressive representations of emotions.

In [85], a framework has been developed which aims to understand when a piece of text contains inflammatory content or not, in order to prevent "trolling" in social web spaces and to block insulting messages. This produces the AffectNet - vocabulary combining common sense knowledge from ConceptNet12 _{and emotional attributes from WordNet represented by} the emotion taxonomy WordNet- Affect [86]. Each concept in the vocabulary is either a common-sense concept or has an affective attribute. AffectNet is then partitioned into four main categories: pleasantness, attention, sensitivity and aptitude, which are further analysed into six basic emotions (with negative to positive valence) each. This modelling is called the Hour Glass of emotion. Concepts which identified in the text and can be matched with ConceptNet are mapped to affective valence in the Hour Glass model of emotions and are given a polarity score to identify "trollness".

Some research work have been done for annotating textual content with the six basic emotions defined by Ekman [27] - anger, disgust, fear, happiness, sadness and surprise. [87-90] constitute a representative sample in this research direction. The methodology being followed includes natural language processing on textual content and classification of text into one of the six basic emotions. Linguistic resources are being used to match term references with affective labels and valence, as well as to construct dictionaries and lexicons for training probabilistic classifiers. Features for classification often include, apart from words, punctuation, emoticons and syntactical rules associations with affective states.

Although richer representations of emotions are being exploited in the aforementioned research outlooks of emotion mining from text, such classification has not been applied to date for user viewpoints modelling. The particulars of affective classes, i.e. the key-words and concepts, which are used to describe the emotion label, have not been used to date to describe user models. Moreover, external resources for enrichment which are used in the classification process have not been considered as domain models to which an opinion or expression in text can instantiate a reflection of the user-contributor.

The feasibility of the approach for annotating emotion is related to the availability of resources which can describe emotion. In Chapter 4 we list state-of-the-art semantic models to represent emotion. For this work we

have exploited WordNet-Affect, a taxonomy of emotion, which was also exploited in previous works, however not for user viewpoint modelling.

Regarding detection and recognition of social signals, a review of methods for capturing and analyzing non-verbal behavioural cues was provided in [91]. These methods involve audio and visual data processing which utilises statistical and probabilistic methods. Little has been done in utilising text UGC to extract body language related concepts. Similarly to [92], we focus on awareness and recognition of social signals for user modelling, but we consider textual content. The significance of the research in this work is based on the semantic augmentation component which is configured for body language in the context of interpersonal communication experimental domain, and the enrichment method that is offered.

Mining body language related terms is made feasible in this work with the design of an ontology for human-activity modelling [93], including body language, in the context of the ImREAL EU project. More details are discussed in Chapter 4.

3.5 Summary

In this Chapter we presented the research context. Firstly, the ViewS framework was outlined with respect to the research questions that this work aims to tackle: viewpoints capturing, representation and analysis. The research methodology used to develop and validate ViewS was then presented. Finally, the domain of experimentation, IC with focus on Social Signals, was discussed.

The following Chapters detail the accomplishment of the methodology steps with respect to the ViewS framework components.