Emotion Classification in a Serious Game for Training Communication Skills

Abstract

We describe the natural language processing component of a new serious gaming project, deLearyous, which aims at developing an environment in which users can improve their communication skills by interacting with a virtual character in (Dutch) written natural language. The virtual characters’ possible dialogue paths are defined by Leary’s Rose, a framework for interpersonal communication. In order to apply this framework, input sentences must be classified into one of four possible “emotion” classes. We tried to carry out this emotion classification task using several machine learning algorithms. More specifically, classification performance was measured using TiMBL –a memory-based learner–, a Naïve Bayes classifier, Support Vector Machines and Conditional Random Fields. Training was done on a relatively small dataset of manually tagged sentences. A large number of different features was extracted from the dataset, and a good feature subset was selected using a combination of a genetic algorithm and various filter metrics. We achieved the best results using the memory-based learner TiMBL, using a combination of word unigrams, lemma trigrams and dependency structures. With this setup, 52.5% of the sentences were classified into the correct emotion quadrant, which is a significant improvement over the statistical baseline (25.15%) and over the scores achieved with a pure bag-of-words approach (41.6%).