Towards explainable prediction of player frustration in video games.

Abstract

Frustration is a key concept in retaining a player interest in both commercial and applied games. In a HCI context, frustration is often seen as a purely negative phenomenon. However, for games to be interesting some amount of frustrating has to be present. As such, dynamically adjusting game elements to ensure optimal frustration levels can be a valuable way to increase player retention. A first step towards such a system is an accurate classifier of frustration. To date, most attempts at frustration classification use models that are relatively hard for a human to understand. In this paper an attempt will be made at creating an explainable predictor of player frustration. To accomplish this, the frustration-aggression theory was used to identify a number of key components that determine the severity of a frustrated response. 135 participants were asked to play a series of Pac-Man levels while being asked about the frustration components. Gameplay features, participant behaviour and participant responses were gathered and used as a dataset to train a number of random forest classifiers. The classifiers were trained to predict player frustration, with accuracy ranging from 66.3% to 83.1% depending on the amount of frustration classes used. Accuracy dropped significantly when excluding participant responses on frustration component questions from the dataset. Furthermore, feature importance analysis revealed the overwhelming importance of the Repeated Failures component, as well as the relatively low importance of all in-game variables. These results suggest that the currently used variable set might not accurately represent the components of frustration. A possible avenue for future research could be the discovery of accurate metrics for these internal component perceptions.