Toadstool: a dataset for training emotional intelligent machines playing Super Mario Bros
Chapter, Conference object, Peer reviewed
MetadataVis full innførsel
OriginalversjonSvoren, Thambawita V, Halvorsen P, Jakobsen P, Garcia-Ceja E, Noori FM, Hammer HL, Lux M, Riegler M, Hicks S: Toadstool: a dataset for training emotional intelligent machines playing Super Mario Bros. In: Alay OA, Toni L. MMSys '20: Proceedings of the 11th ACM Multimedia Systems Conference, 2020. Association for Computing Machinery (ACM) p. 309-314 https://doi.org/10.1145/3339825.3394939
Games are often defined as engines of experience, and they are heavily relying on emotions, they arouse in players. In this paper, we present a dataset called Toadstool as well as a reproducible methodology to extend on the dataset. The dataset consists of video, sensor, and demographic data collected from ten participants playing Super Mario Bros, an iconic and famous video game. The sensor data is collected through an Empatica E4 wristband, which provides highquality measurements and is graded as a medical device. In addition to the dataset and the methodology for data collection, we present a set of baseline experiments which show that we can use video game frames together with the facial expressions to predict the blood volume pulse of the person playing Super Mario Bros. With the dataset and the collection methodology we aim to contribute to research on emotionally aware machine learning algorithms, focusing on reinforcement learning and multimodal data fusion. We believe that the presented dataset can be interesting for a manifold of researchers to explore exciting new interdisciplinary questions.