Gianluca Guglielmo

In this paper, we report on an experiment with The Walking Dead (TWD), which is a narrative-driven adventure game with morally charged decisions set in a post-apocalyptic world filled with zombies. This study aimed to identify physiological markers of moral decisions and non-moral decisions using infrared thermal imaging (ITI). ITI is a non-invasive tool used to capture thermal variations due to blood flow in specific body regions that might be caused by sympathetic activity. Results show that moral decisions seem to elicit a significant decrease in temperature in the chin region 20 seconds after participants are presented with a moral decision. However, given the small sample involved, and the lack of significance in other regions, future studies might be needed to confirm the results obtained in this work.

In this study, we extracted facial action units (AUs) data during a Hearthstone tournament to investigate behavioural differences between expert, intermediate, and novice players. Our aim was to obtain insights into the nature of expertise and how it may be tracked using non-invasive methods such as AUs. These insights may shed light on the endogenous responses in the player and at the same time may provide information to the opponents during a competition. Our results show that player expertise may be characterised by specific patterns in facial expressions. More specifically, AU17 (chin raiser), AU25 (lips apart), and AU26 (jaws drop) intensity responses during gameplay may vary according to players' expertise. Such results were obtained by training a random forest classifier to test whether we can use these three AUs alone to accurately detect players' expertise. The classifier reached 0.75 accuracy on 5-fold cross-validation, after balancing the class weights, and 0.85 after having applied the Synthetic Minority Over-sampling Technique (SMOTE) function. These results suggest that AUs can be effectively used to discriminate different levels of expertise in competitive video game players.

This study aimed to evaluate if eye blinks can be used to discriminate players with different performance in a session of Nintendo Entertainment System (NES) Tetris. To that end, we developed a state-of-the-art method for blink extraction from EAR measures, which is robust enough to be used with data collected by a low-grade webcam such as the ones widely available on laptop computers. Our results show a significant decrease in blink rate per minute (blinks/m) during the first minute of playing Tetris. After having defined 3 groups of proficiency based on in-game performance (Novices, Intermediates, and Experts) we found out that expert players display a significantly lower decrease in blinks/m compared to novices during the first minute of gameplay, which shows that Tetris players' proficiency can be detected by looking at eye blinks/m variations during the early phase of a game session. This difference in blinks/m is observed throughout the entire game session, which supports the general conclusion that proficient Tetris players have a lower decrease in blinks/m, even when playing more difficult levels. Finally, we offer some interpretations of this effect and the relationship that our results may have with the visual cognitive workload experienced during the gameplay.

In this study, we analyzed the blinking behavior of players in a video game tournament. Our aim was to test whether spontaneous blink patterns differ across levels of expertise. We used blink rate, blink duration, blink frequency, and eyelid movements represented by the Eye Aspect Ratio (EAR) to train a machine learning classifier to discriminate between different levels of expertise. Classifier performance was highly influenced by features such as the mean, standard deviation and median EAR. Moreover, further analysis suggests that blinking rate and blink duration are likely to increase along with the level of expertise. We speculate this may be indicative of a reduction in cognitive load and lowered stress of expert players. In general, our results suggest that EAR and blink patterns can be used to identify different levels of expertise of video game players.

In this study, we predict the different levels of performance in a Nintendo Entertainment System (NES) Tetris session based on the score and the number of matches played by the players. Using the first 45 seconds of gameplay, a Random Forest Classifier was trained on the five keys used in the game obtaining a ROC_AUC score of 0.80. Further analysis revealed that the number of down keys (forced drop) and the number of left keys (left translation) are the most relevant keys in this task, showing that by merely including the data from these two keys our Random Forest Classifier reached a ROC_AUC score of 0.83.We conclude that the keylogger data during the early phases of a game session can be successfully used to predict performance in longer sessions of Tetris.

In this paper, we report on an experiment with The Walking Dead (TWD), which is a narrative-driven adventure game where players have to survive in a post-apocalyptic world filled with zombies. We used OpenFace software to extract action unit (AU) intensities of facial expressions characteristic of decision-making processes and then we implemented a simple convolution neural network (CNN) to see which AUs are predictive of decision-making. Our results provide evidence that the pre-decision variations in action units 17 (chin raiser), 23 (lip tightener), and 25 (parting of lips) are predictive of decision-making processes. Furthermore, when combined, their predictive power increased up to 0.81 accuracy on the test set; we offer speculations about why it is that these particular three AUs were found to be connected to decision-making. Our results also suggest that machine learning methods in combination with video games may be used to accurately and automatically identify complex decision-making processes using AU intensity alone. Finally, our study offers a new method to test specific hypotheses about the relationships between higher-order cognitive processes and behavior, which relies on both narrative video games and easily accessible software, like OpenFace.

The aim of this study was to test whether an online video game can be used to investigate planning ability and whether fluid intelligence, objective numeracy, and subjective numeracy are predictive of game performance. Our results demonstrate that fluid intelligence is particularly important, which is in line with previous non-game-based studies that show a relationship between classical planning tests and fluid intelligence. Video games have been previously used for research into cognitive processes and taking them online facilitates data collection on a larger scale. Online video games also afford data collection without the expense and stress of a laboratory environment. For these reasons, using online video games to investigate human cognition is a promising alternative to the classic cognitive paradigms used in laboratories.