Zeulner, T.; Hagerer, G.J.; Mueller, M.; Vazquez, I.; Gloor, P.A. Predicting Individual Well-Being in Teamwork Contexts Based on Multi-Modal Speech Features. Preprints2024, 2024012030. https://doi.org/10.20944/preprints202401.2030.v1
APA Style
Zeulner, T., Hagerer, G.J., Mueller, M., Vazquez, I., & Gloor, P.A. (2024). Predicting Individual Well-Being in Teamwork Contexts Based on Multi-Modal Speech Features. Preprints. https://doi.org/10.20944/preprints202401.2030.v1
Chicago/Turabian Style
Zeulner, T., Ignacio Vazquez and Peter A. Gloor. 2024 "Predicting Individual Well-Being in Teamwork Contexts Based on Multi-Modal Speech Features" Preprints. https://doi.org/10.20944/preprints202401.2030.v1
Abstract
Current methods for assessing individual well-being in team collaboration at the workplace rely often on manually collected surveys. This limits continuous real-world data collection and proactive measures to improve team member workplace satisfaction. We propose a method to automatically derive social signals related to individual well-being in team collaboration from raw audio and video data collected in teamwork contexts. The goal is to develop computational methods and measurements to facilitate the mirroring of individuals’ well-being to themselves. We are focusing on how speech behavior is perceived by team members to improve their well-being. Our main contribution is the assembly of an integrated toolchain to perform multi-modal extraction of robust speech features in noisy field settings and to find which features are predictors of self-reported satisfaction scores. We apply the toolchain to a case study where we collected videos of 22 teams with 56 participants collaborating over a four-day period in a team project. Our audiovisual speaker diarization extracts individual speech features in a noisy environment. As the dependent variable team members filled out a daily PERMA (positive emotion, engagement, relationships, meaning, and accomplishment) survey. These well-being scores have been predicted with speech features extracted from the videos using machine learning. The results suggest that the proposed toolchain is able to automatically predict individual well-being in teams, leading to better teamwork and happier team members.
Keywords
explainable AI; multi-modal speaker diarization; affective computing; social signal processing; team collaboration
Subject
Computer Science and Mathematics, Artificial Intelligence and Machine Learning
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
