Version 1
: Received: 12 January 2024 / Approved: 15 January 2024 / Online: 15 January 2024 (08:24:06 CET)
How to cite:
Koren, L.; Stipancic, T. Generating non-verbal responses in virtual agent with use of LSTM network. Preprints2024, 2024011081. https://doi.org/10.20944/preprints202401.1081.v1
Koren, L.; Stipancic, T. Generating non-verbal responses in virtual agent with use of LSTM network. Preprints 2024, 2024011081. https://doi.org/10.20944/preprints202401.1081.v1
Koren, L.; Stipancic, T. Generating non-verbal responses in virtual agent with use of LSTM network. Preprints2024, 2024011081. https://doi.org/10.20944/preprints202401.1081.v1
APA Style
Koren, L., & Stipancic, T. (2024). Generating non-verbal responses in virtual agent with use of LSTM network. Preprints. https://doi.org/10.20944/preprints202401.1081.v1
Chicago/Turabian Style
Koren, L. and Tomislav Stipancic. 2024 "Generating non-verbal responses in virtual agent with use of LSTM network" Preprints. https://doi.org/10.20944/preprints202401.1081.v1
Abstract
This paper investigates nonverbal communication in human interactions, with a specific focus on facial expressions. Employing a Long Short-Term Memory (LSTM) architecture and a custom-ized facial expression framework, our approach aims to improve virtual agent interactions by incorporating subtle nonverbal cues. The paper contributes to the emerging field of facial expres-sion generation, addressing gaps in current research and presenting a novel framework within Unreal Engine 5. The model's architecture, trained on the CANDOR corpus, captures temporal dynamics, and refines hyperparameters for optimal performance. During testing, the trained model showed a cosine similarity of -0.95. This enables the algorithm to accurately respond to non-verbal cues and interact with humans in a way that is comparable to human-human interac-tion. Unlike other approaches in the field of facial expression generation, the presented method is more comprehensive and enables the integration of a multi-modal approach for generating facial expressions. Future work involves integrating blendshape generation, real-world testing, and the inclusion of additional modalities to create a comprehensive framework for seamless hu-man-agent interactions beyond facial expressions.
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
