Article
Version 1
Preserved in Portico This version is not peer-reviewed
Generating non-verbal responses in virtual agent with use of LSTM network
Version 1
: Received: 12 January 2024 / Approved: 15 January 2024 / Online: 15 January 2024 (08:24:06 CET)
How to cite: Koren, L.; Stipancic, T. Generating non-verbal responses in virtual agent with use of LSTM network. Preprints 2024, 2024011081. https://doi.org/10.20944/preprints202401.1081.v1 Koren, L.; Stipancic, T. Generating non-verbal responses in virtual agent with use of LSTM network. Preprints 2024, 2024011081. https://doi.org/10.20944/preprints202401.1081.v1
Abstract
This paper investigates nonverbal communication in human interactions, with a specific focus on facial expressions. Employing a Long Short-Term Memory (LSTM) architecture and a custom-ized facial expression framework, our approach aims to improve virtual agent interactions by incorporating subtle nonverbal cues. The paper contributes to the emerging field of facial expres-sion generation, addressing gaps in current research and presenting a novel framework within Unreal Engine 5. The model's architecture, trained on the CANDOR corpus, captures temporal dynamics, and refines hyperparameters for optimal performance. During testing, the trained model showed a cosine similarity of -0.95. This enables the algorithm to accurately respond to non-verbal cues and interact with humans in a way that is comparable to human-human interac-tion. Unlike other approaches in the field of facial expression generation, the presented method is more comprehensive and enables the integration of a multi-modal approach for generating facial expressions. Future work involves integrating blendshape generation, real-world testing, and the inclusion of additional modalities to create a comprehensive framework for seamless hu-man-agent interactions beyond facial expressions.
Keywords
LSTM; facial expressions; framework; virtual agent; affective robotics
Subject
Computer Science and Mathematics, Robotics
Copyright: This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Comments (0)
We encourage comments and feedback from a broad range of readers. See criteria for comments and our Diversity statement.
Leave a public commentSend a private comment to the author(s)
* All users must log in before leaving a comment