Matveev, A.; Matveev, Y.; Frolova, O.; Nikolaev, A.; Lyakso, E. A Neural Network Architecture for Children’s Audio–Visual Emotion Recognition. Mathematics2023, 11, 4573.
Matveev, A.; Matveev, Y.; Frolova, O.; Nikolaev, A.; Lyakso, E. A Neural Network Architecture for Children’s Audio–Visual Emotion Recognition. Mathematics 2023, 11, 4573.
Matveev, A.; Matveev, Y.; Frolova, O.; Nikolaev, A.; Lyakso, E. A Neural Network Architecture for Children’s Audio–Visual Emotion Recognition. Mathematics2023, 11, 4573.
Matveev, A.; Matveev, Y.; Frolova, O.; Nikolaev, A.; Lyakso, E. A Neural Network Architecture for Children’s Audio–Visual Emotion Recognition. Mathematics 2023, 11, 4573.
Abstract
Detecting and understanding emotions is critical for our daily activities. As emotion recognition (ER) systems develop, we start looking at more difficult cases than just acted adult audio-visual speech. In this work, we investigate automatic classification of audio-visual emotional speech of children. Our interest is, specifically, in the improvement of the utilization of the cross-modal relationships between the selected modalities: video and audio. To underscore the importance of developing ER systems for the real-world environment, we present a corpus of children’s emotional audio-visual speech that we collected. We select a state-of-the-art model as a baseline for the purposes of comparison and present several modifications focused on a deeper learning of the cross-modal relationships. By conducting experiments with our proposed approach and the selected baseline model, we observe a relative improvement in performance by 2%. Finally, we conclude that focusing more on the cross-modal relationships may be beneficial for building ER systems for child-machine communications and the environments where qualified professionals work with children.
Keywords
Audio-visual speech; emotion recognition; children
Subject
Computer Science and Mathematics, Computer Science
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
