Submitted:
13 January 2025
Posted:
14 January 2025
You are already at the latest version
Abstract
Emotion detection using EEG signals is a rapidly growing field with significant potential in mental health diagnostics and human-computer interaction. This study leverages the DEAP dataset to explore and evaluate various machine learning and deep learning techniques for emotion recognition. Extensive experimentation was conducted with models such as K-Nearest Neighbors (KNN), Support Vector Machines (SVM), Decision Trees (DT), and Random Forests (RF), Bidirectional Long Short-Term Memory (BiLSTM), Gated Recurrent Units (GRU), Convolutional Neural Networks (CNN), Autoencoders, and Transformers, achieving an accuracy of up to 90%. These results demonstrate the effectiveness of advanced neural architectures in decoding emotional states from EEG signals. Furthermore, SHapley Additive exPlanations (SHAP) were employed to interpret the model predictions, enhancing transparency and providing deeper insights into the contribution of individual features to the decision-making process. In addition, A real-time emotion detection system was developed, enabling instantaneous classification of emotional states from EEG signals, which has significant implications for real-world applications such as affective computing, neurofeedback, and human-computer interaction. This real-time capability, combined with the high accuracy and interpretability of the models, ensures that the decisions are not only accurate but also understandable, fostering trust and enabling more informed application in critical areas such as mental health and human-computer interaction.
Keywords:
1. Introduction
2. Related Works
3. Methods And Materials
3.1. Dataset Overview


3.2. Signal Preprocessing And Feature Extraction



3.3. Implementation Tools And Libraries
4. Methodology
4.1. First Implementation
4.2. Second Implementation
- Input Layer: Accepts vectors of size 32.
- Encoding Layers: A dense layer of size 64 followed by a bottleneck layer to compress the data further.
- Decoding Layers: Two dense layers to reconstruct the data back to its original dimension.
- Input Layer: Accepts the input of shape.
- Transformer Encoder: Implements multi-head attention with 4 heads and a feed-forward network with 128 dimensions. Dropout regularization and layer normalization were applied for improved generalization and stability.
- Global Pooling: Reduces the sequence dimension to create a fixed-size feature vector.
- Dense Layers: Includes a 64-unit dense layer with ReLU activation followed by a single-unit output layer with a sigmoid activation for binary classification.
- LSTM Layer: Captures temporal patterns in the input data.
- Transformer Encoder: Adds contextual attention mechanisms on top of the LSTM outputs.
- Global Pooling and Dense Layers: Similar to the Transformer model.
4.3. Third Implementation
4.4. eXplainable Artificial Intelligence (XAI)
5. Results & Discussion
| Models | Training Accuracy | Testing Accuracy |
| Transformers | 79.85% | 64.15% |
| Transformers + LSTM | 84.26% | 69.81% |
| Transformers + LSTM + additive fusion | 90.98% | 73.58% |
| LSTM + Autoencoders | 87.56% | 67.88% |


| Models | Training Accuracy | Testing Accuracy |
| BiLSTM | 85% | 94% |
| GRU | 83% | 93% |
| CNN | 84% | 91% |

6. Conclusions And Future Work
References
- Mauss, Iris B and Robinson, Michael D. Measures of emotion: A review. Cognition and emotion 2009, 23, 209–237. Taylor & Francis. [CrossRef]
- Govindaraju, Vimala and Thangam, Dhanabalan, Emotion Recognition in Human-Machine Interaction and a Review in Interpersonal Communication Perspective, Human-Machine Collaboration and Emotional Intelligence in Industry 5.0, 329–343, 2024, IGI Global.
- Lan, Zirui, EEG-based emotion recognition using machine learning techniques, 2018.
- Jerritta, S and Murugappan, M and Nagarajan, R and Wan, Khairunizam, Physiological signals based human emotion recognition: A review, 2011 IEEE 7th international colloquium on signal processing and its applications, 410–415, 2011, IEEE.
- Adolphs, Ralph, Cognitive neuroscience of human social behavior, Neuropsychologia, 41, 119–126, 2003.
- Böcker, Koen BE and van Avermaete, Jurgen AG and van den Berg-Lenssen, Margaretha MC, The international 10–20 system revisited: Cartesian and spherical co-ordinates, Brain Topography, 6, 231–235, 1994, Springer.
- Cowie, Roddy and Douglas-Cowie, Ellen and Tsapatsoulis, Nicolas and Votsis, George and Kollias, Stefanos and Fellenz, Winfried and Taylor, John G, Emotion recognition in human-computer interaction, IEEE Signal processing magazine, 18, 1, 32–80, 2001, IEEE. [CrossRef]
- Cowie, Roddy and Douglas-Cowie, Ellen and Tsapatsoulis, Nicolas and Votsis, George and Kollias, Stefanos and Fellenz, Winfried and Taylor, John G, Emotion recognition in human-computer interaction, IEEE Signal processing magazine, 18, 1, 32–80, 2001, IEEE. [CrossRef]
- Koelstra, Sander and Muhl, Christian and Soleymani, Mohammad and Lee, Jong-Seok and Yazdani, Ashkan and Ebrahimi, Touradj and Pun, Thierry and Nijholt, Anton and Patras, Ioannis, Deap: A database for emotion analysis; using physiological signals, IEEE transactions on affective computing, 3, 1, 18–31, 2011, IEEE. [CrossRef]
- Chen, JX and Zhang, PW and Mao, ZJ and Huang, YF and Jiang, DM and Zhang, YN, Accurate EEG-based emotion recognition on combined features using deep convolutional neural networks, IEEE Access, 7, 44317–44328, 2019, IEEE. [CrossRef]
- Alhagry, Salma and Fahmy, Aly Aly and El-Khoribi, Reda A, Emotion recognition based on EEG using LSTM recurrent neural network, International Journal of Advanced Computer Science and Applications, 8, 10, 2017, Science and Information (SAI) Organization Limited.
- Li, Youjun and Huang, Jiajin and Zhou, Haiyan and Zhong, Ning, Human emotion recognition with electroencephalographic multidimensional features by hybrid deep neural networks, Applied Sciences, 7, 10, 1060, 2017, MDPI. [CrossRef]
- Xing, Xiaofen and Li, Zhenqi and Xu, Tianyuan and Shu, Lin and Hu, Bin and Xu, Xiangmin, SAE+ LSTM: A new framework for emotion recognition from multi-channel EEG, Frontiers in neurorobotics, 13, 37, 2019, Frontiers Media SA. [CrossRef]
- Pichandi, Sivasankaran and Balasubramanian, Gomathy and Chakrapani, Venkatesh, Hybrid deep models for parallel feature extraction and enhanced emotion state classification,Scientific Reports, 14, 1, 24957, 2024, Nature Publishing Group UK London. [CrossRef]
- Chen, JX and Jiang, DM and Zhang, YN, A hierarchical bidirectional GRU model with attention for EEG-based emotion classification, IEEE Access, 7, 118530–118540, 2019, IEEE. [CrossRef]
- Mehrabian, Albert, Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in temperament, Current Psychology, 14, 261–292,1996, Springer. [CrossRef]
- Soleymani, Mohammad and Lichtenauer, Jeroen and Pun, Thierry and Pantic, Maja, A multimodal database for affect recognition and implicit tagging, IEEE transactions on affective computing, 3, 1, 42–55,2011, IEEE. [CrossRef]
- Newson, Jennifer J and Thiagarajan, Tara C, EEG frequency bands in psychiatric disorders: A review of resting state studies, Frontiers in human neuroscience, 12, 521, 2019, Frontiers Media SA. [CrossRef]
- Bigdely-Shamlo, Nima and Mullen, Tim and Kothe, Christian and Su, Kyung-Min and Robbins, Kay A, The PREP pipeline: Standardized preprocessing for large-scale EEG analysis, Frontiers in neuroinformatics, 9, 16, 2015, Frontiers Media SA. [CrossRef]
- Bhole, L and Ingle, M, Estimating range and relationship of EEG frequency bands for emotion recognition, International Journal of Computer Applications, 78, 13, 16–21, 2019. [CrossRef]
- Mendivil Sauceda, Jesus Arturo and Marquez, Bogart Yail and Esqueda Elizondo, José Jaime, Emotion Classification from Electroencephalographic Signals Using Machine Learning, Brain Sciences, 14, 12, 1211, 2024, MDPI.
- Unde, Sukhada A and Shriram, Revati, Coherence analysis of EEG signal using power spectral density, 2014 Fourth International Conference on Communication Systems and Network Technologies, 871–874, 2014, IEEE.
- Dwivedi, Amit Kumar and Verma, Om Prakash and Taran, Sachin, 2024 11th International Conference on Signal Processing and Integrated Networks (SPIN), EEG-Based Emotion Recognition Using Optimized Deep-Learning Techniques, 2024, , , 372-377, Deep learning;Time-frequency analysis;Emotion recognition;Computational modeling;Brain modeling;Feature extraction;Electroencephalography;EEG;Emotion Recognition;Deep-Learning Methods;Bayesian Optimization, 10.1109/SPIN60856.2024.10512074.
- Tong, Wenhui and Yang, Li and Qin, Yingmei and Che, Yanqiu and Han, Chunxiao, 2022 15th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), EEG-Based Emotion Recognition by Using Machine Learning and Deep Learning, 2022, , ,1-5, Deep learning;Radio frequency;Emotion recognition;Machine learning algorithms;Support vector machine classification;Signal processing algorithms;Forestry;EEG;emotion;support vector machine(SVM);random forest(RF);convolutional neural network(CNN), 10.1109/CISP-BMEI56279.2022.9979849.
- Li, Xiang and Zhang, Yazhou and Tiwari, Prayag and Song, Dawei and Hu, Bin and Yang, Meihong and Zhao, Zhigang and Kumar, Neeraj and Marttinen, Pekka, EEG based emotion recognition: A tutorial and review, ACM Computing Surveys, 55, 4, 1–57, 2022, ACM New York, NY. [CrossRef]
- Gramfort, Alexandre and Luessi, Martin and Larson, Eric and Engemann, Denis A and Strohmeier, Daniel and Brodbeck, Christian and Parkkonen, Lauri and Hämäläinen, Matti S, MNE software for processing MEG and EEG data, neuroimage, 86, 446–460, 2014, Elsevier. [CrossRef]
- Ma, Yahong and Huang, Zhentao and Yang, Yuyao and Zhang, Shanwen and Dong, Qi and Wang, Rongrong and Hu, Liangliang, Emotion Recognition Model of EEG Signals Based on Double Attention Mechanism, Brain Sciences, 14, 12, 1289, 2024, MDPI. [CrossRef]
- Jacob12138xieyuan, EEG-Based Emotion Recognition on DEAP, https://github.com/Jacob12138xieyuan/EEG-Based-Emotion-Recognition-on-DEAP.



| Models | Valence | Arousal | ||
|---|---|---|---|---|
| Training Accuracy | Testing Accuracy | Training Accuracy | Testing Accuracy | |
| KNN | 71.3% | 56.2% | 71.2% | 59.4% |
| SVM | 64.6% | 56.6% | 68.3% | 58.6% |
| Decision Tree | 93.7% | 49.6% | 89.6% | 59.4% |
| Random Forest | 99.3% | 60.9% | 98.9% | 58.2% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).