Spectrogram Contrast Enhancement Improves EEG Signal-Based Emotional Classification

1. Introduction

An electroencephalogram (EEG) records the electrical activity of the brain using surface electrodes attached to the scalp. The EEG has been employed for researching the Brain Computer Interfaces (BCIs) [1]. Neuroscience integrates several subfields, such as psychology & cognitive science, neuroimaging, and artificial intelligence (AI), to explore the functioning of the entire nervous system. Within neuroscience, BCI deals with communication between an individual’s brain signals and an external device not involving oral communication or motor functions [2]. BCI-EEG signals are characterized by their frequency, amplitude, waveform morphology, and spatial placement across scalp electrodes [3]. Typically, EEG signals span a frequency range of 0.1 Hz to 100 Hz and are categorized into five primary bandwidths (Figure 1): Delta (δ) with the range of 0.5–3.5 Hz, associated with deep sleep and comatose states; Theta (θ) with the range of 3.5–7.5 Hz, linked to creativity, stress, and deep meditation; Alpha (α) with the range of 7.5–12 Hz, predominant during relaxed and calm mental states; Beta (β) with the range of 13–30 Hz, observed during focused attention, visual processing, and motor coordination; and Gamma (γ) that has frequencies >30 Hz, which emerges during complex cognitive functions, motor execution, and multitasking [4].

The human brain is functionally divided into four major lobes: the frontal, temporal, parietal, and occipital lobes [6]. Each lobe is associated with a distinct set of structures, which correspond to specific neural functions. Figure 2 shows that the frontal lobe (Fp1, Fp2, AFz, F7, F3, Fz, F4, F8, FC5, FC1, FCz, FC2, and FC6) is primarily responsible for executive functions, including cognitive control, decision-making, and the regulation of emotional responses during task execution. The temporal lobe (T7, TP9, T8, and T10) plays a critical role in auditory processing and the perception of biological motion. The parietal lobe (P7, P3, Pz, P4, P8, PO9, PO10, CP1, CP2, CP5, and CP6) is largely involved in somatosensory processing, spatial representation, and tactile perception. Finally, the occipital lobe (O1, Oz, and O2) is primarily responsible for visual processing, particularly the perception and interpretation of visual stimuli.

Despite recent advancements, EEG-based BCI systems continue to face significant challenges, particularly with respect to low classification accuracy [7] and inter-subject variability [8]. The intra-user variability limitation refers to the phenomenon where EEG signals corresponding to the same cognitive task or thought can vary across different recording sessions for the same individual. In other words, the EEG pattern generated by a specific mental activity may not be identical when that activity is repeated at a later date or time [9].

This study aims to enhance the recognition accuracy of the emotional state of a person for BCI applications. It uses a publicly available EAV (EEG, Audio, Video) dataset [10] as a benchmark. The EAV dataset contains recordings from 42 subjects across five emotional classes: neutral, anger, happiness, sadness, and calmness [10]. Two EEG recordings from channel 0 and channel 5 for subject_1 are randomly depicted in Figure 3. The classification accuracy enhancement is achieved by refining the spectrogram feature extraction methods previously employed with the EEGNet architecture. The proposed methodology leverages the Short-Time Fourier Transform (STFT) that transforms EEG signals into a time-frequency representation. Later, adaptive contrast enhancement is introduced to achieve a better representation, enabling the EEGNet model to more accurately capture both temporal and spectral features.

This research paper is organized into five sections: Section II presents the literature review. Section III explains the research methodology design. Section IV describes the experiments. Section V explains the results and discussion. Finally, Section VI presents the conclusion, followed by the list of references.

2. Literature Review

One of the fundamental components of BCI technology is the ability to recognize emotional states within the brain. This process, often referred to as brain decoding, can be achieved through both invasive [11] and non-invasive [12] methods. Invasive BCIs involve the implantation of microelectronic devices beneath the scalp or directly into neural tissue such as the Electrocorticograph (ECOC) [13], offering high signal fidelity and accuracy. However, these methods present significant challenges, including the risk of infection, high cost, and surgical complexity [14]. In contrast, non-invasive techniques such as those based on EEG or functional near-infrared spectroscopy (fNIRS) [15] are widely adopted due to their safety, portability, and ease of use. While non-invasive approaches offer a more practical solution for everyday applications, they typically yield lower accuracy compared to invasive methods due to signal attenuation and noise. Nevertheless, EEG-based systems, in particular, have become central to the development of second-generation BCI technologies, offering a promising balance between usability and performance [16,17]. The task of decoding emotions from brain activity has attracted considerable attention from researchers. However, achieving high recognition accuracy remains a significant challenge, requiring substantial improvement. Nevertheless, recognizing and classifying EEG bio-signals is a complex task due to several inherent characteristics: high intra-subject variability, high dimensionality, non-stationarity, and a strong susceptibility to noise [18]. These challenges are further compounded when applying deep learning techniques to EEG-based emotion recognition. In particular, two major obstacles persist: the variability of emotional patterns across individuals (intra-subject variability) and the limited availability of labeled EEG datasets. Several studies on emotion recognition using EEG signals have been made publicly available. Notably, Feng et al. [18], proposed a method based on a pre-trained Vision Transformer for emotion recognition, evaluating its performance across four widely-used public datasets: SEED, SEED-IV, DEAP, and FACED. The cross-dataset emotion recognition accuracy achieved 93.14% on SEED, 83.18% on SEED-IV, 93.53% on DEAP, and 92.55% on FACED. The approach utilizes a transfer learning framework known as Pre-trained Encoder from Sensitive Data (PESD).

Another notable study, published in Nature [10], introduced the EAV (EEG-Audio-Video) dataset for emotion recognition in conversational contexts. This multimodal dataset incorporates three modalities—EEG, audio, and video—to model human emotions more comprehensively. Among these, EEG plays a central role. For the EEG component, the authors employed the SEED-IV dataset, which consists of 30-channel EEG recordings. A total of 42 participants took part in the study, each engaging in cue-based conversational scenarios designed to elicit five distinct emotional states: neutrality, anger, happiness, sadness, and calmness. Each participant contributed approximately 200 interactions, encompassing both listening and speaking tasks, resulting in a total of 8,400 interactions across all participants. For EEG data acquisition, the BrainAmp system (Brain Products, Munich, Germany) was used. EEG signals were collected via Ag/AgCl electrodes placed at standardized scalp locations: Fp1, Fp2, F7, F3, Fz, F4, F8, FC5, FC1, FC2, FC6, T7, C3, Cz, C4, T8, CP5, CP1, CP2, CP6, P7, P3, Pz, P4, P8, PO9, O1, Oz, O2, and PO10. Data were sampled at 500 Hz, with reference electrodes placed at the mastoids and grounding via the Afz electrode. The electrode impedance was maintained below 10 kΩ to ensure data quality. The EEG recordings were initially stored in BrainVision Core Data Format and later converted to MATLAB (.mat) format for preprocessing and analysis. Emotion recognition performance for each modality was evaluated using deep neural network (DNN) models. The best classification accuracy achieved for EEG-based emotion recognition using this dataset was approximately 60%. We will use these results as a benchmark as we aim to enhance the classification accuracy of the EAV dataset.

Other researchers have classified EEG into three classes: happy, neutral, and sad, as reported in [19]. Using an SVM classifier with time-frequency features, an accuracy of 88.93% was achieved. Various EEG datasets are designed to study emotional responses under varying experimental conditions. The DEAP dataset [20] records 32-channel EEG from 32 participants during music video stimulation, annotated with continuous dimensions (valence, arousal, dominance, and liking), making it suitable for dimensional emotion modeling. The SEED series [21] employs 64-channel EEG and movie clips to induce discrete emotions such as positive, negative, and neutral; after that, happy, sad, and fear in SEED-IV/V across 15 subjects, facilitating categorical emotion classification studies. In contrast, the DREAMER dataset [22] uses 14-channel EEG from 23 subjects watching film clips, providing self-reported valence, arousal, and dominance ratings, balancing practicality with robust affective annotations. Another MPED dataset (Song et al., 2019) [23] extends emotional descriptors to include arousal, valence, and discrete emotional states (DES) via 62-channel EEG.

3. Materials and Methods

The signal processing and classification of emotional EEG waves involve several processing steps, including band-pass filtering, downsampling, and a reshaping process. Following preprocessing, STFT [24,25] is applied as a feature extraction technique to enhance the signal’s representational characteristics. Unlike a standard Fourier Transform, which assumes signal stationarity, the STFT operates by dividing the continuous EEG signal into brief, sequential time segments using a sliding window function. This allows for the computation of a local Fourier spectrum for each segment, effectively capturing the temporal evolution of spectral power across key frequency bands (delta, theta, alpha, beta, gamma). The resultant time-frequency representation (TFR) provides a highly informative feature set that preserves crucial information about both the timing and the frequency content of neural oscillations and transient events. Then, the adaptive contrast enhancement (ACE) process is applied. Later, a two-dimensional feature matrix is prepared for subsequent advanced analysis within the EEGLab environment, which employs a deep learning training model to classify specific cognitive states. The preprocessing pipeline and the overall model architecture are depicted in the general block diagram as shown in Figure 4.

In this study, EEG data consisting of 42 subjects was retrieved from the public EAV Dataset [10]. Each record has two files: an EEG data file that contains raw EEG signals and a corresponding label file containing class annotations. The EEG signals are represented as $X\in {\mathbb{R}}^{s\times c\times n}$, where $s$ the number of time samples over $20sec$ of recording at 500Hz, $c=30$is the number of channels, and $n=200$ is the number of trials. Labels are stored as a one-hot encoded matrix: $Y\in \left\{\mathrm{0,1}\right\}k\times n$, where $k$ is the number of classes. The input dataset for each subject has dimensions$X_{input}=10000\times 30\times 200$. The data were organized into segments, with each segment representing a trial of EEG recordings across multiple channels. The preprocessing pipeline consisted of several steps to prepare the EEG data for classification. The first step applied bandpass filtering (BPF) to retain frequencies between 3 Hz and 50 Hz, resulting in the filtered signal $X_f$, given by equation:

$$X_f\left(s,c\right)=BPF(X\left(s,c\right),\left[\mathrm{3,50}\right],f_s)$$

(1)

Next, the filtered EEG signal $X_f\left(s,c,n\right)$ were down-sampled from $f_s=500Hz$ to $f_{s2}=100Hz$ using polyphase resampling [26] to reduce computational complexity, as given by equation:

$$X_d\left(s^{\text{'}},c,n\right)=resmaple(X_f\left(s,c,n\right),{\displaystyle \frac{f_s}{f_{s2}}})$$

(2)

Where $\left(s^{\text{'}}\right)$ represents the downsampled signal. Then, the reshaped data is segmented and transposed into a format suitable for analysis, resulting in a shape $X_d\left(s^{\text{'}},c,n\right)=2000\times 30\times 200$. Then, segmentation and reshaping processes are applied, in which the down-sampled signals were segmented into trials and then reshaped into a tensor $X_s\in {\mathbb{R}}^{n\times s^{\text{'}}\times C}$, where $n=200$ is the number of trials, $s^{\text{'}}=2000$ is the number of downsampled time points per trial, and $c=30$. Where $X_s=200\times 2000\times 30$.

The down-sampled signals $X_s$ are reshaped and transposed to align with the segmentation requirements of the classification model. Five specific class labels: $k\in \{\mathrm{1,3},\mathrm{5,7},9\}$ were randomly selected for classification. One-hot encoding was applied to represent the class labels as in equation:

$$Y_{k,i}^{\mathrm{\text{'}}}=\left\{\begin{array}{c}1,ifclasskcorrespondtotriali\hfill \\ 0,Otherwise\hfill \end{array}\right.$$

(3)

In the above, ${\mathrm{Y}}^{\mathrm{\text{'}}}\in {\left\{\mathrm{0,1}\right\}}^{5\times {\mathrm{n}}^{\mathrm{\text{'}}}}$, where $n^{\mathrm{\text{'}}}$ is the number of selected trails. Accordingly, the input dataset $X_s\in {\mathbb{R}}^{t\times c\times s\times 1}$ has the dimensions, $X_s=400\times 30\times 500\times 1$, 400 for trials, $30$ for channels, and $500$ for time points for the EEG signal with the corresponding Labels, $Y_{k,i}^{\mathrm{\text{'}}}=400\times 5$. Next, the STFT was applied to the EEG dataset, as a key methodological improvement that could enhance the EEG signal recognition accuracy by providing time-frequency localization

3.1. Short-Time Fourier Transform (STFT) Feature Extraction

Given an input EEG dataset $x\in {\mathbb{R}}^{t\times c\times s\times 1}$, where $t$ is the number of trials $c$ is the number of channels, $s$ is the number of time samples, the STFT was applied to each trial. The STFT parameters are: $f_s$ (the sampling frequency in Hz), $N_{seg}$ (the segment length), and $N_{overlap}$ (the number of overlapping samples between consecutive segments). The STFT is applied to each one-dimensional EEG signal $x_{t,c}\left(n\right)$, where $x_{t,c}\left(n\right)=x(t,c,n)$ represents the time-domain signal for trial $t\in \left\{\mathrm{1,2},3\dots ,T\right\}$, channel $c\in \left\{\mathrm{1,2},3\dots ,C\right\}$and the sam

$$Z_{t,c}\left(k,m\right)={\sum }_{n=0}^{N_{seg}-1}{x}_{t,c}\left(n+m\cdot \left(N_{seg}-N_{overlap}\right)\right)\cdot w\left(n\right)e^{-{\displaystyle \frac{j2\pi kn}{N_{seg}}}}$$

(4)

ple index $n\in \left\{\mathrm{0,1},2,\dots ,S-1\right\}$. The STFT is defined as equation: where:

$Z_{t,c}\left(k,m\right)$ is the complex-valued STFT coefficient for frequency_bin ($k)$ and time_bin ($m)$;

$w\left(n\right)$ is a window function of length $N_{seg}$;

$k\in \left\{\mathrm{0,1},\dots ,⌊N_{seg}/2⌋\right\}$indexes the frequency_bins assuming positive frequencies for real-valued signals; and,

$m\in \left\{\mathrm{0,1},\dots ,M-1\right\}$indexes the time bins, where$M=⌊{\displaystyle \frac{S-N_{seg}}{N_{seg}-N_{overlap}}}⌋+1$ is the number of time_bins.

To boost the output, the absolute value is obtained from the complex-valued STFT coefficients: $Y_{t,c}\left(k,m\right)=\left|Z_{t,c}\left(k,m\right)\right|$. The resulting STFT magnitude is organized into a four-dimensional array $Y\in {\mathbb{R}}^{t\times c\times k\times m}$, where: $k=\left[N_{seg}/2\right]+1$ is the number of frequency_bins including zero frequency and Nyquist frequency for even $N_{seg}$, and $m$ is the number of time_bins. For each trial $t$ and chancel $c$,the STFT magnitude ${\mathit{Y}}_{\mathit{t},\mathit{c}}\left(\mathit{k},\mathit{m}\right)$is computed, and the result are stacked as:${Y\left(t,c,k,m\right)=Y}_{t,c}\left(k,m\right)$.

It may be noted that the choice of the window function $w\left(n\right)$ and parameters $N_{seg}$ and $N_{\mathit{o}\mathit{v}\mathit{e}\mathit{r}\mathit{l}\mathit{a}\mathit{p}}$ affects the time-frequency resolution trade-off. A larger $N_{seg}$ provides better frequency resolution but poorer time resolution, while a larger $N_{overlap}$ increases the number of time_bins, improving temporal smoothness. The sampling frequency $f_s$ determines the frequency resolution with frequency_bins corresponding to$f_k=k{\displaystyle \frac{f_s}{N_{seg}}}$for $k=\mathrm{0,1},2,\dots ⌊{\displaystyle \frac{N_{seg}}{2}}⌋$.

To facilitate compatibility with the convolutional neural networks (CNNs) within the EEGLAB environment, the 2D time-frequency representation is transformed into a 1D feature vector. This is achieved through a flattening operation, which reshapes the spectrogram matrix, comprising $k$ frequency_bins and $m$ time_bins into a contiguous vector of length $STF{T}_{Cof}=k\times m$. Consequently, the final feature set, denoted as STFT_Cof, encapsulates the entire constellation of magnitude values from the STFT for each single channel in each subject, thereby rendering the rich time-frequency structure into a format suitable for the EEGLAB. Now, the dataset is ready $X_R\in {\mathbb{R}}^{t\times c\times STFT\_Cof\times 1}$ to be fed into the EEGnet for training and building the future reference model. Thus the final tensor $X_R\in {\mathbb{R}}^{400\times 30\times 585\times 1}$.

3.2. Adaptive Contrast Enhancement (ACE)

The adaptive local spectral contrast enhancement (ACE) is used to improve the interpretability and feature salience of time-frequency representations [27,28]. In this study, ACE is used to mitigate the low contrast of the STFT spectrogram features. This is accomplished by normalizing the spectrogram $STF{T}_{Cof}$ based on local statistics within a defined neighborhood $N$.

Let $STF{T}_{Cof}=k\times m$ denote an input spectrogram and $Enhanc\_STF{T}_{Cof}=k\times m$ denote the enhanced spectrogram. Let neighborhood $N_k$ and $N_m$ around each point $(k,m)$ be defined, where the kernel $(k\times m)$ is specified by the neighborhood_size as a hyperparameter. The local statistical estimation includes the local mean $\mathsf{\mu }(k,m)$ and local standard eviation$(k,m)$, which are estimated within this neighborhood using uniform filters. The local mean $\mathsf{\mu }(k,m)$ is computed as equation:

$$\mathsf{\mu }\left(k,m\right)={\displaystyle \frac{1}{N}}{\sum }_{i,j\in (k,m)}STF{T}_{Cof}(\mathrm{I},\mathrm{j})$$

(5)

where $N$ is the number of points in the neighborhood. The local standard $\mathsf{\sigma }\left(k,m\right),$ which measures local spectral contrast (texture) is derived from the local mean of squares as equation:

$$\mathsf{\sigma }\left(k,m\right)=\sqrt{{\displaystyle \frac{1}{N}}{\sum }_{i,j\in \left(k,m\right)}{STF{T}_{Cof}\left(\mathrm{I},\mathrm{j}\right)}^2-{\mathsf{\mu }\left(k,m\right)}^2}$$

(6)

Each point in the spectrogram is then normalized (Z-score) by subtracting the local mean and dividing by the local standard deviation as equation:

$$Enhan{c}_{Norm}\_STF{T}_{Cof}(k\times m)={\displaystyle \frac{STF{T}_{Cof}\left(k\times m\right)–\mathsf{\mu }\left(k,m\right)}{\mathsf{\sigma }\left(k,m\right)+ϵ}}$$

(7)

A small constant ${ϵ=10}^{-8}$ is added for numerical stability and in some cases to prevent division by zero. This operation effectively stretches the local dynamic range, boosting components that stand out from their local background. To map the enhanced data $\mathrm{E}\mathrm{n}\mathrm{h}\mathrm{a}\mathrm{n}{\mathrm{c}}_{Norm}\_STF{T}_{Cof}(k,m)$back to a physically meaningful range, a min-max rescaling is applied to normalize within the range [0, 1] as equation:

$$Enhan{c}_{Scaled}\_STF{T}_{Cof}(k,m)={\displaystyle \frac{Enhan{c}_{Norm}\_STF{T}_{Cof}(k,m)-Min(Enhan{c}_{Norm}\_STF{T}_{Cof})}{Max(Enhan{c}_{Norm}\_STF{T}_{Cof})–Min(Enhan{c}_{Norm}\_STF{T}_{Cof})}}$$

(8)

It is then rescaled to the original amplitude range of the input spectrogram $STF{T}_{Cof}(k,m)$, to preserve the global amplitude relationships while maintaining the enhanced local contrast as equation:

$$Enhanc\_STF{T}_{Cof}(k,m)=Enhan{c_{Scaled}STFT}_{cof}(k,m)\bullet \left(Max\left(STF{T}_{Cof}\right)-Min\left(STF{T}_{Cof}\right)\right)+Min\left(STF{T}_{Cof}\right)$$

(9)

3.3. Deep Learning EEGNet Classifier

After extracting the features for all subjects and channels, the enhanced data $Enhanc\_STF{T}_{Cof}(k,m)$ is fed into the CNN, which consists of 14 layers arranged in two blocks. Table 1 details the sequential layer architecture of EEGNet [29], a convolutional neural network (CNN). Each block consists of a sequence of layers including convolution, normalization, ReLU activation, and average pooling, with their corresponding hyperparameter configurations specified in the table.

The model is trained with the Adam optimizer and categorical cross-entropy loss function. To maintain the cross-validation, the STFT dataset is split into two sets: 50% for training and the other 50% for testing. The 50% training dataset is further subdivided into training (70%) and validation (30%) sets. The model was trained for 100 epochs with a batch size of 32, and performance was monitored on the validation set. The choice of window function $w\left(n\right)$ and parameters $N_{seg=128}$ and $N_{overlap}=64$ affects the time-frequency resolution trade-off. These parameters were selected following experimentation for achieving better accuracy. The Model’s performance was evaluated on the blind test set using accuracy and the weighted F1-score. A confusion matrix was computed for each subject to assess classification performance across the five classes. The confusion matrices were summed across all subjects to obtain an aggregate performance metric. Average accuracy and F1-scores were calculated to summarize the model’s effectiveness. The results were averaged across all 42 subjects. The summed confusion matrix provided insights into the model’s classification performance across the selected classes. In the experiments, the adjustable parameters include the number of classes Nclasses = 5, dropout rate δ = 0.5, filters F1 = 8, depth multiplier D = 2, filters F2 = 16, and normalization rate η = 0.25. Dropout type is either SpatialDropout2D or Dropout.

4. Results and Discussion

Spectrograms based on the STFT for selected EEG signals, such as those from subjects 2 and 3, are shown in Figure 5. In which Figure 5 (a) depicts EEG signals related to subject 2 across three channels: 0, 5, and 10. Their spectrogram for each channel is represented as a time bin on the x-axis and a frequency bin on the y-axis, with spectral power indicated by color intensity. From the image, it is obvious that the higher amplitude signals have higher power spectrum signals in their frequency domain. This process can represent the signals more effectively and provide an abstract representation of the signals’ temporal changes over time, leading to a better understanding by the classifier. Similarly, Figure 5 (b) shows the EEG signals for subject 3, using the three channels 0, 5, and 10, in both the time and frequency domains. Remarkably, this method’s capacity underscores revealing energy distribution across frequency bands over time that offers an informative abstraction of temporal dynamics, enhancing feature discriminability for subsequent classification.

A limitation of the STFT technique is its Fixed Time-Frequency Resolution since the STFT uses a fixed window size $N_{seg}$. The choice of window function mitigates but does not eliminate this issue. Also, it could have sensitivity to parameter selection because the performance of the STFT depends heavily on the choice of $N_{seg}$ and, $N_{overlab}$ for the window function. Suboptimal parameters can lead to poor resolution or artifacts, requiring domain expertise or empirical tuning. Eventually, tensor $Y\in {\mathbb{R}}^{T\times C\times K\times M}$ can be high-dimensional, especially for large $T$,$C$ or $STF{T}_{Cof}$, that is increasing memory and computational requirements for downstream processing

4.1. Signal Enhancement Using STFT with ACE

Figure 6 visualizes the effect of the adaptive spectral contrast enhancement on the STFT original $STF{T}_{Cof}$ and enhanced STFT $Enhanc\_STF{T}_{Cof}$ of subject 2 and subject 3 using randomly selected channels 5 and 10 from each subject. The ACE has three main effects as follows:

• First, the increased contrast, which means that the high and low frequency bin regions in the spectrogram are clearer in the enhanced versions. This makes it easier for the machine learning to distinguish between different frequency components and their changes over time.
• Secondly, spectral features are sharper or more defined in the enhanced plots. And The third effect is background noise elimination by amplifying the relevant signal components while relatively suppressing the less important background noise.

Furthermore, it is easily assessed that the difference between Figure 6 (a), the original STFT, and Figure 6 (b), the enhanced STFT, is subjective. Indeed, ACE has limitations that need to be considered. For example, the normalization process can create artifacts around strong, isolated features, where the local standard deviation is very low just outside the feature's boundary. Besides, the ACE method assumes local stationarity within the window. It may perform poorly with highly non-stationary EEG interfering signals.

The choice of the hyperparameter named neighborhood_size $N,$ is essential for improving the EEG signal representations. It is a trade-off, because if a small window, for instance, $N=3$, the process captures very fine-grained, high-frequency texture. Therefore, this is useful for enhancing narrow spectral lines but may also extract high-frequency noise. Conversely, if large windows are configured, for instance, $N=15$, the process captures broader spectral trends. This is effective for enhancing larger structures, such as a formant's spectral envelope, but may overlook finer details. Overall, the enhancement aims to make the important spectral characteristics of the EEG data more visually prominent, which can be beneficial for subsequent analysis or input into a machine learning model like EEGNet.

4.2. Recognition Accuracy

The classification accuracy is assessed by exploiting the confusion matrix (CM) [30].In the case of two classes, CM has four parameters: True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN), as illustrated in Table 2.

In this paper, the proposed classifier is evaluated by its accuracy and F1 score, in which their formulas are listed below equations:

$$Accuracy={\displaystyle \frac{TP+TN}{TP+FP+TN+FN}}$$

(10)

$$F1score={\displaystyle \frac{TP}{TP+0.5(FP+FN)}}$$

(11)

As this research involves five emotional categories, the standard confusion matrix is extended from two to five classes. The idea is to consider one class as true and the remaining four as false. For example, if the second class is considered true, the first, third, fourth, and fifth are deemed false. Figure 7. presents a comparative analysis of classification accuracy for 42 subjects, contrasting the performance of original EEG data (black), pre-processed EEG dataset using Short-Time Fourier Transform (STFT) feature extraction (orange), and the STFT with the ACE processing (green). The x-axis represents the subject index (1–42), while the y-axis denotes accuracy values ranging from 0 to 1.

The plot reveals variability in the accuracy across subjects, with some showing significant improvement after STFT with the ACE dataset processing, while others exhibit minimal change or slight degradation. Figure 8 presents a comparative analysis of classification accuracy based on F1-score for 42 subjects, contrasting the performance of original EEG data (black), pre-processed EEG dataset using STFT feature extraction (orange), and the STFT with the ACE processing (green). The x-axis represents the subject index (1–42), while the y-axis denotes accuracy values ranging from 0 to 1. There is an improvement after STFT with the ACE dataset processing compared with the original EEG dataset, while others exhibit minimal change or slight degradation.

4.3 Comparison of Classification Accuracies

Table 3 contains the recognition accuracy results of an EEG for five emotional classes: neutrality, anger, happiness, sadness, and calmness. Eight experiments were run for training and testing, and their averages have been calculated. Table III demonstrates a clear performance where the proposed method (STFT+ Adaptive Contrast Enhancement + EEGNet) consistently achieves the highest accuracy with an average of up to 72.5%, outperforming both the approach consisting of STFT + EEGNet at an average of up to 70.84%, and the baseline EEGNet alone has the classification accuracy as an average of up to 59.94% (which is similar to the accuracy published in original dataset [10]).

The substantial increase from the baseline to STFT-enhanced models confirms the critical importance of time-frequency features for EEG analysis, while integrating a small improvement from adding ACE signal preprocessing as an effective refinement technique that enhances feature discriminability in spectrograms. The resulting low variance across all eight experimental runs confirms the statistical robustness of these improvements, solidifying the conclusion that each processing stage (particularly the ACE approach) meaningfully contributes to more accurate and reliable EEG classification.

Table 4 illustrates the confusion matrix related to the STFT with the ACE preprocessing. For the 42 subjects, each of which has 1680 samples per class, and 1 subject has 40 samples per class, as listed in the table. Because each subject has 200 instances entered in the testing (40 instances/class). The model demonstrates strong overall performance for a complex 5-class problem. The high values along the main diagonal (1195, 1195, 1287, 1294, 1130) summed to 6,101 correct predictions, while off-diagonal elements represent misclassifications. The overall accuracy is given as 6,101/8,400 = 72.6%, which reflects the correct predictions divided by the number of samples (5 classes × 1680 instances).

Figure 9 depicts the evaluation across all 42 subjects, featuring three distinct confusion matrices. The first CM corresponds to the model testing on the 200 instances for each subject's original dataset, establishing a baseline performance. The second matrix presents the testing results utilizing STFT pre-processing on a held-out set of 200 instances, representing a blind testing scenario. The third matrix illustrates the accuracy achieved by integrating STFT with ACE pre-processing. A comparative analysis reveals that the principal diagonal of the third confusion matrix (proposed method) contains the highest instances of correct predictions, demonstrating a superior classification accuracy and an enhanced emotional recognition rate attributable to the combined STFT-ACE preprocessing pipeline.

The graph in Figure 10, illustrates the relationship between classification accuracy and neighborhood size as a hyperparameter associated with ACE preprocessing. As it is shown that the accuracy demonstrates notable sensitivity to this parameter, initially increasing to an apparent optimum between neighborhood sizes of 10 and 11, where peak performance of approximately 73% is attained. Beyond this peak, a consistent decrease in accuracy is observed when the neighborhood size increases to 14, suggesting that larger neighborhoods introduce non-discriminative information that degrades the model's performance.

4.4. SHAP Channel Importance Analysis

To highlight which active EEG channels contribute more to building the model than others, SHAP (Shapley Additive exPlanations) analysis is applied. SHAP is based on mean absolute SHAP values to provide a quantitative assessment of the contribution of EEG channel to the predictive output of the trained model [31]. According to Figure 11, channels with taller bars are more influential in determining the model's predictions compared to channels with shorter bars. A high mean absolute SHAP value for a channel suggests that the information contained within the signal from that channel significantly contributes to the model's ability to discriminate between the different classes. Conversely, channels with low mean absolute SHAP values are less effective for the model's performance.

Based on the mean absolute SHAP values presented in Figure 10, and the sorted list of channels by importance, it is clear that channels: channel 15, channel 29, channel 10, channel 1, and channel 2 exhibit higher mean absolute SHAP values compared to other channels. This means that the information acquired by these specific channels is more critical for the model to classify the emotional EEG data. On the contrary, channels with shorter bars, such as channel 9, channel 21, and channel 26, have lower mean absolute SHAP values, indicating they are less influential in the model's decision-making process for this task and dataset. Therefore, the dataset could be reduced without a big impact on the model accuracy. Analyzing the spatial distribution of these important channels on an EEG cap could help neuroscientists to locate which brain regions (channels) are expected to be relevant for emotion processing. In addition, SHAP can be used in the future to adjust the accuracy by applying channel reduction based on SHAP mean absolute values, and then to pipeline it with the STFT along with the ACE for better accuracy and efficient computation.

5. Conclusions

Despite recent improvements in BCI, the biggest potential challenge in EEG-based BCI systems remains the limited recognition rate and the ability to represent an individual's neural patterns for accurate and reliable classification. This paper proposed an efficient methodology aimed at enhancing classification accuracy through a robust pipeline for EEG signal processing and classification. The proposed approach leverages the STFT of non-stationary EEG signals with ACE to amplify high-frequency details by attenuating low-frequency components. The proposed method achieved an average accuracy of 72.50%. This represents a 12.55% improvement over the baseline accuracy of 60% reported in the original EAV study. The proposed methodology has the potential to enhance EEG recognition across multiple fields. Future work will focus on optimizing key parameters, such as STFT windowing and ACE neighborhood size, and explore the use of SHAP-based channel interpretation for feature selection to facilitate the deployment of this methodology on lightweight embedded systems.