FFT and Embedding-Constrained EEG Architectures for Minimal-Channel Semantic Decoding

A. Participants

• Number & demographics: For single-participant / participant-level analyses I report results per subject. Similar to Sun et. al. and Wandelt et. al., few participants were used (n=2) (due to focus on single participant decoding) and most models were trained on both datasets independently (evaluation metrics were averaged) (.
• Note: Cross participant evaluations were not tested yet this should be an area of future research. From a practical standpoint however, these models can be easily trained and fine-tuned to individual participants.

B. Hardware & Acquisition

• EEG hardware: OpenBCI Ganglion (and an OpenBCI flat & comb electrodes). Four active channels were used and optimized for semantic decoding (schematic provided in Figure 1). For both tasks, electrodes were placed at TP7, TP8, F7, and F8 in accordance with the universal 10-20 system (see Figure 1) (Maskeliunas R, Damasevicius R, Martisius I, Vasiljevas M. Consumer-grade EEG devices: are they usable for control tasks?). For sampling, LSL (lab streaming layer) was used to stream data from the OpenBCI GUI.
• Sampling rate: Sampling rate was 200 Hz over 5-second epochs (1000 points/sample), with both time-series and FFT data (60 bins, 250 snapshots) collected per trial.”

C. Impedance & Quality Control: Channels Checked Before Each Session; Trials with High Impedance Levels Were Ignored

D. Stimulus Presentation: Words Displayed Centrally on a Monitor with a Fixation Cross Between Trials. Timing: Stimulus Duration = 5 s, Inter-Stimulus Interval ~ 1-2 s.

E. Stimuli & Experimental Design

• Word lists: Two primary semantic tasks: emotional valence (words labeled negative / neutral / positive) and part-of-speech (noun vs. verb). Experiments described in the introduction used sets of 50 unique words (each).
• Trial structure: Each word was presented for each trial; participants were asked to ponder the word (in addition to passive viewing). Each word had 0 repetitions
• Total trials & augmentation rationale: For the FFT-based method I treated every other FFT “window” or “snapshot” from a single epoch as separate training samples (see Feature Extraction). For example, 50 words × 125 FFT samples per word → 6250 FFT samples; with 100 words → 12500 FFT samples.

F. EEG Preprocessing

• Bandpass filtering: 0.5–100 Hz
• Line-noise removal: notch filter at 50/60 Hz
• Normalization: Percentile-based scaling: for each channel, values were normalized using the 5th–95th percentile range to reduce the influence of extreme values. This approach improves strength across sessions/participants and is used prior to data entrance into the model.

G. Feature Extraction & FFT Sampling

• Raw time-series
  • The epoch time-series (shape: channels, timepoints) after preprocessing was used directly as input to time-domain CNN architectures. For the EEGNet-style baseline the input tensor shape was (channels, timepoints, 1). Note: An extra dimension was added for compatibility with the EEGNet inspired architecture.
• FFT-based frequency-domain sampling (data amplification trick; see Appendix B: Flowchart 2)
  • Rationale: FFT representations can expose certain features (delta/theta/alpha/beta/gamma bands). Because 250 “snapshots” of this FFT data occurred over a different time interval (~2 seconds) I can treat each FFT “plot” as its own sample. In this case, every other snapshot was treated as a unique training sample with the same label as the parent sample (leaving 125 unique samples per trial). Important: It was ensured that trial-level splits were implemented to prevent data-leakage.
  • Parameters used:
    • Frequency bins: 1–60 Hz
    • Number of snapshots per sample: 125
    • Normalization: Each FFT snapshot was normalized using the same percentile scaling as for the time-series.
    • Representation: For CNNs I formatted each FFT snapshot as a 2D “shape” (channels × frequency bins) so that depthwise/separable convolutions could learn spatial-frequency filters.

H. Model Architectures & Development Pipeline (see Appendix A for Detailed Architectures)

• EEGNet-style baseline
  • Based on the EEGNet design¹ (compact model with multiple filters)
  • Key layers: Depthwise spatial filter (per-channel) and separable convolution (to capture temporal patterns).
• Explainability & LIME-guided channel reweighting
  • After training EEGNet-style models, I extracted early-layer filter weights and used LIME (Local Interpretable Model-agnostic Explanations) to estimate channel contributions to predictions (and to discern artifacts/noise)
  • Channels were then weighted accordingly via multiplicative channel scaling in the input
• Fused multi-head model (fine-tuning fusion)
  • Individual single-task models (emotion, POS) were trained together to demonstrate the efficacy of a fusion model. Early convolutional weights were extracted and concatenated into a fused encoder, and task-specific heads were attached.
• Embedding-Constrained EEG Architecture (primary proposed model)

• Generate word embeddings using OpenAI’s text-embedding-3-small for the full stimulus set (n = 100 words) (He T, Boudewyn MA, Kiat JE, Sagae K, Luck SJ. Neural correlates of word representation vectors in natural language processing models…)
• Cluster embeddings into K semantic archetypes (K manually chosen; in the experiments K = 2). Archetypes include groups like emotional, concrete/sensory, abstract, noun/verb etc
• Clustering group categories are determined by evaluating embeddings for words closest to the centroid (and then used as representations for the cluster as a whole)
• For each grouping I created a specialized processing branch with convolutional filters tuned to frequency bands expected for that archetype (neuroscience-informed choices: emotion → emphasis on delta/theta/alpha; noun/verb → include beta/gamma etc) (Pulvermuller F, Preissl H, Lutzenberger W, Birbaumer N. Brain rhythms of language: nouns versus verbs)^,(Gkintoni E, Aroutzidis A, Antonopoulou H, Halkiopoulos C. From Neural Networks to Emotional Networks: A Systematic Review of EEG...). Specifically, temporal convolution sizes and filter sizes are then utilized that bias each branch to the target frequency bands (see Table A1, A2, A3 for detailed architecture).
• Each branch processes FFT input (channels × frequency bins). Branch outputs are concatenated and passed to shared dense layers, and then to task-specific heads.
  • Key property: embeddings are used only for architecture construction, not at inference. In this way, the model is forced to learn EEG related patterns and not over-rely on embedding information (it was demonstrated that the model often prioritizes the embeddings over the noisy EEG data when embeddings were involved in training).

5.

Embedding-as-regularizer (alternative strategy)

• Includes an auxiliary loss (extra loss function) forcing learned EEG embeddings to correlate with word embeddings (i.e, cosine similarity). The goal was to create a joint-embedding space that represented both EEG data and word embeddings. I found this strategy often caused optimization conflicts, over-reliance on the embeddings and poorer performance. Nevertheless, it was implemented for comparison and to explore alternate uses of word embeddings.

I. Loss Functions & Optimization

• Primary loss: Categorical cross-entropy with focal loss (gamma=1.5, alpha=0.25) for imbalanced emotion classes (20 pos/15 neu/15 neg); POS was balanced.
  • For FFT data, trial level split was implemented to ensure integrity.
• Optimizer: Adam with initial learning rate = 1e−3 for training from scratch; 1e−4 for fine-tuning fused models.
• Regularization: Dropout in dense layers (e.g., 0.4).
• Batch size: (example) 32 for FFT snapshots, and smaller (16) for time-series
• Epochs & early stopping: maximum epochs = 100 with early stopping based on validation loss/patience = 20 epochs.

J. Training & Evaluation Protocol

• Data splits: Performing train/validation/test splits, Stratified Shuffle Split was utilized: n_splits = 1, random state = 42, test size = 0.2
• Within-participant fine-tuning: For fused and embedding-constrained experiments, I report both (a) models trained from scratch per participant and (b) models built via transfer/fine-tuning from other participants’ weights (to simulate realistic use where a pretrained model is personalized).

K. Evaluation Metrics & Statistical Tests

• Primary metric: classification accuracy. Baseline chance levels: emotion (3-class) = 33.3%, POS (2-class) = 50%.
• Secondary metrics: confusion matrices, LIME analysis, and convolutional weight visualizations where applicable.
• Evaluations: Model accuracies were evaluated by, after training the model, shuffling the validation/testing set, randomly sampling 40-80%, evaluating the accuracy and repeating 100 times. The accuracies were then averaged to ensure it represented the models true performance. Classification accuracies are reported with 95% Wilson binomial confidence intervals, computed relative to the number of test trials.

L. Explainability & Weight Analysis

• Weight visualization: plot early convolution (depthwise & separable) filters and their spatial maps to inspect learned spatial & temporal patterns.
• LIME: applied to trained EEGNet-style models to estimate per-channel/frequency contributions to single-sample predictions. LIME outputs guided channel reweighting: channels consistently identified as artifactual or noisy were downweighted (by a multiplicative scalar) and models fine-tuned. Results of reweighting are reported.
• Other studies: evaluating performance after removing reweighting, removing/adding noise, etc.

M. Reproducibility & Code/Data Availability

• Code: all preprocessing, model definitions, training scripts, and analysis notebooks are available in the project repository (to be provided). Versions of libraries (TensorFlow, NumPy, etc.) will be included.
• Data: raw EEG data will be shared via the same repository in a pkl file. FFT snapshots and model weights will not be provided explicitly but the code to extract such weights will be available.

A. Data Quantity, Preprocessing, and Effective Sample Amplification

B. Preprocessing Outcomes and Channel Quality

C. Representation Comparison: Time-Series vs FFT Snapshots

D. Explainability, Channel Reweighting, and Single-Participant Fine-Tuning

E. Fused Multi-Head Model (Fine-Tuning Fusion)

F. Embedding-Constrained EEG Architecture (Proposed Model; see Appendix B: Flowchart 2)

G. Embedding-as-Regularizer Strategy (Alternative)

H. Summary of Mean Test Accuracies over 100 Shuffled Evaluations (Accuracies Are Reported with 95% Wilson Binomial Confidence Intervals; Note: For Time-Series Models, CI Will Be Large Due to Small Testing Sizes but This Is Mitigated Through Shuffling and Averaging for Confidence (see Methods)

Appendix A: Model Architectures

Table A1. Model Architecture Specifications for proposed embedding-constrained model.

Table A2. Training Config. for embedding constrained model.

Table A3. Architecture Design Principles.

Table A4. Architecture summary for baseline FFT and time-series models (single-task). This architecture mirrors that of EEGNet, using both separable filters and depthwise filters (utilized as it is applied to each channel separately).

Table A5. Architecture summary for the multi-head fused model. Note, “attention” was implemented to properly and flexibly combine both pre-trained feature vectors. These models were subsequently weighted with the goal of achieving higher accuracies. Shown is the model specs before weighting.

Appendix B: Extended Results, Graphs, & Confusion Matrices

Flowchart 1. Introduction summary and outline for experiment processes.

Flowchart 1. Introduction summary and outline for experiment processes.

Flowchart 2. Left to right: (1) Visualization of FFT data sampling amplification procedure. (2) Visualization of proposed embedding-constrained model.

Flowchart 2. Left to right: (1) Visualization of FFT data sampling amplification procedure. (2) Visualization of proposed embedding-constrained model.