From Segment-Level Discrimination to Unseen-Subject Transfer in Motor Imagery EEG: A Subject-Wise Study of a GAF–PLV Parallel CNN

1. Introduction

Electroencephalography (EEG) remains one of the most practical modalities for brain–computer interface (BCI) research because it is non-invasive, portable, and temporally precise [1,2]. Within EEG-based BCI, motor imagery (MI) is particularly important because it supports assistive control, neurorehabilitation, and human–machine interaction without requiring external stimulation [1]. Yet MI-EEG decoding remains difficult. The signals are weak, non-stationary, noisy, and highly variable across individuals. These properties make it challenging to distinguish genuine task-relevant structure from subject-specific nuisance structure.

Over the last decade, deep learning has become increasingly prominent in MI-EEG analysis because it can learn highly nonlinear features directly from data [2,3,4]. Parallel and transformed-representation models have further shown that spatio-temporal coding, connectivity-aware decoding, and image-like representations can all improve raw benchmark performance when the evaluation protocol is permissive enough [5,6,7]. However, the field has also inherited a methodological problem that is now too important to ignore: model performance depends not only on the representation or architecture, but also on the unit of data partitioning and the transparency of cohort selection. Recent work on Electroencephalography deep learning indicates that sample-based cross-validation may yield optimistic estimates because subject-specific characteristics can compromise fold independence, while even subject-based validation may remain vulnerable when validation and test partitions are not strictly separated [8]. In short, validation strategy is not a minor implementation detail; it changes the scientific meaning of the reported accuracy.

This problem is especially visible in the MI-EEG literature on the PhysioNet EEG Motor Movement/Imagery dataset [9]. A substantial subset of papers on this dataset has relied on segment-level, image-level, or time-resolved validation [6,10,11,12], while many others have used fewer than all available volunteers because of annotation problems, missing trials, computational tractability, quality filtering, or deliberate reduced-cohort benchmarking during model development [4,13,14,15,16]. None of these practices automatically implies scientific misconduct. Some are understandable, and some are useful at the representation-design stage. But if protocol strength is not distinguished from raw predictive performance, then direct comparison across papers becomes unreliable. Strong reported accuracy may reflect a strong model, an easier split, a smaller and more homogeneous cohort, or some combination of all three.

We recently proposed a GAF–PLV parallel CNN as a proof-of-concept segment-wise benchmarking framework for MI-EEG [7]. In that setting, the representation family achieved 99.73% binary accuracy on a 10-subject subset and 99.18% on a 30-subject subset. Those results established the promise of the spatio-temporal encoding for capturing nonlinear and non-stationary MI-EEG structure. The present study extends that line of work by testing the same framework under a subject-wise inference target, asking how much of that benchmark performance remains when the test subject is wholly unseen and every retained fold is disclosed.

For this reason, the current study adopts leave-one-subject-out cross-validation (LOSOCV) for the binary left-versus-right MI task. The analysis is performed on the 105 held-out subject folds for which subject-wise rerun outputs were available in this study. These folds correspond to subject IDs 1–105. Because the available outputs do not include fold results for subjects 106–109, the present subject-wise study is reported on the retained executable cohort rather than on a hypothetical full 109-subject rerun. This transparency matters: the paper’s scientific claim is tied to the exact folds that are actually available, not to unverified assumptions about missing outputs. The goal is therefore twofold: to provide a stronger estimate of cross-subject performance for the GAF–PLV framework, and to show directly how evaluation regime changes the scientific meaning of performance in MI-EEG research. In methodological terms, the paper aims to extend a strong proof-of-concept benchmark into a more rigorous subject-wise reference against which later claims of subject-independent MI-EEG performance can be judged.

The main contributions of this paper are as follows.

• We provide a fold-complete subject-wise LOSOCV study of our recently proposed GAF–PLV representation family on the retained executable cohort.
• We report a transparent retained-cohort analysis on the 105 held-out subject folds for which subject-wise rerun outputs were available in this study and pair it with bootstrap and inferential statistics so that cross-subject uncertainty is visible rather than implicit.
• We provide a direct protocol-sensitive comparison with our earlier pooled-window proof-of-concept benchmark, thereby showing how the same representation family changes meaning when the validation unit changes from windows to unseen subjects.
• We quantify that protocol-sensitive shift using both the cohort mean and held-out subject anchors rather than treating the drop as a single fixed number.
• We convert a strong proof-of-concept benchmark result into a more rigorous field reference by making the retained cohort, fold-level variability, uncertainty, and scope boundaries explicit.
• We distil the empirical findings into a practical reporting standard for future subject-independent MI-EEG studies and add exploratory classical-baseline context while making cohort mismatch explicit.

This paper therefore focuses on the scientific impact of protocol change: it shows how a strong proof-of-concept MI-EEG framework behaves when the inference target shifts from benchmark-oriented segment-level discrimination to subject-independent evaluation. Its importance lies in providing a more trustworthy reference for future method development, evaluation, and translational interpretation in MI-EEG.

2. Related Work

MI-EEG classification has generally developed along two broad directions: traditional machine learning pipelines based on handcrafted features, and deep learning pipelines that either learn directly from raw signals or operate on transformed representations [1,2]. Traditional approaches typically combine preprocessing, handcrafted feature extraction, and classification using methods such as LDA, SVM, KNN, or random forests, whereas deep learning has shifted the field toward convolutional, recurrent, and graph-based architectures that aim to learn discriminative features more directly from the data [3,4,5,6]. Yet the field’s methodological maturity has lagged behind its architectural creativity.

A major source of that mismatch is data partitioning. As Del Pup et al. show, EEG deep learning results can vary substantially depending on whether cross-validation is performed at the sample level, the subject level, or within a nested subject-based design. Sample-based approaches can place segments from the same recording across model-development and evaluation folds, which may inflate apparent accuracy because EEG contains strong subject-specific structure. Even standard subject-based approaches can remain optimistic if the same held-out data are used both to monitor training and to estimate final performance. Their large-scale comparative study therefore argues for subject-based strategies, and more specifically for nested subject-based strategies, whenever unseen-subject generalisation is the target of inference [8]. This point is highly relevant to MI-EEG, where within-subject benchmarking and cross-subject deployment are often conflated.

A separate but related problem is cohort definition. On the PhysioNet motor imagery dataset, many studies use fewer than the full 109 volunteers. In some cases this is due to missing trials, corrupted annotations, or documented data anomalies. In other cases it reflects deliberate reduced-cohort benchmarking, often motivated by computational cost or early-stage model development. Again, this practice is not automatically illegitimate. Indeed, reduced-cohort experimentation has been common on this dataset. The problem arises when cohort selection is not reported transparently, when exclusions are outcome-driven, or when reduced-cohort benchmarking is rhetorically upgraded into a claim of subject-independent superiority.

These two issues – validation protocol and cohort transparency – should therefore be treated explicitly in the literature review rather than hidden in implementation details. Table 1 summarises representative PhysioNet MI studies whose validation profiles should be interpreted with care when the target claim is unseen-subject inference. The point of the table is not to invalidate these papers. It is to clarify how directly each reported result supports cross-subject interpretation and practical subject-independent use.

Protocol heterogeneity is only one side of the problem. The other is selective or reduced cohort usage on the same dataset. Table 2 summarises representative studies that used fewer than all 109 PhysioNet volunteers. The important point here is subtle. The literature shows that reduced-cohort experimentation has been common, but this does not make it methodologically optimal. What matters is why fewer subjects were used, whether the rule was explicit, and whether the retained cohort was then analysed comprehensively or selectively.

Within this broader landscape, the present paper takes a narrower and more specific position. We do not argue that early-stage reduced-cohort benchmarking is without value; indeed, it helped establish the recently proposed GAF–PLV representation. Our point is that when the target claim is cross-subject transfer, protocol choice and retained-cohort disclosure must be explicit. Our own study therefore uses a simple rule: analyse the retained executable cohort transparently, report all available LOSOCV folds, and interpret performance in light of the stronger inference target. This is the methodological stance on which the rest of the paper is built. Segment-wise and pooled-sample studies can still be valuable for rapid representation screening, ablation testing, and early benchmark development. They are not, however, practical proxies for plug-and-play subject-independent MI-BCI, because they do not test whether a trained model transfers to a wholly unseen individual. What the literature still lacks, and what the present paper seeks to supply, is a transparent subject-wise reference object whose scientific meaning is difficult to misread and whose methodological lessons can be reused by later studies.

3. Methodology

3.1. Study Design and Rationale

The present study is a subject-independent evaluation of our recently proposed GAF–PLV spatio-temporal MI-EEG framework. The central representational idea is retained: a temporal branch based on GAF-derived channel correlation and a spatial branch based on PLV-derived functional connectivity are decoded jointly by a dual-input CNN. What changes is not the descriptor family but the inference target. Rather than treating pooled windows as the unit of validation, the present study uses a subject-wise LOSOCV design so that every reported test fold corresponds to an unseen individual. The resulting evidence therefore speaks directly to cross-subject transfer and to the size of the generalization gap between segment-level benchmarking and unseen-subject testing. More specifically, the study is designed to answer three tightly related questions: how large the protocol-sensitive performance drop is when evaluation moves from pooled windows to unseen subjects, whether the representation retains any above-chance transferable signal under that stricter test, and how much subject-wise heterogeneity is hidden by a single cohort mean.

3.2. Dataset and Problem Formulation

The experiments are based on the PhysioNet motor imagery EEG dataset acquired with 64 electrodes at a sampling rate of 160 Hz [9]. The underlying temporal and spatial feature-construction stages follow our recently proposed GAF–PLV pipeline, whereas the present paper changes the evaluation regime from proof-of-concept pooled-window benchmarking to subject-wise LOSOCV [7]. Figure 1 now provides the consolidated end-to-end workflow used in the present paper. Reading from left to right, the figure first defines the retained cohort and the binary task, then shows 8–30 Hz filtering, CAR, per-trial z-scoring, and segmentation of each 4 s trial into ten non-overlapping 0.4 s windows. The upper branch constructs one GAF matrix per channel and integrates the 64 channel-wise GAF descriptors through Spearman correlation to form the temporal image, whereas the lower branch computes the PLV-based spatial image from the same window. Both descriptors are then resized to $256\times 256\times 3$ before being processed by matched CNN branches. The right-hand side of Figure 1 also clarifies that the shown window-level class outputs are illustrative, that the final trial decision is obtained by the mode of the ten window predictions, and that the available evaluation uses a subject-wise LOSOCV design with one held-out test subject and one separate validation subject per outer fold. Each MI trial lasts 4 s and therefore contains 640 temporal samples per channel. In the currently available study outputs, the subject-wise analysis is binary and uses 105 valid subjects. Let the EEG trial from subject s, trial r, channel c, and time index t be denoted by

$$x_{s,r,c}\left(t\right),s\in \mathcal{S},r\in \mathcal{R},c\in \{1,\dots ,64\},t\in \{1,\dots ,640\}.$$

(1)

The binary label is

$$y_{s,r}\in \{0,1\},$$

(2)

where class 0 denotes left-hand MI and class 1 denotes right-hand MI.

3.3. Subject Inclusion, Exclusion, and Reporting Transparency

The official PhysioNet EEG Motor Movement/Imagery dataset contains recordings from 109 volunteers [9]. However, the available binary LOSOCV rerun outputs used in this study contain subject-wise fold results only for subject IDs 1–105. In other words, the retained executable cohort is known exactly, whereas fold outputs for subjects 106–109 are absent from the available study records. We therefore report the study on the basis of what is verifiably available: 105 held-out subject folds, all of which are included in the analysis. This is stricter than inferring undocumented exclusions from memory or from narrative convenience. It also prevents a common weakness in EEG reporting, namely, quietly changing the effective cohort without showing the exact fold-level evidence. Accordingly, the present paper does not claim more than the files support. It claims a transparent LOSOCV study on the 105 retained subject folds for which subject-wise outputs are available, and every one of those folds contributes to the reported summary metrics and figures.

3.4. Preprocessing and Temporal Partitioning

Consistent with the earlier GAF–PLV pipeline, each trial is filtered in the 8–30 Hz motor-rhythm band before common average reference is applied independently at each time point:

$${\tilde{x}}_{s,r,c}\left(t\right)=x_{s,r,c}\left(t\right)-{\displaystyle \frac{1}{C}}\sum _{k=1}^C{x}_{s,r,k}\left(t\right),C=64.$$

(3)

Each channel is then z-scored independently within the trial:

$$z_{s,r,c}\left(t\right)={\displaystyle \frac{{\tilde{x}}_{s,r,c}\left(t\right)-{\mu }_{s,r,c}}{{\sigma }_{s,r,c}+\epsilon }},$$

(4)

with

$${\mu }_{s,r,c}={\displaystyle \frac{1}{T}}\sum _{t=1}^T{\tilde{x}}_{s,r,c}\left(t\right),$$

(5)

$${\sigma }_{s,r,c}=\sqrt{{\displaystyle \frac{1}{T-1}}{\sum }_{t=1}^T{\left({\tilde{x}}_{s,r,c}\left(t\right)-{\mu }_{s,r,c}\right)}^2},$$

(6)

where $T=640$.

Each 4 s trial is divided into ten non-overlapping windows of length 0.4 s. Since the sampling rate is 160 Hz,

$$L=0.4\times 160=64,$$

(7)

so each trial contains

$$W={\displaystyle \frac{640}{64}}=10$$

(8)

windows. The w-th window is denoted by ${\mathbf{Z}}_{s,r}^{\left(w\right)}\in {\mathbb{R}}^{L\times C}$. The ten-window partitioning is a fixed preprocessing rule used for every trial before temporal and spatial image construction.

3.5. Temporal Representation Using GAF and Spearman Integration

For a channel signal ${\mathbf{u}}_c={[u_c\left(1\right),u_c\left(2\right),\dots ,u_c\left(L\right)]}^{\top }$, the temporal descriptor is built using the Gramian Angular Field formulation of Wang and Oates, followed by the same channel-integration strategy used in the earlier GAF–PLV pipeline [23]. Min–max normalisation to $[-1,1]$ is performed as

$$u_c^{\prime }\left(i\right)=\left\{\begin{array}{cc}2{\displaystyle \frac{u_c\left(i\right)-min\left({\mathbf{u}}_c\right)}{max\left({\mathbf{u}}_c\right)-min\left({\mathbf{u}}_c\right)}}-1,\hfill & max\left({\mathbf{u}}_c\right)\ne min\left({\mathbf{u}}_c\right),\hfill \\ 0,\hfill & max\left({\mathbf{u}}_c\right)=min\left({\mathbf{u}}_c\right).\hfill \end{array}\right.$$

(9)

The angular encoding is

$${\varphi }_c\left(i\right)=arccos\left(u_c^{\prime }\left(i\right)\right),i=1,\dots ,L,$$

(10)

and the Gramian Angular Field matrix for channel c is

$${\mathbf{G}}_c(i,j)=cos\left({\varphi }_c\left(i\right)+{\varphi }_c\left(j\right)\right),i,j=1,\dots ,L.$$

(11)

Since one GAF matrix is produced per channel, a given window initially yields 64 GAF matrices. These are integrated into a single temporal summary matrix by Spearman correlation between vectorised GAF matrices:

$$\mathbf{T}(c,d)={\rho }_s\left(\mathrm{vec}\left({\mathbf{G}}_c\right),\mathrm{vec}\left({\mathbf{G}}_d\right)\right),c,d=1,\dots ,64.$$

(12)

The resulting temporal image descriptor $\mathbf{T}\in {\mathbb{R}}^{64\times 64}$ is resized and stored as a $256\times 256\times 3$ image. Figure 2 illustrates this construction for Subject 22: representative single-channel GAF matrices are shown together with the final integrated Spearman matrix that serves as the temporal branch input to the CNN.

3.6. Spatial Representation Using PLV

For the same window, the analytic signal of channel c is computed by the Hilbert transform:

$$a_c\left(t\right)=u_c\left(t\right)+j\mathcal{H}\left\{u_c\left(t\right)\right\},$$

(13)

with instantaneous phase

$${\theta }_c\left(t\right)=arg\left(a_c\left(t\right)\right).$$

(14)

For channels c and d, the phase difference is

$$\Delta {\theta }_{c,d}\left(t\right)={\theta }_c\left(t\right)-{\theta }_d\left(t\right).$$

(15)

The phase-locking value is then

$$\mathrm{PLV}(c,d)=\left|{\displaystyle \frac{1}{L}}\sum _{t=1}^L{e}^{j\Delta {\theta }_{c,d}\left(t\right)}\right|.$$

(16)

This yields the spatial connectivity matrix $\mathbf{P}\in {\mathbb{R}}^{64\times 64}$, which is also resized and stored as a $256\times 256\times 3$ image. Following the earlier GAF–PLV pipeline, the PLV matrix is binarised using the upper quartile of off-diagonal weights so that the functional graph retains only stronger synchronisation edges. Figure 3, Figure 4, and Figure 5 show the corresponding PLV matrices, their binarised thresholded forms, and the resulting functional brain graphs for representative left- and right-imagery windows from Subject 22.

3.7. Dual-Input Parallel CNN

The temporal and spatial descriptors are decoded by a dual-input parallel CNN that follows our recently proposed GAF–PLV model family, interpreted here under a subject-wise LOSOCV regime rather than a pooled-window regime. As summarised in Figure 1, the descriptors used by the executable pipeline are resized image inputs; however, the layer-wise schematic in Figure 6 is drawn at the native $64\times 64$ descriptor scale in order to show the branch architecture, kernel orientation, channel expansion, and fusion stages more explicitly.

The temporal branch uses asymmetric early convolutions with $1\times 2$ kernels, whereas the spatial branch uses complementary $2\times 1$ kernels. This asymmetry is intended to bias the first stage of feature extraction toward directional local structure that differs between the GAF-based temporal representation and the PLV-based spatial representation. In both branches, three initial convolutions expand the feature depth from 32 to 64 channels before the first max-pooling stage. Two deeper $3\times 3$ convolutions then increase the representation to 256 channels, followed by a second pooling stage and a final $3\times 3$ convolution with 256 channels before the third pooling stage. According to the schematic in Figure 6, each branch is then flattened to a 9216-dimensional vector and projected through a branch-specific fully connected layer with 1024 units.

The two 1024-dimensional branch embeddings are concatenated into a 2048-dimensional fusion vector,

$$\mathbf{h}=[{\mathbf{h}}^{\left(T\right)};{\mathbf{h}}^{\left(S\right)}],$$

(17)

which is then passed to a shared fully connected layer with 512 units before final softmax classification:

$$\widehat{\mathbf{y}}=\mathrm{softmax}(\mathbf{W}\mathbf{h}+\mathbf{b}).$$

(18)

Training uses sparse categorical cross-entropy,

$$\mathcal{L}=-\sum _{k=0}^1\mathbb{1}(y=k)log{\widehat{y}}_k,$$

(19)

with the Adam optimiser, maximum 100 epochs, batch size 64, and learning rate ${10}^{-5}$. Figure 6 should therefore be read as a compact layer-by-layer architectural explanation of the branch design, while Figure 1 remains the authoritative end-to-end summary of how descriptor construction, resizing, branch decoding, and subject-wise evaluation interact in the present study.

3.8. Subject-Wise LOSOCV

Let $\mathcal{S}=\{1,2,\dots ,N\}$ denote the set of valid subjects with $N=105$. For each outer fold, one subject $s^{★}$ is held out for testing:

$${\mathcal{S}}_{\mathrm{test}}=\left\{s^{★}\right\}.$$

(20)

A different subject from the remaining pool is chosen for validation, and the rest are used for training:

$${\mathcal{S}}_{\mathrm{train}}=\mathcal{S}\setminus \left({\mathcal{S}}_{\mathrm{test}}\cup {\mathcal{S}}_{\mathrm{val}}\right).$$

(21)

Thus, one subject appears only in train or validation or test, never in more than one split in the same fold.

Window-level predictions are aggregated back to the trial level by majority vote, exactly as visualised in Figure 7. The schematic shows the split into training, validation, and test subjects together with ten explicit window-level predictions and the resulting trial-level majority-vote rule:

$${\widehat{y}}_{s,r}=\mathrm{mode}\{{\widehat{y}}_{s,r,1},{\widehat{y}}_{s,r,2},\dots ,{\widehat{y}}_{s,r,10}\}.$$

(22)

3.9. Performance Metrics

At the trial level, the following metrics are reported:

$$\mathrm{Accuracy}={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}},$$

(23)

$$\mathrm{Balanced}\mathrm{Accuracy}={\displaystyle \frac{1}{2}}\left({\displaystyle \frac{TP}{TP+FN}}+{\displaystyle \frac{TN}{TN+FP}}\right),$$

(24)

$$\mathrm{Precision}={\displaystyle \frac{TP}{TP+FP}},\mathrm{Recall}={\displaystyle \frac{TP}{TP+FN}},$$

(25)

$$F_1={\displaystyle \frac{2·\mathrm{Precision}·\mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}}.$$

(26)

Cohen’s kappa is computed as

$$\kappa ={\displaystyle \frac{p_o-p_e}{1-p_e}},$$

(27)

where $p_o$ is the observed agreement and $p_e$ is the expected agreement by chance.

4. Results and Discussion

4.1. Cohort-Level LOSOCV Performance

Across the 105 retained held-out subject folds, the binary subject-wise study produced a mean trial-level accuracy of 58.07% ± 8.27%, Macro-F1 of 53.48% ± 11.19%, macro precision of 60.97% ± 14.45%, and Cohen’s kappa of 0.1615 ± 0.1654. The median accuracy was 57.14%, the interquartile range was 52.38% to 64.29%, the minimum was 38.10%, and the maximum was 78.57%. These values show that the central empirical result is not a single headline score but the breadth of fold-to-fold behaviour once the held-out subject is fully excluded from model fitting.

These cohort-level values matter because they quantify performance after subject recurrence has been removed from the evaluation logic. In a protocol-sensitive literature, a lower but better-specified estimate is more informative for cross-subject inference than a near-ceiling result obtained under pooled partitions [8,10,11,12]. For real subject-independent use, the latter should be read as benchmark-oriented results rather than as deployment-ready performance estimates. Figure 8, Figure 9 and Figure 10 should therefore be read together: the first condenses the non-redundant cohort-level metrics, the second shows the full held-out subject accuracy profile, and the third visualises the subject-wise behaviour of Macro-F1 and kappa.

4.2. Statistical Analysis of Subject-Wise Performance

To make the subject-wise LOSOCV interpretation less impressionistic, statistical analysis was performed directly on the 105 held-out subject folds. For each metric, we report mean, standard deviation, median, quartiles, minimum, maximum, and a 95% bootstrap confidence interval (CI) of the mean obtained from 20,000 bootstrap resamples. Normality of the subject-wise distributions was examined with the Shapiro–Wilk test. Accuracy and Macro-F1 were then tested against the balanced binary baseline of 0.5, while Cohen’s kappa was tested against 0. One-sample t-tests, Wilcoxon signed-rank tests, and Cohen’s d were used so that both parametric and nonparametric evidence were available.

Figure 11. Mean performance with 95% bootstrap confidence intervals for the three non-redundant cohort-level metrics. The confidence intervals do not cross the relevant baselines for accuracy, Macro-F1, or kappa, which is consistent with the inferential tests in Table 3.

Figure 11. Mean performance with 95% bootstrap confidence intervals for the three non-redundant cohort-level metrics. The confidence intervals do not cross the relevant baselines for accuracy, Macro-F1, or kappa, which is consistent with the inferential tests in Table 3.

Table 3. Subject-wise descriptive and inferential statistics across the 105 LOSOCV folds. CI denotes the 95% bootstrap confidence interval of the mean.

Metric	Mean ± SD	Median	Q1–Q3	Min–Max	95% CI	Baseline	t(104)	Wilcoxon p	Cohen’s d
Accuracy	0.5807 ± 0.0827	0.5714	0.5238–0.6429	0.3810–0.7857	0.5649–0.5964	0.5	10.00 ($p<{10}^{-16}$)	$2.50\times {10}^{-13}$	0.976
Macro-F1	0.5348 ± 0.1119	0.5524	0.4312–0.6050	0.3226–0.7846	0.5139–0.5562	0.5	3.19 ($p=0.0019$)	$2.27\times {10}^{-3}$	0.311
Cohen’s kappa	0.1615 ± 0.1654	0.1429	0.0476–0.2857	$-0.2381$–0.5714	0.1306–0.1927	0	10.00 ($p<{10}^{-16}$)	$5.84\times {10}^{-14}$	0.976

Table 3. Subject-wise descriptive and inferential statistics across the 105 LOSOCV folds. CI denotes the 95% bootstrap confidence interval of the mean.

Metric	Mean ± SD	Median	Q1–Q3	Min–Max	95% CI	Baseline	t(104)	Wilcoxon p	Cohen’s d
Accuracy	0.5807 ± 0.0827	0.5714	0.5238–0.6429	0.3810–0.7857	0.5649–0.5964	0.5	10.00 ($p<{10}^{-16}$)	$2.50\times {10}^{-13}$	0.976
Macro-F1	0.5348 ± 0.1119	0.5524	0.4312–0.6050	0.3226–0.7846	0.5139–0.5562	0.5	3.19 ($p=0.0019$)	$2.27\times {10}^{-3}$	0.311
Cohen’s kappa	0.1615 ± 0.1654	0.1429	0.0476–0.2857	$-0.2381$–0.5714	0.1306–0.1927	0	10.00 ($p<{10}^{-16}$)	$5.84\times {10}^{-14}$	0.976

The statistical results sharpen the descriptive interpretation. Accuracy was significantly above the balanced binary baseline of 0.5 under both parametric and nonparametric testing, with a large effect size. Macro-F1 was also significantly above 0.5, but with a much smaller effect size, which indicates that class-balanced performance is materially weaker than raw correctness. Subject-wise accuracy was significantly higher than subject-wise Macro-F1 (mean difference 0.0459; paired t(104)=10.23, $p<{10}^{-16}$; Wilcoxon $p=2.91\times {10}^{-19}$; Cohen’s $d_z=0.999$), confirming that accuracy alone paints an overly favourable picture of stability. Cohen’s kappa was significantly above zero, yet its magnitude remained low and negative values were observed for a subset of subjects, which is consistent with weak chance-corrected agreement in the lower-performing tail. In the present balanced binary setting, kappa was also a deterministic affine transform of accuracy ($\kappa =2\times \mathrm{Accuracy}-1$ to machine precision), so it should be interpreted as a chance-corrected restatement of the same subject-level pattern rather than as wholly independent evidence.

4.3. Subject-to-Subject Variability

To make the subject-wise behaviour explicit, this paper reports the full held-out subject profile rather than only a cohort mean. The held-out subject profile reveals a wide spread between the lowest-performing fold (Subject 21: 38.10% accuracy, 34.38% Macro-F1, $-0.238$ kappa), a representative central fold (Subject 7: 57.14% accuracy, 56.25% Macro-F1, 0.143 kappa), and the strongest retained fold (Subject 22: 78.57% accuracy, 78.46% Macro-F1, 0.571 kappa). Figure 9 shows the full unsorted subject-wise pattern, while Figure 10 shows the corresponding Macro-F1 and kappa traces. These three points are reported as anchors within a heterogeneous cohort so that the breadth of cross-subject behaviour is visible alongside the cohort summary.

The LOSOCV results therefore show substantial heterogeneity across individuals. This variability is visible directly in Figure 8, Figure 9, and Figure 10. Some held-out subjects are classified reasonably well, whereas others remain close to chance. Quantitatively, 90 of 105 subjects (85.7%) are at or above 50% accuracy, 39 subjects (37.1%) reach at least 60%, only 8 subjects (7.6%) reach at least 70%, and just 3 subjects (2.9%) exceed 75%. At the lower end, 15 subjects (14.3%) fall below 50% accuracy, and one subject falls below 40%. This heterogeneity is not a nuisance detail; it is one of the main scientific results of the paper.

The same conclusion appears in the companion metrics. Macro-F1 ranges from 32.26% to 78.46%, and kappa ranges from $-0.238$ to 0.571. Thus, the model’s errors are not merely random noise around a single mean. Rather, there is a structured subject-wise dispersion that reflects genuine instability of cross-subject transfer. This observation is consistent with the broader EEG partitioning literature. Subject-specific properties of EEG can be learned and exploited by deep networks, which is precisely why subject-based validation is necessary when the target claim is cross-subject generalisation rather than within-subject adaptation [8]. The present fold-by-fold spread therefore supports a cautious interpretation: the proposed representation retains some cross-subject signal, but it is not yet sufficiently invariant to individual EEG idiosyncrasies.

4.4. Class-Wise Performance

Class-wise performance is shown in Figure 12. The left class achieved precision 0.5711, recall 0.6481, and F1-score 0.6072, whereas the right class achieved precision 0.5933, recall 0.5134, and F1-score 0.5504. The main asymmetry therefore lies in recall: left-hand imagery is recovered more reliably than right-hand imagery, whereas right-hand precision is only slightly higher.

This imbalance is not catastrophic, but it reinforces the wider subject-wise instability visible in Figure 9 and Figure 10. A deployment-oriented subject-independent MI-EEG model would ideally be both stronger overall and more symmetric across the two motor states. The explicit class-wise profile therefore helps localise where further gains in cross-subject robustness are most likely to arise, particularly in right-hand recall.

4.5. Protocol Sensitivity Relative to the Earlier Pooled-Window Proof-of-Concept Study

The present paper keeps the underlying GAF–PLV feature construction and parallel-CNN decoding logic, but changes the inference target. In the earlier proof-of-concept benchmark, 0.4 s windows were generated before data partitioning, reduced cohorts of 10 and 30 subjects were used, and a 9:1 sample-level split with five-fold sample-level cross-validation was reported for the binary task. The present study instead evaluates 105 retained subjects under LOSOCV with one fully held-out subject per outer fold. Table 4 summarises that contrast.

These rows are not intended as a like-for-like leaderboard because cohort size, task scope, and split logic differ. They are, however, highly informative about protocol sensitivity. When the same GAF–PLV representation family is moved from pooled-window benchmarking to unseen-subject evaluation, the apparent performance landscape changes materially. Using the 99.73% proof-of-concept benchmark as the common anchor, the mean generalization gap to the retained-cohort LOSOCV result is 41.66 points. Crucially, that shift is not one fixed number across the cohort: it is 21.16 points for the best held-out subject (78.57%), 42.59 points for the median held-out subject (57.14%), and 61.63 points for the lowest-performing held-out subject (38.10%). Reporting those anchors alongside the cohort mean makes the protocol-sensitive behaviour of the framework much more interpretable.

4.6. Why Subject-Wise Evaluation Reveals the Generalization Gap

The drop from the earlier segment-level proof-of-concept benchmark to the present LOSOCV result should not be interpreted as evidence that the representation is useless. Rather, it indicates that the evaluation target has become much harder and much more realistic. Several plausible explanations can account for this gap. First, the network can learn subject-specific EEG signatures that help discrimination when pooled windows from the same subject recur across development and evaluation, but those signatures do not transfer reliably to a wholly unseen individual. Second, although the GAF and PLV descriptors are physiologically motivated, they are not automatically invariant to inter-subject differences in anatomy, rhythm expression, noise profile, and task execution style. Third, non-overlapping windows extracted from the same trial are correlated observations rather than independent experimental units, so permissive window-level splitting can overstate apparent generalizability even when the representation itself is informative.

These explanations are consistent with the wider literature on EEG partitioning and with the fold-by-fold heterogeneity seen in the present paper. The key point is therefore not simply that the percentage drops, but that the scientific meaning of the number changes. Figure 13 makes this visible in two ways: the cohort mean falls by 41.66 points, and the held-out subject gaps span 21.16–61.63 points. Under pooled-window evaluation, the result primarily reflects benchmark-oriented discrimination under a permissive partition. Under subject-wise LOSOCV, the remaining performance is a direct estimate of how much transferable structure the representation retains once the held-out individual is fully excluded from model fitting.

4.7. Protocol Trust and the Interpretation of Performance

A key contribution of this paper is methodological rather than architectural. Its originality lies in the evaluation logic and reporting transparency rather than in a new model block. The present LOSOCV results should therefore be read alongside the earlier pooled-window proof-of-concept benchmark and the broader literature showing that validation design can dominate the apparent ranking of MI-EEG models [8,10,11,12]. The gap between segment-level benchmarking and the present LOSOCV result is not just a change in points; it is a change in the meaning of the evaluation. Under segment-, image-, or time-resolved partitioning, a network may benefit from recurring subject-specific structure across folds. Under LOSOCV, the test subject is absent from model fitting and checkpoint selection. Any performance that remains must therefore come from structure that generalises across individuals rather than from subject recurrence.

For that reason, the present manuscript should be read as a study of generalization rather than as a simple leaderboard exercise. Its main contribution is to show how representational promise changes once the evaluation target becomes the unseen subject. The GAF–PLV representation remains physiologically motivated and interpretable: GAF captures nonlinear temporal correlation and PLV captures synchronisation structure between electrodes [23]. However, the LOSOCV results show that these properties alone do not ensure robust cross-subject transfer. The practical implication is straightforward: a representation that performs extremely well under pooled-window benchmarking can remain only moderately effective under unseen-subject testing, so segment-wise results should not be treated as direct evidence of plug-and-play subject-independent usability.

The present interpretation also aligns with Del Pup et al., who show that subject-based evaluation is more reliable when cross-subject inference is the goal [8]. In that context, the current study contributes a stronger subject-wise reference than segment-level benchmarking because it removes the most obvious subject-overlap pathway, uses a held-out subject in every outer fold, and reports all retained folds transparently.

4.8. Field Significance of the Present Benchmark

The importance of the present result lies not in claiming that 58% accuracy is competitively high in absolute terms, but in showing what a more defensible subject-wise estimate looks like when the full retained cohort is exposed to scrutiny. For future MI-EEG studies, a moderate but transparently derived subject-wise score can be more scientifically valuable than a near-ceiling pooled-window result whose inference target is ambiguous. In that sense, the present paper contributes a calibration point for later architectures, domain-adaptation strategies, compact models, and subject-invariant learning methods.

The benchmark also has practical significance for how claims are framed and reviewed. Segment-level studies can still justify representational promise, proof-of-concept screening, and ablation analysis, but they do not support the same translational claim as subject-wise evaluation. By separating those claim levels explicitly, the present paper reduces the risk that benchmark discrimination is mistaken for plug-and-play readiness and helps align algorithmic reporting with realistic expectations for assistive and neurorehabilitation-oriented BCI development.

4.9. Closest Same-Dataset Subject-Independent Comparisons

Same-dataset subject-independent comparisons are only meaningful when protocol matching is made explicit. Table 5 therefore reports not only headline performance, but also cohort size, validation style, and whether the paper visibly exposes the held-out fold profile. The goal is not a leaderboard; it is a fair description of what each reported number can and cannot support.

Two points follow from Table 5. First, the present accuracy is plainly below the strongest same-dataset subject-independent reports. That fact should not be hidden. Second, those higher numbers do not answer exactly the same methodological question. Some use transfer learning or stronger pretraining, some use subject-based grouping rather than strict LOSO, and several emphasise only cohort-level mean ± standard deviation instead of exposing the full held-out subject-wise profile.

The exploratory classical row is useful for a narrower reason. Altamirano’s CSP-based preprint reports a best LOSO average accuracy of 51.11% with SVM on a five-subject subset of the same PhysioNet resource. Because the study is both unpublished and cohort-mismatched, it cannot serve as a definitive benchmark. Even so, it suggests that the retained-cohort GAF–PLV result captures somewhat more cross-subject structure than a simple classical CSP+SVM baseline, while still leaving substantial headroom for robust subject-invariant decoding.

Recent same-dataset studies continue to show this heterogeneity rather than eliminating it. Some recent works remain based on restricted subject subsets or segmentation-based sample construction, whereas others move closer to explicit subject-independent or LOSO-style evaluation [27,28,29]. This newer literature therefore supports the same interpretive point made throughout the present paper: same-dataset accuracy should be read together with cohort definition, partitioning logic, and fold-level reporting transparency, not in isolation.

The present manuscript is therefore best read as a high-transparency subject-wise benchmark for this representation family. Its distinguishing strengths are explicit retained-cohort definition, complete held-out fold disclosure, and joint reporting of cohort-level and fold-level variability. Relative to the studies in Table 5, those properties make the present comparison set more interpretable even when the raw accuracy is lower. Higher numbers obtained under stronger architectures, partially different task formulations, or less explicit subject-wise reporting do not invalidate the present result; they show that unseen-subject MI-EEG performance depends jointly on model class, training regime, cohort definition, and protocol strength.

4.10. Transparent Cohort Reporting and the Absent Four Subjects

A second methodological issue is the difference between the official dataset size and the retained executable cohort. The official PhysioNet MI resource contains 109 volunteers [9]. However, the available study outputs on which the present paper is based contain fold-level results only for subjects 1–105. We therefore know exactly which four subjects are absent from the available output set: 106, 107, 108, and 109. What we do not know from the available files alone is a fully documented machine-readable reason for their absence. This distinction matters. It would be easy to repeat a familiar but unsupported phrase such as “four subjects were excluded due to data inconsistencies” without showing file-level evidence. We chose not to do that.

Instead, the paper adopts a stricter reporting stance. The scientific claim is tied to the 105 retained folds that are actually available in the study outputs, and the absent subjects are stated explicitly rather than hidden. This is more transparent than studies that use reduced cohorts without naming the retained subject IDs or change the effective cohort without showing the fold-level outputs. The same-dataset literature demonstrates how common this ambiguity has become [4,13,15,16]. In that context, exact fold disclosure is part of the core evidence base, not a minor administrative detail.

4.11. Computational Cost

The experiment was run on an NVIDIA GeForce RTX 4090. The average time per epoch was 215.03 s, the average number of trained epochs per fold was 15.70, the approximate runtime per fold was 0.94 h, and the approximate total runtime across 105 folds was 99.05 h. This computational burden helps explain why reduced-cohort benchmarking has persisted historically. More importantly, it shows that even a nearly 100-hour subject-wise rerun on a modern GPU yields only moderate cross-subject performance. That observation underscores why protocol stringency should be considered alongside raw % accuracy.

5. Study Scope and Next Steps

The present study has a clearly defined scope and several next-step priorities. First, the executable rerun package currently supports only the binary task, so the four-class LOSOCV extension should not be inferred from the reported results. Second, although LOSOCV removes the most obvious subject-overlap pathway, the implementation is not nested; nested subject-based selection would provide a stronger estimate when model selection and final testing must be strictly separated [8]. Third, each outer fold uses only a single validation subject, which is statistically fragile. Fourth, strong protocol-matched subject-wise baselines such as CSP+LDA, EEGNet, and other compact MI-EEG references are not yet included in the executable comparison. The exploratory CSP+SVM preprint cited in this paper is useful only as a small-subset external reference, not as a substitute for a matched baseline rerun. In addition, the absent subjects 106–109 are known from the supplied files, but the reason for their absence is not yet machine-readably documented.

These scope boundaries define the natural next steps: nested subject-wise validation, matched classical and compact-deep baselines, formal ablations, and methods aimed explicitly at subject-invariant learning. Domain adaptation, representation alignment, calibration, and confidence-aware decision rules are especially relevant directions. Accordingly, the present manuscript should be read as a transparent subject-wise benchmark for this representation family rather than as a final estimate of deployable subject-independent performance.

These scope boundaries also motivate a practical reporting standard for future MI-EEG studies. At a minimum, authors should:

• state the validation unit explicitly (window, trial, subject, or nested subject-based design);
• report benchmark-oriented discrimination claims and subject-independent transfer claims as separate evidence levels rather than presenting one as proof of the other;
• report the full subject-wise fold profile rather than only a cohort mean;
• disclose cohort inclusion and exclusion rules with exact subject identifiers whenever possible;
• include chance-corrected and/or class-balanced metrics alongside raw accuracy; and
• provide confidence intervals or equivalent uncertainty estimates for the main performance statistics.

None of these requirements is methodologically extravagant, but together they make it much harder to confuse benchmark discrimination with unseen-subject generalization and much easier to build cumulative, internationally interpretable evidence.

6. Conclusion

Deep learning results in MI-EEG depend not only on model design but also on what the evaluation protocol actually asks the model to do. This paper examined how the scientific meaning of performance changes when our recently proposed GAF–PLV parallel CNN is moved from segment-level benchmarking to subject-wise LOSOCV. On the 105 retained executable subjects, the model achieved 58.07% ± 8.27% mean accuracy and 53.48% ± 11.19% macro-F1, with substantial subject-to-subject variability.

Read together with the earlier 99.73% segment-level benchmark, these results reveal a clear generalization gap between permissive pooled-window discrimination and unseen-subject transfer. The mean gap is 41.66 points, while the best-, median-, and worst-held-out subject gaps are 21.16, 42.59, and 61.63 points, respectively. The implication is not that segment-wise studies are useless; they remain valuable for rapid representation screening and early method development. The implication is that they should not be interpreted as direct evidence of plug-and-play subject-independent MI-BCI performance.

By making the retained cohort explicit, reporting every available held-out fold, and quantifying uncertainty and heterogeneity rather than hiding them behind a single headline value, this study provides a stronger subject-wise reference for future work on this representation family. The next priorities are nested subject-wise validation, matched classical and compact-deep baselines, and methods designed explicitly for subject-invariant learning. More broadly, the paper’s contribution is to show how a strong proof-of-concept benchmark becomes scientifically more informative and interpretively clearer when examined under subject-wise evaluation. A model that reaches 99.73% under segment-level splits can still fail to provide robust performance for many new users. If MI-EEG research is serious about plug-and-play subject-independent BCI, within-subject discrimination and cross-subject generalization must be reported separately rather than conflated.