Investigating the Impact of Semi-Supervised Learning Methods to Improve the Quality of Diagnosis of Retinal Diseases from OCT Images

1. Introduction

Retinal diseases encompass a range of conditions that can significantly impair vision and even lead to blindness. Retinopathies, such as glaucoma, macular holes, diabetic macular edema (DME), and age-related macular degeneration (AMD), often result in complications like macular edema and retinal detachment. AMD, a complex retinal disorder, is one of the leading causes of irreversible vision loss in industrialized countries, particularly affecting individuals over the age of 60 [1,2]. AMD is clinically categorized into two main subtypes: dry (non-exudative) and wet (exudative). In the early stages of dry AMD, yellowish-white deposits known as drusen form between the retinal pigment epithelium (RPE) and Bruch's membrane (BM), marking the onset of the disease [3,4]. As these deposits accumulate, choroidal capillaries extend into the RPE and BM, leading to pathological changes characterized by abnormal choroidal neovascularization (CNV) beneath the retina. This progression can cause exudation, bleeding, scarring, and eventual loss of central vision [5,6,7]. The introduction of anti-VEGF therapies has significantly improved outcomes for wet AMD, helping to preserve visual acuity [1,8,9]. Early detection, however, remains crucial for preventing advanced retinal damage and reducing treatment costs.

Retinal disorders are commonly diagnosed using imaging techniques such as fundus photography, fluorescein angiography, and optical coherence tomography (OCT and OCTA) [5]. OCT, a noninvasive and noncontact modality, captures high-resolution, cross-sectional images of retinal layers through low-coherence light. Its high speed, resolution, and nondestructive imaging capabilities make OCT a valuable tool for diagnosing and monitoring retinal diseases [10,11]. OCT’s utility extends beyond ophthalmology; recent studies have highlighted its relevance in detecting neurological conditions like Alzheimer’s, multiple sclerosis, and Parkinson’s disease [12,13,14]. However, OCT interpretation poses challenges. As the population ages and AMD and other chronic retinal conditions increase, the demand for OCT evaluations is straining healthcare systems. Manual OCT analysis, especially for detecting subtle or early-stage lesions, presents significant challenges, as it can be both complex and prone to misinterpretation, particularly for conditions lacking distinct visual markers in the retina. Additionally, the process can be time-consuming, with notable delays in obtaining tests, reporting results, and associated costs, which may place a burden on patients and healthcare systems. Limited data availability for rare conditions further exacerbates the difficulty in achieving accurate diagnosis and effective treatment planning. These challenges highlight the limitations of relying solely on traditional manual OCT analysis to address the growing demands of retinal care. Figure 1 provides an illustration of OCT B-scans for AMD cases

Artificial Intelligence (AI) encompasses technologies that simulate human cognitive functions, enabling machines to perform tasks requiring human-like understanding and pattern recognition Within AI, Machine Learning (ML) focuses on developing algorithms that learn from data, allowing systems to make predictions or decisions without explicit programming [15,16,17].

AI has made significant strides in multi-disease classification for healthcare, particularly in medical image analysis [18,19]. In ophthalmology, DL models have shown remarkable accuracy in detecting AMD-related lesions using OCT images, with notable studies advancing this application [6,10,20]. For example, He et al. proposed a two-stage DL model combining ResNet-50 with a local outlier factor algorithm to detect AMD from OCT volume scans, achieving high accuracy and AUC [21]. Alenezi et al. utilized a weighted fusion approach to combine outputs from models like ResNet and EfficientNet with attention mechanisms, achieving 91.88% accuracy in AMD classification [22]. SL models have also demonstrated efficacy across various tasks in OCT image analysis, including automated classification and segmentation of lesions [23,24]. Wang et al. used DenseNet, ResNet, and other architectures to classify OCT images into AMD, DME, and normal classes [25]. Meanwhile, Zapata et al. deployed a 24-layer CNN to differentiate between AMD and glaucomatous optic neuropathy using fundus and OCT images [26]. Other models, such as Dong et al.'s joint CNN detector using Yolov3, identified AMD among 11 retinal conditions with 88% sensitivity and 98% specificity [27]. In another study, Mathews et al. used a lightweight CNN to classify OCT images with 100% sensitivity and specificity across AMD, DME, and normal categories [28]. These advancements underscore the growing role of SL in enhancing OCT analysis, contributing to accurate and efficient diagnostics in retinal healthcare.

A major challenge in deploying SL models for disease detection lies in the limited availability of labeled data, especially for rare or specific conditions. SL algorithms require large labeled datasets to achieve robust performance. The process of annotating and diagnosing data is resource-intensive and time-consuming for medical professionals. While routine screenings can generate raw data, labeling these datasets remains a barrier. Semi-supervised learning (SSL) approaches offer a promising solution by leveraging large amounts of unlabeled data alongside smaller labeled datasets to build effective classifiers. This makes developing automated systems capable of assessing and distinguishing diseases with minimal labeled data essential for advancing medical diagnostics.

Recent studies highlight the challenge of limited labeled data in medical imaging, particularly for rare or early-stage eye diseases, and emphasize the potential of SSL in addressing this issue. SSL has been widely explored for detecting retinopathies in OCT and fundus images [29,30]. Dali Chen et al. proposed an SSL approach combining U-Net with an iterative dataset updating strategy, improving blood vessel segmentation in retinal images [31]. Amir Rahdar et al. leveraged unlabeled data for retinoblastoma tumor segmentation, using a Gaussian mixture model (GMM) to detect abnormalities in fundus images [32]. Additionally, Andres Diaz-Pinto et al. improved glaucoma prediction accuracy by integrating a Deep Convolutional Generative Adversarial Network (DCGAN) with an SSL-based retinal image synthesizer, achieving an AUC of 0.9017 on 86,926 retinal images [33]. Xi Wang et al. employed a Student-Teacher architecture for diabetic macular edema (DME) detection in OCT images, utilizing confidence-based pseudo-labeling and consistency regularization to address label noise [34]. Further, Sixu Duan et al. proposed a semi-supervised graph-attentional convolutional neural network (GACNN), combining CNN and graph convolutional networks (GCN) with attention mechanisms to leverage both labeled and unlabeled fundus images [35]. Zhicong Tan et al. implemented a transfer learning method with sub-domain adaptation (TLSDA), achieving impressive classification accuracies of 93.63% and 96.59% on private datasets, outperforming conventional supervised approaches. These studies collectively demonstrate the efficacy of SSL in improving retinal disease classification using limited labeled data [36].

The primary aim of this study is to investigate methodologies for evaluating the diagnosis of diseases such as neurological disorders, Alzheimer's disease, or eye diseases like Retinitis Pigmentosa (RP), for which very limited datasets are available, in order to propose an optimal performance of DL models, particularly SSL models. To achieve this, various experiments are conducted based on use of small dataset of OCT images to identify AMD diseases, including cases where the dataset is moderately reduced, slightly reduced, severely limited, and minimally available. This investigation is conduced across four categories: 70%, 50%, 20%, and 5% of the train dataset volume, to introduce the best SSL method based on the results. In this study, DL models are evaluated on 16822 OCT scans of Noor Eye Hospital (NEH) to reach optimized model for discriminating normal, dry and wet AMD.

2. Material and Methods

2.1. Database

This study utilized the publicly available Noor dataset, which consists of 16,822 retinal OCT images collected from 441 patients, comprising a total of 554 OCT volumes. The dataset covers various stages of age-related macular degeneration (AMD). Of the 16,822 B-scans, 8,584 images are normal, 4,998 depict drusen, and 3,240 represent cases of choroidal neovascularization (CNV). At the volume level, there are 187 normal cases, 194 drusen cases, and 173 CNV cases out of the 554 total volumes. The OCT images were captured using the Heidelberg SD-OCT imaging system at Noor Eye Hospital and annotated by retinal specialists. The dataset includes patients aged 50 years or older with no other retinal pathologies and with good image quality (Q ≥ 20). Some patients had both eyes scanned, with each OCT volume containing an average of 30 B-scans.

2.2. Data Preprocessing and Augmentation

During the preprocessing stage, all images are resized to a consistent dimension to ensure uniform input for the model, facilitating efficient training. Additionally, normalization is applied to all images to standardize pixel intensity values, ensuring a zero mean and unit variance. This normalization step helps accelerate the convergence of the model during training by ensuring the inputs are on a comparable scale. To further enhance the diversity of the dataset and improve the model's generalization ability, several data augmentation techniques are employed. These include random cropping to simulate different field-of-view variations, horizontal flipping to increase variability in orientation, and rotation within a controlled range. Additionally, affine transformations such as translation and shear are applied to simulate spatial variations, further enriching the training set. These augmentations help improve the model's robustness by exposing it to a wider range of potential input variations, thereby reducing the risk of overfitting and enhancing its performance on unseen data.

2.3. Proposed Model

The aim of the first step of this study is to achieve best performance to distinguish mentioned dataset OCT images into normal, drusen and CNV categories using SL models. Among the SL models, EfficientNet B0, part of the EfficientNet family of deep learning models, has employed as supervised learning model to classify different stages of AMD due to its design and architecture benefits. EfficientNet B0 is specifically optimized to achieve a good balance between model accuracy and computational efficiency. This means it can deliver competitive performance on tasks while requiring less computational resources compared to larger models. The architecture of EfficientNet is designed to scale efficiently to different sizes (B0 to B7), allowing flexibility in choosing a model size based on the available compute resources and the specific requirements of the task. Also, due to pre-training on large datasets like ImageNet, it can be effectively used for transfer learning. This allows for faster training convergence and better performance on new tasks with smaller datasets. Moreover, it has demonstrated good generalization capabilities, meaning it can perform well across a variety of domains and datasets, making it versatile for different computer vision tasks [37,38]. Furthermore, based on several experiments and investigating the other models including ResNet, EfficientNet B0 and Attention, it has been concluded that the model exhibits superior performance in AMD classification on NEH dataset [22].

In the next stage, only defined portion of the train dataset volume is considered as labeled data with the remaining unlabeled data. The SL model is trained on this labeled dataset and classified OCT images. The accuracy percentage of disease detection in each image by the model in this stage is crucial. For training the SSL model, the Teacher/Student method has been utilized. The Teacher-Student (TS) approach is a widely used strategy in semi-supervised learning, aiming to enhance model performance by utilizing both labeled and unlabeled data. This method involves two distinct neural networks: the teacher and the student. Typically, the teacher network is a pre-trained model that generates pseudo-labels for the unlabeled data. The student network, which is undergoing training, learns from both the valid labeled data and the pseudo-labels produced by the teacher [39]. Initially, the teacher network is trained on the labeled dataset. Once trained, it predicts labels for the unlabeled data, and the confidence in these predictions helps create pseudo-labels. The confidence parameter essentially controls how much trust is placed in the pseudo-labels generated by the teacher model for the unlabeled data. Each prediction comes with an associated confidence score, which reflects how certain the teacher model is about the predicted label. The confidence parameter sets a threshold for these confidence scores. Only the pseudo-labels that exceed this threshold are used for training the student model. This helps ensure that only high-confidence (more reliable) pseudo-labels are incorporated into the student model’s training process. The primary purpose of the confidence parameter is to balance the trade-off between the quantity and quality of the pseudo-labels used for training. By carefully adjusting this parameter, the semi-supervised learning process aims to improve the student model’s accomplishment by leveraging the additional data while minimizing the risk of learning from incorrect or uncertain labels.

The student network is subsequently trained using both the labeled and pseudo-labeled data, with the goal of improving classification performance. This technique capitalizes on the availability of unlabeled data, which is often easier and more cost-effective to obtain than labeled data. Utilizing pseudo-labels allows the student network to potentially generalize better to new, unseen data. The success of the student network is significantly dependent on the quality of the pseudo-labels provided by the teacher network, as low-quality pseudo-labels can hinder training. Therefore, it is essential to maintain a balance in the performance between the teacher and student networks to prevent issues such as overfitting.

In this study, an Iterative Teacher-Student (ITS) approach is employed to enhance the training of deep learning models for OCT image analysis. The process begins with training a teacher model on the labeled portion of the dataset. The teacher model then generates pseudo-labels for the unlabeled OCT images, which are used to expand the training set. To ensure the reliability of the pseudo-labels, a confidence threshold is applied: only pseudo-labels with a confidence score above a certain threshold are added to the labeled dataset for the next iteration. This threshold helps ensure that only high-confidence predictions are incorporated into the training data, improving the quality of the pseudo-labels and preventing the model from learning from uncertain or incorrect labels. The student model is trained on both the original labeled data and the pseudo-labeled data from the teacher, refining its parameters through multiple iterations. This iterative process allows the model to progressively improve by leveraging unlabeled data, while the confidence threshold maintains control over the quality of the pseudo-labels used in each iteration, enhancing the model’s generalization capabilities and performance on the AMD classification task. Figure 2 provides a visual representation of the ITS entire process.

3. Experiments

To enhance computational efficiency, manage memory usage, and facilitate effective preprocessing and data augmentation in deep learning networks, all OCT images are resized to a uniform size of 224x224 pixels. Various batch sizes (16, 32, and 64) are tested to optimize computational resources and ensure the model generalizes well to new data. The training process typically spans 100 epochs to refine model parameters and avoid overfitting, with strategies such as early stopping, model checkpointing, and learning rate reduction (with a patience of 15 epochs) employed to preserve the best model. Data augmentation is applied to improve model performance, reduce overfitting, and enhance generalization. Additionally, the models are initialized with pre-trained weights from large image datasets, enabling faster convergence and transfer learning for AMD classification. The Adam optimizer, known for its adaptive learning rate capabilities, is used alongside categorical cross-entropy loss for multi-class classification tasks.

For model training, the dataset is divided into training, validation, and test sets, consisting of 10,811 training images, 2,839 validation images, and 3,235 test images from a total of 16,822 OCT images. The test set remains constant across all experiments to ensure consistent evaluation, while training data varies based on the strategies investigated and dataset sizes. To prevent information leakage, data is partitioned patient-wise, ensuring no overlap between training and test sets. If class imbalance exists, appropriate techniques (e.g., oversampling or class weighting) are applied to address it. Data augmentation is employed on the training set to enhance model generalization.

In the second phase of the evaluation, the model's performance is assessed using progressively smaller subsets of the dataset. Specifically, subsets containing 70%, 50%, and 20% of the original training data are created, with the remaining data excluded from the training process. The SL model, configured with parameters (obtained in previous phase), is then trained and tested on these subsets. During this phase, various parameter configurations are explored to identify the optimal settings based on performance metrics. The impact of reducing the training dataset size on model performance is analyzed, and the results are compared to those from the later SSL phase to evaluate the added value of the semi-supervised approach.

In the SSL phase, a confidence threshold for labeling images is defined at 0%, 90%, 95%, and 99% for each unlabeled image. The quantity and quality of the pseudo-labeled data significantly influence model performance. In contrast to the second phase, the model is presented with a mix of labeled and unlabeled data, where labeled images account for 70%, 50%, 20%, and 5% of the total training data, while the rest are treated as unlabeled. As the model makes predictions on the unlabeled images, those with a confidence level exceeding the specified threshold are added to the labeled set as pseudo-labeled images. The model is then retrained with this expanded dataset to evaluate the effectiveness of the SSL approach in improving accuracy as more data becomes available.

4. Results

4.1. SL Training Phase

Initially, the complete dataset was used to train the SL model, optimizing key parameters such as batch size, data augmentation, and the inclusion of pre-trained weights. The experiments showed no significant performance differences across batch sizes, although smaller sizes increased training time. Based on these results, a batch size of 64 was selected as it provided a good balance between performance and training efficiency. Transfer learning substantially improved classification accuracy compared to models trained from scratch, highlighting its role in leveraging pre-trained feature representations. Moreover, fine-tuning pre-trained layers yielded a 7–10% increase in detection accuracy over freezing them, underscoring the importance of adaptable feature extraction for disease classification. Table 1 summarizes the SL model’s performance under different parameter settings.

4.2. Limited Data in SL Models

To evaluate the impact of data scarcity, SL models were trained on progressively smaller subsets of the training dataset: 100%, 70%, 50%, 20%, and 5%. A consistent decline in performance was observed as the data volume decreased, with accuracy dropping sharply from 88.44% at 70% of the dataset to 77.05% at 20%, and further reductions seen with only 5% of the data. Similar trends were evident across precision, recall, and F1 score, underscoring the sensitivity of SL models to training data size. These results are visualized in Figure 3, where each metric is represented by a distinct color, highlighting the critical importance of data availability in maintaining robust model performance.

Figure 3. Performance of SL models (accuracy, precision, recall, and F1 score) under varying training dataset sizes (100%, 70%, 50%, 20%, and 5%). Each metric is represented by a unique color, illustrating the decline in performance with reduced data availability.

Figure 3. Performance of SL models (accuracy, precision, recall, and F1 score) under varying training dataset sizes (100%, 70%, 50%, 20%, and 5%). Each metric is represented by a unique color, illustrating the decline in performance with reduced data availability.

4.3. Performance of Teacher-Student (TS) Models

The TS approach was evaluated using datasets with varying proportions of labeled data (70%, 50%, 20%, and 5%) alongside unlabeled data. Pseudo-labeling was employed to generate additional labeled examples, with different confidence thresholds determining the inclusion criteria for pseudo-labeled images.

Based on the results from all the experiments, it is observed that the best outcomes were achieved with a 95% confidence criterion when the dataset size was reduced to 20%. The accuracy at 70% was 88.71, and at 50% it was 88.32. Besides, defining confidence threshold 95% significantly enhances the model’s achievement across all metrics: accuracy, precision, recall, and F1 score. This improvement indicates that the integration of a substantial volume of pseudo-labeled data, even with lower confidence, positively impacts model capabilities. Performance at the 20% dataset size across various metrics is shown in Table 2.

With a significant reduction in dataset size to 20% and 5%, the model’s diagnostic competency obviously diminished. However, by increasing the strictness of the confidence criterion, particularly at the 5% dataset size, better results were achieved. At this reduced size, with a confidence level of 95%, the accuracy was 49.34%, and with a 99% confidence level, the accuracy improved to 55.88%.

4.4. Performance of Iterative Teacher-Student (ITS) Models

The proposed ITS model, considering the improvement of the TS model’s results by adding pseudo-labeled data, aims to refine and enhance the model’s detection quality in performance metrics. Applying this method can improve the severe decrease in accuracy at lower volumes, especially in specific diseases with limited labeled data. Based on the results obtained from the TS model, the best model performances at 20% and 5% volumes have been included in the iteration process. Refer to figure 3 for the results of the iteration process at 20% and 5% of train dataset.

Based on the experimental results, the model's efficiency in the iterative process positively affected its accuracy and precision. Compared to the TS model results, the diagnostic accuracy of the model at a 20% dataset volume shows a 2.5% increase in the first iteration. Additionally, at a 5% dataset size, an upward trend is observed in the iterative process, particularly in the second iteration, where the model’s accuracy increased by 10%, reaching 64.15%.

Figure 3. ITS model results at 20% and 5% of dataset volume in 4 iterations.

Figure 3. ITS model results at 20% and 5% of dataset volume in 4 iterations.

The iterative process demonstrates that while the addition of high-confidence pseudo-labels initially improves model accuracy and reduces test loss, excessive pseudo-labeling can lead to diminishing returns and potential overfitting. The model performs best with a balanced amount of pseudo-labeled data, as seen in the first and second iteration, and shows decreased performance with further iterations.

4.5. Confusion Matrix

To provide a detailed assessment of the classification performance of each model, the four confusion matrices (CMs) corresponding to SL and our proposed ITS Model are presented. Each confusion matrix is a 3 × 3 table that displays the distribution of true labels versus predicted labels for the three AMD classes: normal, drusen, and CNV across all datasets (see Figure 4).

By comparing the CMs in the SL and ITS models, it can be observed that at a 20% volume, the misclassification rate across different AMD classes has significantly decreased. At a 5% of dataset volume, despite diagnostic errors in drusen, the ITS model has shown good performance in classifying normal and CNV classes, indicating that the proposed model has performed very well in distinguishing AMD cases.

4.6. ROC Curve Analysis

To thoroughly assess the performance of the optimized supervised learning (SL) model and the proposed iterative teacher-student (ITS) model in detecting and distinguishing AMD cases at reduced dataset volumes (20% and 5%), Figure 5 presents the receiver operating characteristic (ROC) curves. These include ROC curves for the AMD classes (normal, drusen, and CNV), along with the micro-average and macro-average ROC curves.

The micro-average curve combines the true positives and false positives rates across all classes, and then calculates performance metrics based on these combined totals. This provides a comprehensive assessment of the model’s overall performance and gives an insight into how both the SL and ITS models perform across all classes. In contrast, the macro-average curve computes the performance metrics like ROC curve for each AMD class individually, and then averages these values, giving equal weight to each class. By comparing these curves across both SL and ITS settings, we can evaluate the effectiveness of the proposed method in differentiating between AMD cases.

5. Conclusion

The study demonstrates the critical role of parameter selection, transfer learning, and data augmentation in achieving optimal performance in supervised learning (SL) models for OCT images classification. Experiments revealed that batch size had no significant impact on performance but influenced training time, with smaller batch sizes increasing computational costs. Incorporating transfer learning significantly enhanced classification accuracy, particularly when pre-trained layers were fine-tuned. Models trained with frozen pre-trained layers exhibited a 7–10% decrease in accuracy, emphasizing the importance of enabling further learning within these layers. The SL model achieved its best performance with a batch size of 64, transfer learning, unfrozen pre-trainable layers, and effective data augmentation.

As the training dataset size decreased from 70% to 20% of the total data, a consistent decline in performance metrics, including accuracy, precision, recall, and F1 score, was observed. Accuracy dropped from 88.44% with 70% of the training data to 77.05% with 20%, highlighting the sensitivity of model performance to dataset size. Despite this, the robustness of the chosen parameter settings across varying data volumes was evident. However, the results underscore the need for larger training datasets to ensure optimal model performance, particularly in scenarios where high accuracy is critical. Decision-makers must carefully weigh the trade-offs between computational efficiency and model performance when determining dataset sizes.

The findings also underscore the importance of pre-trainable layers in SSL models. The inclusion of these layers led to significant improvements across all metrics, as they enhanced the model’s ability to generalize and make accurate predictions. For SSL models, unfrozen pre-trainable layers contributed to a 14.05% improvement in accuracy compared to frozen layers, a greater benefit than the 5.40% gain observed in SL models. This suggests that SSL methods are better equipped to leverage additional fine-tuning, making them more adaptable to exploiting learnable parameters and improving overall performance.

In TS model, when the dataset is reduced, the 95% confidence level can be effective. However, when the dataset size is drastically reduced, increasing the strictness of the confidence threshold yields better results.

Pseudo-labeling in SSL was shown to be an effective strategy for leveraging unlabeled data, with accuracy improving from 77% in the teacher model to 82.15% in the student model using pseudo-labeled data. The iterative process revealed that a balanced approach to adding high-confidence pseudo-labels was essential for maintaining model performance while minimizing test loss. Excessive pseudo-labeling led to diminishing returns and potential overfitting, emphasizing the need for careful management of pseudo-labeling strategies. Despite these challenges, the results demonstrated that pseudo-labeling, even with lower-confidence data, positively impacted model performance by providing valuable supplementary information.

Overall, this study highlights the distinct benefits of SSL over SL, particularly in scenarios where labeled data is limited. While both SL and SSL benefit from unfrozen pre-trainable layers, SSL shows a more pronounced improvement, suggesting its superior adaptability to fine-tuning. The findings emphasize the importance of careful parameter tuning, dataset size considerations, and pseudo-labeling strategies in optimizing model performance. These insights can guide future research and practical applications, paving the way for more effective machine learning systems in disease classification and beyond.