Fast Contactless Detection of Post-COVID-19 from Eye Microvascular Images Using Deep Convolutional Neural Networks

1. Introduction

Artificial intelligence (AI) has rapidly evolved into a central tool in modern medicine, transforming imaging, diagnosis, pathology, surgery, treatment planning, patient monitoring, treatment personalization, and healthcare efficiency. From a broader healthcare perspective, AI systems are expected to demonstrate improved resource allocation reducing costs [1].

Machine learning (ML) can be considered a subfield of artificial intelligence (AI) and deep learning (DL) a subfield of ML that uses artificial neural networks with multiple layers. In diagnostic medical imaging, DL with its exceptional ability to process complex visual data, reduces diagnostic errors [2]. Meta-analysis has demonstrated that AI can match clinicians in medical imaging accuracy in the fields of ophthalmology, respiratory imaging and breast imaging [3]. Convolutional Neural Networks (CNNs), a subtype of artificial neural networks, can classify skin cancers with a level of accuracy comparable to board-certified dermatologists [4].

The seminal paper that introduced the modern concept of a CNN and its application to image recognition was published in 1998 [5]. The next big step was AlexNet, a significantly deeper and larger network than any previous CNN, which won the 2012 ImageNet Large Scale Visual Recognition Challenge (ILSVRC-2012) by an impressive margin, improving the error rate from 26.2% (the previous best) down to 15.3% [6]. Two years later, researchers at Google presented GoogLeNet (later renamed Inception-v1), a CNN with a core innovation the “inception module”, which executed multiple different convolutions and pooling operations in parallel [7]. GoogLeNet submission to ILSVRC-2014 used 12 times less parameters than AlexNet while being significantly more accurate [7]. One year later, at ILSVRC-2015, He et al. [8] introduced residual networks that are easier to optimize and gain accuracy from increased depth. Afterward, the combination of the inception architecture with the residual connections was studied and the hybrid versions “Inception-ResNet-v1” and “Inception-ResNet-v2” were presented [9]. In 2019, Tan & Le [10] proposed a new scaling method that balances network depth, width, and resolution and introduced a new family of models called “EfficientNets” which were significantly more efficient than previous CNNs.

For a human level performance in visual recognition tasks, apart from the CNN architecture, access to a sufficiently large training dataset, is of crucial importance. The rise of multi-million-item sets has improved dramatically the AI performance in classification of both objects (object recognition) and scenes (scene recognition). The scene or place in which an object appears may be equally or more important than the object itself, depending on the application. Scenes such as a field, park, runway, and a lobby are composed of parts and like objects have specific attributes [11].

In 2009, Deng et al. [12] introduced “ImageNet”, a large-scale database of images, consisting of 12 subtrees containing 3.2 million images, to be used as a training resource and a benchmark dataset for object recognition. In 2014, Zhou et al. [13] introduced the “Places” scene-centric database, a collection of over 7 million scene photographs. A few years later, they published [11] an improved version of “Places” with 10 million scene photographs, labeled with 434 scene semantic categories, comprising about 98 percent of scene types a human can encounter in the world.

The emergence of large datasets containing millions of items has allowed machine learning algorithms to achieve classification performance on visual object and scene recognition tasks that approaches human-level accuracy. The “transfer learning” process in deep learning, refines (fine-tunes) feature extraction of a pretrained network to be more specific to the current application, and has the advantage of increased flexibility with less effort. All CNNs used in this work were fine-tuned networks which had been pretrained on subsets of the aforementioned databases (ImageNet or Places).

The purpose of this work was to examine if fine-tuned CNNs can discriminate efficiently healthy individuals from post-COVID-19 patients, using conjunctival images acquired by high magnification conjunctival video capillaroscopy. The results demonstrated, for the first time, that a virus affects the microvascular topology in ways that only the sophisticated nature of a deep CNN can discriminate. On surplus, there is now evidence that AI could provide a painless, risk-free, contactless, fast, and accurate post-COVID-19 detection method that does not depend on the optical clarity of the eye.

2. Materials and Methods

2.1. Study Design

This machine learning, case-control study was based on images acquired at the University Hospital of Larissa from April 2021 to February 2022 with a protocol approved by the University Hospital of Larissa Scientific Committee (11155/2021). The research was performed according to the Declaration of Helsinki and informed consent was obtained from all participants in the study.

2.2. Subjects

At the temporal side of the bulbar conjunctiva of the right eye, microvascular images were recorded from 12 normal men volunteers (named “Control Group”) and 12 COVID-19 men survivors shortly after exiting the hospital (named “COVID-19 Group”). Image recordings were performed in a temperature controlled environment (21 to 23 °C) after waiting 30 minutes for adaptation.

Inclusion criteria for the Control Group were no systemic or ocular disease, no alcohol or smoking habit. Control subjects were not under any medication and had never been tested positive for SARS-CoV-2.

Inclusion criteria for the COVID-19 Group were hospitalization for COVID-19, oxygen therapy, Acute Respiratory Distress Syndrome (ARDS) with HI (Horowitz Index) less than 300 mmHg and a 28-day maximum time interval between hospital exit and image recording (duration from discharge). Exclusion criterion for the COVID-19 Group was rheumatic disease.

Clinical data for both groups, including body height and body weight, diastolic and systolic arterial pressure (DP and SP), and cardiac beats per minute (BPM), were measured on site before and after image recording. Body surface area (BSA, Du Bois formula) and body mass index (BMI) were calculated from the body height and weight. Mean arterial pressure (MP) was calculated from DP and SP (MP = DP + (SP-DP)/3). Subjects with diastolic blood pressure greater than 95 mmHg were excluded from the study (both groups) as hypertensive.

2.3. Experimental Set Up

The image acquisition experimental setup consisted of a slit lamp (Nikon FS-3V) connected with a high-speed CCD camera (12 bit, PCO Computer Optics GmbH, Germany) and a PC (Pentium 4, 3 GHz) [14]. The conjunctival video capillaroscopy system produced digital images of 320 x 240 pixels at a frame rate of 96 frames per second (fps) with an enhanced maximum magnification of 242x and a digital resolution of 1.257 ± 0.004 μm/pixel.

Image processing and deep learning in-silico AI experiments were performed on a desktop with an Intel CPU (i9-9900K) running at 3.6 (1.9 – 4.7) GHz, with 48 GB of RAM running at 2.13 GHz, and a 250 GB SSD. The image processing and deep learning AI software was MATLAB R2024a.

2.4. Deep Learning Training by Transfer Learning

A variety of pretrained convolutional neural networks (CNNs) were initially tested and 4 of them were selected for fine-tuning by transfer learning on our imaging data. Three CNNs, namely GoogLeNet, InceptionResNet-v2, and EfficientNet-b0, were pretrained on the ImageNet-1K dataset, a subset comprising roughly 1.2 million images from the full ImageNet database [12], enabling classification across 1,000 distinct object categories. One CNN, namely GoogLeNet-Places365 was pretrained on the Places365 dataset [11], which includes approximately 1.8 million images from the Places database, allowing it to classify images into 365 different scene categories. The output layers of the pretrained networks were modified to match the new binary classification task of 2 output classes (CONTROLS and COVID-19).

2.4.1. Architectural Hyperparameters and Size

Some of the architectural hyperparameters of the 4 pretrained deep neural networks (CNNs) used in this work, such as number of layers (NOL), depth (D), input image size (IIS), and input image channels (IIC) are shown in Table 1. GoogLeNet and GoogLeNet-Places365 are effectively the same network pretrained on different datasets, so, architectural hyperparameters are exactly the same.

Architectural hyperparameters are quantifiable characteristics of the network’s design, and correlate directly with how many learnable parameters (model parameters) the network will have. The amount of model parameters (size) of the pretrained deep neural networks (CNNs) used in this work, are shown in Table 2. It is clear that EfficientNet-b0 is lighter in comparison to the other networks, requiring only 20 MB of disk drive memory.

2.4.2. Importing and Splitting Image Data

30 to 52 images were selected for each subject from image recordings of various microvessel fields of the same side (temporal side) of the bulbar conjunctiva. Fields were selected with as many microvessels as possible because the emphasis was in the microvessel topology. Images were examined to rule out repetitions of the same microvascular field but large translocations of the same field, if appeared, were included in order to incorporate as much information as possible.

Images were organized by class into two separated folders: one for the Control Group (CONTROLS) and one for the COVID-19 Group (COVID-19). There was no image pre-processing. A MATLAB image datastore was created for labelling images. From each group datastore, ninety percent (90%) was used for training and ten percent (10%) for testing, after randomly shuffling the datastore images. An augmented image datastore was created to convert the size and channels of the training and testing images to the input image size and channels of the corresponding pretrained CNN network (see Table 1).

2.4.3. Evaluation Criteria

Two evaluation criteria were used to assess the speed of each convolutional network. Training time (TT) was defined as the amount of time required to train the pretrained convolutional neural network and is measured in minutes. Decision time (DT) was defined as the amount of time required for the trained network to decide the category of a single input image, and is measured in seconds.

The following evaluation criteria were applied for assessing classification performance. Sensitivity (Se) or true positive rate or recall is defined in percent:

$$Se={\displaystyle \frac{TP}{TP+FN}}100(\%)$$

(1)

where TP and FN is the number of true positive and false negative outcomes, respectively.

Precision (Pr) or positive predictive value is defined in percent:

$$Pr={\displaystyle \frac{TP}{TP+FP}}100(\%)$$

(2)

where FP is the number of false positive outcomes.

F1 score or Dice similarity coefficient is defined as the harmonic mean of Se and Pr in percent:

$$F1={\displaystyle \frac{2TP}{2TP+FP+FN}}100(\%)$$

(3)

Specificity (Sp) or true negative rate is defined in percent:

$$Sp={\displaystyle \frac{TN}{TN+FP}}100(\%)$$

(4)

where TN is the number of true negative outcomes.

Accuracy (Ac) is defined in percent:

$$Ac={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}100(\%)$$

(5)

2.4.4. Fine-Tuning with Training Hyperparameters

Training hyperparameters used for fine-tuning in each in silico experiment were learning rate (LR), mini batch size (MBS), and maximum number of epochs (MEP). Initial LR, MBS, and MEP values were 0.01, 128, and 20, respectively. Initial LR and MBS were gradually reduced, and initial MEPs were gradually increased in random combinations. The performance of each combination for each pretrained network was assessed by the evaluation criteria presented in the previous Section 2.4.3. When a good combination of training hyperparameters was spotted, at least one experiment with validation data was performed to identify the onset of overfitting (optimum MEP). In these cases, the image data split was eighty-one percent (81%) for training, nine percent (9%) for validation, and ten percent (10%) for testing.

2.5. Statistical Analysis

Microsoft Excel 2016 (Professional edition) and SOFA (version 1.4, Paton-Simpson & Associates Ltd) were used for statistical analyses. To account for all variable distributions, normal or not, differences between groups were examined using the Mann-Whitney U test. The significance level was set at 0.05. After identifying the optimum maximum number of epochs (MEP), 10 successive in silico experiments were performed with the randomized image data split reported in Section 2.4.2 (90% for training and 10% for testing). In this manner, the average performance of each fine-tuned CNN was evaluated across 10 data splits using the criteria outlined in Section 2.4.3.

The overall discriminatory power of the fine-tuned CNN with the best average performance was evaluated by the area under the curve (AUC) of the receiver operating characteristic (ROC) curve. An AUC value of 1.0 represents perfect discriminatory power, while a value of 0.5 represents random guessing. For each data split, the ROC curve was generated by plotting the true positive rate (sensitivity) against the false positive rate (1 - specificity) across the full range of classification thresholds. This was repeated for all 10 train-test data splits to account for variability due to randomized data splits, and the mean AUC with a 95% confidence interval (CI) was calculated using the t-distribution.

For evaluating the performance at the Youden-optimal point, the Youden index (J) was estimated for every point on the ROC curve:

$$J=Se+Sp-100(\%)$$

(6)

For each split independently, the optimal classification threshold was determined by maximizing J, which is equivalent to finding the ROC point furthest from the diagonal line of chance. At this per-split Youden optimal threshold, the model was evaluated by the criteria of Section 2.4.3. Averages and standard deviations of these criteria were estimated across repeated experiments.

3. Results

3.1. Group Characteristics

A summary of the demographic and clinical statistical data for the 2 groups is presented in Table 3. There was no statistically significant difference in age, body mass index (BMI), body surface area (BSA), cardiac beats per minute (BPM), arterial mean pressure (MP), and number of images per subject (NI) between the 2 groups. In each group, the changes in BPM and MP before and after the recording procedure were not significant.

A summary of the clinical and pharmacological profile of the COVID-19 Group during and after hospitalization is presented in Table 4. All patients (COVID-19 Group) had pneumonia with different lung distribution, and needed oxygen therapy with High Flow Nasal Cannula (HFNC). During hospitalization, the average Horowitz Index (HI) of the COVID-19 Group was 134 mmHg (Table 4), which implies moderate ARDS. During hospitalization, 11 patients (91%) received remdesivir and 6 patients (50%) received tocilizumab due to rapid clinical deterioration and high levels of C-reactive protein and ferritin (Table 4). In addition, 11 patients (91%) received LMWH (Low Molecular Weight Heparin) in antithrombotic prophylaxis dosage. Two of them received a full dose of enoxaparin since they developed atrial fibrillation and a high CHADS2 (Congestive heart failure, Hypertension, Age ≥75 years, Diabetes mellitus, Stroke [double weight]) score. One of them was also on antiplatelet therapy with ASA (AcetylSalicylic Acid) because of coronary artery disease (Table 4).

After hospital discharge, all patients had extensive radiological findings with deteriorated pulmonary function due to previous ARDS. Six patients (50%) needed oxygen therapy at home for at least one month (Table 2). All patients were on antithrombotic prophylaxis until fully mobilized (LMWH is recommended up to 40 days in severely and critically ill patients after discharge). Those 2 patients with atrial fibrillation who were receiving a therapeutic dose of LMWH continued their antithrombotic therapy after discharge with a full dose of rivaroxaban (Table 4).

3.2. Fine-Tuning and Performance Comparison

Different training combinations of the learning rate (LR), mini batch size (MBS), and maximum number of epochs (MEP) were tested for each of the 4 CNNs: GoogLeNet, GoogLeNet-Palces365, InceptionResNet-v2, and EfficientNet-b0. To take into account the effect of the data split, many different in silico experiments were made with randomized train-test splits for each training hyperparameter combination. The training hyperparameter combinations LR, MBS, and MEP that gave the best average performance on testing data are shown in Table 5 for each of the 4 pretrained CNNs.

In Table 6, the best average performance of each fine-tuned pretrained CNN on testing data is presented, after using the combinations shown in Table 5. The best testing accuracy of 92% (± 3%) was achieved by the GoogLeNet-Places365 network, followed by GoogLeNet and InceptionResNet-v2 with 88% (± 5%). It is also noted, that GoogLeNet-Places365 was the network with the shortest training time (TT) (average value of 23.5 minutes) and the fastest decision time (DT) (average value of 0.56 seconds).

3.3. Discrimination Performance

The network with the best testing accuracy in Table 6 (GoogLeNet-Places365) demonstrated stable discriminative power across 10 repeated train–test splits (in silico experiments). The mean AUC was 0.971, with a 95% confidence interval (CI) of [0.954 – 0.988], indicating robust separation between COVID-19 and non–COVID-19 cases across repeated experiments and limited sensitivity to data partitioning (Figure 1).

3.4. Performance at the Youden-Optimal Threshold

Using the Youden-optimal threshold per-split, the GoogleNetPlaces365 achieved a mean sensitivity of 92% (95% CI: 88% – 95%) and a mean specificity of 95% (95% CI: 93 – 98) across experiments (Figure 2). The corresponding mean ± sd (standard deviation) precision, F1-score, and accuracy were 95 ± 3.5%, 93 ± 3.4%, and 93 ± 3.3%, respectively.

4. Discussion

AI will revolutionize the way we practice medicine, transforming traditional approaches in many medical specialties and healthcare applications. In 2011, more than 1000 AI medical papers were uploaded to PubMed. Since then, the number has increased spectacularly, reaching approximately 33,000 papers in 2025.

Regarding COVID-19 (COronaVIrus Disease 2019), an acute respiratory disease caused by the SARS-CoV-2 virus (Severe Acute Respiratory Syndrome CoronaVirus 2), it emerged in late 2019 and rapidly spread on a worldwide scale, causing the 2019-2023 global pandemic. The pandemic had an overwhelming impact worldwide, leading to vast disruptions in healthcare systems, education, and social life. It ignited an extraordinary level of scientific international collaboration, enabling the rapid deployment of novel mRNA vaccines and antiviral treatments. Today, the virus continues to circulate, but it is typically managed through vaccination and public health surveillance.

Despite the effective management of the disease, some people cannot fully recover, presenting long-term symptoms which are usually named “Long-COVID-19” or “post-acute COVID-19 syndrome” without a universally accepted explanation. However, hemodynamic and thrombotic effects have been observed in the blood and microvessels of COVID-19 patients during [16,17] and after [15] the disease. Long-lasting microvascular thrombosis is functionally equivalent to microvascular loss reported by many quantitative case–control studies from multiple tissues [18,19,20,21]. Based on these findings, a pathophysiological framework for the explanation of Long COVID was grounded in persisting microvascular blood supply reduction [22], and 76% of the total incidence rate of Long COVID symptoms was directly attributed to this mechanism [23].

A key aspect of the smallest microvasculature that has not been examined yet in post COVID patients, is the microvessel network “spatial distribution” or “topological structure” (topology). The fundamental emerging question is whether COVID-19 affects the microvessel network topology. Based on this question, the central target in this work was to consider the microvessel topology as a “scene” and examine whether the microvessel topological structure of post-COVID patients can be discriminated from that of healthy individuals using the superior capabilities of artificial intelligence in image classification.

For that purpose, a variety of pretrained deep convolutional neural networks were assessed and the best average testing accuracy across 10 random data splits of 92% (± 3%) was achieved by the GoogLeNet-Places365 network. GoogLeNet-Places365, a scene-centric network, excelled on all other performance criteria of Table 6, and performed much better than object-centric CNNs with a 4% difference in average testing accuracy from the second-best performance. Larger differences were observed in other evaluation criteria (Table 6), such as a 7% difference in Sensitivity (92% for GoogLeNet-Places365 versus 85% for InceptionResNet-v2) and Specificity (91% for GoogLeNet-Places365 versus 84% for EfficientNet-b0).

Overall, the GoogLeNet-Places365 exhibited both excellent discriminative ability and consistent performance at clinically relevant conditions with an average ROC AUC of 0.971 across 10 train-test splits (Figure 1). At the Youden-Optimal operating point, the high values and limited dispersion of sensitivity and specificity under varying data partitions indicates stable operating characteristics (Figure 2).

Regarding input image size (IIS), it is generally accepted that the bigger the better for testing accuracy (Tan and Le, 2019). Inception ResNet-v2 has a much larger IIS (299x299) than GoogLeNet and EfficientNet-b0 (224x224) (Table 1). In addition, Inception ResNet-v2 is significantly deeper (Table 1) and larger in size (Table 2) than the other networks. However, these architectural advantages of Inception ResNet-v2 did not result in better classification performance over the other networks (Table 6). GoogLeNet-Places365 performed significantly better at all classification evaluation criteria. Therefore, for this particular case, it is concluded that the critical factor was not depth and size, but the pretraining on a dataset of scenes and backgrounds (Places365) instead of objects.

Comparing the decision times (DT) of the 4 fine-tuned pretrained networks in Table 6, we observe that all average DTs are less than a few seconds, and the most accurate network (GeegLeNet-Places365) requires less than 1 second to make the final prediction using only one image. These decision times are significantly shorter (more than a thousand times) than the approximately 15 minutes needed by current antigen (rapid) COVID-19 detection tests (detection of viral proteins). They are also 4 orders of magnitude shorter than the hours required by the current gold standard of molecular PCR tests (detection of the virus’s genetic material). The future prospect of a much faster COVID-19 test is reinforced by the noncontact nature of this AI imaging method in contrast to the requirement of liquid human samples (nose, throat, saliva) by the current tests. On surplus, since the images come from the bulbar conjunctiva and not the retina, the outcome does not depend on the optical clarity of the eye, which may be reduced significantly in cases of cataract or vitreous hemorrhage.

A limitation of the study was the small sample size and the subsequent medium to small image dataset.

5. Conclusions

In conclusion, it was shown that post-COVID effects on the eye microcirculation can be detected by deep CNNs with a relatively high average accuracy and excellent discriminative power. The critical factor for achieving higher accuracy was not depth and size, but the pretraining on a dataset of scenes and backgrounds. This AI-based post-COVID detection method could revolutionize the way we detect viral infections since it is risk-free, painless, contactless, fast, accurate, and does not depend on the optical clarity of the eye.