Development of The AI Pipeline for Corneal Opacity Detection

1. Introduction

Despite the global increase in the number of ophthalmologists, a significant shortage remains in developing countries [1].This shortage is compounded by the limited access to appropriate surgical technologies and diagnostic tools [2,3]. Deployment of local ophthalmologists is considered a cost-effective solution. However, the scarcity of professionals in developing countries remains a challenge [4].

The Smart Eye Camera (SEC) [5] used in this study to photograph the anterior segment of the eye was invented and developed by an active ophthalmologist to solve the problems encountered in ophthalmology treatment in Japan and developing countries, an ophthalmic medical device that has been successfully put into practical used as a medical device. SEC is a smartphone attachment that enables observation of various anterior segment structures of the eyes, including the eyelid, conjunctiva, cornea, anterior chamber, iris, lens, and anterior vitreous. This device mirrors the functionalities of conventional slit-lamp microscopy [6,7]. Furthermore, SEC facilitates the preliminary estimation and identification of several anterior segment pathologies such as cataracts [8], primary angle closure [9], allergic conjunctivitis [10], and dry eye disease [11,12]. Its integration with smartphone technology not only enhances accessibility, but also potentially expands the scope of ophthalmologic diagnostics in various settings. Additionally, an image-filing system using a dedicated application was used to enable remote ophthalmology treatment. The development of SEC has made it possible for anyone to perform eye examinations at any time regardless of location. We are diagnosing videos of the anterior segment of the eye sent via the cloud, and we are conducting research and development to perform the diagnosis using AI to support the diagnosis of ophthalmologists.

Deep learning has been applied in various ways to diagnose conditions that affect the anterior segment of the eye. Applications range from detecting angle-closure in anterior segment optical coherence tomography (AS-OCT) images to diagnosing dry eye disease (DED) and identifying peripheral anterior synechia (PAS). For instance, a deep learning system was developed for angle-closure detection in AS-OCT images, which surpassed previous methods by utilizing a multilevel deep network that captured subtle visual cues from the global anterior segment structure, local iris region, and anterior chamber angle (ACA) patch [13]. Another study evaluated a deep learning-based method to autonomously detect DED in AS-OCT images, which showed promising results compared to standard clinical dry eye tests [14]. Deep learning classifiers have also been used to measure peripheral anterior synechia based on swept-source optical coherence tomography (SS-OCT) images, demonstrating good diagnostic performance for gonioscopic angle closure and moderate performance for PAS detection [15]. In addition, deep learning classifiers have been developed to detect gonioscopic angle closure and primary angle closure disease (PACD) based on a fully automated analysis of AS-OCT images, showing effective detection capabilities [16]. Another study focused on the diagnostic performance of deep learning for predicting plateau iris in patients with primary angle-closure disease using AS-OCT images, which revealed a high performance in predicting plateau iris [17]. Finally, a deep learning model was developed for automated detection of eye laterality in anterior segment photographs, which achieved high accuracy and outperformed human experts [18]. In summary, deep learning has shown significant potential in the diagnosis of various anterior eye conditions, offering automated, accurate, and noninvasive methods that could enhance clinical evaluations and improve access to eye care in high-risk populations [13,14,15,16,17,18]. Deep learning models have shown significant promise in the field of biomedicine, particularly for the diagnosis of systemic diseases. However, there are several challenges associated with their application. One of the primary concerns is the need to guarantee the performance of deep-learning systems once they are deployed in a clinical setting. The inherent flexibility and strength of deep learning also present difficulties in ensuring consistent and reliable outcomes [19]. Moreover, there is a critical need to establish trust among stakeholders, including clinicians and regulators who require transparent and interpretable decision-making processes. The complexity of deep learning models often leads to a ’black box’ scenario, where the rationale behind their predictions is not easily understood or explained. This lack of transparency can hinder the adoption of deep learning in clinical practice [19]. In ophthalmology, deep learning has demonstrated potential in automated image analysis for detecting diseases, such as diabetic retinopathy, age-related macular degeneration, and glaucoma. Despite the high accuracy in the initial studies, further testing and research are necessary to validate these technologies clinically. This highlights the challenge of moving from research and development to practical and clinical applications [20]. A systematic review and meta-analysis comparing the diagnostic accuracy of deep learning algorithms and healthcare professionals found that, while deep learning models can match the performance of healthcare professionals, there is a scarcity of studies that provide externally validated results. Additionally, the review identified the prevalent issue of poor reporting in deep learning studies, which undermines the ability to reliably interpret diagnostic accuracy. The establishment of new reporting standards that address the unique challenges of deep learning is essential for improving the quality of future studies and fostering greater confidence in technology [21]. In summary, the challenges of applying deep learning models to diagnose systemic diseases include ensuring reliable performance, establishing trust through transparency and interpretability, clinically validating technology, and improving the quality of reporting in deep learning studies [19,20,21]. Deep learning models have shown significant promise in the field of ophthalmology, particularly for the detection and diagnosis of ocular diseases. However, these models have several limitations that must be considered. One of the primary limitations of this study was the need for further testing and clinical validation. Although deep learning models have demonstrated high accuracy in automated image analysis of fundus photographs and optical coherence tomography images, additional research is required to validate these technologies in clinical settings [22]. Another limitation is the lack of disease specificity and the public generalizability of the models. Despite the decent performance reported in previous studies, most deep learning models developed for identifying systemic diseases based on ocular data lack the specificity required for individual diseases and are not yet generalizable to the broader public for real-world applications [23]. Furthermore, deep-learning models can be computationally expensive, and deploying them on edge devices may pose a challenge. This is particularly relevant when considering the variety of available models and the potential need for a combination of models to solve a given task. The computational demands of these models may limit their practicality in certain clinical settings [24]. Lastly, while deep learning models can predict the development of diseases such as glaucoma with reasonable accuracy, they may miss certain cases, especially those with visual field abnormalities, but not glaucomatous optic neuropathy. This indicates that although DL models are powerful tools, they may not yet be able to fully replace the nuanced judgment of trained medical professionals [25]. In summary, while deep learning models hold great potential for revolutionizing the diagnosis of ocular diseases, their limitations in terms of clinical validation, disease specificity, computational demands, and potential to miss certain cases must be addressed before they can be fully integrated into clinical practice.

The purpose of this study was to develop an AI pipeline to determine the presence of corneal opacity using anterior segment videos captured using a portable sitting microscope and deep learning techniques.

2. Materials and Methods

This study was conducted in strict accordance with the principles of the Declaration of Helsinki. Ethical approval for the study protocol was obtained from the Institutional Ethics Review Board of the Minamiaoyama Eye Clinic, Tokyo, Japan (IRB No. 15000127. Approved No. 202101). Owing to the retrospective design of the study and the use of anonymized data, the board waived the requirement for written informed consent from the participants.

Anterior segment videos were captured using a portable slit-lamp microscope (Smart Eye Camera; SEC. SLM-i07/SLM-i08SE, OUI Inc., Tokyo, Japan; 13B2X10198030 101/13B2X10198030201) (Figure 1). By attaching this device to a smartphone, it will be possible to perform eye examinations in the same way as with existing slit-lamp microscopes. There is evidence that they do not require battery replacement or charging, are easy to carry, and exhibit the same performance and safety as existing medical devices in several regions [6,7].

Data acquisition for this study was centralized at a singular ophthalmological facility, namely, the Yokohama Keiai Eye Clinic. The recordings, systematically obtained between July 2020 and December 2021, were subsequently collated on a dedicated cloud server, thus constituting the dataset for this study. The recording process entailed skilled ophthalmologists directing SEC towards the cornea of the patients, leveraging the device’s feature of emitting a white diffused light to facilitate clear visualization. The video capture protocol was aligned with the methodologies conventionally associated with slit-lamp microscopes, ensuring standardization of the visual data. To further emulate the conditions of routine clinical assessments, patients were advised to avoid blinking during the video recording phase to enhance the consistency and clinical relevance of the collected video data.

First, 30 diffuse light videos were decomposed into images, and the images that could be used as validation data were selected. A total of 5996 images, 1617 positive frames and 4379 negative frames were used to detect corneal opacity. The resolution of all images was 1280 pixels horizontally and 720 pixels vertically. Using these verified images, we attempted to detect corneal opacity by performing image classification. EfficientNet-B4 [26] was used as the convolutional neural network (CNN) model, and cross-validation was performed, but almost no correct positive frames could be detected.

Therefore, to improve the detection accuracy, we manually annotated the cornea, which is the region of interest (ROI), extracted an image of only the cornea, adjusted the contrast using contrast-limited adaptive histogram equalization (CLAHE) [27], and used a convolutional neural network. Image classification was performed again using EfficientNet-B4 as a (CNN) model. The data structures used for the learning and prediction are presented in Table 1. The proportion of underlying diseases in the data is shown in the pie chart in Figure 2, the cornea extraction procedure is shown in Figure 3, and the contrast changes before and after CLAHE image processing are shown in Figure 4.

This confirmed that corneal opacity could be detected in the anterior segment images. However, to incorporate it into an AI automated prediction system, which we call an AI pipeline, it is necessary to automate the manual annotation of the process of extracting the cornea and ROI from the anterior eye image.

The hyperparameters at training were 30 for the number of epochs, 8 for the batch size, and 0.0001 for the learning rate, with default values for EfficientNet-B4 for the rest. Data augmentation during training included resizing (512 × 512), flipping up and down with a 1/2 probability, and flipping left and right with a 1/2 probability.

Therefore, we performed semantic segmentation learning to segment the cornea from the anterior segment image by reusing the annotation mask used to extract the cornea from the anterior segment image and original extracted image as learning data. U-Net [28]/EfficientNet-B4 was adopted as the semantic segmentation model.

The hyperparameters for learning the semantic segmentation were 40 epochs and 10 batch sizes. The learning rate started at 0.001 and decreased to 0.0001 after 25 epochs. All other values were set to the U-Net/EfficientNet-B4 default. Data Augmentation during training included resizing (256x256), flipping left/right with 1/2 probability, Affin transformation, padding the edges according to image size, cropping at random, Gaussian noise with a probability of 1/5, and perspective transformation with a probability of 1/2. In addition, one set of augmentations among the following three was performed with a probability of 9/10:1st set includes CLAHE, brightness adjustment, and gamma transformation, 2nd set includes sharpening, blur, and motion blur, 3rd set includes contrast adjustment and hue, saturation, and luminance change.

This study was conducted on a Windows 11 system with the following specifications: CPU: Intel Core i7-11700KF, memory: 128GB, and GPU: RTX 4070.

In this way, an AI pipeline was completed that uses semantic segmentation to extract the ROI and cornea from anterior segment images, and uses deep learning to classify images to detect corneal opacity.

3. Results

Table 2 shows the results of manually annotating the cornea as a region of interest (ROI), extracting only the cornea, adjusting the contrast with CLAHE, and learning with CNN (EfficientNet-B4).

Table 3 lists the metrics derived from the outcomes predicted by the model. The evaluation yielded commendable results across several key indicators: sensitivity, specificity, accuracy, and the Area Under the Curve (AUC). The values obtained were as follows: sensitivity of 0.96 (95% Confidence Interval [CI]: 0.97–0.99), specificity of 0.96 (95% CI: 0.97–0.99), accuracy of 0.96 (95% CI: 0.97–0.99), and an AUC of 0.98 (95% CI: 0.98–0.99).

Figure 5 depicts the Receiver Operating Characteristic (ROC) curve, illustrating the diagnostic ability of the model across various threshold settings.

Table 4 shows the outcomes of the corneal semantic segmentation, as predicted by the model. The Dice coefficient, also referred to as the F1 score, had a substantial value of 0.94. Furthermore, Intersection over Union (IoU), another critical metric for segmentation performance, similarly registered a notable value of 0.94.

The Dice coefficient is called the "Sørensen-Dice index" or the "Sørensen-Dice coefficient.” The Dice coefficient DSC(A,B) for set A and set B is defined by the following equation:

$$DSC(A,B)=\frac{2|A\cap B|}{\left|A\right|+\left|B\right|}$$

(1)

The Dice coefficient represents the ratio of the average number of elements in the two sets to the number of elements they have in common and is a value between zero and one. The larger the Dice coefficient, the more similar the two sets are.

The Intersection over Union (IoU) is an evaluation metric used in object detection and represents the percentage of image overlap. Specifically, it has a maximum value of 1 when the detected and true areas completely overlap and a minimum value of zero when there is no overlap at all. The IoU for Regions A and B can be calculated using the following formula:

$$IoU=\frac{|A\cap B|}{|A\cup B|}$$

(2)

4. Discussions

We believe that these three approaches contributed to the improved prediction accuracy for corneal opacity. First, the cornea, the ROI, was extracted from the anterior segment image; second, the image resolution was reduced by changing the input image from the entire anterior segment image to the corneal image, the ROI, thereby reducing the reduction in image features; and third, CLAHE was applied as a contrast optimization.

It was also a good idea to reuse the anterior segment image used in the model training phase of corneal opacity prediction and the mask image used to extract the cornea to perform corneal semantic segmentation. The corneal semantic segmentation model eliminates the need to manually extract the cornea and allows it to be integrated into the AI pipeline. The corneal opacity prediction AI pipeline begins with the selection of anterior segment image frames from the video that were deemed suitable for diagnosis, followed by the extraction of the cornea through semantic segmentation, resulting in an accurate diagnosis of corneal opacity.

Despite the constraints presented by the limited size of the sampling dataset (comprising 5996 frames, with 1617 positive and 4379 negative frames), this study successfully developed a model with high diagnostic accuracy for corneal opacity. It is noteworthy that previous research in the domain of ocular image analysis often utilized datasets that exceeded thousands of annotated cases [29,30]. Conversely, the current study leveraged video data as the primary raw material, capitalizing on the potential to extract multiple image frames from a single video sequence. This methodology aligns with the techniques employed in prior research focused on the development of automated diagnostic AI systems [31,32], wherein methods such as cropping, flipping, and other forms of data augmentation are utilized to effectively expand the dataset from a single image. The implementation of these techniques, particularly the strategic use of video data for frame extraction and image amplification [33], is posited as a pivotal factor contributing to the development of a high-performance model despite the relatively modest size of the dataset.

The current study had several limitations. First, limited scale of the sample size was small. Despite the retrospective nature of the study, wherein the use of video recordings served to augment the dataset, the scope of the data remained relatively constrained. To develop robust and adaptable AI models, particularly those pertinent to imaging analysis, there is a significant need for more extensive datasets. Therefore, the limited sample size in this study may represent a significant impediment to the generalizability and comprehensive applicability of the derived AI models. In addressing the aforementioned limitation, this study drew inspiration from prior literature, which demonstrated enhanced detection of keratitis through the augmentation of single anterior segment images [34]. Such augmentation involves a six-fold increase in data quantity achieved by methods such as flipping, rotating, and cropping [34]. Similarly, our approach involved meticulous recording of digital anterior segment videos, thereby facilitating amplification of the volume of raw data [31,32]. Second, the dataset in this study was exclusively sourced from a single medical institution, which may limit the external validity and generalizability of the findings. To ensure broader clinical applicability and substantiate the robustness of the conclusions, it is essential for future research endeavors to incorporate and validate the models against external datasets. Ideally, this validation process should involve a large-scale cohort comprising data from multiple medical facilities. This comprehensive approach will be instrumental in enhancing the reliability and relevance of AI models in diverse clinical settings.

In the context of developing diagnostic AI programs for medical applications, determining the optimal performance benchmarks, particularly for diagnostic goals, presents a substantial challenge. This is exemplified in the realm of ophthalmology, where certain diseases are leading causes of blindness globally. Previous investigations, including our own, have underscored the potential of AI to achieve diagnostic accuracies comparable to, if not surpassing, those of human specialists. For instance, our prior research indicated that AI-based diagnostics could achieve over 95% accuracy in comparison with evaluations conducted by ophthalmologists in the context of a disease with a significant worldwide blindness burden [29]. Furthermore, Hu et al. reported an impressive diagnostic accuracy of 93.5% with an AUC of 0.9198 [35], indicating a high level of diagnostic precision. Additionally, a cross-sectional study by Son et al. demonstrated AI’s robust diagnostic performance, with an accuracy of 90.26% and an AUC of 0.9465 [36], further evidencing AI’s capability to accurately diagnose medical conditions. Moreover, recent studies provide compelling evidence on the efficacy of AI algorithms in distinguishing between infectious keratitis and immunological keratitis through image analysis. A notable report highlights the exceptional performance of the AI algorithm, as evidenced by AUC values of 0.986 for infectious keratitis and 0.960 for immunological keratitis [37]. These findings underscore the algorithm’s broad applicability not only in the identification of keratitis subtypes, but also in its performance across a range of ocular conditions, including corneal scars, ocular surface tumors, corneal deposits, acute angle-closure glaucoma, cataracts, and bullous keratopathy [37]. The deployment of this technology in ophthalmology clinics for professional use signifies a significant advancement in the field. It enables healthcare providers to more accurately identify the underlying causes of ocular diseases, thereby facilitating the determination of appropriate differential treatment methods. This development represents a pivotal step toward integrating AI into clinical practice, offering a promising tool for enhancing diagnostic accuracy and improving patient outcomes in ophthalmology. These findings collectively suggest that high diagnostic accuracy should be a key consideration in establishing performance benchmarks for AI systems aimed at diagnosing corneal opacity. Such evidence supports the argument for setting ambitious yet achievable accuracy goals in the development and evaluation of AI diagnostics, thereby enhancing their utility and reliability in clinical settings.

In the existing literature, there is a scarcity of studies employing deep learning methodologies for the identification of corneal opacity through images acquired via slit-lamp microscopy. Consequently, this study is pioneering in its endeavor to develop a highly accurate model for the detection of corneal opacity. Moreover, the application of AI to the diagnosis of ocular pathologies from medical examination videos remains a nascent field. This research, therefore, holds significance because of its innovative approach to both the development of a precision model for corneal opacity detection and its exploration of AI-based diagnostic methodologies in ophthalmology. In addition, research on using AI to diagnose eye diseases from medical examination videos is new, and we believe that this research is significant in these two respects.

5. Conclusions

The accuracy of the prediction model was improved by manually extracting only the cornea, which is the ROI, from the images of the anterior segment of the eye to train the model to predict the presence or absence of corneal opacity.

Furthermore, to incorporate it into the AI pipeline, rather than manually extracting the cornea, we succeeded in performing semantic segmentation using the original and mask images that were used to train a model to predict the presence or absence of corneal opacity.

The corneal opacity detection AI pipeline can seamlessly execute the process of extracting still images from videos, extracting corneas from still images using semantic segmentation, and classifying whether the cornea has opacity.

Additionally, if we develop and replace the corneal opacity detection module in this AI pipeline with a detection module for other anterior segment diseases, there is potential for the future construction of a general-purpose anterior segment diagnosis AI pipeline. If this can be achieved, it will be possible to triage large numbers of anterior eye segment videos taken during health checkups in a short period of time, which we believe will lead to a reduction in the burden on ophthalmologists.

To improve the completeness of this corneal opacity detection AI pipeline, we believe that the following three issues need to be addressed. These include increasing the amount of training data to improve the classification accuracy of the corneal opacity detection model, selecting images suitable for diagnosis from images decomposed from videos, and classifying whether the anterior eye in the image suitable for diagnosis is the right eye or the left eye.

We plan to continue this research to complete an anterior segment diagnostic AI pipeline. References