Deep Learning–Based Choroidal Boundary Detection in Geographic Atrophy Using Spectral-Domain Optical Coherence Tomography

1. Introduction

The role of the choroid in age-related macular degeneration (AMD) pathophysiology remains debated, including the sequencing of choriocapillaris loss and retinal pigment epithelium (RPE) degeneration. In geographic atrophy (GA), the advanced stage of AMD, this temporal relationship between choriocapillaris dropout and RPE degeneration is correlated to longitudinal OCT-based quantification of choroidal structure [1]. Growing evidence highlights the role of the choroid in AMD and GA, reflecting its critical contribution to retinal metabolic support. [2] Choroidal thinning has been consistently reported in eyes with GA, while a distinct subset, referred to as pachychoroid GA, has also been identified. [3] Similarly, other choroidal biomarkers in geographic atrophy have also been explored, including the choroidal vascularity index (CVI) [4] and total choroidal volume, both of which have been shown to be reduced in GA.

Manual segmentation of the choroidal inner boundary (CIB) and outer boundary (COB) on high-density optical coherence tomography (OCT) volumes is prohibitively time-consuming for both clinical trials and routine practice. Precise delineation of these boundaries forms the foundation for deriving key choroidal biomarkers, including volumetric and sectoral choroidal thickness, choroidal volume, choroidal vascularity index, choroidal contour [6] and three-dimensional reconstruction of choroidal vascular architecture. [7] Deep-learning models can delineate the choroid reliably in many healthy eyes, but GA changes the imaging problem itself. With RPE loss, SD-OCT often shows marked hyper-transmission that can wash out the choroidal–scleral interface (CSI) or create “tailing,” so the boundary is no longer cleanly expressed in the intensity profile [8,9]. In this setting, accuracy reported in healthy or non-GA AMD cohorts may not carry over, because the disease introduces GA-specific contrast shifts and local ambiguities, most notably for COB detection [10,11]. Recent GA imaging work echoes this point, emphasizing that RPE loss fundamentally reshapes OCT contrast at the CSI in ways that can mislead automated boundary estimation [12].

Currently, there is no automated algorithm available that can reliably segment the choroid in GA eyes without supervision. Given the persistence of these artifacts, the role of AI must be defined as assistive rather than autonomous. A "human-in-the-loop" approach is critical, where manual verification serves as a mandatory quality control step to correct errors caused by hyper-transmission, ensuring that efficiency does not come at the cost of clinical validity. Accordingly, no AI-based method is expected to be 100% error-free in this setting. In this study, we evaluated a deep learning pipeline for the automated choroidal segmentation of GA eyes imaged with SPECTRALIS® OCT system (Heidelberg Engineering, Heidelberg, Germany) [13,14]. Unlike lesion-detection work, we perform a "failure analysis" of boundary definition, hypothesizing that hyper-transmission artifacts affect COB more than CIB. We also quantify efficiency of an AI-assisted approach versus manual grading to see if time savings justify expert artifact correction.

2. Materials and Methods

Data and Imaging

This retrospective study included 5723 scans from 59 volumes (each 97 scans) with GA secondary to dry AMD. The study was conducted at the UPMC Vision Institute, from October 2023 to September 2024 following approval from the Institutional Review Board. The study adhered to the tenets of the Declaration of Helsinki. A “waiver of informed consent” was obtained considering the retrospective nature of the study.

Imaging Acquisition

Volumetric SD-OCT imaging was performed using the SPECTRALIS® OCT system (Heidelberg Engineering, Heidelberg, Germany). Each volume covered a 6x6 mm area and consisted of 97 horizontal B-scans covering macular regions. It can scan up to 40,000 A-scans per second, has a wavelength of 870nm and an axial resolution of about 3.9 µm in tissue. The SD-OCT software’s integrated scoring system called the quality score or Q-score, evaluated the quality of the scans. Only scans with a quality score of 25 or higher on a scale of 0-40 met the criteria for inclusion in the analysis. For each volume, the SPECTRALIS® system also acquired a co-registered infrared (IR) fundus image used for scan localization during acquisition. Scan quality was reviewed, and volumes with significant motion artifacts or signal loss were excluded from analysis.

Two trained readers (SL and JW) independently assessed the OCT images and graded the degree of inaccuracy for each B-scan, while also recording the number of completely accurate B-scans within each volume. Manual correction was performed in a subset of volumes, and the time required for full manual segmentation was compared with the time required for manual correction following automated segmentation. (Figure 1)

Segmentation Model and Pipeline

We used the previously validated NMI-ChoroidAI algorithm (Netramind Innovations Inc., Pittsburgh, PA), a deep convolutional neural network based on a ResUNet (Residual U-Net) architecture, to segment the choroidal inner and outer boundaries in the OCT volumes. [15] The segmentation of choroid layer boundaries involves identifying two critical interfaces: the RPE–choroidal interface (CIB) and the choroid–sclera interface (COB/CSI). Manual delineation and correction of CIB and COB was performed using the manual segmentation feature on the NMI ChoroidAI tool.

Shadow compensation was applied to each B-scan to enhance visualization of the choroidal region. The preprocessed B-scan was then passed to the ResUNet, which produced pixel-wise probability maps for the CIB and COB. These probability maps were converted into continuous boundaries by selecting the highest-probability pixel for each boundary, followed by minimal post-processing (e.g. Robust Locally Estimated Scatterplot Smoothing). For visualization and quality control, the predicted CIB and COB lines were overlaid on the original B-scans.

Performance Evaluation and Classification Metrics

Performance was assessed by comparing automated segmentations against manual ground truth with slice-wise error analysis. Each B-scan’s boundary deviation was graded: 0 = accurate, 1 = minor (≤33% of the B-scan deviated), 2 = moderate (33–66%), 3 = major (>66% or missed) (Figure 2). The percentage quantified the portion of each B-scan where the automated segmentation diverged from the ground-truth boundary. Manual segmentation paid close attention to distinguishing the true scleral interface from hyper-transmission artifacts. We also identified error causes such as low image quality, motion, or lesion morphology.

We computed confusion matrices and derived accuracy, sensitivity, precision, and F₁ score for CIB and COB. Specificity and negative predictive value were not reported since every B-scan contains true CIB/COB boundaries. Metrics were calculated separately for CIB and COB to assess performance under different error tolerance levels. Inter-reader agreement for identifying number of accurate scans from each volume was assessed using intraclass coefficient (ICC; two-way mixed-effects model, absolute agreement).

Workflow Efficiency Assessment

To evaluate the clinical utility of the AI model despite its limitations, we compared the time required for two workflows in three volumes. "Manual-Only" segmentation; the time taken for an expert to segment the full choroidal volume (CIB and COB) from scratch. "AI-Assisted" workflow; the time taken for the model to generate initial boundaries plus the time required for a human expert to verify and correct all segmentation errors (specifically correcting COB displacements in areas of signal washout). Timing was recorded from the start of each workflow to completion and summarized across volumes.

Statistical Analysis

Normality was tested with the Shapiro–Wilk test. The ANOVA test was used for normally distributed data and the Kruskal-Wallis test for non-normally distributed data. The Chi-square test was employed to compare the two methods. A confidence interval (CI) of 95% and a p-value of ≤0.05 were considered statistically significant. All statistical procedures were executed using IBM SPSS Statistics software, version 26.

3. Results

Choroidal Boundary Segmentation Performance and Failure Analysis

Across 5,723 B-scans, CIB was highly robust, with a strict accuracy (Grade 0) of 94.8%, a weighted F₁-score of 0.97, and a precision of 1.0. The remaining were minor deviations at 2.5%, and moderate and major errors at 1.01% and 1.64%, respectively.

In contrast, COB accuracy dropped to 81.0% (weighted F₁-score: 0.89). There were minor deviations in 13.2% of scans and moderate-to-major errors in about 6% (Categories 2 and 3). Confusion matrices (Figure 3) indicate many more Grade-3 errors for COB (202 scans) compared to CIB (94 scans). However, considering minor deviations (≤33% deviation is considered acceptable) increased COB sensitivity to 94.2%, suggesting many errors consisted of minor spatial deviation rather than outright failures. Inter-grader reliability for identifying the number of accurate B-scans per volume was excellent, with ICCs of 0.938 (95% CI: 0.874–0.970, p < 0.001) for COB and 0.923 (95% CI: 0.847–0.962, p < 0.001) for CIB. Inter-grader reliability for identifying the number of accurate B-scans per volume was excellent, with ICCs of 0.938 (95% CI: 0.874–0.970, p < 0.001) for COB and 0.923 (95% CI: 0.847–0.962, p < 0.001) for CIB.

Failure Mode Characterization

The causes of inaccuracy are detailed in Table 1. While CIB errors were primarily driven by image quality issues, COB failures were largely attributed to hypertransmission artifacts specific to GA area. Specifically, signal washout and hyper-transmission defects accounted for approximately 40% of COB errors, where the increased light penetration through the atrophic RPE obscured the choroidal-scleral interface.

Workflow Efficiency and Clinical Utility

Despite the limitations in COB accuracy requiring human verification, the AI-assisted workflow demonstrated a dramatic improvement in efficiency compared to manual grading. Manual-only segmentation of the full choroid (CIB and COB) across a complete 97-slice volume was extremely labor-intensive, requiring an average of approximately 7 hours per volume by a single experienced ophthalmologist. In contrast, the "AI-assisted" workflow, comprising automated inference followed by expert correction of artifacts, took an average of 45 minutes per volume by a single experienced ophthalmologist. Although the model frequently misidentified the COB in areas of hyper-transmission (necessitating human correction), the AI provided a foundational structure that reduced the total human effort by approximately 90%.

4. Discussion

In this study, we performed a failure analysis of automated choroidal boundary detection in GA eyes using a validated a choroid segmentation tool (NMI-CHoroidAI), focusing on the markedly different behavior of the model when estimating the CIB versus the COB. While the CIB remained highly detectable even in advanced GA (94.8% grade-0 accuracy), COB definition consistently deteriorated in areas of hyper-transmission, reaching only 81.0% strict accuracy.

Compared with prior studies, which have predominantly evaluated choroidal boundary detection in healthy eyes or non-GA AMD, our findings demonstrate a critical gap in literature that is current automated choroidal segmentation methods rarely account for GA-specific optical artifacts. Several high-performing deep-learning models have reported strong choroidal and retinal boundary accuracy under normal imaging conditions [10,11], yet these results do not necessarily translate to GA eyes, where loss of the RPE results in marked hyper-transmission and signal washout. Consistent with this, recent research on GA imaging has emphasized that RPE loss fundamentally alters OCT contrast characteristics [12] which result in producing structural ambiguities at the CSI that complicate automated COB estimation. Our strict COB accuracy of 81% reflects a pathology-driven limitation rather than a model flaw alone. However, when scans with minor inaccuracies were included, the model demonstrated improved performance, achieving an overall accuracy of 94.2%. Only 13.2% of scans required minor manual corrections, substantially reducing the time required and overall workload.

Accurate and reproducible delineation of the choroidal inner and outer boundaries is clinically crucial; as choroidal thickness and vascular metrics are increasingly investigated as imaging biomarkers and potential exploratory endpoints in AMD and GA research, as well as interventional trials [2,16,17]. Manual slice-by-slice segmentation is prohibitive in volume, explaining why AI-assisted workflows are appealing. In this study, AI-assisted processing reduced the total time to about 45 minutes per volume, representing nearly a 90% reduction in human effort. This pattern is consistent with prior AI-based OCT pipelines, which have demonstrated substantial time savings in clinical trial settings while still relying on human verification rather than fully autonomous deployment [18,19,20]. In this context, our model should therefore be considered as a workflow accelerator, providing a plausible anatomical estimate whose COB accuracy requires expert verification before use in trials.

The strong performance on the CIB and the weaker performance on the COB arise from fundamentally different signal characteristics within SD-OCT volumes. The CIB corresponds to a high-contrast RPE–Bruch’s membrane transition, which remains anatomically recognizable even in eyes with advanced cRORA [12,21]. This structural stability allows deep networks to detect the CIB reliably across varying image quality and patient-dependent fluctuations [10]. In contrast, the COB does not represent a discrete optical interface in GA eyes, but rather a gradual attenuation zone whose appearance varies with the degree of RPE loss, choroidal vascular density, and local scleral reflectivity. As a result, the model frequently encounters regions where no clear boundary exists at all, making errors unavoidable regardless of architectural complexity.

Another factor influencing performance is the directionality of OCT signal propagation. The hyper-transmission that accompanies RPE loss increases light penetration into deeper tissues but simultaneously reduces the local contrast between the choroid and sclera. This “signal dilution” effect decreases the sharpness of the true boundary and increases the probability that the model converges on an anatomically invalid but visually dominant tailing artifact. This behavior is consistent with histopathologic and imaging observations describing the fading of the CSI in cRORA [12]. These effects can also influence boundary consistency across adjacent B-scans: while slice-to-slice stabilization steps may improve the continuity of the CIB, they offer limited benefit for the COB because its visibility fluctuates along the scan. Some preprocessing strategies have been reported to stabilize retinal and choroidal boundaries in automated OCT analysis [10,11]. In regions where hyper-transmission abruptly increases or posterior ciliary vessels locally dominate the intensity profile, preprocessing cannot reconstruct a boundary that is physically absent.

NMI-ChoroidAI tool integrates manual editing within the automated segmentation workflow, allowing users to refine results in real time while viewing OCT images. Importantly, corrections are required only for boundary segments that deviate from the true anatomical layer, rather than redrawing the entire contour. This selective adjustment approach greatly improves efficiency and enables rapid generation of high-quality segmentations.

Furthermore, architectural elements such as residual connections and attention mechanisms help the model focus on coherent, layer-specific patterns rather than isolated noise. Residual learning has been shown to improve detection of fine structural transitions in medical image segmentation [22,23], while attention mechanisms enhance the network’s ability to prioritize relevant anatomical features and suppress background clutter [24,25,26]. While this improves CIB detection, it also means the model tends to ignore faint or discontinuous deep signals, which in GA lesions are often the only remaining cues for COB localization. Thus, architecture inadvertently reinforces the performance gap: robust for structures with preserved reflectivity, fragile for structures whose visibility deteriorates under disease-specific optical conditions.

Limitations

Several limitations of this study should be acknowledged. First, the analysis was performed exclusively on eyes with established GA that met Classification of atrophy meetings (CAM) criteria for cRORA. Earlier stages such as iRORA, which exhibit only partial RPE-photoreceptor disruption, were not included. Because the optical visibility of the CSI degrades progressively along the iRORA to cRORA continuum, the present model may not generalize to earlier atrophic stages where the boundary is partially preserved and the pattern of hyper-transmission differs [21].

Second, although the model demonstrated high reliability for the CIB, COB performance was strongly constrained by OCT physics rather than by network architecture. In regions of pronounced hyper-transmission, the choroid–sclera interface becomes attenuated or disappears entirely, resulting in true signal ambiguity. As described in structural and histopathologic studies of GA, this washout of the deeper interface is an inherent consequence of RPE loss [12]. In such regions, even a theoretically ideal model would lack sufficient reflectivity cues to localize the COB consistently, limiting the achievable performance ceiling.

Third, the dataset consisted of SD-OCT volumes from a single imaging platform (Heidelberg Spectralis), all with relatively high acquisition quality. Device-specific differences such as axial resolution, sensitivity roll-off, and noise characteristics may affect the appearance of the COB. Previous work has shown that automated OCT algorithms may degrade when transferred across devices without retraining [9]. Broader validation across additional devices, including swept-source OCT with deeper penetration profiles, will therefore be required.

Finaly, all segmentations were evaluated in a curated dataset without the full range of real-world image artifacts (motion, low SNR, shadowing from media opacities). While registration and curvature normalization can reduce inter-scan variability, they cannot reconstruct a boundary that is not visible due to washout. Consequently, automated COB estimation in lower-quality OCT scans may be more error-prone.

Prospective Trajectories

Future work should focus on addressing the optical limitations that underline COB failures. Because the CSI progressively fades along the CAM-defined iRORA to cRORA spectrum, incorporating stage-specific annotations may help models learn how boundary visibility deteriorates with advancing atrophy in GA patients [21]. Additional progress will require artifact-aware or physics-informed approaches capable of distinguishing true boundary loss from hyper-transmission–related washout [12]. Broader training across multiple OCT platforms is also needed, as automated algorithms often show reduced performance when transferred between devices [9]. Together, these steps may reduce analysis time further, but human verification will still be required for clinical-grade outputs.

5. Conclusions

This study provides a failure analysis of automated choroidal boundary detection in GA eyes imaged with Spectralis SD-OCT. While the model reliably identified the CIB, COB detection was substantially limited by GA-related optical artifacts, particularly hyper-transmission in cRORA lesions. Despite these inaccuracies, the AI-assisted workflow reduced segmentation time from approximately seven hours to forty-five minutes per volume (1/10th), underscoring its value as an efficiency tool rather than an autonomous solution. Improved handling of deep signal loss, stage-specific modeling across the iRORA–cRORA spectrum, and broader cross-device training will be required before automated COB measurements can be used reliably in future GA studies.