Augmented Reality Device for Precision-Guided Tumor Resection via Depth-Based Markerless Registration

1. Introduction

Tumor resection with clear surgical margins is critical for reducing recurrence in oncologic surgery, but existing intraoperative guidance methods often fall short of providing sub-millimeter accuracy in real time. Augmented reality (AR) has emerged in recent years as a promising solution: for example, in AR-guided open liver surgery, multimodel segmentation methods combining RGB-D depth sensing with semantic segmentation and tracking achieved IoU of ~71-74% at 11-13 fps, showing feasibility but not yet achieving margin accuracy at surgical scales [1]. Similarly, EasyREG proposed a depth-based, markerless registration and tracking framework using the HoloLens 2, demonstrating robust registration and pose estimation using depth sensor corrections, human-in-the-loop region filtering, and curvature-aware sampling [2]. EasyREG has also illustrated the transformative role of AR devices in precision-guided interventions, providing the conceptual basis for applying depth-based markerless registration to oncology surgery [3]. Efforts such as deformable registration for head and neck tumor resection have integrated both pre- and post-resection surfaces to reduce target registration error by up to 33%, yet still suffer from relatively coarse relocation of specimen margins [4]. Hyperspectral imaging systems (e.g. SLIMBRAIN) have delivered real-time tissue classification overlapped onto point clouds to help delineate tumor boundary in neurosurgery, but their spatial registration and depth precision are constrained by camera resolution and latency [5]. AR reviews of perioperative visual guidance technologies have repeatedly identified key challenges: reliable image-to-patient registration without external fiducials, handling of tissue deformation, latency, and the trade-off among segmentation accuracy, computational cost, and frame rate [6]. Depth sensor benchmarking studies indicate that commercial depth sensors for intraoperative use can produce substantial error especially near surface edges or in presence of specularities, limiting their direct use for precise surgical margin guidance [7]. Moreover, although SLAM (Simultaneous Localization and Mapping)-based tracking has seen growth in AR navigation, its integration with refined depth data (e.g. via generative adversarial network (GAN)-based enhancement) remains unexplored in many surgical domains [8,9]. Taken together, past work has pushed forward AR guidance in tumor surgery in segmentation, tracking, deformation modeling, and markerless registration; yet gaps remain: many systems have not achieved sub-millimeter margin accuracy; experimental settings are often phantoms or small-animal models; latency and residual registration error in complex, moving, or deforming tissues remain problematic; and comprehensive evaluations quantifying margin preservation (residual tumor volume, margin accuracy) in in vivo trials are rare.

2. Materials and Methods

2.1. Sample and Study Area Description

This study used eight porcine models because their anatomical structures are similar to human tissue in tumor resection. Each animal was implanted with simulated tumor nodules sized 1.0–2.5 cm in diameter to serve as resection targets. All operations were performed in a controlled operating room environment under uniform lighting and sterile conditions. The AR headset system was applied during surgery, and tumor margins were defined by preoperative CT images registered with intraoperative surface geometry. All procedures were reviewed and approved by the institutional animal ethics committee.

2.2. Experimental Design and Control Setup

Two groups were included: the experimental group, which used the AR headset with depth-based markerless registration (n = 8), and the control group, which used conventional navigation with manual visual inspection and ruler-based guidance (n = 8). The control group was designed to reproduce the standard clinical workflow and provide a baseline for comparison. Both groups shared identical conditions in anesthesia, operating environment, and surgical staff. Trials were randomized, and the surgeon was not informed of the group assignment until the start of the operation.

2.3. Measurement Methods and Quality Control

Margin accuracy was determined by comparing histological measurements with pre-defined tumor boundaries. Residual tumor volume was calculated by three-dimensional reconstruction of excised specimens. Resection time was measured from the initial incision to the end of tumor removal. System latency was continuously recorded by the AR software. Quality control was ensured by using the same CT protocol for all cases (slice thickness 0.5 mm, resolution 512 × 512). The AR headset was calibrated before each trial. Two independent observers verified the measurements, and all records were double-checked.

2.4. Data Processing and Model Formulation

Data were processed with MATLAB and Python. Statistical analysis compared the experimental and control groups using paired t-tests for margin accuracy, resection time, and residual tumor volume. A linear regression model was used to describe the effect of system latency on margin accuracy [10]:

$$\mathrm{Y}={\mathsf{\beta }}_0+{\mathsf{\beta }}_1\mathrm{X}+\mathsf{ϵ}$$

where $\mathrm{Y}$ is margin accuracy, $\mathrm{X}$ is system latency, and $\mathsf{ϵ}$ is the error term. The residual tumor reduction rate was calculated as [11]:

$$\mathrm{R}={\displaystyle \frac{{\mathrm{V}}_{\mathrm{c}}-{\mathrm{V}}_{\mathrm{e}}}{{\mathrm{V}}_{\mathrm{c}}}}\times 100\mathrm{\%}$$

where ${\mathrm{V}}_{\mathrm{c}}$ is the mean residual tumor volume of the control group and ${\mathrm{V}}_{\mathrm{e}}$ is that of the experimental group. These calculations provided quantitative evaluation of the AR system’s effect on surgical performance.

3. Results and Discussion

3.1. Margin Accuracy and Residual Tumor Volume

The AR-guided group achieved a mean margin accuracy of 94%, compared with 78% in the control group (Figure 1). Residual tumor volume decreased by approximately 30% in the AR group. These results indicate that AR guidance improves the precision of resection. Similar observations were reported in Cancers, where tool-tip tracking enabled quantitative estimation of residual tumor and demonstrated clear differences in resection completeness between methods [12]. Our findings extend this evidence by validating depth-based AR registration in live animal trials.

3.2. Latency and Resection Efficiency

Latency was consistently below 40 ms in all AR trials, and mean resection time was reduced by 18% compared with controls. Figure 2 shows that margin accuracy decreased with higher latency, highlighting the importance of responsiveness. Sensors also emphasized the influence of system latency, where HoloLens-based ultrasound visualization suffered accuracy losses when delays exceeded tens of milliseconds. Our data confirm that low latency is crucial to maintain stable overlays and effective intraoperative guidance.

3.3. Comparison with Previous Studies

Most earlier AR studies relied on phantom or cadaver experiments and rarely reported residual tumor volume alongside latency and efficiency. In contrast, our work combines margin accuracy, residual volume, resection time, and latency into a unified evaluation. The Cancers paper provided residual quantification but did not include latency analysis, while the Sensors paper measured latency but did not quantify tumor margins[13,14]. By integrating both dimensions, our study bridges a gap between technical system performance and oncologic outcomes [15].

3.4. Clinical Implications and Future Perspectives

These results suggest that AR systems with markerless registration and depth refinement can increase surgical precision, reduce residual tumor, and shorten operation time. However, some residual tumor volumes remained, indicating challenges in handling irregular margins and tissue deformation [16]. Practical barriers include ergonomics of the headset, long-term calibration stability, and robustness in varied surgical environments. Future research should expand to larger preclinical or cadaveric cohorts, test diverse tumor geometries, and integrate intraoperative modalities such as ultrasound or MRI for deformation correction. Long-term studies will be necessary to establish whether these intraoperative improvements translate into reduced recurrence and better survival outcomes.

4. Conclusion

This study verified that an augmented reality headset with depth-based markerless registration, SLAM tracking, and GAN-refined depth data improved the outcomes of tumor resection. In vivo animal experiments showed higher margin accuracy, lower residual tumor volume, and shorter resection time, while system latency was kept below 40 ms. The combination of real-time depth refinement and markerless registration achieved sub-millimeter precision and provided clear scientific value for intraoperative navigation. These findings indicate potential application in clinical oncology where accurate margin preservation is essential to reduce recurrence. At the same time, difficulties remain in handling irregular tumor boundaries, tissue deformation, and device ergonomics. Further work should include larger samples, different tumor types, and integration with intraoperative imaging to enhance accuracy and reliability.