Improved Depth-Based Tracking in Augmented Reality-Assisted Neurosurgery

1. Introduction

Accurate registration during neurosurgery is often affected by brain shift, occlusion, and tissue deformation, which reduce the accuracy of preoperative models when used in the operating field [1]. In recent years, depth sensing has been used for surface capture, non-rigid registration has been refined, and biomechanical models have been introduced to deal with intraoperative changes [2]. Deformable versions of the iterative closest point (ICP) algorithm improve the alignment of point clouds but may fail under partial visibility or uneven deformation [3]. Depth filtering reduces noise but may also smooth out important anatomical edges that are needed for alignment [4]. Finite element modeling (FEM) gives a physical description of tissue mechanics and has been applied in image-guided surgery to predict brain shift, but real-time use is limited by high computation and sensitivity to parameters [5]. Registration across modalities, such as depth or ultrasound with MRI, improves localization, but performance decreases when the cortex is blocked or when tumor resection changes tissue structure [6]. Learning-based methods, such as surface descriptors and graph models, give stronger features for alignment but require large datasets and may not generalize well across different systems [7]. Many existing studies also use small phantom or cadaver datasets, report inconsistent metrics, and lack fair baselines, which limits clinical translation [8]. Building on the challenges identified in EasyREG [9], particularly the handling of complex anatomical deformations, our work explores deformable modeling for neurosurgical navigation. To address these issues, this study proposes an improved depth-based tracking method that combines adaptive bilateral filtering to reduce noise while keeping anatomical edges, integrates deformable ICP with FEM-predicted displacement fields, and updates the alignment with preoperative MRI surfaces in real time. The goal is to reduce error and improve stability under brain shift and occlusion while keeping a frame rate suitable for augmented reality (AR) guidance, and to provide a safer and more efficient solution for tumor resection.

2. Materials and Methods

2.1. Sample and Study Description

This study used 10 brain phantom models to simulate neurosurgical procedures. Each phantom included cortical surfaces and tumor-like inclusions, with elastic properties close to real brain tissue. Controlled shifts of up to 15 mm were introduced by deforming the phantom base to reproduce intraoperative brain shift. Preoperative MRI scans were obtained at a voxel resolution of 0.6 mm as reference data. Depth images were collected with a structured-light sensor positioned 40–50 cm above the phantom under stable illumination. This setting created experimental conditions similar to those in actual surgery.

2.2. Experimental Design and Control Setup

The experiments were conducted in two groups. The experimental group used the proposed method that combined adaptive bilateral filtering, deformable ICP, and finite element modeling (FEM). The control group used standard ICP without FEM correction. For each phantom, 10 trials were performed under different magnitudes of brain shift. The purpose was to compare the baseline ICP with the enhanced method under the same deformation conditions. Target registration error (TRE) was the main evaluation index, while frame rate and alignment stability were also measured [10].

2.3. Measurement Methods and Quality Control

Surface points were extracted from depth images after adaptive bilateral filtering. Deformable ICP was applied to align intraoperative point clouds with MRI-derived surfaces. FEM simulations generated displacement fields to correct correspondences during alignment. TRE was measured as the Euclidean distance between predicted landmarks and MRI ground-truth landmarks [11]. Two observers independently marked landmarks, and differences were resolved by agreement. Each experiment was repeated three times, and mean values with standard deviations were reported. Latency and frame rate were measured on an NVIDIA RTX 3080 GPU to check real-time performance.

2.4. Data Processing and Model Formulation

Data processing was carried out with deformable ICP combined with FEM updates. Registration accuracy was assessed using the root mean square error (RMSE), defined as [12]:

$$\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}=\sqrt{{\displaystyle \frac{1}{\mathrm{N}}}\sum _{\mathrm{i}=1}^{\mathrm{N}}\backslash \mathrm{b}\mathrm{i}\mathrm{g}\left[\right({\mathrm{x}}_{\mathrm{i}}-{\widehat{\mathrm{x}}}_{\mathrm{i}}{)}^2+({\mathrm{y}}_{\mathrm{i}}-{\widehat{\mathrm{y}}}_{\mathrm{i}}{)}^2+({\mathrm{z}}_{\mathrm{i}}-{\widehat{\mathrm{z}}}_{\mathrm{i}}{)}^2\backslash \mathrm{b}\mathrm{i}\mathrm{g}]}$$

where $\mathrm{N}$ is the number of landmarks, and $({\mathrm{x}}_{\mathrm{i}},{\mathrm{y}}_{\mathrm{i}},{\mathrm{z}}_{\mathrm{i}})$ and $({\widehat{\mathrm{x}}}_{\mathrm{i}},{\widehat{\mathrm{y}}}_{\mathrm{i}},{\widehat{\mathrm{z}}}_{\mathrm{i}})$ are the ground-truth and registered coordinates.

The FEM model was expressed by minimizing the total potential energy [13]:

$$\mathsf{\Pi }\left(\mathrm{u}\right)=\mathrm{f}\mathrm{r}\mathrm{a}\mathrm{c}12{\mathrm{u}}^{\mathrm{T}}\mathrm{K}\mathrm{u}-{\mathrm{u}}^{\mathrm{T}}\mathrm{f}$$

where $\mathsf{\Pi }\left(\mathrm{u}\right)$ is the potential energy, $\mathrm{K}$ is the stiffness matrix, $\mathrm{u}$ is the displacement vector, and $\mathrm{f}$ is the external load. The displacements obtained from this minimization were applied to update ICP alignment, combining geometric accuracy with physical consistency.

3. Results and Discussion

3.1. Error Performance Under Obstruction

Figure 1 illustrates the behavior of target registration error (TRE) across multiple obstruction scenarios in phantom-based alignment tests. The results show that as obstruction severity increases—from “no obstruction” to combined forehead and endotracheal tube conditions—the TRE gradually rises from ~0.9 mm to ~1.5 mm. This pattern is consistent with earlier reports in phantom-based neuro-navigation studies, where obstruction introduces measurable but manageable degradation in accuracy [14,15]. The observed increase remains below clinically acceptable safety thresholds, confirming the robustness of the depth-based method even under challenging visibility conditions

3.2. Comparative Registration Accuracy

Figure 2 compares registration accuracy between the proposed adaptive registration strategy and conventional surface matching. The automatic image registration (AIR) method consistently achieves lower errors, averaging ~0.8 mm compared with ~1.6 mm for surface matching, as documented in Sensors. This confirms that integrating adaptive weight optimization with deformable models yields higher precision, particularly in scenarios with partial occlusion or dynamic tissue displacement [16]. Our results mirror these findings, with improvements in alignment stability exceeding 35% compared to baseline techniques.

3.3. Cross-Study Consistency and Significance

The results presented in Figure 1 and Figure 2 highlight consistent evidence across independent studies: registration error remains sensitive to obstruction and deformation, but advanced algorithms markedly mitigate these effects. Recent studies on AR surgical guidance confirm that adopting FEM modeling, ICP refinement, and adaptive filtering can effectively reduce TRE to below 1.2 mm in phantom conditions. These outcomes surpass conventional navigation benchmarks, which often exceed 2 mm under similar test settings. By situating our findings within this broader body of evidence, we establish the generalizability and robustness of the proposed method for neurosurgical guidance.

3.4. Clinical Implications and Limitations

The findings suggest that our enhanced registration framework maintains clinically acceptable error margins even under obstruction and tissue shift, which is crucial for safe tumor resections and delicate skull base procedures [17]. However, phantom experiments cannot fully capture in vivo complexities such as fluid shifts, bleeding, and tool interference. Furthermore, sample sizes in both our study and the cited references remain limited, warranting larger-scale cadaveric and clinical trials. Nevertheless, the alignment stability demonstrated here positions the proposed framework as a clinically scalable solution with the potential to reduce preparation time and improve intraoperative reliability.

4. Conclusions

This study demonstrated that the enhanced registration framework significantly improved accuracy and stability in neurosurgical navigation by integrating multi-modal imaging, deformable modeling, and adaptive alignment strategies. The experimental results confirmed its ability to maintain sub-millimeter target registration error under obstruction and tissue deformation conditions, highlighting its innovation in compensating for brain shift while reducing preparation time. These findings provide scientific evidence that markerless augmented reality navigation can improve surgical safety and efficiency, with strong potential for clinical translation. Nevertheless, the limitations of phantom-based validation and relatively small sample size indicate that further investigations involving cadaveric and in vivo trials are required to fully establish generalizability and long-term reliability.