Surgical Phase Duration in Robot-Assisted Partial Nephrectomy: A Surgical Data Science Exploration for Clinical Relevance

1. Introduction

Surgical video recordings have become an indispensable tool in surgery and its use can be categorized into 3 main groups [1]:

• Surgical Education and Training: Video recordings provide a valuable resource for trainees to observe and learn surgical procedures from experts. By capturing critical moments during surgery, videos also allow for retrospective analysis and discussions among surgical teams and aid in complex decision-making. Literature reports have documented the utility of surgical video review in assessing anatomical landmarks, identifying surgical errors, refining surgical techniques [2], and the development [3] and assessment [4] of objective surgical skill metrics which can be used for training. Additionally, video-based platforms and telemedicine applications have enabled remote surgical education and mentorship, further expanding access to surgical training opportunities.
• Patient Outcomes Assessment: Surgical video recordings provide objective documentation of surgical procedures, enabling detailed analysis of surgical techniques, complications, and their impact on patient outcomes. Retrospective video analysis has been utilized in several studies to investigate patient outcomes [5].
• Quality Improvement Initiatives: By reviewing surgical videos, surgical teams can identify areas for improvement, refine surgical approaches, evaluate adherence to established protocols, and enhance patient safety[6]. Furthermore, video-based quality assurance programs have been implemented to monitor surgical performance, benchmark outcomes, and ensure standardized practices across institutions [7].

Uniform surgical phase definition and assessment is an enabling factor in many of the applications above. Surgical phase assessment allows for consistent and standardized indexing across procedures which facilitates video review, analysis and sharing. As video analysis, computer vision, artificial intelligence, and data-analytics (so called “surgical data science”) enter surgical theaters, the importance of correct and uniformly defined surgical phases increases. This is especially true for complex surgeries which require a decomposition into simpler phase blocks for objective assessment and review [8]. Previous work has also shown that phase definitions, even for non-complex procedures, are often not uniformly defined [9] and there is a clear need for common ontologies to unlock the full potential of surgical data science [10]. Up to now, researchers have been focusing on shorter and often linear procedures such as laparoscopic cholecystectomy[11], whilst more varying and complex procedures, which might benefit most from new data science insights, remain poorly investigated.

One such longer procedure with more inherent variation is robot assisted partial nephrectomy (RAPN). Partial nephrectomy itself is the gold standard treatment for patients with T1 renal cell carcinoma [12]. RAPN has become increasingly popular, amongst others because of its low morbidity and early convalescence when compared to open surgery [13] and its shorter learning curve when compared to laparoscopy [14].

Apart from the usefulness of surgical phase indexing for video manipulation and case revision, surgical errors are often related with, and dependent on, specific surgical phases [15]. As such, surgical phases form an initial and crucial step towards achieving automated and objective surgical skill assessment. A skill assessment system should first identify the phase it is in, before focusing on the errors inherent to that phase. Error reduction and skill enhancement through proficiency based progression as compared to classical training has proven to be beneficial for surgical skill acquisition and quantification [16]. Surgical phase duration has not yet been shown to correlate with surgical skill. When assessing clinical outcomes, full surgical duration has been investigated for correlations [17,18,19]. To our best knowledge, specific surgical phase duration and possible correlations with patient-specific clinical parameters has not been performed.

In this work, we firstly propose a vision-based framework for objective, uniform, and precise surgical phase definition during RAPN. The definition is based on previously developed and clinically validated metrics for a proficiency-based curriculum[4].

Secondly, we manually analyze 100 RAPN procedures to refine and optimize the previously defined visual cues. We compare our annotations to a commercially available online platform (Touch Surgery™ – Medtronic, Minneapolis, USA) which provides phase information when uploading videos to its online library without user input. We evaluate its correctness by comparing manually performed in-house annotations.

Finally, we perform the first statistical exploration of phase durations and patient characteristics/clinical outcomes in RAPN.

2. Materials and Methods

2.1. Data collection and visual cue definition

During January 2018 - June 2022, 100 transperitoneal RAPN procedures were collected in OLV Aalst Hospital (80 procedures) and Ghent University Hospital (20 procedures) after IRB approval. All procedures were performed on Intuitive Xi robotic systems (Intuitive™, California, USA).

We derive 16 visually distinctive phases from the validated RAPN metrics framework as developed by Farinha et al [3,4]. We define exact visual cues as starting points for surgical phases. Table A1 (Appendix) depicts the exhaustive phase list and corresponding visual cues. We adapt the existing framework, defined for left-sided RAPN, for common phases as present in both left and right-sided RAPN. We refer to this annotation framework as the Orsi-framework.

An in-house developed manual annotation tool is used, consisting of an in-browser viewer which allows to load a video, precisely define starting points of selected phases and export the timepoints with millisecond precision afterwards. An example of the manual annotation tool can be found in the Appendix (Figure A1). All videos were assessed by three medical students (MD, CVS, TO) after consensus on the visual definitions with a consultant urologist (RDG). The final manual student annotations were subsequently double-checked by a second consultant urologist (MPL).

2.2. Comparison to commercial software

The precise and more nuanced Orsi definitions are concatenated in 13 phases to be consistent with the Touch Surgery (TS) phase definitions for a one-to-one comparison. The phases consistency can be found in Table 1. Two medical students (KVR, AW) reviewed all videos for correctness after automated, rule-based concatenation.

All uploaded videos were automatically anonymized with removal of all possible identity clues, which include removal of out-of-body segments, as well as removal of segments with TilePro™ function (Intuitive™, California, USA) while they display patient-specific information such as CT, MRI or 3D model data [20]. The anonymization did not alter the case duration as these parts were blacked out, rather than cropped. Subsequently, videos were uploaded to the TS platform (software version February 2023). The Orsi-analysis was performed on full non-anonymized videos, under IRB approval.

After upload and processing, TS timepoints were manually extracted from the online commercial platform.

The Touch Surgery platform does not provide details on the phase definition rules. The videos underwent automated similarity analysis in which the manual Orsi annotations are compared to Touch Surgery annotations by use of a predefined loss function. This function objectifies how differently phases were annotated and detects procedures who have least similarity.

We base the loss function on a commonly used metric called Intersection-over-Union (IoU). For each phase, a binary vector is created which indicates 1 for seconds that belonged to that phase for the given annotation method and 0 elsewhere. The total sum of all phase vectors equals the total procedure time in seconds. The concept is visualized in Figure 1. The IOU per phase is then calculated as follows:

$$\mathit{I}\mathit{o}\mathit{U}(\mathit{p}\mathit{h}\mathit{a}\mathit{s}\mathit{e}\mathit{X})=\frac{\mathit{s}\mathit{u}\mathit{m}({\mathit{v}}_1 \mathit{A}\mathit{N}\mathit{D} {\mathit{v}}_2)}{\mathit{s}\mathit{u}\mathit{m}({\mathit{v}}_1 \mathit{O}\mathit{R} {\mathit{v}}_2)}$$

Where ${\mathit{v}}_1$ is the vector with the number of seconds spent in a certain phase X in the manual annotation and ${\mathit{v}}_2$ is the vector with the amount seconds spent in phase X according to the Touch Surgery annotation. Both are linked in a one-to-one temporal fashion as depicted in Figure 1.

As such, the numerator holds the number of seconds where both procedures “agree” on the investigated phase for the full procedural. The overlaps are summed as phases can reoccur. The denominator counts the number of seconds where the phase was indicated by at least one of both annotation methods. A perfect correspondence between both annotation approaches results in an IoU(phase) of 100%. To determine the final loss, we multiply all IoU(phases) and take the negative logarithm.

The loss function is described below:

$$\mathit{L}\mathit{o}\mathit{s}\mathit{s}=-\mathbf{ln}(\prod _{\mathit{p}\mathit{h}\mathit{a}\mathit{s}\mathit{e}\mathit{s}}\mathit{I}\mathit{o}\mathit{U}\left(\mathit{p}\mathit{h}\mathit{a}\mathit{s}\mathit{e}\right))$$

2.3. Clinical analysis

We explored correlations between phase durations and the following clinical parameters: age, gender, tumor complexity (PADUA score), tumor size, prior abdominal surgery, BMI, pT stage, histology, preoperative and postoperative renal function as measured by eGFR and creatinine. We also examined postoperative complications. Durations are assessed for both TS and manual (M) definitions. Correlation examination was performed by use of the Spearman’s, Kendall’s tau tests, as appropriate. In case of statistically significant correlation, a linear regression model was then built to analyze the relationship between the identified variables. All the analysis were performed by using Jamovi software v2.3.

3. Results

3.1. General Phase data analysis and comparison

Touch Surgery does not provide phase assessment in case of cystic renal lesions, despite it being a similar technical approach. As such, the resulting final dataset for the side-by-side comparison was diluted to a total of 86 RAPN procedures. Table 2 shows the descriptive statistics for our concatenated manual (M) phase analysis and the TS analysis for 86 analyzed procedures.

At first sight, phase duration in both platforms are quite different. When looking at similarity between phase definitions on the full dataset, the 15 RAPN procedures with the least similarity (consistent with a loss lower than 29) were analyzed in depth for phase assessment differences. Figure 2 shows the loss for all procedures.

The following recurring discrepancies were withheld in poorly corresponding videos. Although port insertion and surgical access was not always recorded, it was not annotated in TS in half of the cases, leading to a longer duration of ‘Colon Mobilization’. From ‘Colon Mobilization’ onwards, we do note that in general the same number of phases are annotated. A clearly differing visual definition is present in ‘Kidney mobilization’ and ‘Tumor identification’. Ultrasound use and guided demarcation forms an important part of tumor identification and in certain procedures, ultrasound (US) is initiated before opening of Gerota's fascia. In the Manual phase analysis, this US usage is considered a subphase of 'Tumor identification' and thus marks the beginning of this phase. Touch Surgery does not always recognize US usage as part of 'Tumor identification' yet, resulting in a missed phase initiation. Touch Surgery recognizes the start of the 'Tumor identification' phase when the US is used for the second time when the renal parenchyma with tumor zone is already freed from peri-renal fat. This also results in longer ‘Tumor Identification’ phases in the manual assessment. As the manual assessment is more nuanced and the definition is different, a side-by-side comparison on accuracy is irrelevant. This differing definition impacts all phase durations before the onset of hilar clamping and tumor excision, which makes phase duration comparison for other phases before ‘Hilar Clamping’ unreliable.

We do note that combining all surgical manipulations before ‘Hilar Clamping’, this is the combination of ‘Port Insertion and Surgical Access’, 'Colon Mobilization', 'Identification of Anatomical Landmarks', 'Hilar Dissection' and ‘Kidney Mobilization’, does have very similar mean and median durations (means are 76,5 and 76,3 minutes, medians are 62 and 64,9 minutes for respectively M and TS assessment).

The next important surgical landmark entails tumor excision, which is most often performed after hilar clamping. We see here that of all 100 procedures, 77 procedures were performed off-clamp. Touch Surgery did not indicate tumor excision and tumor retrieval in 2 cases due to our anonymization protocol. Both cases had patient-specific CT scan or 3D model info in the console view which was anonymized before upload. Despite being a clear drawback for comparison, these phases should not impact median times, where we note similar trends. The combined duration of ‘Hilar clamping’ and ‘Tumor Excision’ is very similar. We find mean and median combined values of respectively 9,2 and 6,9 minutes for manual assessment; 9,9 and 8,6 minutes for TS assessment. The main factors attributing to longer ‘Tumor Excision’ times in TS are twofold. TS includes application of hemostatic agents, which is excluded in our manual assessment and the TS ‘Tumor Excision’ phase starts as soon as the kidney is manipulated after clamping, where in our assessment it only starts when the blunt dissection takes place. This also results in a longer manual ‘Hilar Clamping’ duration.

After tumor excision, we enter the last procedural part in which the kidney is reconstructed after specimen retrieval. ‘Specimen Retrieval’ has very similar durations, nonetheless we note that, despite our best efforts, both our manual assessment and TS missed to annotate one specimen retrieval, albeit in different procedures. Similar to ‘Hilar Clamping’, the ‘Hilar Unclamping’ phase is once more shorter in TS. The main attributing factor is the rennoraphy starting point definition. In TS, this phase starts when needles are brought closeby the parenchyma and thus this ends the unclamping. In our assessment, this phase only starts at the first parenchymal suture. This is also reflected in shorter renorraphy times for our assessment and confirms the narrower definition. ‘Specimen retrieval and Closure’ has a similar time range and includes the renal retroperitonealisation. The ‘Operation Finished’ definition is more precise in our assessment and marks when the camera is removed from the abdomen for the last time, whilst for Touch Surgery this is simply the end of the video, resulting in an empty phase duration.

3.2. Clinical data correlation exploration

Prior abdominal surgery did not significantly impact any of the phases’ duration, independent of the phase assessment methodology (TS or M). Likewise, no significant impact was found for age, gender, pT stage, histology type or pre/post-operative renal function. The post-operative complication rate was too low to find any correlations.

Spearman’s test showed a significant correlation (p=0.011) between patients’ BMI and the ‘Tumor Identification’ duration for manually assessed cases only. Figure 3 shows the linear regression model. Increased BMI results in increased ‘Tumor Identification’ periods (p=0.011).

When assessing tumor complexity according to the PADUA scores, we see that ‘Renorrhaphy’ duration has a significant positive correlation with PADUA score (p<0.001 for both Touch Surgery and manual assessment). Likewise, ‘Tumor Excision’ was significantly correlated with PADUA scores, nevertheless only in manual assessment (p<0.001). At linear regression, PADUA score confirmed a positive correlation with ‘Renorrhaphy’ duration (p<0.001 for both TS and M assessment, Figure 4a and 4b, respectively). Figure 4c depicts the positive correlation between manual assessment of ‘Tumor Excision’ and Padua score (p<0.001).

4. Discussion

• Visual clue definition

Defining visual clues for phase analysis is a repetitive process in which the multicentric approach allowed for a more robust and precise definition. Several iterations where required before agreeing on a final template as provided in Table A1, which was then used for full dataset investigation. Despite our best efforts in checking all procedures fivefold, post-hoc data analysis revealed we still missed annotating one phase. As such, phase annotation is a time-intensive effort which reveals the clinical need for intelligent systems which can support this automatically and consistently.

• Platform comparison

When comparing our manual platform and the Touch Surgery platform for RAPN phase assessment, we identify three key items:

Firstly, the lack of analyzing renal cysts in Touch Surgery is an apparent drawback as it immediately discards 14% of the cases, despite having identical surgical technique and approach.

Secondly, when analyzing both platforms for the remaining 86 procedures, we identify the lack of uniform and transparent definition as the main inhibitor for in depth side-by-side comparison. Touch Surgery does not provide details on the phase analysis process. It is unknown whether the assessment happens fully autonomous, semi-autonomous or fully manually. In the latter case, nothing is known about the expertise level of annotators, their training, or their interrater reliability. The platform provides no information on the used visual phase cues. On the other hand, no public consensus exists for objectively defined visual phase cues. Despite having based our definition on a published consensus for phase metrics, defining visual cues requires more details and as such there is a clear mismatch between both definitions.

Thirdly, assessment of the most differing procedures as identified by the loss function, shows that phase combinations and evaluation of specific subphases results in greater similarity. As such, large surgical entities are annotated in a similar fashion. When assessing the full procedure, 3 major operative parts (preparation for tumor excision, tumor excision and the hemostasis and closure) were found to be similar. Nevertheless, they are insufficiently granular for immediate outcome correlation research.

• Clinical outcomes

BMI and ‘Tumor Identification’ duration correlate positively, which can be suspected as higher BMI often involves more intraabdominal fatty tissue. As the kidney is surrounded by perinephric fat, tumor identification on the renal surface can indeed be more tedious. This correlation was not withheld in the TS definitions. This can be attributed to the apparent TS definitions of only annotating ‘Tumor Identification’ when the kidney is already fully exposed. Another attributing factor is our removal of our echography TilePro™ segments in the TS group for anonymization purposes. Nevertheless, these segments were often correctly picked-up inside the TS platform, starting when ultrasound entered the body and ending upon ultrasound removal.

When tumor complexity rises, e.g. because of larger size or endophytic nature, as expected tumor excision duration also increases. Likewise, ‘Renorraphy’ duration increases with tumor complexity. Big or deep tumor can indeed leave large or difficult to reach renal resection beds, which can effectively be more difficult to reconstruct or obtain adequate hemostasis. Nonetheless, it is noteworthy that these correlations are only withheld in manual assessment. The missing correlation for TS might be attributed to the broader ‘Tumor Excision’ definition which includes hemostatic agent applications. Touch Surgery retrieved statistical significance with renorraphy duration only. We do note that 2 out of 86 cases were irrelevant for tumor excision comparison, given the pre-upload anonymization. Furthermore, the manual data analysis was performed on 14 cases more.

5. Conclusions

When assessing surgical phase duration for clinical relevance, a very nuanced description and definition of phase starting points is required. Metrics based training curricula can help in identifying the clinically most relevant phases. Sufficient granularity in surgical phases annotation is a stringent requirement for thorough exploration of clinical relevance. Currently, due to its blackbox nature and its omission of cystic lesions, the Touch Surgery platform cannot support research aimed at evaluating clinical outcomes or surgical performance in RAPN using proficiency-based metrics. Commercially generated annotations can however provide general surgical information and are expected to facilitate surgical video navigation outside clinical trials.

Assessment according to our metrics-based definitions resulted in more clinical correlations. Firstly, this strengthens our idea that, despite their practicality and ease of use, commercial annotations are not ready for primetime use in investigating possible clinical correlations. Secondly, this suggests that expert consensus methods for choosing phase definitions such as derived from metrics-based training curricula can help in standardization. Given the lack of visual phase definition consensus, this manuscript might act as a guide towards better standardization for future phase analysis projects in RAPN. Likewise, we hope it inspires future projects in robot-assisted surgery assessment.

As this work explores surgical phase duration correlation with patient characteristics, it is important to note that correlations do not imply causations and that the value of this work is more hypothesis generating. The retrospective nature of this study further adds to this.

Lastly, this work can serve as the necessary basis for automated phase detection in RAPN. Here, deep learning computer vision algorithms take in phase duration data combined with video data to automatically assess and define surgical phases. This is in turn an enabler towards fully automated surgical scene understanding [21] which unlocks a myriad of other possible applications. Given the time-intensive nature of manual annotations, automated phase detection can also pave the way for larger patient cohort investigations. This is a needed step towards gathering more evidence.

6. Patents

Not Applicable.