4.1. Curriculum Development – The Apprentice Model
Traditionally, open surgical training represented an apprenticeship model. Trainees observed and assisted their surgical mentors in performing a procedure. Through mimicry and direct supervision, trainees gradually acquired operative skill and increasing independence. Competence was subjectively determined by surgical mentors throughout training, with little in the way of objective procedure-specific assessment of technical proficiency. However, modern surgical training poses significant challenges to this model. Restricted working hours, increased subspecialisation and a rapid expansion in technology and procedures dictate that this model is no longer able to guarantee training adequacy. Furthermore, as established surgeons adopt new technologies, it is imperative that gold standard outcomes are maintained through thorough training. These issues are compounded by a lack of standardisation in hospital accreditation processes and pressures from patients, peers, media, medical technology companies and healthcare administrators. Thus, surgical training models must adapt in order to maintain standards of care. Zorn and colleagues highlighted the extent of the problem in 2009, noting that an estimated 85% of radical prostatectomies in the US the preceding year were performed with robotic assistance despite the lack of any formalised accreditation or training process [
1].
Currently, Intuitive Surgical provides a recommended framework for robotic accreditation, including online learning, in-service training, bedside assistance and primary operating [
2]. Trainees subsequently provide a letter supporting their robotic competence from their supervisor, at which point Da Vinci will issue a certificate of system training. Completion of Da Vinci surgical simulator skills sessions are also recommended, though not required. However, one must note that this certificate is only intended to show competence in the use of the robot itself. The ability to perform a range of procedures requires far greater surgical experience with in-depth knowledge of anatomy, tissue handling and pathophysiology.
In the past decade there has been a deluge of studies reporting on the robotic learning curve and its potential utility in determining competence. A recent systematic review by Soomro et al. showed that the majority of literature was of poor quality with a large variety of outcome measures and methodologies rendering it difficult to make any meaningful conclusions for the purposes of implementing safe training in robotic MIS [
3]. The challenge lies in determining at what point an individual has reached sufficient proficiency to practice independently without undue harm to patients and it seems that case numbers alone are insufficient to determine this.
Therefore, considerable effort has been placed into the development of structured training programmes for the safe implementation of robotic surgery. The Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) and the Minimally Invasive Robotic Association (MIRA) first published a consensus statement regarding guidelines for training and credentialing in 2008 [
4]. These guidelines emphasised the importance of a combination of didactic teaching, live case observation and hands-on experience, both of simulations and in vivo, though specific indicators of proficiency were not discussed. Lee and colleagues expanded on this concept, publishing best practice guidelines for robotic training and credentialing in 2011 [
5]. They divided the training process into preclinical and clinical phases. Preclinical training involved didactic teaching and online learning modules followed by acquisition of basic robotic skills through dry labs and simulation. Trainees subsequently graduated to the clinical phase of teaching which progressed through case observation, live cases and expert discussion, bedside assisting, and finally operating as console surgeon with procedural breakdown into measurable steps of increasing complexity. Critically, the authors recognised the inadequacy of using case numbers to determine competence. They therefore recommended an objective, outcomes-based assessment of proficiency prior to credentialing. These recommendations form the backbone of modern robotic training curricula.
This has been further refined over time. In 2014, 14 multinational surgical societies developed the “Fundamentals of Robotic Surgery” (FRS) curriculum [
6]. This programme is web-based and has the formidable aspiration of generalisability to any robotics platform and any surgical discipline. It includes 25 perioperative outcome measures and is divided into 3 sections: cognitive skills, psychomotor skills and team training and communication. Assessment is competency-based rather than time-based, with trainees required to reach benchmark “pass” values to complete the course. A multicentre RCT showed that significant improvements in task completion time and error rate were made following FRS training. These were comparable to a control group consisting of surgeons who had completed their own local institution specific training, confirming that the FRS was at least comparable with other common forms of training [
7].
Alternative validated training curricula include the Robotic Training network (RTN) and the Fundamental Skills of Robotic Surgery (FSRS). However, these have their own limitations including limited international availability and, for FRS, requirement for the specific RoSS surgical simulator [
8,
9].
While online curricula are more accessible and offer greater flexibility to participants, a perceived benefit of on-site training is the ability of supervisors to give advice on correcting technique and thus improve efficiency. However, this supposition has been challenged by a study comparing expert preceptorship with an educational video on skills acquisition [
10]. Both groups showed significant improvements with training and no significant differences between groups. Thus, although direct comparisons between each training curricula have not been performed, this is unlikely to be of clinical significance. The most important factor is the completion of one form of validated, proficiency-based skills curriculum prior to embarking on further robotics training. The ‘best’ programme for any given surgeon is likely that which they have ready access to.
The robotic section of the EAU, known as ERUS, ultimately published the first standardised 12-week robotic training curriculum in 2015 [
11]. The curriculum included an initial e-learning module on principles of robotics followed by operative observation and assisting, simulation-based training incorporating VR simulation, dry and wet lab activities and supervised, modular training with a progression through increasingly complex steps as proficiency increased. All participants showed significant improvement in dVSS simulator performance through the course of the training program. Face, content and construct validity were all confirmed. However, only 80% of participants were deemed competent to independently perform robot assisted radical prostatectomy (RARP) by their mentors at the conclusion of the programme. Independent assessors similarly scored 80% as safe and competent. Further, expert mentors felt that only 30% were capable of safely and independently completing a complex case. This once again highlights the inherent variability in learning curve and the importance of using objective performance-based outcomes over case numbers to determine competency.
The following year, the Clinical Robotic Surgery Association (CRSA) published specific recommendations on structured training in colorectal surgery [
12]. These guidelines focused on a stepwise objective assessment of competency. Basic training is divided into sequential stages and each must be passed objectively before being allowed to progress to live operating. Each operation is broken down into steps and trainees are rated as independent, requiring prompting, or unable to perform for each step. Procedures are taught in order of increasing complexity until the trainee is ultimately deemed independent and credentialed.
Numerous surgical societies have subsequently followed suit and published their own guidelines for safe introduction of robotic surgery. However, the recommendations of the CRSA arguably represent the greatest paradigm shift toward objective, procedure specific, proficiency-based accreditation to date.
4.2. Novel Training Modalities
Having now defined the aims of robotic training, one must examine the most effective methods for achieving proficiency while minimising risk of harm to patients. Fortunately, robotic surgery is uniquely suited to innovative new training models.
4.2.1. Virtual Reality
While VR is not a new concept, having first arisen in the 1980s, imaging technology and processing power have only recently advanced sufficiently to allow virtual reality to enter the mainstream. Driven by video game development, there are now numerous open surgical simulators based on the oculus and HTC platforms. However, much of the skill in open surgery centres around tissue handling and manipulation that is well beyond the capabilities of current platforms, severely limiting their utility in open surgical training. Similarly, in performing laparoscopic surgery there remains a significant component of haptic feedback and off-screen movement that influences surgical technique and is difficult to simulate. By comparison, modern robotic surgery lacks haptic feedback. While this is often described as a limitation in current robotic surgery, for the purposes of simulation and training, these become strengths. Technique is entirely guided by on-screen visual cues and, therefore, simulation has the potential to most accurately reproduce the robotic operative environment.
First generation VR simulators include the DaVinci Skills Simulator (dVSS), Mimic DaVinci trainer (dVT) FlexVR, the ProMIS simulator, the Simsurgery Educational Platform (SEP) and the Robotic Surgical Simulator (RoSS) [
13]. These platforms focus primarily on basic skill acquisition through skills drills, though a limited degree of operative simulation is offered. Second generation platforms include the RobotiX Mentor (RM), the RoSS II/II lite, and SimNow by Da Vinci [
13,
14,
15]. These next generation platforms greatly expand the utility of simulation. Photorealism is vastly improved and it is now possible to perform complete operative procedures in the simulated environment, either in guided fashion or free-hand with in-depth assessment scores on completion. RoSS also offers an alternative unique module known as Hands-on Surgical Training (HoST). Here, surgical videos from real robotic procedures are displayed on screen. Through haptic feedback of the controls, the trainee is able to experience the exact hand movements of the operating surgeon in synchrony with the surgical footage [
14].
In order to be of value, simulation must lead to a demonstrable improvement in surgical skills. This may be described across several validity domains
(Table 1).
The efficacy of virtual reality simulation in robotic skills training is well established. Each of the aforementioned simulators has demonstrated face, content and construct validity in randomised controlled trials across individuals with varying levels of surgical ability [
13,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29,
30,
31,
32].
In considering the superiority of one simulator over another, few head-to-head comparisons have been performed. Hertz and colleagues recently compared face and content validity between dVT, dVSS and RM. The dVSS was found to be superior to dVT, while no other significant differences were identified between platforms [
33]. This potentially reflects the aging nature of dVT, now in use since 2007, and the use of the master console with dVSS, improving simulation realism. A further comparison performed by Tanaka et al. compared face, content and construct validity between dVT, DVSS and RoSS [
34]. dVSS and dVT both significantly outperformed RoSS in face and content validity. Additionally, dVSS and dVT both showed good construct validity, while RoSS was unable to distinguish between novice and expert users. Further support for the dVSS simulator was published by Ahmad and colleagues who showed that fellows trained on the dVT scored significantly lower on both pre and post-test assessment, had a lower average curriculum score and spent more time completing each assessment.
These results appear to translate to meaningful skill acquisition. Hoogenes et al. compared the performance of junior and senior trainees in performing vesicourethral anastomoses following completion of an identical simulator-training curriculum on dVT or dVSS [
35]. Junior trainees showed significantly better performance following dVSS training compared with dVT training, while senior trainees who had greater previous robotic experience showed no difference in performance between programmes. Therefore, it seems that dVSS may represent a superior training tool when compared to dVT, particularly in robotic novices. Unfortunately, there is no data comparing the next generation simulators described above with which to make a similarly informed decision.
Of note, the majority of aforementioned studies focus on face, content and construct validity in performing basic skills such as object manipulation and knot tying. While such tasks are attractive for research purposes due to their simplistic nature and ease of outcome comparison between groups, they do not necessarily reflect the transfer of skills to the operating room (concurrent and predictive validity) which is of critical importance in determining the value of simulation in training. A recent meta-analysis by Schmidt et al. has examined the role of simulator skills transfer to live operating, identifying 8 studies for review [
36]. They concluded that VR skills acquisition is transferrable to the OR and that simulator performance on DVT and dVSS demonstrates concurrent validity, although their findings were limited by the small number and heterogeneous nature of included studies.
Furthermore, logic dictates that realistic operative simulation is likely to dramatically increase the utility of virtual reality platforms. This technology is already available and will only continue to advance in coming years as processing power and imaging technology allow true photorealistic re-creation. To date, this has only been assessed in one randomised trial by Raison and colleagues [
37]. Novice participants received no training (n=9), basic simulation training (n=13) or procedural simulation training (n=13). Subsequently, each participant performed robotic radical prostatectomy on a cadaver and was assessed by blinded expert reviewers. Completion of either training model resulted in significantly higher scores than no training, demonstrating concurrent validity. Furthermore, procedural training resulted in significantly higher scores than basic training, demonstrating superiority as a training tool.
4.2.2. Animal/Cadaver Models
Of course, there are alternatives to virtual reality simulation that must also be considered in determining the most appropriate teaching tool for robotic surgery. Animal and cadaveric models are well recognised in surgical training [
38,
39]. These models represent the most realistic form of surgical simulation in terms of anatomical authenticity and tissue handling properties. This is particularly true of living tissue handling in animal models, though at the expense of anatomical variance. To bridge this divide, cadaveric models may be enhanced with re-establishment of simulated perfusion, as outlined in a systematic review by Bellier et al. [
40]. However, despite their longstanding usage and validated nature, direct comparisons between animal, cadaveric and VR simulation in MIS are severely limited. Just two papers comparing laparoscopic VR with cadaveric simulation have been published [
41,
42]. Both focused strongly on subjective participant satisfaction, finding a preference for cadaveric training. VR simulation found greater acceptance for basic task training with junior trainees, and was found to be less complex than cadaveric dissection in procedural assessment.
Specific to robotics, literature is similarly limited. Bertolo and colleagues conducted a single robotic training session on fresh-frozen human cadavers for surgical residents with limited robotic experience [
43]. They found a high degree of satisfaction amongst participants, who showed subjective and objective skill improvement following the session. Furthermore, participants rated the activity as superior to both VR and porcine training sessions.
Thus, very limited evidence may suggest an advantage to cadaveric simulation, particularly for more complex procedural tasks. However, as noted above, VR technology has advanced dramatically in the past decade and the relevance of the above results when compared to more modern VR simulators is unknown.
4.2.3. 3-D Printing
Another novel alternative has been the construction of artificial models through 3D printing and polymer moulding. Several studies have recently assessed the value of 3D printed models in robotic training. These have demonstrated the face, content and construct validity of such models [
44,
45,
46].
As with VR, modelling techniques have greatly advanced in recent years. Significant focus has been placed on the realism of tissue reproduction, paving the way for valuable, high fidelity surgical simulation. Models can now conduct diathermy in a realistic manner and can ‘bleed’ due to artificial perfusion with solutions of similar viscosity to blood. In addition to creating realistic models for training purposes, patient specific models have been developed from CT reconstructions, allowing 3-dimensional tumour assessment to assist in operative planning, and even to rehearse procedures. This was first demonstrated by von Rundstedt et al. who showed that operative times and tumour characteristics were remarkably similar when comparing 3D printed complex renal tumours to the in vivo specimen [
47].
Further work was performed by Ghazi and colleagues in establishing the validity of such high-fidelity organ models. Their models allow ultra-realistic simulation of entire surgical procedures within a replica abdomen or pelvis [
48]. Reviewers consistently rated these models as superior to porcine or cadaveric models and perfusion was considered to be a particularly important element. Experts significantly outperformed novices in performing RALPN over a variety of validated scoring systems, confirming the construct validity.
Each of these papers serve as an impressive demonstration of how far 3D printed simulation has progressed and its great potential for teaching and operative planning. However, animal, cadaveric and 3D models are not without other practical limitations and VR simulators demonstrate several benefits here. Training in animals and cadavers is costly, there is limited availability and there are many ethical considerations. A complete Da Vinci robot must be available in a wet laboratory environment and instruments must also be made available at considerable cost. As a result, access to training with these models is greatly restricted, and is primarily only available in dedicated teaching institutions. In the case of 3D printed models, the requirement for a complete robot and training instruments remains a limitation, though instrument sterility and infection control issues are eliminated. Models can only be used once, consume considerable time and resources in preparation and generate significant waste. The more complex and realistic the model, the greater the effort required in manufacture. In fact, Witthaus et al. noted that each model took a skilled biomedical engineer approximately 5.5hrs to construct. Therefore, commercialisation is likely required in order to develop mainstream utility. Unfortunately, this often comes at considerable additional cost, particularly in the case of medical technology. By comparison, although initial purchase costs are high, VR simulators may subsequently be made available 24 hours a day, 7 days a week. They can be reused an unlimited number of times with virtually no operational costs. The main limitation is in the fidelity of procedural simulation. As VR comes ever closer to achieving photorealism, it is highly likely to declare itself as the predominant training modality.
4.2.4. Dual-Console Training
Following graduation from simulation, trainees must transition to in vivo console operation. During the early transition phase, the presence of an on-site preceptor is considered critical. This individual should be an experienced robotic surgeon, able to offer guidance and supervision to maximise patient safety. With the initial Da Vinci platform, this was a cumbersome process, as described by doctors Crawford and Dwyer [
49]. In addition to the provision of verbal advice, the preceptor would often act as bedside assistant, providing the ability to assist or point laparoscopically. However, the preceptor was commonly required to “break scrub” and temporarily act as console surgeon, a time consuming and frustrating process. Fortunately, progressive technological advancement has significantly enhanced the means of interaction between supervisor and mentee since this time.
Hanly and colleagues first described the concept of dual console platforms in 2006, linking two surgeon consoles together via a special-purpose connection [
50]. This enabled collaborative teaching through the fluid exchange of instrument control between the trainer and trainee, in addition to providing a form of haptic feedback, allowing both surgeons to simultaneously feel the movement of the instruments. Thus, the trainee could be guided in the performance of precise tasks such as intracorporeal suturing. This teaching model was much more akin to open teaching techniques than those employed at laparoscopy and was subjectively perceived as highly advantageous. It offered vast improvements in the ability to teach robotic surgery efficiently while reducing some of the associated anxiety involved in teaching MIS. Several studies have subsequently reported the safety and training benefits of the dual-console model [
51,
52,
53].
4.2.5. Augmented Reality
The second major advance in robotic teaching was the introduction of telestration [
49]. This likely represented the first practical application of augmented reality (AR) in robotic surgery. Telestration enabled the preceptor to direct the trainee by marking on the laparoscopic image on a touch-screen display. The markings were then reproduced on the surgeon console display. Alternatively, in the dual console model, the preceptor was given the ability to control a virtual pointer on-screen in real time to serve the same purpose, but with the added benefit of 3D. Therefore, the trainee could be provided with visual guidance without need to leave the console, greatly improving communication and efficiency.
Jarc et al. further advanced telestration techniques with the introduction of “ghost tools” [
54,
55]. Ghost tools offered 3D telestration abilities to proctors through use of a 3D pointer, 3D hands with the ability to point or simulate grasping, and 3D instruments that could be manipulated in similar fashion to actual operating instruments. Both proctors and trainees demonstrated a preference for 3D hands and 3D instruments over traditional 2D telestration, finding them to be more effective demonstration tools despite an increase in complexity of use for the proctors. Further objective research demonstrated proctors made good use of the enhanced manipulation abilities on offer [
55].
Though not specific to teaching, there are several other current and future augmented reality technologies that offer benefit to both trainee and mentor alike. First amongst these is image enhancement through the use of immunofluorescence. The technology, known as “Firefly”, involves the administration of indocyanine green (ICG) followed by filtering the endoscope image for near infrared light wavelengths. The technique can be used to highlight underlying critical vasculature, biliary structures and ureters, to delineate hepatic tumours and to assess tissue perfusion during anastomosis. Since its introduction in 2011 its use has become commonplace in many surgical procedures [
56].
Recently, Activ Surgical have released an augmented reality endoscope attachment that allows real-time assessment of tissue perfusion without the need to inject dye [
57]. Though this is currently designed for laparoscopic surgery, there is little doubt that robotic offerings will be quick to follow and further refinement may see this become the new standard of care.
3D reconstructions of staging imaging can prove particularly helpful in providing a surgeon or trainee with a greater understanding of the patient’s anatomy preoperatively, enhancing surgical planning and surgical safety. Currently, the TilePro function of the Da Vinci platform allows the operator to display and manipulate this imaging intraoperatively alongside the endoscopic display to guide dissection. Intraoperative ultrasound may also be displayed on TilePro, allowing sonographically detectible lesions in solid organs to be marked out, allowing maximal preservation of critical neurovasculature while reducing the risk of an involved resection margin [
58]. The next phase of augmented reality involves overlaying this information onto the surgical field in real time for enhanced identification of critical anatomy and improved efficiency. Proof of concept has already been successfully demonstrated in urologic and hepatic surgeries, with recent work suggesting improved accuracy in resection margins for hepatic tumours compared to the current gold standard of intraoperative ultrasound [
59,
60,
61,
62,
63,
64]. The next major hurdle lies in adjusting for real-time tissue deformation and manipulation intraoperatively.
In the near future, augmented reality will offer even greater value through the use of artificial intelligence (AI) and machine learning. Machine learning may be supervised, whereby a human input labelled data into a programme to teach it to differentiate between structures, or unsupervised, where unlabelled data is fed into the algorithm, which then attempts to identify the abnormality. In re-enforcement learning, the AI is then set a task and gains further data points based on its successes or failures [
65]. Essentially, these methods allow AI programs to ‘learn’ to analyse data and identify the desired abnormality with increasing accuracy. In minimally invasive surgery, this technology can be applied to display an intraoperative on-screen visual representation of areas of safe dissection and “no go” zones containing underlying critical anatomy in order to improve patient safety. Such algorithms have already been successfully applied to laparoscopic cholecystectomy to provide on-screen guidance around safe dissection of Calot’s triangle and avoidance of portal structures with high levels of efficacy [
66,
67]. While the application of AI to surgical training is still in its infancy, machine learning holds the ability to advance at a rate far outstripping human learning and this will no doubt become an extremely powerful surgical tool.
4.2.6. Telementoring
Since the advent of robotic surgery there has been great interest in the potential applications of telesurgery in revolutionising healthcare. The master-slave nature of the Da Vinci platform is perfectly suited to the performance of telesurgery, whereby the operating surgeon controls the robotic instruments from a remote location. Potential applications included the delivery of healthcare to poorly serviced areas or adverse environments (e.g., warzones), to offer highly subspecialised services from a central “institute of excellence” without requiring the patient or surgeon to travel and, in the age of Covid-19, to reduce the risk of surgeon exposure to transmissible diseases [
68]. The first telerobotic cholecystectomy was performed in 2001 [
69]. However, adoption and practical application has been limited due to concerns over network stability, latency times, medicolegal issues, the risk of cybersecurity threats and establishment costs [
68]. Optimal latencies are considered to be below 200-300ms, while latencies greater than 700-1500ms make surgical performance challenging and likely unsafe [
70,
71,
72,
73].
Fortunately, these limitations are less restrictive in the case of telementoring, and this technology has been utilised with good effect in robotic surgical training. A systematic review by Bilgic et al. has confirmed the safety and efficacy of telementoring in surgery [
74]. The authors examined papers comparing on-site mentoring with telementoring and found 11 studies of 453 cases that were suitable for inclusion. No differences in perioperative complication rates were encountered in any study. 90% reported comparable operating times between groups, with 1 study showing a longer operating time due to telementoring. Technical difficulties were encountered in 3% of telementored cases. Subjective analysis of trainee satisfaction revealed no difference between on-site mentoring and telementoring, while objective improvements in operating times across the learning curve were also comparable between groups.
Subsequent publications by Papalois et al. and Artsen et al. have placed further support behind telementoring. Papalois and colleagues developed a surgical curriculum delivered in mixed reality through use of Microsoft’s Hololens [
75]. The curriculum focused on surgical decision-making, operative anatomy and expert “tips and tricks”. 93% of students and 100% of tutors felt that virtual mentorship was of use in future surgical training, while 73% agreed/strongly agreed that their understanding of anatomy and decision-making rationale was improved by the module. Artsen et al. compared a series of teleproctored robotic gynaecologic cases with historical controls performed with in-person proctoring. They found high satisfaction rates amongst surgeons and no change in perioperative complication rates [
76].
Several new commercial platforms have recently become available that significantly improve on existing telementoring software. These include Orpheus medical (recently acquired by Intuitive Surgical,) Proximie, and Reacts (recently acquired by Philips) [
77,
78,
79]. Each of these platforms allows real-time collaboration and consultation for the purpose of live telementoring and utilises augmented reality overlays for advanced telestration and annotation.
Telementoring therefore offers great potential. As the technology matures, it may be utilised at any ability level, from residents receiving didactic teaching and supervised simulation training to a specialist surgeon consulting peers regarding a particular intraoperative quandary. This will aid in the standardisation of gold-standard surgical techniques globally by vastly increasing access to world leaders in any given subspecialty. This is particularly important in an age where new procedures and technologies are in constant development. In order to maintain currency, there is a strong requirement for the establishment of global surgical collaborative networks.
4.2.7. Surgical Videos
In the past two decades, online data transfer rates have accelerated dramatically. This has greatly improved the ability to share high-definition surgical footage globally. This footage offers a valuable learning tool, particularly in the case of robotic surgery. As discussed previously, due to haptic feedback limitations of current robotic platforms, the surgeon is entirely guided by on-screen visual cues. Thus, observers have access to identical sensory inputs afforded to the operating surgeon. This may be capitalised upon for learning purposes.
Video-based learning has become an extremely common method of surgical learning. In a recent survey of residents and surgeons, Mota et al. found that 98.6% of respondents had made use of videos in preparation for surgery [
80]. Furthermore, 57% noted that surgical video was their preferred method of surgical preparation. This was particularly true of younger, less experienced respondents.
Video material may be used at different stages of teaching. Demonstration videos can be effective in teaching simulation skills for robotic surgery, as shown by Shim et al. [
10]. Educational video was shown to be as effective as on-site mentoring in mastering robotic vesicourethral anastomosis, while both learning techniques were superior to self-directed learning. Video based learning has also shown superiority when compared to hands-on practical training in the case of laparoscopic cholecystectomy. Pape-Koehler and colleagues performed a randomised controlled trial comparing a multimedia, video-based learning module with practical training or no training [
81]. Participants who underwent multimedia-based training showed a significant improvement in surgical performance compared to both their practically trained and untrained colleagues.
In the near future, surgical video may be further enhanced through immersive footage that incorporates haptic feedback. Pandya and colleagues developed a novel recording system that synchronized robotic arm and surgeon-console interactions with operative footage [
82]. The material could then be replayed to an observer at the surgeon console, allowing them to feel the movements of the operating surgeon’s hands to gain a deeper appreciation of correct operative technique and shorten the learning curve associated with complex procedures. This concept has already been applied to simulation training in the form of the HoST platform on the RoSS virtual simulator described earlier [
14].
Huynh and colleagues offer a word of caution in the application of surgical videos [
83]. The authors reviewed the most viewed 50 YouTube videos relating to MIS inguinal hernia repair and rated their surgical performance in comparison to the “9 commandments” of safe technique as defined by Daes and Felix [
84]. Only 16% of videos demonstrated all 9 commandments, with significant differences between laparoscopic and robotic approaches. Furthermore, 46% of videos were considered to display unsafe techniques through dangerous mesh fixation, risks to critical structures or inappropriate tissue handling.
To address the variable quality of online material, an international multidisciplinary consensus group have recently published guidelines on the appropriate reporting of educational videos [
85]. These guidelines cover 36 recommendations including requirement for video introduction and information about the authors, case presentation and staging imaging, robotic setup, procedural demonstration +/- stepwise teaching and telestration of relevant anatomy, review of postoperative outcomes, and confirmation of high-quality footage. This standardised approach to reporting aims to improve the general quality of online material as a teaching resource. Knowledge of these guidelines will also help a prospective student to select appropriate videos from which to learn.
Several comprehensive clinical media platforms are now available that facilitate data sharing in the surgical community. Examples include Orpheus, Proximie and Touch Surgery Enterprise. These platforms simplify the process of video storage, editing and retrieval, while offering rapid, secure, de-identified sharing of material between colleagues. Furthermore, they each possess AI algorithms for the automatic segmentation and labelling of surgical footage [
77,
78,
86]. This further improves the utility of videos through rapid access to relevant sections of a procedure.
4.3. Technical Assessment and Tracked Metrics
As previously described, most modern robotic surgery curricula focus on objective assessment of proficiency prior to graduation. This is critical to patient safety. Technical skills assessment from operative video has been directly associated with perioperative complications, morbidity and mortality [
87,
88]. Assessments in robotic surgery publications have most commonly been delivered through validated scoring systems by mentors or independent reviewers. Common examples include GEARS, R-OSATS and PACE scores [
89,
90,
91]. However, this process is time intensive and relies on considerable goodwill on behalf of assessors. Therefore, several potential alternatives have been explored. Crowdsourcing is one such example. This technique utilises the ready availability of large numbers of individuals in an online forum. Non-surgical crowdworkers are briefly trained in video evaluation through an online module and then score performances using a rating scale such as GEARS. Crowdsourcing results in much more rapid responses than the use of expert reviewers at minimal cost and studies have shown good correlation between scores by expert reviewers and crowdworkers, even in the assessment of complex procedures such as prostatectomy [
92,
93,
94,
95].
Automated performance metrics (APMs) represent an increasingly valuable means of assessment. APMs are a set of data points relating to various aspects of an operation that are routinely collected throughout a robotic procedure. They are readily available with minimal effort or cost. For the purposes of training, each surgical simulation platform automatically assesses user data and provides performance scores following the completion of a task, as compared to expert benchmarks. In order to be of value, these metrics must be shown to hold practical utility. Several studies on the validity of simulation have shown that automated metrics correlate highly with expert GEARS scores in simulation exercises. Importantly, tracked metrics were also shown to correlate well with subsequent intraoperative performance assessment [
96].
Chen et al. have also proven the ability of intraoperative APMs to differentiate between expert and novice robotic surgeons in the performance of a vesicourethral anastomosis during RARP [
97]. By combining the APM data with clinicopathologic characteristics in a deep learning algorithm, they were subsequently able to accurately predict postoperative continence rates [
98]. This supports the value of APMs and deep learning algorithms in assessing surgical quality. Multiple subsequent studies of deep learning algorithms have unanimously proven to be highly accurate in predicting surgical skill level when compared to structured assessments by expert reviewers [
99].
Currently, this remains a new and novel technology and how best to incorporate it into proficiency assessment has yet to be clearly defined. Nevertheless, APMs and deep learning algorithms are an area of burgeoning research. Due to the rapid availability of results, cost-efficacy and accuracy, it is highly likely that these will ultimately become the primary method of proficiency-based assessment for surgical trainees and will have a significant role in hospital accreditation and credentialing processes.
4.4. Artificial Intelligence, Machine Learning and Big Data
Several major device manufacturers have identified the potential of tracked metrics and machine learning. Previously discussed media platforms such as Orpheus, Proximie and Touch Surgery currently offer their users procedural analysis based on APMs. In their current states, the information produced is useful for benchmarking and identifying areas for individual improvement. However, their true potential lies in future applications. The routine recording and upload of cases will rapidly lead to vast troves of data, knows as “big data”. In combination with electronic medical records, these can be analysed by deep learning algorithms such as convolutional neural networks to identify associations between surgical technique and outcomes that cannot be appreciated by traditional research methods or simple observation. As a result, trainees and surgeons will be provided with a constant source of feedback on subtle areas for self-improvement, to the benefit of their patients.
Optimal patient management will also become far more standardised globally based on best evidence from hundreds of thousands of cases, overcoming many of the current limitations of surgical research. While big data is already utilised in the production of treatment algorithms such as CeDAR or NELA, these only utilise a handful of data points to guide clinical decision making. Deep learning algorithms will have the ability to advise on the most appropriate treatment for a patient based on innumerable individualised data points in a far more personalised manner and will have a profound effect on the way we practice surgery.
An additional significant benefit arises in cost and time savings. Traditional research methods such as randomised controlled trials frequently take years to produce clinically relevant conclusions at great cost and often with limited generalisability. The field of surgery is rapidly expanding with procedures undergoing constant evolution. Traditional clinical trial models can no longer keep up with this rate of advancement. By comparison, machine learning possesses the ability to produce clinically relevant outcome data at a rate that keeps pace with surgical advancement, guiding clinical practice in a current and meaningful manner.
The storage of big data does raise ethical and medicolegal issues around the potential identification of the underperforming surgeon. However, it is currently of critical importance that the information produced is utilised for self-improvement purposes alone, rather than in any disciplinary manner. Deep learning algorithms are entirely dependent on the purity of input data in order to produce accurate results. The fear that participation may result in negative repercussions may otherwise lead to data contamination through inaccurate data input or data omission. This would severely compromise the reliability of results and potentially even lead to patient harm. As a self–improvement tool, all surgeons are afforded the opportunity to improve their own practice, thus providing widespread benefits to patients.