Development and Implementation of a Mixed Reality-Based System for Planning, Simulation, and Real-World Clinical Performance of Laparoscopic Myomectomy

1. Introduction

Uterine fibroid is the most common benign tumor in women of childbearing age [1,2]. The symptoms of uterine fibroids include abnormal uterine bleeding, urgency of urination, and constipation. Fibroids may cause reduced fertility [3]. Up to 70% of premenopausal women will develop uterine fibroids, and 25% of these will require treatment [4]. Surgical options for treating uterine fibroids include myomectomy and total hysterectomy. Myomectomy is performed more frequently, as most women prefer to preserve their uterus and their ability to have children. Hysteroscopic and laparoscopic surgical approaches are common [5,6]. The International Federation of Gynecology and Obstetrics classifies uterine fibroids into nine types based on their location [7,8], types 3-8 are typically removed via laparoscopic myomectomy.

However, laparoscopic myomectomy provide a relatively narrow field-of-view and render touch of uterus and fibroid texture difficult. Residual tumors and postoperative recurrence are relatively common clinical problems [9,10]. The percentage of fibroid left after laparoscopic myomectomy is as high as 20% [11]. The reported 1-, 3-, 5-, and 8-year rates of fibroid recurrence after laparoscopic myomectomy are 11.7%, 36.1%, 52.9%, and 84.4%, respectively[12]. Despite significant advances in medical imaging technology, especially magnetic resonance imaging (MRI) [5,6,13,14], laparoscopic myomectomy still faces numerous challenges. There is considerable difference in interpretation and understanding of two-dimensional (2D) medical images between individual surgeons, these differences can affect the skillful operation. Improper or incorrect surgical planning and performance may result in a high volume of intraoperative bleeding and/or incomplete excision. In addition, trainee surgeons and young surgeons require preoperative simulation training tools, which are currently lacking. Furthermore, patients frequently have a poor understanding of the details of the laparoscopic surgery, making doctor–patient communication rather abstract.

In the field of medical engineering integration, Beihang University [15] has designed a ball-chain hybrid deformation algorithm, which is used in a virtual laparoscopic surgery training system. Mixed reality technology combines the real world with the virtual world, integrating computer-generated three-dimensional (3D) images with intraoperative scenes to assist in completing medical research. Augmented reality systems are already being used in maxillofacial surgery and orthopedic surgery applications[16,17]. However, intraoperative 3D visualization of soft tissue structures such as the uterus and uterine fibroids is technically difficult. Nonetheless, case reports have described the use of 3D MRI reconstructions to explore the integration of imaging and mixed reality technology in laparoscopic myomectomy [18]. Additionally, Ochi et al. [19] reported a preliminary attempt at applying mixed reality technology to intraoperative navigation during laparoscopic myomectomy. However, implementation of combining imaging and mixed reality relies completely on artificial intelligence experts and imaging specialists. It is not yet feasible for large-scale routine clinical application.

This study aims to develop an intelligent, individualized, real-time, and portable gynecological surgical assistance system that will provide 1) preoperative planning, 2) surgical simulation, and 3) intraoperative real- time 3D visualization. Hopefully, the system will assist surgeons in planning, simulating and performing uterine fibroid operations in a rapid and efficient manner with lower rates of surgical complications.

2. Study Setting

The study will be conducted at Beijing Shijitan Hospital, affiliated with Capital Medical University, where subsequent clinical trials will also be performed. Beihang University will collaborate and provide technical support.

3. Study Contents

3.1. Anatomical Visualization of Uterine FIBROIDS Based on 3D MRI Reconstructions

Imaging data acquisition and image segmentation: Based on the backend data system of our hospital's information center, pelvic MRI data will be obtained. Using a conditional generative adversarial network combined with a variable reconstruction module, image segmentation will be completed to extract anatomical data regarding the uterus and fibroids within.

3D model construction: Based on a uterine fibroid MRI database established in our previous research [20], a reconstruction module with prior anatomical knowledge will be developed, which ultimately transforms the acquired two-dimensional imaging data into a 3D network model. The model will visualize fibroid anatomical details including size, location, and blood supply distribution.

3.2. Preoperative Planning and Surgical Simulation of Laparoscopic Myomectomy

Preoperative planning: Based on the 3D model, a mixed modeling method will be used for precise modeling of the myometrium, uterine cavity, and fibroids. For each fibroid, it will automatically evaluate size, classification, and relationship with the uterine wall, as well as the blood supply. By constructing high-dimensional mathematical equations, the shortest distance between the uterine fibroid and the uterine serosa will be calculated, and the location of the optimal point for uterine incision will be indicated. The required length of the uterine incision will also be calculated. Combining data from the model with 3D vascular imaging data will clarify the source, course, and richness of the blood supply, which will allow the surgeon to avoid important blood vessels and reduce intraoperative bleeding.

Surgical simulation: A dynamic simulation model of uterine fibroid resection surgery will be constructed using metaballs and meshless mixed method simulation technology. Motion modeling and force feedback devices will be integrated to achieve tactile feedback function, allowing the surgeon to perceive the force and tactile sensations of the surgical instruments in real-time during the process of cutting and pulling the uterine fibroid in simulated operations. With the support of the preoperative intelligent planning scheme, operations can be conducted during surgical simulations using the recommended uterine incision, allowing full simulation of different fibroid removal sequences. The predicted uterine deformation present after fibroid removal will be simulated, as well as its impact on the position of other fibroids and the uterus. Based on each patient's MRI, comprehensive preoperative individualized simulation training will be conducted. The simulation process can also generate animations that can be played intraoperatively for the surgeon's reference.

3.3. Intraoperative 3D Visualization Based on Mixed Reality Technology

First, mixed reality technology will be used to integrate the patient's uterine fibroids into the real environment as a 3D image. Through real-time rendering technology, it enables observation and tracking of the surgical field during the operation, achieving a fusion of virtual and real scenes. This helps the surgeon to 'see through' the internal structure of the uterus, clearly identifying the size, position, anatomical adjacency, and blood supply distribution of the fibroids, thereby facilitating surgical operations. Secondly, the 3D view can be adjusted in real-time, including rotation and scaling, to obtain 3D images of uterine fibroids from different angles and depths. Before surgically excising the fibroid, the simulation animation can be played in real-time to reference the uterine deformation after a certain fibroid excision, assisting in intraoperative decision-making regarding the incision site and length.

3.4. Conduct Clinical Trials for Efficacy and Safety Verification

The system’s efficacy and safety will be verified in a randomized controlled clinical trial comprising experimental and control groups. The experimental group patients will undergo conventional preoperative preparation and uterine fibroid MRI imaging with 3D reconstruction; surgeons will use the system for surgical planning and simulation and perform surgery with intraoperative mixed reality imaging assistance. In contrast, patients in the control group will undergo surgery according to the traditional myomectomy process. The differences in surgical outcomes and complications between the two groups will be evaluated to determine the system’s safety and effectiveness.

4. Core Technology

4.1. Establish a 3D Imaging Model of Uterine Fibroids and Locate Incision Indicator Points

Conditional generative adversarial network technology will be used for segmentation and reconstruction of uterine fibroids based on medical imaging. Interactive or automated multi-task segmentation networks and attention mechanisms will be developed to capture the uterus and surrounding organs, providing more accurate and realistic 3D reconstruction. High-dimensional mathematical equations will be constructed to calculate the shortest distance and path from the uterine serosa to the fibroid and automatically locate incision indicator points. Additionally, conditional variables will be incorporated in the path equations (such as vascular distribution and adjacent organs) to develop an optimal uterine incision planning scheme.

4.2. Construction of a Surgical Simulation System Using Physical Simulation Technology

Construction of a physical model based on metaballs and meshless mixed method technology is a key research focus of this project. Such a model determines the deformation effects of soft tissues in surgical simulation and is a core module of the Virtual Surgery Engine. The simulation of basic surgical operations such as cutting, tearing, and electrocautery based on this hybrid model is a challenging aspect of this project and another key technology of the Virtual Surgery Engine.

4.3. Implementation of Intraoperative Real-Time Imaging Using Mixed Reality Technology

Mixed reality technology integrates a 3D model of a uterine fibroid into the real environment in the form of images. Through real-time rendering technology, observation and tracking of the surgical field are enabled during surgery, achieving fusion of virtual and real scenes. This helps the surgeon 'see through' the internal structure of the uterus and perform surgical operations. The surgeon is able to interact with the 3D view using specific input devices to rotate, zoom, or slice the view, obtaining different angles and depths of view.

4.4. Technical Roadmap

The project aims to develop an individualized uterine fibroid myomectomy surgical planning, surgical simulation, and intraoperative interactive 3D visualization prototype system. The technical roadmap is shown in Figure 1.

5. Study Procedures

5.1. D Reconstruction of Uterine Fibroids

Based on the uterine fibroid MRI imaging dataset established in our previous research, a 3D network model of uterine fibroids will be formed. We will also develop software that is adaptable to more than 95% of physician office computers, allowing input of patient imaging data used for 3D reconstruction. As shown in Figure 2, the research steps include two stages. First, the Spatial Attention Module is used to extract temporal information from the MRI sequences. Second, data regarding the uterus, surrounding organs, and vessels are captured through a multi-task segmentation network and attention mechanism, providing a more accurate reconstruction of uterine morphology.

5.2. Surgical Planning and Preoperative Simulation

After obtaining the individualized uterine fibroid 3D model, surgeries will be modeled and simulated to provide tactile feedback using a hybrid meta-spheres and meshless method, as shown in Figure 3:

The content within the blue box represents the offline functional module, while the content within the yellow box represents the online real-time functional module. The content in the yellow box, together with the 'construction of the physical model,' forms the core of the Virtual Surgery Engine. This requires researching efficient physical simulation algorithms, especially simulation methods for soft tissue deformation and cutting. Simultaneously, it is also necessary to study how to integrate force feedback technology with physical simulation to provide realistic tactile feedback. In addition, it is necessary to optimize the system's rendering efficiency to ensure real-time and visual authenticity of the simulation process. After completing the above research, we can achieve physical deformation simulation of the uterus composed of the uterine cavity, blood vessels, and fibroids based on an individualized 3D model of uterine fibroids, and handle the mesh topology changes caused by surgical instrument cutting on the tissue surface during the myomectomy simulation process, while ensuring the system's real-time performance and visual realism.

5.3. Intraoperative 3D Visualization of Uterine Fibroids

The project intends to develop an intraoperative visualization system based on mixed reality, as shown in Figure 4, using surgical field visualization as a foundation. Interactive tools will be developed, and ultimately, the system will enable surgical planning and preoperative simulation to assist surgeons in avoiding surgical complications and communicating with patients.

5.4. Evaluation of the Effectiveness and Safety of the Simulation and 3D Visualization System

After the system has been completed, its safety and effectiveness will be verified for use in clinical practice.

5.4.1. Research Design

A prospective randomized controlled trial will be conducted with uterine fibroid patients as subjects. Seventy-two patients will be recruited and randomly assigned to the experimental or control group in a 1:1 ratio. Written informed consent will be obtained from all participants. The study will be conducted in accordance with the principles of the Declaration of Helsinki. The following data will be recorded from an electronic medical record system: demographics, general clinical information, physical examination indicators, laboratory testing, imaging, pathology, and postoperative follow-up.

5.4.2. Research Subjects

The research subjects will be patients scheduled for laparoscopic myomectomy at Beijing Shijitan Hospital. Inclusion criteria: (1) menorrhagia caused by fibroids leading to secondary anemia; (2) dyspareunia or abdominal pain caused by pedunculated fibroid torsion; (3) large fibroid causing symptoms via compression of adjacent structures; (4) infertility or recurrent miscarriages caused by fibroids; (5) growing fibroids in postmenopausal women not receiving hormone replacement therapy; and (6) suspected sarcomatous transformation. Exclusion criteria: (1) severe comorbidities such as severe coagulopathy; (2) acute pelvic inflammation, vaginal infection, and/or body temperature exceeding 37.5°C; (3) active uterine bleeding or severe anemia; (4) pregnancy; (5) malignant tumor of the reproductive organs; (6) genital tuberculosis without appropriate anti-tuberculosis treatment; (7) diaphragmatic hernia; (8) postoperative pathology not consistent with uterine fibroid; and (9) not suitable for uterine preservation (total hysterectomy deemed necessary).

5.4.3. Sample Size Estimation

Because of the exploratory nature of the study, the data needed for traditional sample size estimation is lacking. Therefore, sample size was selected based on the following considerations:

(1) The stability of the sample mean supported by the central limit theorem

The central limit theorem states that when the sample size is ≥30, the distribution of the sample mean approaches a normal distribution, and the standard error decreases as the sample size increases. Therefore, a sample size of 30 per group was set to provide a stable estimate of the mean and sufficient data for exploratory analysis. Considering a dropout rate of 20%, the sample size was set at 36 per group.

(2) Research conditions and practical limitations

The study design requires that all surgeries be performed by gynecologists with associate senior titles or above. The research team has a limited number of surgical physicians, and extending the research period may introduce confounding that affects the accuracy of the results. We considered that a sample size of 36 per group would ensure the study is completed within a reasonable time frame.

After verification by statistical experts, clinical experts, and project team members, the final sample size was determined to be 72, with 36 in the experimental and control groups, respectively.

5.4.4. Random Grouping

An independent statistical expert will use statistical software to generate 72 random numbers by setting a fixed random seed number. The generated random numbers will be sorted by size, and the sorted numbers divided into two groups in sequence: numbers 1 through 36 are assigned to the control group, and 37 through 72 are assigned to the experimental group. According to the randomization results, the numbers and corresponding group information will be placed into numbered envelopes. marked only with a number (1 to 72) on the outside; the corresponding group is indicated inside. The envelopes will be filled and sealed by an independent third party or by members of the research team who are not involved in patient recruitment, treatment, follow-up, or data analysis. For each participating patient, an envelope will be drawn to determine the group (control or experimental) based on its contents. Once the grouping has been determined, communication regarding treatment between the control and experimental groups will be prohibited to avoid research bias.

5.4.5. Experimental Scheme

The experimental process is shown in Figure 5. The experimental group will complete the 3D reconstruction of patient MRI data using the mixed reality-based system and surgical planning will be performed accordingly. Furthermore, the system will be used to simulate the planned operation. Using force feedback devices and associated equipment, virtual laparoscopic myomectomy based on physical modeling will be completed to verify the effectiveness of planning, which will be revised as needed based on the simulation results. During the real-world clinical operation, doctors will use mixed reality equipment to visualize the patient's 3D uterine fibroid model and manipulate it in real-time as needed. This will assist surgeons in selecting fibroid incisions and the appropriate sequence of tumor removal.

In the control group, preoperative surgical planning will be performed by studying the number, position, and size of uterine fibroids using 2D MRI. During the operation, surgeons will directly observe the surgical field through a 2D display and perform the planned myomectomy. The entire surgical process in both the experimental and control groups will be recorded and reviewed later for evaluation.

5.4.6. Observation Indicators

To quantitatively evaluate surgical outcomes, duration of surgery will be the primary observation indicator. Secondary observation indicators will include blood loss volume, Global Operative Assessment of Laparoscopic Skills score, postoperative gastrointestinal exhaust time, pelvic drainage volume, postoperative fever, length of hospital stay, and cost of hospitalization. The chief surgeon will complete the National Aeronautics and Space Administration Task Load Index questionnaire after each operation (Table 1) to assess workload, and evaluate the system in terms of usability and effectiveness.

5.4.7. Statistical Analysis

Statistical analyses will be performed using SPSS software version 29.0 (IBM Corp., Armonk, NY, USA). Continuous data will be described using means with standard deviation or medians with quartiles, as appropriate. Categorical data will be described using frequencies with percentage. Statistical comparisons will be performed using the chi-square or Fisher's exact test for categorical data and the t-test or rank-sum test for continuous data, as appropriate. P <0.05 will be considered significant.

5.4.8. Quality Control Measures

A quality control team will be established to provide uniform training for all project members. Strict control will be maintained at each stage, with timely recording and handling measures implemented. Cases will be selected strictly according to inclusion and exclusion criteria. Participants and researchers will be monitored to ensure that selected treatment plans are not discussed with each other. Clinical guidelines will be strictly followed throughout the study. Any adverse reactions will be recorded and handled promptly. If a patient's condition progresses to the point where they are no longer suitable for continued participation, they will be withdrawn and appropriate treatment provided to ensure his or her safety. Patients will be followed after surgery as required and communication maintained to minimize loss to follow-up. Study data will be managed by a designated person to ensure its timely recording, integrity, accuracy, and preservation.

6. Expected Outcomes

The expected outcomes of this project are as follows: 1) develop a laparoscopic surgery planning and simulation prototype system, and validate procedural safety and effectiveness, 2) deliver the system to medical device manufacturers to produce an engineering prototype, and 3) implement the system in clinical, teaching, and research scenarios.

7. Discussion

Although several case studies of segmentation and 3D reconstruction of uterine fibroids have been previously reported, systematic technological development is lacking, and large-scale application has not yet been achieved. Therefore, this study aims to achieve automated annotation and segmentation of uterine fibroids and develop organ model reconstruction software. The promotion and application of this software should improve the universality, accuracy, and reliability of preoperative MRI diagnosis. Then, addressing the challenges of tissue deformation and movement of uterine soft tissue during surgery, a highly realistic simulation system fo myomectomy will be constructed with the aid of force feedback and physical simulation technology, allowing for comprehensive preoperative simulation and training. Additionally, animations can be generated for surgeons to reference during surgery. Mixed reality technology will enable the following: intraoperative 3D visualization and fusion of virtual and real scenes, referral to preoperative planning, and review of preoperative simulation animation. Furthermore, it will assist the surgeon with intraoperative decision-making. As a result, the duration of surgery is shortened and intraoperative bleeding is reduced.

The research team in this study has a solid foundation of preliminary work, and the research objectives and content are set with both technological foresight and feasibility in mind. The research plan is based on previous work, with ample room for innovation, and is reasonable and feasible. Individuals from multiple disciplines and backgrounds comprise the research team. The feasibility analysis for product registration and market launch has already been completed. The product has technological, resource, and industry experience advantages. Therefore, it has potential market scale and expected industrialization prospects. We anticipate that the system will meet real clinical needs.

The implementation of the project has three major points of significance: First, it will improve the diagnosis and treatment of uterine fibroids, and is expected to significantly reduce duration of surgery, avoid surgical complications, reduce intraoperative bleeding, and improve the fibroid excision rate (thus reducing the incidence of residual/recurrent tumor and the need for further surgery). In addition, we expect it to help eliminate unnecessary wasting of medical resources, and have an important and far-reaching impact on women’s health. Furthermore, the technical approach taken in the project can be extended to other gynecological surgeries in the future. Finally, implementation of this project can greatly enhance the training process for laparoscopic myomectomy, ensuring that safe and effective uterine fibroid care can be provided from community hospitals to tertiary medical centers.