PRONOBIS:A Robotic System for Automated Ultrasound-Based Prostate Reconstruction and Biopsy Planning

1. Introduction

Prostate cancer is the second most common malignant neoplasm in men worldwide and the fourth leading cause of cancer-related death [1]. Diagnosis relies on histopathological evaluation of prostate biopsies, typically performed following an elevated prostate-specific antigen (PSA) level and/or an abnormal digital rectal examination (DRE), even though most patients are asymptomatic. The prostate is anatomically divided into four main zones—peripheral, transitional, central, and anterior—with approximately 70% of prostate cancers arising in the peripheral zone [2].

Multiparametric MRI (mpMRI) is now recommended before biopsy to detect suspicious lesions, which are subsequently targeted using either cognitive or fusion approaches; current evidence suggests no significant difference in accuracy between these methods [3]. However, a key challenge remains: the precise targeting of lesions, as targeted prostate biopsy is highly operator-dependent and subject to significant inter-operator variability [4]. This variability raises concerns regarding the consistent application of the mpMRI diagnostic pathway and the risk of improper diagnosis.

To address these challenges, current initiatives aim to standardize the entire diagnostic workflow—including MRI acquisition, interpretation, biopsy planning, and execution—through robust quality assurance and control measures [4]. One proposed solution is the integration of robotic prostate platforms and artificial intelligence to minimize inter-operator bias and reduce the learning curve [5], although studies on this approach have yielded conflicting results [6].

Robotic biopsy techniques that combine transrectal ultrasound (TRUS) with MR imaging have been introduced. Although designs vary, these systems typically fuse mpMRI-identified lesions with TRUS images to enable a robotic arm to target the areas of interest accurately. The system determines the needle’s penetration angle and depth by positioning a needle guide equipped with a stop bar; the surgeon then manually inserts and fires the needle gun at the preset location and depth [7,8]. The targeted biopsy results have higher success rates than systematic biopsy [9,10]. Therefore, the novel approach that enables clinicians to pick the exact positions for biopsy needle impact exclusively via ultrasound, is crucial. Incorporating targeted biopsy into the workflow requires an end-to-end software solution that enables biopsy procedure planning.

One of the challenges for robotic biopsy are safety measures and robustness of the robots that operate in close proximity to the human body. By using a redundant 7 DOF robot arm it is possible to optimize each trajectory avoiding joint limits and singularities [11], crucial to perform complex movements in space restricted areas. Zhang et al. presented a design of a 7-DOF manipulator for TRUS probe [12]. The manipulator was evaluated in an environment that indicated the mechanism can reach any position in working space of the biopsy procedure. The robotic arm structure of continuous body type for prostate biopsy procedures in confined spaces has been introduced [13]. Two sets of drive lines run through the robot and are fixed to the end nodes. The end position is controlled by adjusting the angle of the servo motor. Experiments indicate the introduced robot design achieves 2.5 mm positioning error which is a significant improvement for robotic biopsy procedures.

The robotic platform for prostate biopsy, the iSR’obot Mona Lisa™ (Biobot Surgical, Singapore) [14], allows clinicians to create biopsy procedure planning by fusing MRI and ultrasound images. Additionally, a novel Vector MRI/ultrasound fusion transperineal biopsy technique, utilizing electromagnetic needle tracking, has been developed [15]. Ipsen et al. [16] demonstrated 4D (volumetric) ultrasound image acquisition with motion compensation over longer periods of time. The study presented the approach evaluated on five volunteers for periods of 30 minutes with dynamic pressure adjustments with target force of 10N, to determine the feasibility and safety of the proposed approach.

A transperineal prostate biopsy approach that proposes a robotically guided fusion of MRI and ultrasound images [17] has been introduced. After scanning the phantom used in the experimental phase, it creates a 3D model, enabling biopsy planning with the average error of 1.44 mm, which is below the clinical threshold required to distinguish cancerous from healthy tissue. Furthermore, the approach is similar to traditional biopsy approaches and can be adopted in a clinical setting. Another high precision robot introduced by Stoianovici et al. is designed for transperineal prostate biopsy with an accuracy of 2.55 mm [18]. The promising accuracy results of targeting the exact positions for the biopsy needle using robots provide the foundation for our research.

The robotically assisted biopsy needle positioning study resulted in a robot, guided by an MRI image, that achieved average precision of 2.39 mm, compared to 3.71 mm with the manual approach [19]. Although a robot can eliminate human error, it introduces challenges such as the robot positioning and the pressure applied to the patient, that causes prostate deformation. A recent study addressed the challenges by introducing robot that minimizes prostate deformation [20] with 4 DOF probe manipulator used for TRUS; needle targeting was achieved with a precision of 1 mm. To minimize prostate deformation, the approach involved three essential aspects; optimization of the biopsy trajectory, optimization of the order of biopsy points, and a prostate coordinate system that the team developed.

The ultrasound image segmentation plays a crucial role to ensure adequate precision in prostate scanning, 3D reconstruction, and robot-assisted biopsy procedure planning, The ultrasound image intensity inhomogeneities and weak object boundaries make segmentation difficult for conventional image processing methods [21], such as simple standard or adaptive thresholding, Otsu’s binarization [22] or contour detection that do not completely fulfill the requirements of high accuracy, robustness and real-time performance [23]. Deep learning-based approaches demonstrate superior accuracy and robustness due to ability to learn the local and global image features. The utilization of Convolutional Neural Networks (CNN) for medical image segmentation presents a relevant solution. One of the most significant CNN architectures for medical image segmentation is U-Net [24], which is the encoder-decoder network, capable of capturing local image features and producing accurate high-resolution segmentations. The latest research utilizes the Transformer architecture for image processing. The Vision Transformer (ViT) architecture for classification [25] demonstrates remarkable performance that surpasses state-of-the-art CNN models. One of the most relevant algorithms that combines CNNs and ViT models for medical image segmentation is TransUNet [26]. The TransUNet hybrid architecture combines U-Net and ViT to eliminate the limitations of separate solutions, precisely, it enables segmentation of images with significant inter-patient variations in terms of specific object size, shape and tissue texture. One of the current state-of-the-art segmentation models for micro-ultrasound prostate images is MicroSegNet model [27], based on the TransUNet architecture with a modified annotation-guided binary cross-entropy (AG-BCE) loss function and multi-scale deep supervision (MSDS) that modifies the segmentation performance at different decoder layers, assuring precise segmentation globally and locally.

The proposed article presents robotically navigated prostate scanning and biopsy treatment planning. Our method integrates robotics, ultrasound imaging, and deep learning algorithms for image analysis, with the objective of generating visualizations for biopsy needle placement. A key challenge addressed is achieving an ultrasound-only reconstruction process, comparable to hybrid MR-ultrasound methods. The ultrasound-only prostate reconstruction and biopsy treatment is fast, cost-effective, and less physically demanding for the patient. Fully automated, robotic prostate sweep scanning ensures positional tracking of sweep slices for precise prostate reconstruction. The utilization of deep learning algorithms with image post-processing enables real-time, precise segmentation across the whole prostate sweep. For precise segmentation performance, the model is trained on a hybrid dataset consisting of real prostate and phantom prostate images, annotated using our Medical Annotation Platform (MedAP), that utilizes zero-shot CNN architectures. To select the optimal deep learning algorithm for the ultrasound medical prostate images, DAF3D and MicroSegNet architectures are trained on a hybrid dataset and compared, with the selection of MicroSegNet. 3D visualization and prostate reconstruction, based on the segmentation results and robotic positional information, are evaluated on two prostate phantoms (CIRS Prostate Training Phantom Model 070L and CIRS Tissue Equivalent Ultrasound Prostate Phantom Model 053L) achieving accurate dimensional and volumetric reconstructions. The 3D prostate reconstruction takes approximately 3 seconds and enables detailed insight for biopsy needle position planning.. The proposed system is planned to be integrated as part of the RONNA robotic system for neuronavigation which was already clinically validated in brain biopsies and EVD placement [28,29]

2. Materials and Methods

The proposed robotic approach integrates multiple components to create a comprehensive solution. The goal is to seamlessly incorporate the robotic system into the existing prostate biopsy procedure, maintaining simplicity for clinicians while enhancing patient comfort. The complete workflow of the procedure is illustrated in Figure 1.

2.1. Hardware and Software

KUKA’s LBR iiwa collaborative industrial robot, is utilized for this research. Although custom-built robotic systems, such as the Mona Lisa [14], provide significant design flexibility and a tailored approach to robotic architecture and procedural flow, they require extensive development. In contrast, using a medical robot arm KUKA LBR greatly reduces the time needed for development, offering a streamlined, cost-effective, and easily deployable solution with proven performance in industrial and medical environments. In this stage of development, the clinical ultrasound system P25 and BCL10-5 TRUS probe, developed by SonoScape are utilized. The TRUS probe transmits a 7.5 MHz signal and receives between 6 and 16 MHz signals. The acquired ultrasound image is 60 mm wide, with depth between 3 and 90 mm. The robot with an attached TRUS probe is shown in Figure 2 and Figure 3.

Robot operating system 2 (ROS2) [30] is used for robot control and communication within the system components; robot, ultrasound, segmentation and visualization. ROS2 provides integration of system components without compromising the integrity or functionality of existing elements. Integration between the robot and ROS2 Humble framework is achieved by using LBR-Stack [31]. LBR-Stack takes KUKA’s network communications protocols and integrates them into the ROS2 framework to enable seamless usage of these robots.

The proposed end-to-end software was developed using Python programming language version 3.10 for calibration, robot control and segmentation, while for the process of reconstruction and visualization version 3.12 was used because it offers a speed improvement. The deep learning algorithms used for segmentation purposes require CUDA enabled PyTorch 2.5.1. To implement reconstruction and visualization, we used the following libraries: OpenCV 4.10. and PyQt6. The complete solution uses the HP Z4 workstation with Intel Xeon W-2245 processor, 64 GB RAM memory, Quadro RTX 5000 GPU with 16 GB VRAM memory.

2.1.1. Robot Setup

After the TRUS probe is mounted on the robot, calibration of the TCP must be performed, for which KUKA’s four-point algorithm was used. Since the TRUS image needs to be calibrated with the TCP of the robot, a digital cross-hair was created to position a 3D printed cone in the center of the image, as shown in Figure 2(b).

Upon successful calibration, the TRUS probe is placed inside the specified prostate phantom using the robot’s compliance mode. The segmentation starts the process and the robot rotates the TRUS probe around its axis to acquire the US images. Once the segmentation process finds no prostate in the image, the algorithm sends the end signal and the robot changes its rotational direction to scan the other half. Prostate images, fused with the synchronized robot arm data are constantly being published via ROS2 for visualization purposes. Once the whole prostate sweep is complete, the program stops. The detailed image of the prostate scanning equipment and the prostate sweep scanning procedure is shown in Figure 3.

2.2. Prostate Segmentation

Prostate segmentation processes a raw ultrasound image and outputs a binary segmentation mask. Our segmentation algorithm utilizes a hybrid CNN-ViT model to achieve robust image segmentation by combining the strengths of both approaches. It enables automated analysis of ultrasound prostate images by combining local feature extraction via CNN with global context extraction via ViT. Trained on a large, diverse and well-balanced dataset, this combination delivers state-of-the-art segmentation performance.

2.2.1. MedAP

As part of the segmentation training dataset development, we have developed the Medical Annotation Platform (https://github.com/CRTA-Lab/MedAP), an advanced annotation framework designed for semi-automatic segmentation of medical images, shown in Figure 4. MedAP leverages state-of-the-art, zero-shot segmentation models to achieve rapid and accurate extraction of objects from medical images. The platform combines deep learning models with human annotation expertise in case the deep learning models struggle to segment complex structures, such as granular-appearing tissues in medical ultrasound images.

The zero-shot segmentation model used for MedAP is Meta AI’s Segment Anything Model (SAM) [32], a state-of-the-art transformer-based model designed to segment objects in images and videos with minimal user input. The model represents an innovative approach to the segmentation process by utilizing various prompt-driven annotations. SAM is selected for use in the MedAP due to its versatility in prompting options such as positive and negative Point prompting, Bounding Box prompts and Free-form text.

The MedAP annotation pipeline is presented in Figure 5. The process begins with importing ultrasound prostate sweeps, followed by MedAP-assisted annotation using prompts or manual tools to ensure precise segmentation and high dataset quality. The annotation platform stores the original image and segmentation mask pair, alongside the image metadata, including the image parameters and probe angle information required for 3D reconstruction. The created dataset is used to evaluate two deep learning segmentation models, DAF3D and MicroSegNet, on prostate phantom images in order to determine the most suitable solution for robust segmentation. DAF3D and MicroSegNet are state-of-the-art ultrasound segmentation models with specialized architectures, presented in this article and evaluated on the prostate phantom segmentation task, crucial for performing our prostate phantom reconstruction.

2.2.2. Deep Attentive Features for Prostate Segmentation (DAF3D)

DAF3D is a 3D CNN with built-in attention modules that leverage information encoded in the different layers of the CNN called multilevel features [33]. The feature extraction portion of the network is based on the 3D ResNeXt architecture [34]. The network also employs 3D atrous spatial pyramid pooling for resampling the attentive features at different rates in order to achieve a more accurate prostate representation. The model uses a hybrid loss function that combines binary cross-entropy loss and Dice loss to preserve the boundary details while producing a compact segmentation result. The model achieves values of 0.90 and 0.82 for the Dice score and the Jaccard score in the test set, respectively. After the segmentation step, the predicted masks are saved and their contours are plotted on the original image.

2.2.3. MicroSegNet

The MicroSegNet model [27], designed for micro ultrasound prostate segmentation, has introduced the annotation-guided binary cross entropy loss AG-BCE, which enables training attention on hard to segment regions. The model also introduces multi-scale deep supervision that enables robust performance, invariant to object appearance on the image. Although MicroSegNet achieved a Dice score of 0.939 and a Hausdorff distance of 2.02 $mm$, this approach struggles with the challenge of annotating the edges of the prostate and producing empty masks if the prostate is not present in the image. To address these challenges, MicroSegNet combined with the image post-processing algorithm eliminates false positive segmentation results. The image post-processing portion includes a filtering algorithm that calculates the area and the center of the segmentation mask at the prostate edges.

As part of our research, the Graphical User Interface (GUI), that enables intuitive prostate procedure planning was developed in collaboration with clinicians (T.H, T.K, T.Z.), who indicated the base requirements and useful tools for our software. The application shows the reconstructed prostate and allows the clinician to pick the biopsy target locations. The clinician can pick an ultrasound slice with suspected lesions and choose the exact target location to perform the biopsy. Upon choosing the target(s), they will appear on the reconstructed 3D visualization model.

3. Results

3.1. Model Performance Evaluation

To achieve real-time ultrasound prostate scanning and segmentation, crucial for 3D visualization and biopsy treatment planning, the first step is to compare existing deep learning solutions based on their segmentation performance. Following a thorough literature review based on the Dice score coefficient and Jaccard index, two state-of-the-art models, DAF3D and MicroSegNet, are further evaluated on the dataset that combines the open-source MicroSegNet dataset (https://zenodo.org/records/10475293) consisting of 75 prostate sweeps ranging from 40 to 60 slices per sweep, and our six sweep dataset consisting of CIRS 053L and CIRS 070L prostate phantoms with up to 300 slices per sweep (https://www.kaggle.com/datasets/lukaiktar/crta-pronobis-prostate-phantom-ultrasound-dataset). Each model was trained on the five uniform hybrid dataset splits to analyze the performance on non-biased sets of images with different augmentations. The best performing model is then trained on complete dataset and used for real-time segmentation.

The Dice score, commonly known as the Dice coefficient is defined as (1) and Jaccard score or Jaccard index is defined as (2). For binary image segmentation, y represents the ground truth segmentation and $\tilde{y}$ represents the model segmentation prediction:

$$\begin{array}{c}\hfill D(y,\tilde{y})={\displaystyle \frac{2|y\cap \tilde{y}|}{\left|y\right|+|\tilde{y}|}}\end{array}$$

(1)

$$\begin{array}{c}\hfill J(y,\tilde{y})={\displaystyle \frac{|y\cap \tilde{y}|}{|y\cup \tilde{y}|}}\end{array}$$

(2)

DAF3D and MicroSegNet models are evaluated on the hybrid dataset splits, consisting of 55 training and 20 testing real prostate sweeps from the open-source MicroSegNet dataset, along with 3 sweeps from the 053L phantom and 3 sweeps from the 070L phantom. As phantom sweeps consist 300 slices, they are randomly divided into the smaller sets and allocated to the train and test datasets. The dataset is split into five equal folds with data augmentation, to perform uniform and non-biased training evaluation. Each fold consists of 18 054 original and augmented images divided as follows: 80 % for training, 10 % for validation and 10% test images. The MicroSegNet model is trained on images of size 128x128 pixels, for 30 epochs with batch size 8, and adaptive learning rate that starts with 0.001. The DAF3D model is trained on the same folds with the same image size, for 50 epochs, adaptive batch size and mixed precision training in order to reduce computational costs. A learning rate scheduler is also used so that the learning rate is reduced once the Dice score plateaus to achieve optimal model performance. The different training parameters are derived from the optimal hyperparameter values from the respective research papers [27,33]. The training results across the folds are presented in Table 1:

MicroSegNet performed better on the hybrid dataset, thus the MicroSegNet is chosen for further training process and exploitation. The chosen model is fine-tuned on a complete hybrid dataset consisting of 2 152 real prostate images and 3 042 phantom prostate images, further enhanced with 5 194 augmented images. The augmentation procedure includes random vertical and horizontal flipping, random rotations in the range of ± 10° and random Gaussian noise addition. The model is trained on images size 224x224 pixels for 30 epochs with batch size 8 and adaptive learning rate 0.001, patch size 16 pixels and weight for the hard regions equals 4. The model achieved a Dice score of 0.943988 and a Jaccard score of 0.886911. The examples of segmentation results on prostate phantoms 035L and 070L are shown in Image 6. The model performance has been enhanced with a postprocessing algorithm that filters the segmentation of the prostate edges by applying the curvature factor threshold and area threshold, as described in Algorithm 1. A postprocessing step is needed for the complete prostate sweep segmentation because of frequent false positive model predictions on the edges, caused by unusual prostate shapes and indistinctive outer contours. The overall system processing speed is 30 Hz, which is considered real-time for transformer-based segmentation performance.

Algorithm 1 Postprocessing procedure for segmentation images

Require: List of segmentation masks M, probe angles A, angle threshold $\theta$, area threshold $T_a$, curvature threshold $T_c$ Ensure: Filtered segmentation masks $M^{\prime }$  for each $(m_i,a_i)$ in $(M,A)$ do      if $|a_i|>\theta$ then          $area\leftarrow \mathrm{computeArea}\left(m_i\right)$          if $area>T_a$ then             $m_i\leftarrow \mathrm{emptyMask}\left(\right)$          end if      end if      $curvature\leftarrow \mathrm{computeCurvature}\left(m_i\right)$      if $curvature>T_c$ then          $m_i\leftarrow \mathrm{emptyMask}\left(\right)$      end if      Append $m_i$ to $M^{\prime }$  end for  return  $M^{\prime }$

The state-of-the-art model results on prostate sweep with included edges and background images are primarily achieved by area and curvature filtering on the edges. Segmentation results fused with the robot arm positional information are stored in the DICOM format for further analysis, such as mask contour extraction and 3D visualization. An intuitive and detailed 3D prostate visualization, enriched with the list of recorded 2D slices, is required for biopsy treatment planning.

3.2. Prostate Reconstruction

After the segmentation of each ultrasound slice, the prostate gland is reconstructed using the contours of segmented masks. As discussed in Section 2, the robot arm, along with the ultrasound image, sends information about the angle of the probe, synchronized in time with image acquisition. The obtained slices, used for reconstruction and prostate visualization are not parallel because the probe rotates around its axis. The 3D reconstruction is achieved by transforming each slice of the sweep from its coordinate frame (the slice is assumed to be positioned in the xy plane) to the visualization frame $x^{\prime }{y}^{\prime }{z}^{\prime }$ by using equations (3 - 5). The visualization frame is aligned with the frame of the initial slice, which will be in the middle of the prostate due to such probe placement in standard clinical practice.

$$\begin{array}{cc}\hfill x^{\prime }& =x\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill y^{\prime }& =cos\left(\varphi \right)\ast y\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill z^{\prime }& =sin\left(\varphi \right)(y+r)\hfill \end{array}$$

(5)

Using this method, we calculate the point-cloud representation of the prostate gland’s outer surface. Ultrasound sweep and the corresponding prostate representation enable the clinician to analyze suspect lesions and mark biopsy targets, as shown in Figure 7. This approach is tested and evaluated on prostate phantoms, as the 3D reconstructed phantom’s dimensions are compared to the specification measurements of the CIRS 053L and CIRS 070L phantoms. CIRS 070L phantom specifications do not provide the dimensions, so it is evaluated only on the specified volume. The reconstruction was tested on 10 phantom sweeps performed with the same pressure force applied perpendicular to the surface plane, along the image y direction, of 6.5 N for CIRS 053L and 9 N for CIRS 070L. The reconstructed point clouds are used to generate a surface approximation in MeshLab, then measured and compared with the datasheet specifications to assess reconstruction accuracy. For the dimensional measurements we use the outer dimensions of the bounding box aligning with $x^{\prime }{y}^{\prime }{z}^{\prime }$ coordinate frame, and the volume is calculated by SolidWorks software. After 10 measurements, the average dimensions of the CIRS 053L prostate phantom, given in Table 3, are 55.9x42.9x37.3 mm and the volume of 54 058.2 $m{m}^3$, for the given dimensions of the fantom of 50x45x40 mm and the volume of 53 $c{m}^3$. The volume difference of the reconstructed models is within 2% of the specified volume. There are larger differences in dimensions, which stem mostly from the deformations caused by the presence of the probe but are also affected by the inaccuracies in probe calibration, inaccuracies in segmentation, and differences in measuring methodology. The average volume of CIRS 070L is 53 217.6 $m{m}^3$, with a given volume of 49 $c{m}^3$. This phantom has a larger difference in volume than the first one, most probably due to the unclear prostate dimensions used for volume calculation. The presented robotically navigated ultrasound-only prostate scanning and 3D reconstruction evaluated on our phantoms demonstrated results comparable to the volumetric and dimensional results of MR-ultrasound solutions [17].

Table 2. Reconstruction performance over 10 sweeps - phantom CIRS 053L.

	Length / [mm]	Width / [mm]	Height / [mm]	Volume / [${\mathit{m}\mathit{m}}^3$]
Mesured	55.9 ± 0.47	42.9 ± 0.42	37.3 ± 0.61	54058.2 ± 652.4
Ground truth	50	45	40	53000

Table 2. Reconstruction performance over 10 sweeps - phantom CIRS 053L.

	Length / [mm]	Width / [mm]	Height / [mm]	Volume / [${\mathit{m}\mathit{m}}^3$]
Mesured	55.9 ± 0.47	42.9 ± 0.42	37.3 ± 0.61	54058.2 ± 652.4
Ground truth	50	45	40	53000

Table 3. Reconstruction performance over 10 sweeps - phantom CIRS 070L. Dimensional ground truth data is not specified.

	Length / [mm]	Width / [mm]	Height / [mm]	Volume / [${\mathit{m}\mathit{m}}^3$]
Mesured	58.0 ± 0.16	43.9 ± 0.43	37.4 ± 0.19	53217.6 ± 546.6
Ground truth	-	-	-	49000

Table 3. Reconstruction performance over 10 sweeps - phantom CIRS 070L. Dimensional ground truth data is not specified.

	Length / [mm]	Width / [mm]	Height / [mm]	Volume / [${\mathit{m}\mathit{m}}^3$]
Mesured	58.0 ± 0.16	43.9 ± 0.43	37.4 ± 0.19	53217.6 ± 546.6
Ground truth	-	-	-	49000

Figure 7. Graphical User Interface. The program is divided into three different interactive visualizations. In the top left corner, a reconstructed 3D model of the prostate gland is shown. The green cylinders represent biopsy targets picked by the clinician. The clinician can zoom in/out and rotate the model (using mouse and keyboard) to examine it. Next to the model is a small representation of the male human body, to check the prostate orientation. The bottom part contains all the US images recorded during the sweep, while the right top corner shows the selected US image on which the clinician can select an area from which a sample will be taken via biopsy.

Figure 7. Graphical User Interface. The program is divided into three different interactive visualizations. In the top left corner, a reconstructed 3D model of the prostate gland is shown. The green cylinders represent biopsy targets picked by the clinician. The clinician can zoom in/out and rotate the model (using mouse and keyboard) to examine it. Next to the model is a small representation of the male human body, to check the prostate orientation. The bottom part contains all the US images recorded during the sweep, while the right top corner shows the selected US image on which the clinician can select an area from which a sample will be taken via biopsy.

4. Discussion

This paper presents a robotically navigated proof-of-concept system for prostate scanning and biopsy treatment planning. It utilizes a robot arm with attached TRUS probe for real-time sweep scanning, followed by deep learning-based segmentation, necessary for 3D prostate reconstruction. The accurate prostate reconstruction enables thorough analysis and biopsy needle path planning. On contrary to hybrid methods that combine MR and ultrasound imaging, our ultrasound-only, scanning and prostate reconstruction system, evaluated on CIRS Prostate Training Phantom Model 070L and CIRS Tissue Equivalent Ultrasound Prostate Phantom Model 053L, results in 92% and 98% volumetric accuracy for 070L and 053L, respectively. This paper utilizes a hybrid dataset of open-source prostate sweeps and our prostate phantom sweeps to enable fully automated real-time prostate scanning and segmentation at 30Hz. To create a balanced dataset, we developed MedAP, a Medical Annotation Platform utilizing zero-shot segmentation models. State-of-the-art ultrasound segmentation models were evaluated, and the best-performing algorithm was enhanced with postprocessing to discard outliers at prostate edges, achieving a Dice score of 0.944 and a Jaccard score of 0.887.

To facilitate the selection of target points for prostate needle path planning, we have developed a GUI that helps the clinician examine the prostate and select the exact points to target during the biopsy procedure. The presented end-to-end software system is conceptualized, designed and validated with valuable insights from urological clinicians who the system is aimed for. Our work so far presents foundation for a more sophisticated approach that aims to address the challenges of patient movement, like presented in [35], as well as prostate deformation, especially tissue inflammation caused by biopsy needle penetration. Furthermore, an important consideration for future work is the validation of biopsy precision and accuracy by measuring the distance between the target and location where the sample is taken from.

Also, a vital consideration for the future is to automate the process of targeting locations i.e. lesions and other areas of interest. Furthermore, human robot interaction in surgical scenarios will be considered in the next step to facilitate intuitive interaction with the system [36].