Deployment of an Optimised Brain Tumour Segmentation Model on the ZCU104 FPGA Using Xilinx Vitis AI

1. Introduction

Brain tumour segmentation from magnetic resonance imaging (MRI) remains a challenging task due to the variability in tumour appearance, size, and location across patients. Accurate delineation of tumour sub-regions, including whole tumour (WT), tumour core (TC), and enhancing tumour (ET), is required to support diagnosis, treatment planning, and monitoring of disease progression. Manual segmentation is time-intensive and subject to inter-observer variability, which has led to the development of automated methods based on deep learning [4,5].

Convolutional neural networks have achieved strong performance on brain tumour segmentation benchmarks, particularly on the Brain Tumour Segmentation (BraTS) dataset [4]. Encoder–decoder architectures, such as U-Net [6], learn spatial representations that enable accurate localisation of tumour regions. Subsequent work has incorporated attention mechanisms and multi-scale feature aggregation to improve segmentation of small and irregular tumour structures. RAAGR2-Net [7] extends this line of work by integrating recurrent residual blocks, attention gates, and atrous spatial pyramid pooling within a modified Y-Net backbone, reporting competitive Dice scores on the BraTS 2019 dataset.

Despite these advances, deploying segmentation models on edge devices poses challenges. Many architectures are developed and evaluated on GPU platforms, where memory and compute resources are readily available. In contrast, clinical environments may require models to operate under constraints on power consumption, cost, and physical infrastructure. Field-programmable gate arrays (FPGAs) provide an alternative deployment platform, offering hardware acceleration with lower power requirements and predictable latency, making them suitable for edge-based diagnostic systems [8].

Adapting deep neural networks for FPGA deployment requires reducing model complexity while preserving segmentation performance. Network pruning is commonly used for this purpose by identifying parameters that contribute little to model output and removing or suppressing them [9]. Magnitude-based pruning removes weights below a threshold, but this criterion does not account for the role of features in segmentation. Sensitivity-based approaches provide an alternative by evaluating how changes in network parameters affect model output. Perturbation-based sensitivity analysis, which measures the gradient of output differences under input perturbations, provides a label-independent estimate of channel importance that can be applied without relying on ground truth annotations [10].

Limited work has examined the interaction between pruning strategies and FPGA deployment in medical image segmentation. In particular, the use of perturbation-based sensitivity analysis for channel selection in brain tumour segmentation has not been explored in the context of hardware deployment. This work addresses this gap by applying three pruning strategies to RAAGR2-Net trained on the BraTS 2019 dataset.

The final approach applies sensitivity-based channel selection within a controlled pruning framework, supporting stable feature representation during pruning.

The pruned model is deployed on the Xilinx ZCU104 FPGA using the Vitis AI 3.0 toolchain. Results show that weight masking is compatible with INT8 quantisation and DPU compilation, and that segmentation accuracy across WT, TC, and ET regions is maintained or improved after pruning.

Section 2 describes the hardware platform and deployment pipeline. Section 3 presents segmentation and inference results. Section 4 discusses the findings. Section 5 concludes the paper.

2. Materials and Methods

The work presented in this section focuses on the deployment of an optimised RAAGR2-Net brain tumour segmentation model [7], on an embedded FPGA platform. The segmentation network was first trained using brain MRI data and later reduced in complexity through channel pruning by masking. It should be noted that the training stage was carried out using a reduced training subset of 551 slices from the available 5,512 training slices, representing approximately 10% of the training data. This was done because of computational and hardware constraints during model optimisation and deployment preparation. Although the validation set was retained for performance evaluation, training on only 551 slices is a significant limitation because it may restrict the diversity of anatomical and tumour appearances seen by the model during learning. Therefore, the reported results should be interpreted as deployment-oriented findings based on a limited training subset, rather than as a fully trained clinical segmentation model. This optimisation stage suppressed redundant channels through weight masking, yielding a slight improvement in segmentation performance while leaving the network dimensions unchanged. The deployment served to verify that masked channels do not contribute to inference before structural pruning is applied

The model pruning process was designed to maintain segmentation stability while removing redundant channels in selected layers of the network. As a result, the optimised network became suitable for deployment on resource-constrained hardware platforms such as the Xilinx ZCU104 evaluation board. The objective of this deployment stage was therefore to evaluate how the optimised RAAGR2-Net segmentation model performs when executed on an FPGA-based deep learning accelerator using the Vitis AI toolchain. The deployment pipeline included preparation of the host environment, model calibration, post-training quantisation, compilation for the Deep Processing Unit (DPU), and runtime inference on the target board.

2.1. Hardware Platform

The deployment experiments in this study were carried out on the Xilinx ZCU104 evaluation board [11], which is based on the Zynq UltraScale+ MPSoC architecture. The platform integrates programmable logic with an embedded processing system, enabling hardware acceleration of deep learning models through the Deep Processing Unit (DPU). This heterogeneous architecture makes the board suitable for edge artificial intelligence applications where computational efficiency and low power consumption are important. In this work, the ZCU104 board served as the target device for executing the optimised RAAGR2-Net brain tumour segmentation model after model quantisation and compilation using the Vitis AI 3.0 toolchain [11]. The ZCU104 development board is built around the Zynq UltraScale+ MPSoC, which integrates an embedded processing system with reconfigurable FPGA logic on the same device. The platform includes a quad-core ARM Cortex-A53 processor together with programmable logic resources and several high-speed communication interfaces that support embedded computing tasks. This architecture allows control operations to be managed by the ARM processor while computationally demanding parts of the neural network are executed in the programmable logic. For deep learning deployment, Xilinx provides a configurable Deep Processing Unit (DPU) that runs in the FPGA fabric and performs operations such as convolution, pooling, and activation layers. In this study, the DPU was used as the hardware accelerator responsible for executing the optimised RAAGR2-Net segmentation network. The board was prepared using the official Vitis AI system image for Zynq UltraScale+ platforms, which includes the required runtime libraries, drivers, and utilities for running compiled neural network models on the device [11].

Figure 1. Xilinx ZCU104 development board used for the deployment experiments.

Figure 1. Xilinx ZCU104 development board used for the deployment experiments.

2.2. Host Development Environment

The host development environment used in this work was prepared on a Windows-based computer running Ubuntu Linux inside a VirtualBox virtual machine. This configuration provided a stable Linux environment required for the installation of the Xilinx Vitis AI development tools. The Vitis AI environment distributed by Xilinx is provided through Docker containers [2], which simplify the installation of the required libraries, compilers, and deep learning tools needed for model preparation and deployment on FPGA platforms. Within the Ubuntu environment, the project repository containing the RAAGR2-Net segmentation model and associated scripts was downloaded and prepared for deployment. The model and dataset were organised within the workspace so that the training outputs could be processed for FPGA execution using the Vitis AI toolchain. The Vitis AI development environment provides tools for quantisation, compilation and deployment of deep neural networks on FPGA devices [11]. The host machine and Vitis AI software was used to perform model preparation tasks including dataset organisation, model conversion, quantisation preparation, and compilation for the target fpga device. Using this environment, the optimised RAAGR2-Net model, which had previously undergone channel pruning for computational reduction, was prepared for deployment through the Vitis AI workflow. This included preparing the model files, organising calibration datasets, and configuring the compilation tools required to generate a deployable model for the Deep Processing Unit (DPU) on the target hardware platform [11,12]. DPU for the target architecture is in Figure 2 below.

The Deep Learning Processing Unit (DPU) is a hardware accelerator designed to support deep neural network inference. It is implemented as configurable intellectual property (IP) cores on programmable logic, enabling efficient execution of convolutional neural network operations without requiring low-level hardware design by the user.

The DPU is used within the Vitis AI framework, which provides a dedicated instruction set for deploying trained models onto supported hardware platforms. This enables practical acceleration of common computer vision tasks such as image classification, object detection, and semantic segmentation.

The architecture supports a range of widely used neural network models, including VGG, ResNet, GoogLeNet, YOLO, SSD, and MobileNet. It can be deployed across multiple hardware platforms, including Zynq UltraScale+ MPSoCs, Kria devices, Versal architectures, and Alveo accelerator cards, allowing deployment from edge systems to data centre environments [11].

Figure 3. Vitis AI Integrated Development Environment workflow for preparing and deploying deep neural networks on FPGA platforms.

Figure 3. Vitis AI Integrated Development Environment workflow for preparing and deploying deep neural networks on FPGA platforms.

2.3. ZCU104 FPGA Platform Setup

The hardware platform used for executing the deployed segmentation model was the Xilinx ZCU104 evaluation board, which is based on the Zynq UltraScale+ MPSoC architecture. This platform integrates a processing system together with programmable logic on a single device, enabling hardware acceleration of deep neural network inference through the Deep Processing Unit (DPU). The board was configured using the appropriate system image and prepared for communication with the host development machine. It was powered and the connection was established through the serial communication port.

For board initialization, the required boot image was written to a MicroSD card and inserted into the SD card slot of the ZCU104 board. The boot configuration switches were then set to enable booting from the SD card. Once the board was powered using the external 12V power supply, a USB cable was connected between the host computer and the board to allow serial communication through a terminal interface.

An Ethernet connection was also established between the ZCU104 board and the local network to allow remote access from the host machine. A terminal emulator (putty software) was used to access the board through the serial interface in order to monitor the boot process and identify the network configuration of the device. Once the board had successfully booted, the network address of the device was obtained using system commands on the board. This IP address was later used to establish a remote connection from the host machine to the FPGA platform, allowing deployment and execution of the compiled neural network model.

2.4. Project Workspace Preparation

After the Vitis AI development environment was successfully started inside the Docker container, the project workspace was prepared for the deployment process. The repository containing the segmentation model and associated scripts was first downloaded to the host machine and extracted within the working directory of the Vitis AI container. This workspace served as the central location where the model files, dataset, and deployment scripts were organised. The trained model produced during the training stage was placed inside the workspace together with the supporting scripts required for quantisation, compilation, and inference. The dataset used for calibration and testing was also copied into the same environment to allow the quantisation tools to analyse representative input samples. Organising the files in this way allowed the Vitis AI toolchain to access the trained model, calibration data, and configuration scripts required to prepare the model for execution on the FPGA platform.

2.5. Deployment Workflow for the Optimised RAAGR2-Net Model

This work focuses on the deployment of an optimised RAAGR2-Net brain tumour segmentation model on FPGA hardware using the Vitis AI 3.0 toolchain. The model used in this study had previously been trained on the BraTS dataset and later optimised through channel pruning in order to increase sparsity while maintaining segmentation accuracy. The pruning approach applied in this work aimed to remove redundant channels while preserving the stability of the segmentation network. The deployment process consists of a series of stages beginning from model training, preparation on the host machine and ending with execution on the FPGA-based Deep Processing Unit (DPU). These stages include model export, preparation of calibration data, model quantisation, compilation for the target hardware architecture, and deployment to the ZCU104 FPGA platform. The following stages summarise the deployment pipeline implemented in this study:

1.

Model preparation The trained RAAGR2-Net segmentation model was prepared on the host development environment installed on the virtual machine. The model weights obtained from the training stage were put together with the required preprocessing scripts and configuration files so that they could be processed by the Vitis AI 3.0 toolchain.

2.

Calibration dataset preparation A representative subset of MRI images from the BraTS dataset was selected to serve as calibration data. This dataset is required during quantisation, the end result of quantization is converting floating-point operations to lower precision representations.

3.

Model quantisation The initial trained model based on floating-point values, was processed using the Vitis AI quantisation tools to convert the network into an INT8 representation suitable for FPGA execution. This in effect reduces computational cost and memory requirements while maintaining acceptable segmentation accuracy. .

4.

Model compilation After quantisation, the model was compiled using the Vitis AI compiler. The compiler transforms the quantised model into an executable representation compatible with the Deep Processing Unit (DPU) architecture available on the ZCU104 FPGA platform.

5.

Deployment on FPGA The compiled model is transferred to the ZCU104 board where inference will be executed using the Vitis AI 3.0 runtime environment. This stage allows the evaluation of inference latency, throughput, and computational efficiency of the optimised RAAGR2-Net model on edge hardware.

The pruned RAAGR2-Net model used in this deployment was trained on 551 samples (10% of the BraTS 2019 dataset of 5,512 slices used) over 50 epochs with a batch size of 16. Prior to deployment, the pruned model was evaluated against the base RAAGR2-Net on 131 held-out samples. The pruned model achieved a Tumour Core (TC) Dice of 0.8578, Enhancing Tumour (ET) Dice of 0.8131, and Whole Tumour (WT) Dice of 0.8635, compared to 0.8169, 0.7896, and 0.8424 for the base model, respectively. Mean IoU also improved from 0.7900 to 0.8212. These results confirm that the pruning process did not degrade the segmentation performance (Dice scores across TC, ET, and WT regions) prior to FPGA deployment.

3. Results

This section reports segmentation performance of the base and pruned RAAGR2-Net models evaluated on 131 BraTS validation samples, followed by inference performance recorded during deployment on the Xilinx ZCU104 FPGA

3.1. Segmentation Performance Before Deployment

Table 1 summarises the segmentation performance of the base and pruned RAAGR2-Net models on CPU, evaluated on 131 BraTS samples before FPGA deployment.

The pruned model recorded higher Dice scores than the base model across all three tumour sub-regions. TC Dice increased from 0.8169 to 0.8578 (+0.0409), ET Dice from 0.7896 to 0.8131 (+0.0235), and WT Dice from 0.8424 to 0.8635 (+0.0211). Mean IoU also improved from 0.7900 to 0.8218 (+0.0318), and the overall Dice score increased from 0.9858 to 0.9883 (+0.0025). The largest gain was observed in TC Dice (+0.0409), suggesting improved representation of core tumour structures after channel suppression. Although the Dice-based validation metrics improved, the validation loss increased slightly from 0.0573 to 0.0637.

Table 2 presents the inference performance of the original and pruned RAAGR2-Net models on CPU using 131 validation samples. The original model achieved an average latency of 63.08 ms and a throughput of 15.85 FPS, while the pruned model achieved an average latency of 66.01 ms and a throughput of 15.15 FPS. These results indicate that the masked pruned model did not provide a direct CPU speed-up. This is expected because the pruning approach applied channel masking rather than structural channel removal, so the network architecture and tensor dimensions remained unchanged during inference. However, the pruned model showed a lower standard deviation and a smaller maximum latency than the original model, suggesting more stable execution behaviour on CPU.

On CPU, the original model achieved slightly lower average latency and slightly higher throughput than the pruned model. This indicates that masked channel pruning did not produce a direct CPU speed-up. Since the pruning method used weight masking rather than structural channel removal, the computational graph remained unchanged during inference. However, the pruned model showed lower latency variation and a lower maximum latency, suggesting more stable execution behaviour.

Table 3 presents the GPU inference performance of the original and pruned RAAGR2-Net models. The pruned model achieved a lower average latency of 67.58 ms compared to 75.28 ms for the original model, and a higher throughput of 14.80 FPS compared to 13.28 FPS. In addition to improved average performance, the pruned model demonstrated significantly more stable execution behaviour. The standard deviation of latency decreased from 84.79 ms to 9.24 ms, and the maximum latency reduced from 1040.28 ms to 100.91 ms. This substantial reduction in worst-case latency indicates that the pruned model provides more consistent inference performance on GPU. These results suggest that although channel masking does not structurally reduce the network, it can still influence execution characteristics on parallel hardware platforms.

Masked pruning does not guarantee speed-up on CPU, but can improve execution behaviour and stability on parallel hardware, while maintaining segmentation performance.

3.1.1. FPGA Inference Results

Following quantisation and compilation through the Vitis AI 3.0 toolchain, the pruned model was deployed on the Xilinx ZCU104 board and inference was executed on all 131 test samples. Table 4 reports the inference performance recorded on the DPUCZDX8G hardware accelerator.

Across 131 samples, the Vitis AI runtime recorded a total inference time of 9.06 seconds (9,060 ms), giving an average latency of 69.16 ms per sample (9,060 / 131). Throughput is calculated as 131 / 9.06 = 14.46 FPS. On the ZCU104, the runtime measures elapsed time from input submission to the DPU through to output return, averaged across all samples. The standard deviation of 0.08 ms confirms stable execution across inputs, consistent with fixed-function hardware where scheduling variance is lower than on GPU platforms.

The average inference latency of 69.16 ms per sample with a standard deviation of 0.08 ms indicates consistent execution across all samples.

At 14.46 FPS, the model processes approximately 14 MRI slices per second on the ZCU104, which is sufficient for clinical workflows where scans are submitted and processed individually on demand. Strict real-time processing is not required in this context, as MRI acquisition itself is not a continuous stream.

The low variance between minimum and maximum latency (68.96 ms to 69.57 ms) confirms stable execution of the quantised model on the ZCU104 platform.

Zeroed weights passed through the Vitis AI quantisation and compilation stages without error, and inference completed successfully on all 131 samples. The results show that masked channels did not disrupt compilation or execution on the DPUCZDX8G. This suggests that the deployment pipeline processes zero-valued weights without requiring structural modification, which has implications for gradual pruning strategies that precede structured pruning. The low variance between minimum and maximum latency (68.96 ms to 69.57 ms) confirms stable execution of the quantised model on the ZCU104 platform. While sparsity is typically associated with reduced hardware efficiency, the results show that masked channels did not disrupt compilation or execution on the DPUCZDX8G. This suggests that the deployment pipeline processes zero-valued weights without requiring structural modification, which has implications for gradual pruning strategies that precede structured pruning.

Table 5 compares inference performance across CPU, GPU, and ZCU104 FPGA platforms. On CPU, the original model achieved slightly lower average latency than the pruned model, indicating that masked channel pruning did not result in direct execution speed-up. This is expected because pruning was applied through weight masking rather than structural channel removal, leaving the computational graph unchanged.

On GPU, the pruned model achieved lower average latency and higher throughput compared to the original model. In addition, the pruned model demonstrated significantly improved execution stability, with a reduction in latency standard deviation and a substantial decrease in maximum latency from 1040.28 ms to 100.91 ms. This indicates more consistent execution behaviour on parallel hardware.

The deployed pruned model on the ZCU104 FPGA achieved an average latency of 69.16 ms and a throughput of 14.46 FPS, with very low latency variation. These results confirm that the masked sparse model can be successfully deployed on edge hardware while maintaining stable and efficient inference performance.

Overall, the results demonstrate that although masked pruning does not guarantee speed improvements on CPU, it can improve execution characteristics on parallel and hardware-accelerated platforms, while preserving segmentation performance.

These results reinforce the novelty of the proposed stability-constrained pruning approach, where channels are selected based on perturbation-aware importance and stability criteria.

The method enables controlled sparsification that maintains segmentation quality while improving execution consistency, making it suitable for reliable deployment and a preparatory step toward structural pruning for edge systems.

3.2. Deployability of the Pruned Model

The deployment experiment confirms that weight masking applied during pruning does not prevent successful quantisation, compilation, or inference on the ZCU104 FPGA. The model was processed through the full Vitis AI pipeline, including INT8 post-training quantisation and DPU compilation, without modification to the pruned weight structure. Inference completed successfully on all 131 samples, producing valid segmentation predictions. Combined with the segmentation results in Table 1, Overall, the results indicate that channel-level masking can be integrated into the Vitis AI deployment pipeline without restructuring the network, while maintaining segmentation accuracy and enabling execution on FPGA hardware.

4. Discussion

The experimental results address two related points: whether sensitivity-based weight masking preserves segmentation accuracy, and whether the masked model can be quantised, compiled, and executed on an FPGA platform. Both are supported by the observed outcomes.

The pruned RAAGR2-Net achieved higher Dice scores than the base model across all tumour sub-regions on the 131-sample evaluation set. TC Dice increased from 0.8169 to 0.8578, ET Dice from 0.7896 to 0.8131, and WT Dice from 0.8424 to 0.8635. The increase in Mean IoU from 0.7900 to 0.8218 is consistent with these improvements. In contrast, the overall loss rose from 0.0573 to 0.0637. This difference reflects the interaction between weighted loss components and the distributional changes introduced by pruning. The loss function aggregates multiple outputs, and a marginal increase in one component can raise the total value even when segmentation overlap improves across all target regions. Further analysis is required to isolate the contribution of each loss component under pruning.

A subset of the dataset was used during training, with 551 samples selected from the 5,512 available BraTS slices to reduce computational cost during deployment evaluation. This introduces a limitation in terms of generalisation. Models trained on reduced data may not capture the full variability present in the dataset. Full dataset training is currently in progress to assess whether the observed performance gains remain consistent at scale.

Deployment results show consistent execution on the ZCU104 FPGA. The average latency of 69.16 ms per sample, with a standard deviation of 0.08 ms, indicates stable inference across inputs. The throughput of 14.46 FPS aligns with batch-based clinical processing, where scans are analysed individually rather than as continuous streams. Direct comparisons with FPGA-based brain tumour segmentation models are limited in the available literature. This reflects the relatively small body of work combining pruning with FPGA deployment for this task, providing context for the contribution presented here.

The deployment pipeline did not require modification of the masked weight structure. Zeroed weights passed through the Vitis AI quantisation and compilation stages without error, and inference completed successfully on all evaluation samples. These results indicate that weight masking is compatible with the Vitis AI 3.0 toolchain and the DPUCZDX8G architecture on the ZCU104 platform.

Two areas require further investigation. Structural pruning, which removes channels rather than masking weights, would reduce tensor dimensions and may lead to measurable reductions in inference latency and memory usage on the FPGA. The current results provide a baseline for evaluating such changes. In addition, power consumption during inference on the ZCU104 has not been measured and remains necessary for a complete assessment of efficiency in edge deployment scenarios.

5. Conclusions

Deployment of a pruned RAAGR2-Net model for brain tumour segmentation was examined on the Xilinx ZCU104 FPGA using the Vitis AI 3.0 toolchain. Pruning was implemented through weight masking, where channels identified as redundant were suppressed by setting their values to zero without modifying the network structure. The evaluation focused on whether such a model remains functional when transferred to resource-constrained hardware.

The pruned network was successfully quantised to INT8 precision, compiled for the DPUCZDX8G accelerator, and executed on the ZCU104 platform. Across 131 evaluation samples, the average inference latency was 69.16 ms per sample, with a throughput of 14.46 FPS. The latency standard deviation of 0.08 ms indicates consistent execution across inputs.

Performance was assessed prior to deployment by comparing the masked model with the baseline RAAGR2-Net on the same evaluation set. TC Dice increased from 0.8169 to 0.8578, ET Dice from 0.7896 to 0.8131, and WT Dice from 0.8424 to 0.8635. These results show that suppressing low-importance channels did not reduce segmentation accuracy, with improvements observed across all tumour sub-regions.

The results indicate that sensitivity-guided channel pruning can be integrated into the Vitis AI deployment pipeline without disrupting execution on FPGA hardware. Structural channel removal, full dataset training, and power measurement on the ZCU104 remain as next steps toward a complete efficiency assessment.