Boundary Segmentation of Vascular Images in Fourier Domain Doppler Optical Coherence Tomography Based on Deep Learning

1. Introduction

Microscopic and ultra-microscopic vascular anastomosis is extremely challenging, which is the basis of plastic surgery and organ grafting [1]. The intraoperative vascular status assessment is of great importance. It can provide effective evaluation parameters such as thrombus formation, lumen restoration for potential early re-intervention [2]. The early intervention can not only improve the success rate of the surgery, but also reduce the expense of patients. The Doppler optical coherence tomography (DOCT) is a high-resolution non-invasive 3D imaging modality. As an assisting tool, it has demonstrated its special advantages in surgical evaluation [3,4].

Boundary segmentation of vessel wall from Doppler OCT intensity image and the segmentation of the inner lumen contour from Doppler OCT phase image are essential for the 3D morphology reconstruction of the vessel. Due to the strong scattering of red blood cells and other medium in vessel, the images suffer from low contrast and blurred boundary especially at the greater depth of the vessel. And the OCT images contain severely speckle noise and random noise, which increases the complexity of the boundary segmentation. Therefore, the algorithms for accurate boundary segmentation are demanded, which should be robust to the OCT noise and very good at the weak boundary segmentation.

There exist many boundary segmentation approaches for OCT vessel images. Earlier studies usually focus on pixel intensity changes, gradient information, Chan-Vese model, graph-based method. The pixel intensity changes along the A-Scan contour were used to achieve image segmentation [5,6,7,8], while most segmentation methods using intensity information are based on intensity changes of backscattered signals [9,10,11,12,13,14]. Gasca et al. [15] extracted the boundary in intensity images, but it was generally difficult to extract the weak boundary accurately. Sihan et al. [16] used Canny filter for edge detection, and then used link step for segmentation. However, due to the fixed threshold, this method has poor segmentation effect on noisy images. Yang et al. [17] proposed a segmentation algorithm based on dual-scale gradient information, which simultaneously uses the global gradient information and local gradient information to complement each other. And it adopts the shortest path search algorithm to optimize the edge in combination with the local Canny edge detection method, which has high accuracy and repeatability for the segmentation of 3D OCT volume data. However, this method is not robust to noise. Guimaraes et al. [18] used intensity images to achieve retinal segmentation, and divided different thresholds according to different tissues to achieve retinal layer segmentation. The computational efficiency of this method is very high. But due to the influence of the intensity variation in the layer, the intensity inconsistency causes artifacts in the blood vessels during the imaging process. Most of the segmentation methods based on image intensity information are used for eye image segmentation, which the extraction ability of weak boundary is insufficient. Chan-Vese (C-V) model is a classical region-based geometric active contour model, which can segment images without obvious edges well. The method based on C-V model first needs to set up an initial contour curve, and use the gradient information to get the optimal result. In 2006, Grady applied the random walk method to the field of image segmentation [19], introducing regularization into the segmentation process. This method can extract the weak boundary of images, but in the weak boundary extraction process of OCT images, the staircase effect will appear, resulting in incorrect segmentation. Roy et al. [20] used the random walk algorithm and signal attenuation model of OCT imaging process to track the maximum optical backscatter of each A-scan. And then used global gray scale statistics to optimize and achieve image segmentation. Huang et al. [21] used the Laplacian operator to perform iterative calculation by changing the calculation operator of the random walk, which showed good robustness to the weak boundary of OCT images. The graph-based approach is used to achieve segmentation of retinal in time domain OCT images [22,23,24] and OCT coronary vessel images [25]. Garvin et al. [26,27] also extended this method to spectral OCT images, adding flexible and diverse constraints and greatly improving segmentation accuracy. Nabila et al. [28] segmented retinal vessels of OCTA with generalized Gauss-Markov-Gibbs random field model and Markov-Gibbs random field model. Ruchir et al. [29] used the method of 3D map cutting to segment the skin surface layer. In 2022, Mittal et al. proposed to use random walk and interframe flattening algorithm to process OCT images. They used N-ret layer segmentation method to simplify OCT image segmentation and improve the accuracy of OCT image segmentation [30].Traditional vascular segmentation methods often rely on shallow image features, such as image gray scale and texture features, which have low representativeness and are easy to be disturbed by noise. Moreover, such features rely on designers’ experience and knowledge accumulation to design extraction algorithms. Practice has proved that the deep features are more representative. Machine learning techniques can extract deep features from massive data. In different application scenarios, more representative features can be extracted from the training data and applied to the blood vessel segmentation task, which can not only improve the accuracy of blood vessel segmentation, but also improve the segmentation efficiency. . The sparse representation and discriminant dictionary learning methods were used to vascular segmentation. According to the feature vector of pixels, each pixel is classified into blood vessels and non-blood vessels, so as to achieve the segmentation and extraction of retinal blood vessels [31,32,33]. In recent years, the powerful segmentation ability of deep learning has gradually been widely applied to the segmentation field of medical images. Roy et al. [34] proposed RelayNet for end-to-end segmentation of retinal layers and fluid masses in ocular OCT images. Fan et al. [35] used a multi-channel FCN to automatically segment coronary arteries from angiographic images. Hamwood et al. [36] studied the impact of batch size and network structure on retinal segmentation results. Girish et al. [37] applied the FCN model to the segmentation of the inner retinal capsule membrane. This plays an important role in ophthalmic diagnosis and quantification of retinal abnormalities. In recent years, many scholars have combined traditional methods with deep learning methods to achieve image segmentation. Fang et al. [38] combined deep learning with Graph Search to propose a Convolutional Neural Networks-Graph search (CNN-GS) architecture for automated segmentation of 9-layer boundaries of retinal OCT images. Zhang et al. [39] combined convolutional neural networks with random walk algorithms to segment plaques in OCT coronary artery images. The CFANet method proposed by MaFei et al. has achieved high accuracy in the segmentation of OCTA optic disc and macular region [40]. Yazan et al. [41] used SegNet in combination with conditional random fields to segment the lumen and calcified area of OCT coronary arteries, helping to determine the placement of stents during coronary interventional therapy. Ronneberger et al. [42] proposed full convolutional neural network U-net for the segmentation of neuron structure under electron microscope and the segmentation and extraction of cell contour under light microscope. More recent studies commonly use its variants for layer-wise retina segmentation [43,44,45]. The cascaded U-net architecture was proposed to segment the vessel intensity image and its corresponding phase image for the vessel wall and the blood flow area boundary, respectively [46]. It contains two U-nets, which means that the second segmentation results rely on the first segmentation results. The limitation of this method is that two models are trained and optimized separately, which takes more time to obtain the optimized models. Besides, when integrating the models into the system for online application, it is more complicated to call two models than one.

Due to the strong scattering of blood in the blood vessels, with the increase of imaging depth, the sensitivity of the OCT system decreases. And the signal-to-noise ratio of the images decreases, resulting in the blurring of the lower edge of the blood vessels. At the same time, OCT images contain noise, and noise levels in phase images are generally higher than in intensity images, increasing the difficulty and complexity of accurately segmenting the contour of blood vessel edges. The accurate extraction of the contour of blood vessel wall and blood flow region is particularly crucial for the reconstruction of the three-dimensional structure of blood vessel and the three-dimensional morphology of thrombus.

In this work, we proposed a multi-classification deep learning model to extract the vessel boundary from Doppler OCT intensity images and the lumen area contour from corresponding Doppler OCT phase images simultaneously and automatically. There are two reasons for using the U-net structure in this study. Firstly, the principle of OCT is a fiber optic Michelson interferometer. The interference occurs when the optical path difference between the reference light returned by the mirror and the backscattered light of the measured sample is within the coherence length of the light source. The detector output signal reflects the backscattering intensity of the medium. While the shallow and deep layers of U-Net acquire high-frequency and low-frequency information from the image, respectively, and the information of each level is well preserved through skip connections. Secondly, the scanning mirror can record its spatial position so that the reference light interferes with backscattered light from different depths within the medium. According to the position of the mirror and the corresponding interference signal intensity, the measurement data of different depths (z direction) of the sample can be obtained. U-Net can reconstruct the spatial structure with self-similarity and use the handcrafted image and the structure of the known part to interpolate the unknown region [48].

The pair of intensity image and phase image are preprocessed into one 512×512 grayscale image as the input of the model. The pixel categories of the input image are divided into three categories: background, vessel region and inner lumen area. The average Dice Coefficient (DC) adopted for the quantitative analysis of the segmentation results reaches 96.7%. Based on 250 in-vivo mouse femoral artery images segmentation results, 3D thrombosis morphology and lumen area analysis were adopted for quantitative outcome evaluation.

The paper is organized as follows. Section 2 is the method, which describes the multi-classification deep learning model, data sources and system implementation. Section 3 demonstrates the segmentation results. The discussions and conclusions are drawn in Section 4 and Section 5.

2. Methods

In this work, we regard two binary classification problems as a multi-classification issue. First, we preprocessed the intensity image and phase image as one grayscale image as shown in Figure 1(a). Then, the corresponding label is as shown in Figure 1(b). The background area is regarded as the zero category. The vessel region in intensity image belongs to the first category and the inner lumen area in phase image is classified into the second category.

2.1. Model Architecture

Our architecture is derived from the U-Net architecture described in [42], which is a classical network. Since the U-Net structure was proposed, it has gained widespread attention in the field of OCT vessel imaging. The deep layer contains semantic information of the features and the shallow layer contains high-resolution information of the fine features of the image [47]. We modify and extend the U-net architecture, which works with very few training images and yields more precise segmentation results, as shown in Figure 2. The input is a 512×512-pixel image. The output is the predicted probability map obtained by the SoftMax. We adopted convolutional layers with 3×3 filters. The batch normalization layer is added before each convolutional layer, which is helpful to prevent the gradient explosion and gradient disappearance. The Batch Normalization (BN) effectively normalizes each batch of inputs by re-centering and scaling to alleviate the problem of internal covariate shift and, in some cases, eliminates the need for dropout [49]. The convolutional layers are followed by the rectified linear unit (ReLU) activation function. The convolutional layers followed a pooling layer can extract deeper features in the encoder. The upsampling layers followed convolutional layers in the decoder. The max pooling layers and upsampling layers use 2×2 filters. The number of layers and the feature maps used in each convolution layer are illustrated in Figure 2.

2.2. Dataset and Implementation

The mouse artery was imaged with a Sensemos-OCT-840-70-U3 handheld probe (Suzhou Sensemos Technology Co., LTD ). The system was with each frame size of 512 (lateral)×512 (axial) pixels using a source with a central wavelength of 840 nm and tuning range from 820 nm to 860 nm. In brief, these data were obtained from the exposed femoral artery after anastomosis on six to eight-week-old male BALB/C mice. A 3D volume data of the anastomosed vessel site was acquired covering a volume range of 1.5 mm✕1.5 mm ✕5mm (lateral X lateral Y axial Z). The 3D dataset consists of 250 B-frames. Each B-frame image consisted of 1000 A-scans. All the experiments were performed on a workstation with one NVIDIA GeForce RTX4080 graphics processing units (GPU) and 128GB of RAM. The models were implemented with Python based on Keras deep learning library.The vascular Doppler OCT intensity images and corresponding phase images were first sampled into one 512×512 pixels grayscale images to fit the model. We crop the image by MATLAB software and only select the part with blood vessels. The size of image captured is 512*256. Then we merge the intensity image and phase image. The intensity image is placed on the left and the phase image is placed on the right. So, the size of CNN input image is 512*512. Then image pairs were manually delineated by specialists to form the gold standards of vessel wall boundary and inner blood flowing lumen area contour. The capability of Doppler OCT imaging for detection of thrombosis has been investigated and validated by gold standard histology analysis in previous publication [1]. That’ s the foundation of our drawing on the formation of gold standard. The training dataset consists of 250 images. To teach the network with desired invariance and robustness properties, data augmentation was applied to the training set and their corresponding segmentation labels 20-fold to obtain 5000 pairs of data sets as training inputs for the network. , which was by translation (the coefficient of width, height and zoom set as 0.07, 0.03, 0.05, respectively), rotation (coefficient sets 0.3), shear (coefficient sets 0.1) and flipping.

3. Results

3.1. Training Procedure

The training dataset and corresponding ground truth were delivered into the networks. The categorical Cross-entropy loss and Adam optimizer and learning rate of 0.0001 were adopted in training process. The learning rate is decreased by a fraction of 0.1 if the loss does not decrease for seven consecutive epochs. We used the dropout ratio 0.2 during the training procedure to reduce overfitting. The loss value was altered with the weights map of the convolutional layers. When the loss function value no longer decreases as the training epochs increases, the loss function tends to converge and the optimal classification model can be obtained. An early stopping mechanism is also employed to stop the training process if the loss value does not decrease for 5 consecutive epochs. The proposed architecture is trained for 100 epochs. In Figure 3, the convergence of the learning curves is illustrated over the epochs. The red curves represent the train accuracy and train loss while the black curves stand for the validation accuracy and validation loss. The loss values decreased quickly in the first several epochs and became steady.

3.2. Quantitative Evaluation

The metrics like Area Under the Curve (AUC) is used to evaluate the performance of the proposed network, which are defined as below [51]:

$$Precision=\frac{TP}{TP+FP}$$

(1)

$$Recall=\frac{TP}{TP+FN}$$

(2)

$$F1Score=\frac{2*Precision*Recall}{Precision+Recall}$$

(3)

Where $TP$ represents True Positives, FP represents False Positives, and FN represents False Negatives. The Dice Coefficient (DC) was also applied to evaluate both the segmentation and classification performance. We computed the average of the test dataset to get the final results. The DC is defined as below [39]:

$$DC=2\frac{\left|S_{seg}\cap S_{gt}\right|}{\left|S_{seg}\right|+\left|S_{gt}\right|}$$

(4)

Where $S_{seg}$ and $S_{gt}$ represent the segmented area and the corresponding ground truth. The range of DC value is 0-1. The closer the DC value is to 1, the better the performance of the networks.

3.3. Segmentation Results

To verify the validity of the proposed method, we segmented an 3D volume data contained 250 OCT artery images with the trained model. The several segmentation results are shown in Figure 4 and Table 1. The intensity images with vessel wall marked out with blue curves and the phase images with blood flowing lumen area boundary marked out by red curves. There is different degree of thrombosis occlusion in Figure 4(b-1), (d-1), (e-1), (f-1), (g-1) and (h-1) while no thrombosis is in Figure 4(a-1) and (c-1). We can see that the boundaries were successfully segmented, which demonstrates the effectiveness of the proposed method. Table 1 summarizes the performance metrics for (a-1) - (h-1) frame OCT images from 250 frame OCT images correspondingly. We can see that the precision is above 0.94 and the recall is above 0.96. In addition, the F1Score and the Dice are above 0.95.

We performed registration with subpixel image registration algorithm and 3D reconstruction of 250 segmentation results to visually display vascular morphology, as shown in Figure 5 [51,52]. In order to observe the internal structure of the blood vessel, we divided the blood vessel into two halves according to the blood vessel center line. We can see two clots in the blood vessels. At the location of thrombus 1, it caused a narrowing of the blood vessels to some extent.

4. Discussion

Quantitative analysis of the vessel was performed, which is essential to the objective assessment of the surgical successful rate. We calculated the blood flow region and blood vessel radius along the blood flow direction with 250 segmentation results, which contains a physical range of 1.5mm. We make the comparison with the traditional random walk algorithm and the cascaded U-net architecture. Figure 6(a) shows the blood flowing region variation along the vessel flow axis. It clearly shows that the lumen area becomes narrower at the thrombosis position, which points out by black circle. The average radius of one cross-sectional image in 10 angular directions with their stand deviations is shown in Figure 6(b), which is consistent with the segmentation results in [21,46]. The average radius of the vessel gets smaller at the thrombus point, which is easier to result in turbulence and contribute to the thrombosis formation.

Table 2 shows the performance of the proposed method compared to the traditional random walk method [21] and CU-Net method [46] in terms of the dice coefficient and consuming time. These three methods have high segmentation accuracy for Doppler OCT intensity map and phase map. The segmentation results of phase image with random walk and CU-Net method are slightly lower than that of intensity image. The CU-Net method and proposed method, these two algorithms based on deep learning have greater advantages in time compared with the traditional random walk method. The proposed method took an average time consumption of 0.63s to segment one intensity image and phase image pairs. And it took 157.5s to get all segmentation results, which is faster than the CU-Net method. Compared with the random walk method, the proposed method has a slight improvement in accuracy and a great improvement in time. Compared with the method based on cascaded U-net, the accuracy of the method based on the proposed method is slightly increased. The call of single network is more convenient, and the training of single network is simpler and more efficient in the process of network training. Besides, one model is much simpler and faster than two models for further incorporation into online OCT system for the segmentation process.

5. Conclusions

We proposed a method based on deep learning for Doppler OCT images segmentation. It can obtain the vessel boundary from the intensity image and the lumen area boundary from the corresponding phase image simultaneously, which achieve an average testing segmentation accuracy of 0.967 and an average time consumption of 0.63s. The 3D thrombosis morphology and quantitative analysis of the vessel condition were performed. The proposed method can extract the boundaries obtaining the morphology of vessel for accurate quantitative evaluation. In the future, we will develop a real-time online segmentation algorithm platform for OCT images. It is expected to realize real-time clinical segmentation and provide technical support for intraoperative evaluation, which could promote the application of OCT in clinical analysis assisting the doctors in making decisions.