Automatic Thin Cap Fibroatheroma Detection Using Fusion of Intravascular Ultrasound and Virtual Histology Images

Virtual HistologyIntravascular Ultrasound (VH-IVUS) image is an available method for visualizing plaque component to detect thin cap fibroatheroma. Nevertheless, this imaging modality has considerable limitations to extract the plaque component features and identifying the TCFA plaque. The aim of this paper is to improve the identification of TCFA using fusion of IVUS and VH-IVUS images. In order to generate the automatic technique for reducing the human interaction, a new method namely Active Contour based Plaque Border Detection (ACPB) is proposed. In order to perform the pixel wise classification, hybrid of K-means algorithm with Particle Swarm Optimization and Plaque based Minimum Euclidean Distance (KMPSO-PMED) method is presented to classify the plaque region as well. Moreover, to obtain more significant information of imaging modalities, fusion of two different images consisting of VH-IVUS and IVUS is performed. Therefore, geometric features are extracted from the plaque region and combine with IVUS features. Furthermore, different group of plaque features are divided by means of the histopathological studies. SVM classifiers is applied to detect the TCFA and non-TCFA plaques. The proposed method is applied on 566 in-vivo IVUS and their matching VH-IVUS images obtained from 9 patients. The best result of SVM illustrates the accuracy rates of 99.41% for classification of TCFA plaque. The results prove that the highest accuracy is achieved by integrated features of IVUS and VH-IVUS images.


Introduction
Thin Cap Fibroatheroma (TCFA) plaque is particularly prone to rupture and causes cardiac events [1].Invasive imaging modalities, including Intravascular Ultrasound (IVUS), Virtual Histology-Intravascular Ultrasound (VH-IVUS), and optical coherence tomography (OCT) have been presented for identifying the TCFA plaque [2].Ultrasound imaging has been developed based on the high frequency sound waves to provide the images form body tissues and organs [3].The first commercially available IVUS-based technology, namely VH-IVUS is presented by Nair, Kuban [4] for atherosclerotic plaque characterization.Proposed method has been implemented in Volcano (Rancho Cordova, CA) software to classify four tissue types consisting of dense calcium (DC), necrotic core (NC), fibrotic tissue (FI), fibro-fatty tissue (FF) [5].Plaque characterization is required to accurately describe the vicinity of plaque component in contact with the lumen [6,7].However, the gold standard of plaque vulnerability is not available in clinical assessment [8,9].Some program-based studies have been investigated the diagnosis of the plaque type.In order to plaque type classification, a program was developed by the authors of Margolis, Vince [10].Plaque type classification in IVUS images was performed by means of a decision tree.The authors of [11] attempted to classify the plaque by means of the simple threshold methods.But, their method was not consistent for segmentation of the highly mixed tissue types.The analysis of lesion morphology using baseline and follow-up VH-IVUS images has investigated by Kubo, Maehara [12].However, proposed method only evaluated the plaque burden of non-culprit types.In [13] a program for assessment of atherosclerotic plaque morphology using IVUS/ VH-IVUS images has been developed.Developed software was capable to classify the TCFA and calcified TCFA plaques based on the phantom images.The authors of [14] classified VH-IVUS-defined images to examine the interobserver variability.The effect of different definitions for TCFA plaque were explored as well.Nevertheless, the presented algorithm was unable to recognize the thrombus.Moreover, the analyses were performed on plaques not on patients and provided the different results.Plaque type classification using the baseline images was presented by Zhang et al. [15].Morphology structures of VH-IVUS and IVUS images and biomarkers were presented to classify the TCFA and non-TCFA.Recently, Zhang et al. [1] addressed the prediction of TCFAs using the VH-IVUS data.However, TCFA is identified at a single-frame, while the length of VH-derived TCFA should be in at least three consecutive frames.Despite profound advance in atherosclerosis treatment, preventive and diagnostic tools should be developed to reduce the risk of sudden death [8,9].
In our previous work [16], segmentation, geometric feature extraction, and machine learning based classification were proposed to diagnosis the TCFA plaque.Moreover, in our recently published paper [17], a new hybrid model of K-means with Particle Swarm Optimization (PSO) and a minimum Euclidean distance algorithm (KMPSO-mED) has been proposed to enhance the VH-IVUS images segmentation.Furthermore, geometric and informative texture features were fused to provide a discriminative feature vector for TCFA detection.Different classifiers are applied to obtain the best accuracy result for classifying the plaque into TCFA and non-TCFA.In this paper, in order to generate an automatic technique of plaque border detection, a new method namely Active Contour based Plaque Border Detection (ACPB) is proposed.The delineated plaque area is presented to the pixel wise classification step of K-means PSO-Plaque Based Minimum Euclidean Distance model (KMPSO-PMED) model for improving the accuracy of VH-IVUS image segmentation.Although, many new imaging modalities have been emerged, but ultrasound is applied as a popular imaging modality because it visualizes the most of tissues without damaging them [18].Therefore, plaque features of IVUS and VH-IVUS images are fused to accurately diagnosis of the TCFA.Histopathological features are considered to present the different group of features for the classifier as well.This study is designed into five sections.The proposed methodology for feature extraction using VH-IVUS and IVUS images are presented in Section 2. Section 3 illustrates the classification results by means of different feature sets.The strong points and restrictions of the presented methods are discussed in Section 4 and the conclusion is presented in Section 5.

Proposed Methodology
The proposed approach consisting of plaque borders detection, segmentation, feature extraction using VH-IVUS and IVUS, data set labeling, and TCFA detection.In order to generate the automatic technique for reducing the human interaction, a new technique namely Morphological Lumen Based Initial Contour (MLIC) is proposed.The ACPB algorithm is then proposed to automatically detect the plaque borders.Plaque area is used by KMPSO-mED model to segment the VH-IVUS image.The Extracting Confluent Components (ECC), Determining NC layering (DNCL), and Plaque Burden Measurement (PBM) algorithms are employed to extract the geometric features of VH-IVUS images.Geometric features of IVUS and different group of VH-IVUS feature sets are fed to the classifier.Figure 1 illustrates the block diagram of the proposed approach.

𝐼 =
1 (, ) > 0 0 ℎ Lumen is a hole that is defined as an area of dark pixels surrounded by plaque region.To determine the vessel region, the zero values of the lumen area are converted into one, so vessel region namely IV can be detected by: Lumen region IL is computed by subtracting the IV from IP.After removing the small object in IL, the biggest area is determined as lumen.By extracting the properties of the IL, the smallest rectangle containing the lumen region of bounding box namely BBx is calculated.This matrix specifies the upper-left corner of the BBx and the width of the BBx along each dimension.

Level Set Method
The active contour method is presented to delineate the contour in region of interest (ROI) and detect the vessel and lumen borders [19].Let surface φ to represent the inside of BBx, where φ (i,j)=-1, i=x1,…, x1+ x2 and j=y1,…,y1+y2 .However, the values outside the BBx are positive.In order to enclose the curve within the surface, φ is adjusted iteratively until convergence [20].In VH-IVUS image, by considering the closed lumen, two borders should be detected.Therefore, counter initialization is very important.The detected plaque is considered as ROI that is called the plaque area (PA).The pixels of belong to the internal counter are considered as lumen border (LB).Figure 3 illustrates the result of different iterations for plaque border detection by ACPB algorithm.Figure 4

Plaque Burden Measurement (PBM)
In order to measure the plaque burden, coordinates of plaque border using ACPB algorithm are plotted in a new binary image to represent the plaque area (PA).By applying the morphological operator, closed counter is then filled by '1' to show the vessel area (VA).Lumen area (LA) is obtained by means of subtracting the VA from PA,.The area of VA and LA are measured and stored in fVA and fLA, respectively.

KMPSO-PMED Model for VH-IVUS Image Segmentation
In our previous work, KMPSO-mED model was proposed to enhance the VH-IVUS image segmentation [17].In this study, a new model namely Plaque based Minimum Euclidian Distance Plaque (PMED) is proposed to classify the pixels of testing image.This model executes the K-means algorithm on VH-IVUS image to generate the initial cluster centre and seed one initial particle of swarm.The rest initial particles are set randomly.The PSO is employed to generate the best solution for cluster centres.Based on the RBG colour of each cluster centre, generated clusters are labelled as red, white, dark green, and light green to represent the NC, DC, FI, and FF, respectively [16].The outliers are detected and removed in NC and DC images [22].The RGB values of DC, NC, FF, and FI images are extracted and saved into the color feature matrix namely CF.The matching labels of plaque components, including NC=1, DC=2, FF=3, and FI=4 are determined and retained to the L [16].In order to classify each pixel of test image by PMED algorithm, CF is categorized into NC, DC, FF, and FI classes.In order to extract the PA of testing image, ACPB algorithm is applied.To perform the pixel wise classification, minimum Euclidian distance of pixel xi and four different classes are computed.The lowest minimum distance is provided the class label of xi.Four plaque components are generated using the assigned label.The number of non-zero pixels of NC, DC, FF, and FI images are calculated to show the area features consisting of fNC, fDC, fNC, fFF, and fFI, respectively.

Extracting Confluent Components (ECC) Algorithm
To identify the confluent components as a mass with diameter of greater than 14 pixels in a 400×400 VH-IVUS image, NC image is binarized by the thresholding method [23].In order to detect the NC segments, connected component labeling algorithm is then checked the 8neighborhood of withe pixels.If the segment's diameter ≥14 pixels, then this segment is saved to the confluent NC (CNC) segments, else it is preserved in the scattered NC (SNC) segments [24].Similarity, confluent DC (CDC) and micro calcification (MicroCa) are provided from the DC segments.The area of CNC, CDC, SNC, and MicroCa segments are hold into the fCNC, fCDC, fSNC, and fMicroCa, respectively.For extracting the confluent NC in contact with the lumen (CNCCL), share pixels between CNC and LB are determined.The entire process of Extracting Confluent Components (ECC) algorithm is enumerated as follows [16]: Step 1: The NC image is binarized and then a binary image BNC is produced.Connected component labeling is performed for binary image BNC to produce a labeled matrix LNC of size M×N [25].The LNC matrix involves the labels in range of 1 to n representing each component.In order to divide the n connected components, the coordinates (x,y) of each region of LNC are determined and their corresponding coordinates in BNC are then selected to generate a new binary image namely S for each component.
Step 2: Therefore, binary images S1, S2,… Sn involving only one segment are generated [24].Steps 3 to 6 are repeated n times to calculate the diameter of each object and determine the CNC and SNC sets.
Step 3: The edge of segmenti belonging to the Si is extracted by edge detection algorithm.Let X and Y represent all x and y coordinates of the detected edge.The horizontal hi and vertical vi dimensions of the X and Y are provided by equations ( 6) and (7).
vi= max(Y)-min(Y) Sep 4: The diameter Di of segmenti is assumed to be the maximum value of hi and vi defined by: Step 5: If Di ≥14 pixels then segmenti is considered as CNC [26] and image Si renamed to CNCi, otherwise it is called as SNCi .
Step 6: The coordinates of segmenti in CNCi are stored in a 2D matrix Ci.The Coordinates of LB is then searched to find the possible shared points with Ci.If there is a shared point between LB and Ci, then it can be concluded that Ci is in contact with the lumen and add to CNCCL segments.
Step 7: When the iteration is completed, the area of CNC images and SNC images are calculated by the following equation.Where, b(i,j) with M rows and N columns consisting of '1' for white pixels and '0' for black pixels.
Step 8: The sum of all areas obtained by CNC and SNC images are stored into fCNC and fSNC, respectively.Moreover, the number of CNCCL segments are stored in fCNCCL.

Determining NC Layering (DNCL) Algorithm
TCFA is categorized according to the presence of CNC layers [23].The DNCL algorithm is developed to identify the number of NC layers.Firstly, radial lines are drawn from the center of image in anti-clockwise direction to determine the pixels that are share between drawn lines and CNC segments.If one of the drawn line has share pixels with one segment, it means that NC involve only one layer.At the end, the total of NC layers is calculated and represented as fNCL.Similarity, DC layers are determined and stored in fDCL.The steps 1 to 4 are repeated m times to calculate the NC layers [16].
Step 1: The images CNC1, CNC2,….CNCm are put together in one binary image as CNC.The x and y coordinates of segmenti in CNCi is stored in a 2D matrix namely Ci (m: number of CNC segments).
Step 2: The n radials lines RL= {rl1, …. rln} which start from the center of CNC image are drawn using.By employing the Bresenham's line algorithm, all pixels belonging to the each line are represented as rli= {p1… pn} [27].
Step 3: The points belonging to the line rli are then searched to find the possible shared points with Cj.If there is a shared point, it means that this line passing through Cj that is correspond to the segmentj.
Step 4: The number of segments that rli passes through them are retrieved and stored into the PSi.
Step 5: In order to identify the number of NC layers, the maximum value of PS1, PS2,…, PSn is computed and stored as fNCL feature.
Drawing n radial lines over the CNC image is presented in Figure 8.The red lines represent the radial lines and the white areas depicts the confluent NC segments in binary image.The zoom area indicated by the blue box in Figure 7 (a).As illustrated in this figure, radial lines pass through two segments.Therefore, the confluent NC has multiple layers that represent the previous rupture.Figure 9 presents the x and y coordinates (red color cells) of the shared points belonging to the radial line and coordinates of the CNC segment (white cells).9 The x and y coordinates of shared points between radial lines and segment of Figure 8.

Statistical Analysis
Extracted features construct the VH-IVUS features as FV-VH={fNC, fDC, fFI, fFF, fNCCL, fCNC, fCNC, fCNCCL, fCDC, fNCL, fDCL, fMicroCa, fSNC, fPA, fVA, fLA, fPB}.In order to perform the t-test analysis, geometric features are categorized into three classes: PIT, TCFA, and CaTCFA based on the plaque type classification criteria.Mean, standard deviation (STD), and p-value of all 17 geometrical features are then calculated [28].Table 1 depicts the t-test results (mean ± STD) for three plaque types consisting of PIT, TCFA, and CaTCFA.Regarding the results, p-value <0.0001 are obtained for all features that indicate the discriminating capability of geometric features [29].
The mean value of the primary features consisting of fFI, fFF, fCNC, and fCDC for PIT, TCFA, and CaTCFA classes are illustrated in Figure 10.The x-axis and y-axis indicate the features mean area (mm 2 ) of each feature, respectively.The value of FF should be maximum for PIT class based on the Table 2.As shown in Figure 10, PIT achieves the maximum value of fFF.Also, as depicted in  2 for TCFA and CaTCFA lesions, CNC >10%, but for PIT CNC <10%, so the minimum value of CNC should be provided for PIT lesion.Figure 10 demonstrates that fCNC of PIT obtains the lowest value.Therefore, the requirements are satisfied by the mean values of fFI, fFF, fCNC, and fCDC features.Figure 11 illustrates the mean (mm 2 ) of fFI, fFF, fCNC, and fCDC, and fPA in for PIT lesions.As described in Table 2, PIT involve the mixture of FI and FF (FF ≥15%).The results presented in this figure fulfills these criteria as the mean of fFI and fFF are superior to fCNC and fCDC.Moreover, based on Table 2, the mean of CNC and CDC should less than 10% of the plaque area that is satisfied by the fCNC < 10% fPA and fCDC < 10% fPA in this figure .Figure 12 depicts the mean (mm 2 ) of fFI, fFF, fCNC, and fCDC, fNCCL, fPB and fPA features for TCFA lesions.According to the Table 2, three features are important for TCFA detection, including CNC >10%, NCCL>0, and CDC<10%.As illustrated in the Figure 12, the mean values of fCNC > 10% fPA, fNCCL> 0, and fPB > 40%.Therefore, given criteria of TCFA detection are fulfilled.Figure 13 illustrates mean of (mm 2 ) fFI, fFF, fCNC, and fCDC, fNCCL, fBP and fPA features for CaTCFA lesions.According to the Table 2, four conditions are presented for CaTCFA detection, including CNC >10%, CDC >10%, NCCL>0 and PB ≥40%.As illustrated in this figure, fCNC >10% fPA, fCDC >10% fPA, fNCCL >0, and fPB ≥40%.So, the criteria of CaTCFA are satisfied.
The obtained results indicate that the geometric features that are categorized in PIT, TCFA, and CaTCFA considerably represent the histological criteria of plaque type classification.

IVUS features
Volcano's software can be provided 12 features of gold standard IVUS modality for plaque characterization [30].The area of lumen, fibrous, fibro lipidic, lipid core, calcified, and Media are measured.Moreover, minimum, maximum, and average of lumen diameter and vessel diameter obtained.The area of external elastic membrane (EEM) or vessel is provided by the total area of plaque, lumen, and media [31,32].Plaque burden is expressed as a percentage of the vessel area and computed by means of the difference between two parameters, including vessel area and lumen area [33].

Results
In this section, effects of the extracted features for classification of the plaque is explored.

Data Set Labeling
The data set of this study consisting of 566 Gray-scale IVUS and their corresponding VH-IVUS images were obtained from the 9 patients [35].The feature extraction algorithms are implemented in MATLAB software and feature matrix namely FM of size 566×17are created from VH-IVUS images.
The FM can be divided into different subsets to obtain the most significant and effective features of TCFA and achieve the desired accuracy.Therefore, histopathological features are considered to determine the different group of TCFA features.The feature set 1(FS1) is constructed using all FV-VH features to represent the main criteria of plaque type and feature set 2 (FS2), feature set 3 (FS3), feature set 4 (FS4), and feature set 5 (FS5) are constructed based on the different definition of the TCFA plaque.A few studies have considered the multiple NC layers as a critical feature of TCFA plaque [26,37,38].However, others did not consider the NC layers [14,39].Thus, to investigate the effect of NC layering on classification results, FS2 is built by excluding the NC layering features consisting of fNCL, fDCL.Moreover, some researchers consider a plaque burden ≥40% to detect the TCFA lesion [14,39].Therefore, FS3 is constructed by eliminating the plaque burden features consisting of fPA, fVA, fLA, and fPB to investigate the effect of these features.Furthermore, different studies only introduce the basic features and ignore the plaque burden and layering features.Therefore, FS4 consisting of fCNC, fCDC, fFI, fFF and fNCCL is established by eliminating the plaque burden features and layering features.In addition, FS5 is constructed by the plaque area features consisting of fNC, fDC, fFI, fFF.Table 3 illustrated different feature sets consisting of FS1, FS2, FS3, FS4, and FS5.The K-fold cross validation technique has been employed for evaluating the results of SVM classifier.The first k-1 parts are considered as the training set and the remaining one part is applied to evaluate the trained classifier [29].Different kernels of SVM consisting of linear, quadratic, polynomial, rbf, and mlp are employed.To evaluate the performance, accuracy, sensitivity, and specificity are calculated [41].The features are normalized by the zscore function due to the various range of feature values [43].

TCFA Classification by VH-IVUS Images
Regarding the experimental setup, 70% and 30% of FM matrix of size 566×18 are considered as training and testing sets, respectively.Therefore, FMTrain of size 397×18 and FMTest of size 169×18 are generated.The confusion matrix is calculated, and the averages accuracy, sensitivity, and specificity parameters derived from 10 runs are calculated.Performance are measured using different groups of feature sets.The result of FS1, FS2, FS3, and FS4 are illustrated in Table 4. Based on Table 4, the best result of accuracy is obtained by applying the quadratic kernel for FS1 and linear kernel for FS3.While the worst result is provided by FS2 using the mlp kernel.The highest sensitivity is achieved for FS1 and the lowest result is provided by FS3 using the quadratic kernel.In addition, FS1 illustrates the best result of specificity by employing the quadratic kernel, whereas the worst result is achieved by FS2 and FS4 using the linear kernel.

TCFA Classification by fusion of VH-IVUS and IVUS Images
In order to investigate the power of various feature sets, IVUS and VH-IVUS features are fused.Consider FV-VH and IVUS features are represented as FV-VH ={vh1, vh2,… vh17,l} and FV-IVUS ={ivus1, ivus2,… ivus12}, respectively, where l represents the label of vector.Table 5 depicts the of combination of IVUS features and FS1, FS2, FS3, FS4, and FS5 in terms of accuracy, sensitivity, and specificity.5, the best results of accuracy is obtained by fusion of IVUS and FS5 by applying the linear kernel, while the worst result is provided by IVUS features.In addition, the best result of sensitivity is provided by employing the linear kernel for all feature sets, while the worst result is achieved by IVUS features using rbf kernel.The highest and the lowest specificity are achieved using the IVUS features by rbf and mlp kernel, respectively.Figure 14  As depicted in this figure, the highest and lowest results are achieved by IVUS and FS5-IVUS features, respectively.The fusion of FS1, FS2, and FS3 with IVUS features illustrate the lower result than FS1, FS2, and FS3 features, respectively.While, the fusion of FS4 and FS5 with IVUS show the improved results rather than FS4 and FS5.

Discussion
Automatic plaque type classification need to moderate the examiner observer fault [44].However, because of different criteria for plaque classification, the definition of a TCFA plaque is not clearly standardized [14,37].Moreover, there are different definitions for plaque type classification, but the effects of different plaque definitions have not been studied [14].Therefore, the results of classification are different [45,46].Regarding the clinical research, vessel morphology changes over the time due to the dynamic nature of coronary artery.Nevertheless, intravascular images cannot fully assess the morphology of the plaque.Therefore, imaging-derived predictors to determine the clinical outcomes and assess more reliably plaque morphology should be thoroughly fused.

Conclusions
In this study, automatic detection of TCFA plaque by means of the fusion of IVUS and VH-IVUS images is presented.A new algorithm is developed to extract the plaque borders based on the level set method.Based on the histopathological studies, extracted features were divided into different subsets to obtain the most significant and effective features of TCFA.Highest accuracy of SVM classifier is achieved using the integrated features of IVUS and VH-IVUS images rather than IVUS features.A prognostic assessment of vulnerable plaques contributes to the identification of high-risk patients.However, further research is required to investigate the concept of vulnerable plaque.

Figure 2
Figure 2 illustrates the steps of MLIC algorithm.The BBx counter is illustrate by red color around the lumen (Figure 2. (c)).

Figure 4
Figure 4 Plaque border detection by ACPB algorithm.

Figure 5
illustrates the result of PBM algorithm.The Plaque burden (fPB) is computed based on the next formula[21]:

Figure 8
Figure 8 (a) Drawn lines from the center of image and (b) zoomed area, red pixels illustrate the drawn radial lines and the white areas depict the CNC segments in binary image.

Figure 14
Figure14The maximum accuracy result of different feature sets.

Table 1 T
-test results (Mean ± STD and p-value).In order to investigate the association between the geometric features and histopathological criteria, further analysis is performed for PIT, TCFA, and CaTCFA lesions using the mean values of the FV-VH features.The criteria of PIT, TCFA, and CaTCFA are summarized in Table2.

Table 2 ,
for CaTCFA plaque, CDC > 10% and for others, CDC <10%, so the CDC value of CaTCFA must be greater than the TCFA and CaTCFA.As depicted in this figure, fCDC of CaTCFA has the highest value among the other classes.As illustrated

Table 4 .
The result of different SVM kernels using VH-IVUS features.

Table 5 .
The result of different SVM kernels using VH-IVUS and IVUS features.