Combination of texture, color and shape operators to describe image content: a survey

: In many image processing and computer vision applications, the main aim is to describe image contents. So, different visual properties such as color, texture and shape are extracted to make aim. In this respect, texture information play important role in image description and visual pattern classification. Texture is referred to a specific local distribution of intensities that is repeated throughout the image. Since now different operations or descriptors have been proposed to analysis texture characteristics. In the multi object images specific texture operators usually doesn’t provide accurate results. So, in many cases, combination of texture operators are used to achieve more discriminant features. In this paper, some combination methods are survived to analysis effect of combinational texture features in image content description. Also, in the result part, different related methods are compared in terms of accuracy and computational complexity.


I. INTRODUCTION
The human visual system typically uses color, shape, and texture parameters to identify the contents of images. Therefore, extracting texture, color and shape features is very important in computer vision [1]. So far, various operators have been proposed to extract texture and color information. Experimental results show that in complex images these operators alone cannot provide high detection accuracy. Complex images are images that have a crowded background or a large number of objects and concepts hidden in them. So in the last decade, many researchers have tried to use combination of texture, color, and shape specific operators together. Some examples of complex images are shown in the figure 1. Combining operators does not always provide the desired results. Therefore, it is important to know the operators that can complement each other. Complementarity of operators means that the features that each operator extracts have the following properties: A. Provide high discrimination between samples B. The extracted information of the features does not overlap with each other. Experimental results show that color and shape information alone cannot describe the nature of objects within an image. Therefore, the information and features of the image texture have more resolving power. In most research papers, color and shape information must be combined with texture information. Therefore, image texture analysis operators are more diverse. This paper attempts to review some of the research that uses a combination of different operators to identify the contents of an image. In the selection process of researches, an attempt has been made to make the experimental results of their experiments successful.
II. TEXTURE IMAGE ANALYSIS DESCRIPTORS As mentioned above, the purpose of this article is to investigate the combined methods for describing the contents of the image. Most combination methods use concatenating schema of texture, color, and shape features extracted using single purpose operators. Therefore, before reviewing hybrid methods, it is necessary to review some of the most commonly used single purpose image operators. How intensity is distributed throughout the image forms the texture of the image. Since now, various operators are provided to extract image texture information. In [2], texture analysis operators are divided into four categories as statistical methods, structural methods, filter-based methods and model-based methods. Statistical methods are operators that try to extract the texture features of the image globally and in terms of possible numerical probabilistic parameters. Gray level co-occurrence matrix (GLCM) [3], local binary patterns (LBP) [4], improved local quinary patterns (ILQP) [5], histogram properties, registration-based features, autocorrelation, Haralick features, etc are some of the known statistical texture descriptors. Structural methods usually try to search for specific local structures in the image and claim that the statistical distribution of these structures can be used as feature to describe the contents of the image. Primitive pattern units (PPU) [6], morphological operations [7], skeleton features, edge features, etc are some traditional structural operations. The family of filter-based methods includes methods that first transfer the image from a spatial domain to a new domain such as a frequency, wavelet, curvelet, etc. Then statistical features are extracted in the new domain. Model-based methods are also often methods that try to discover the structure of the repetitive pattern in an image. Fractal analysis operations, texel analysis descriptors and random field model features are some known operations that can be categorized in model-based methods. Some examples of full texture images are shown in the figure 2. Also, some region-based texture images are shown in the figure 3.   [8], a combination of three different efficient texture operator are used to classify texture images. Firstly, input image is filtered by basic Also, edge detector are performed on the output matrixes. Finally, harlick features are extracted to extract discriminant features. Reported results in [8], show that proposed combination approach provide higher accuracy in comparison with basic LBP and GLCM. Structural texture analysis operators make a wide range of texture descriptors. In some of the structural operators, image is discovered to locate some predefined local structures. For example, in [9], primitive patterns unit (PPU) is used to extract discriminant features in joint of statistical methods. Also, this combination method is applied on color channels separately to achieve better results in colored images. Results in [9], show that combination of PPU and statistical features may provide better results than original GLCM and basic LBP operators. Tamura et al., [10] took the approach of devising texture features that correspond to human visual perception. They defined six textural features, such as coarseness, contrast, directionality, line-likeness, regularity and roughness, and compared them with psychological measurements for human subjects. In [10], combination of normalized GLCM, tamura and Gabor features are used to extract discriminant features for feature extraction phase of content based image retrieval (CBIR). Also, Manhattan distance measure is used for similarity matching phase. Morphological operations are widely used today to extract discriminative features especially from binary multiobject images. In [11] combination of uniform local binary patterns (ULBP) are morphological operation for content based image retrieval. First top-hat transform operator is performed on input image to crop non-related background. Next, color channels are separated in RGB color space. Finally, ULBP is performed to extract texture information. Results show that proposed approach in [11], provide precision about 80.68 percent on simplicity dataset. In many cases, shape and color features can be joined to texture features to improve image description accuracy. In [21], a combinational method is proposed for CBIR. In this respect, LBP is used to extract texture features, dominant color operator is used to extract color features and moment invariant operator to describe shape information. Reported experimental results show that joint of color, shape and texture features improve the traditional CBIR systems. A comparison survey is proposed by Selvarajah et al., [13] on texture operators in content based image retrieval. The performance of different combination manners of first order statistical (FOS) features, GLCM, Gray level run length matrix (GLRLM), autocorrelation features and Gabor transform features are compared on a selfcollected dataset of texture images. Results show that combination methods can provide better accuracy in comparison with single operators. Also, combination of GLCM, FOS, GLRLM, and Gabor features provide highest precision about 78 percent on experimental dataset. Hor and Fekri-Ershad [14], used evaluated local binary patterns (ELBP) and Pre-defined pattern units (Pre-PU) to describe image contents. ELBP provide more discriminative feature that basic LBP and MBP. Also, the computational complexity of ELBP is lower in comparison with some lbp-like descriptors. Pre-PU is used in this paper to extract texture information in spatial domain. Experimental results in [14], show that combination of ELBP and Pre-PU provide higher performance on simplicity dataset in terms of precision and recall rate. Vadivel et al., [15], proposed a multi stage method to define discriminative features for CBIR. Modified color difference histogram (MCDH) and edge orientation in Lab color space are used to extract color and texture information jointly. Results in [15] show that combination of edge and MCDH provides high accuracy in image classification. Figure 4. The flowchart of CBIR system in [15] In [16], the effect of different similarity measures is tested on CBIR systems. The proposed CBIR system in [16], uses color histogram features in HSV color space and basic local binary patterns. Experimental results in [16] show that city block provide higher performance in terms of precision rate. Combination of texture operators is used in many medical image processing applications. For example, Fekri-Ershad [17] proposed an efficient approach for cervical cancer diagnosis in pap smear images based on combination of texture information and time series features. In this respect, global significant value (GSV) and selected subset of statistical features are used to extract texture features. Finally, k-nearest neighborhood (KNN) is used to classify image samples. High accuracy and low complexity are advantages of the [17]. As an innovative research, in [18], a compact multi-texture representation is used to detect forgery. In this respect, the image is converted to YCbCr color space. Next, image is decomposed to 10 sub bands. Next, LBP, local phase quantization (LPQ) [18], binarized statistical image features (BSIF) and binary Gabor patterns (BGP) are performed on each sub band to extract texture features. Finally, extracted features are compacted in a sub-set. Experimental results show that combination of these 4 texture descriptors provide higher accuracy in comparison with each possible set of these features, from single to third combination. Combination of texture and shape features may be provide good performance according to the nature of images. For example, in [19], combination of texture and shape feature is used to classify batik images. To obtain texture features, GLCM is applied on the input image and four different statistical features, as contrast, correlation, ASM and IDM, are extracted. To obtain shape features, input image is converted to binary sample. Next morphological operation is performed. Finally, 4 different shape measures such as area, perimeter, major length and convex hull are computed. Experimental results in [19] show that combination of texture and shape features provide high accuracy on batik classification about 92.86 percent. Lbp-like descriptors make a wide range of possible efficient texture image descriptors. In [20], the impulsenoise resistant color local binary patterns (NrCLBP) descriptor is proposed for the first time. HCLBP extract color and texture features jointly, which make it resistant to impulse noise too. Finally, to achieve higher Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 18 December 2020 doi:10.20944/preprints202012.0479.v1 accuracy, significant point selection (SPS) algorithm is proposed to find most key points of the image. SPS algorithm computed the local variations in the images, which can be categorized as local texture descriptor. Results show that combination of HCLBP and SPS classify texture images with higher accuracy in comparison with many previous lbp-like descriptors. As reviewed above, texture information play more important role than color and shape features. So, it is necessary to know up-to-date texture descriptors and texture classification methods. A complete review of texture analysis operators and texture classification methods has proposed in [21].

IV. COMPARISON RESULTS
The main aim of this article is to examine the combined methods for image content analysis and image classification. A variety of methods were examined in the previous section. Experimental results have shown that some of these methods are used for specific applications. Therefore, in this section, the performance of combined texture and color methods is compared. Usually in most image processing and computer vision applications, the state-of-the-art performance evaluation metrics for image analysis and image classification methods are accuracy, precision and recall rate. Accuracy is the most intuitive performance measure and it is simply a ratio of correctly predicted observation to the total observations. One may think that, if we have high accuracy then our model is best. Yes, accuracy is a great measure but only when you have symmetric datasets where values of false positive and false negatives are almost same. Therefore, you have to look at other parameters to evaluate the performance.

Accuracy = TP+TN/TP+FP+FN+TN (1)
Precision is the ratio of correctly predicted positive observations to the total predicted positive observations. Precision = TP/TP+FP (2) Recall (Sensitivity) is the ratio of correctly predicted positive observations to the all observations in actual class. Recall = TP / TP + FN (3) In real applications, to choose a hybrid method, its experimental results in the same problem must be considered. Therefore, in the table below, these methods are compared in terms of application type and database tested. All the results presented in the table below are based on the results reported in the relevant references.

V. CONCLUSION
The main purpose of this paper was to investigate the combined methods in describing the contents of the image. In this regard, efficient operators to extract texture, color and shape properties were survived. Then, wellknown hybrid methods that use a combination of several operators to analyze the image were reviewed. The results of the combined methods were presented in case of various problems based on the reports presented in the related papers. Comparative results show two practical points as follows: A. The combination of efficient operators in most cases provides better results than using a single operator. B. The results also showed that in most computer vision applications, image texture information plays a more important role than color and shape. Therefore, it is necessary to use image texture analysis operators to describe the contents of the image.