Image Databases with Features Augmented with Singular Point Shapes for Enhanced Machine Learning

1. Introduction

The number, the sizes, and the variety of image databases has flourished in the recent decade. These databases are generated to solve problems in medicine, science, security, and industry. The demand for increasing accuracy is high and important. For example, face image databases usually contain hundreds of thousands of images. Hence, a single percentage of error will lead to thousands of incorrectly processed samples. Therefore, embedding additional features into an image carries the potential of enhancing the capabilities of the classifier [1] to correctly identify unknown input samples. This holds because in medical images, like skin lesions, the features are cluttered and chaotic. Hence, embedding image related, additional, features make the classifier more focused, which improves the accuracy [2,3].

VFs are classical tools in hydro-, air- and electrodynamics but in the recent decades they attracted the attention of Computer Vision and Image Analysis scientists. In [4] the authors developed an Ambrosio-Tortorelli scalar field for the purpose of objects partitioning. The authors of [5] developed a method that applies the gradient vector field (GVF) of the solution of the Poisson partial differential equation to extract geometric features and objects. Another application of VFs is in facilitating the iterative matching of features through "consensus" between VFs [6]. The advantage provided by this approach is the reduced number of false matches. It holds because the VF is generated by a bank of directional morphological openings which considers multiple directions of contour and defines ellipses as special structuring elements [7], defining new adaptive filters for image segmentation.

This paper describes a repository of image databases where the geometric image features are augmented with VF features like SPs shapes, trajectories of vectors, and separatrices. In paper [3] we proved that the SPs shapes are invariant according to scaling, translation, and weak rotations. This property shows that the SP shapes are useful features to augment the image features and enhance the statistics of ML classifiers. We validated this advantage with a sparse representation wavelet classification (SRWC) and a sparse representation classification quaternion wavelet (SRCQW) method presented in paper [1] and with five convolutional neural networks (CNNs) in the papers [2,3].

For the purpose of validation we applied the above listed ML classifiers on the public image databases ISIC2018 [19], ISIC2020 [20], COIL100 [16], and YaleB [18]. These databases contain 2D images of different types and sizes. The first two image databases consist of skin lesion images, the third one consists of 100 objects recorded from 72 different angles, and the last database contains the images of 38 faces recorded under 64 different illumination conditions. Note that the ISIC2020 is the largest among the above listed databases and has 32,542 benign and 584 malignant images. In addition, 10,982 non-labeled benign and malignant images are available for testing.

To facilitate the embedding, learning and classification processes the original images were resized to $420\times 420$ for the COIL100 database and to $192\times 168$ for the Yale database. Further, from ISIC2018 were generated two image databases resized to $250\times 250$ and $500\times 500$ pixels, respectively. Additionally, two datasets were generated from ISIC2020 by resizing the original images to $256\times 192$ and $332\times 590$ pixels, respectively.

To augment the image features with SPs shapes we have developed six VFs. In [1,11] we designed three VFs $\nabla \widehat{u}$, $\nabla \widehat{\varphi }$ and $\nabla \widehat{\psi }$ whose SPs have real Eigenvalues and shapes formed by vectors whose main directions are determined by the Eigenvectors of the SPs. Further, in [3] we developed two VFs $v_{\widehat{u}}$ and $v_{\widehat{\varphi }}$ which contain SPs with real and complex Eigenvalues, Eigenvectors, and shapes. In the present study we formulate the sixth VF $v_{\widehat{\psi }}$ which also possess SPs with real and complex shapes.

We proved in [1,11] that the gradient VF (GVF) $\nabla \widehat{u}$ generates two kinds of SPs’ shapes, namely saddle and sinking. The two GVFs $\nabla \widehat{\varphi }$ and $\nabla \widehat{\psi }$ generate three real shaped SPs’: saddle, sinking and springing. Next, we proved in [3] that the VFs $v_{\widehat{u}}$ and $v_{\widehat{\varphi }}$ may create seven SPs shapes which include the three real shaped SPs as well as attracting and repealing (in and out) spirals along with clockwise and counterclockwise orbits.

We embed the six VFs $\nabla \widehat{u},\nabla \widehat{\varphi },\nabla \widehat{\psi },v_{\widehat{u}}$, $v_{\widehat{\varphi }}$ and $v_{\widehat{\psi }}$ with the help of Matlab tools, incorporated into the ELPAC software [12], which was initially developed for automatic image segmentation with an active contour guided by the VF $\nabla \widehat{u}$. By embedding a VF into every image of a database we built up a new image database, where the image features are augmented with SPs shapes. Thus, by applying ELPAC on the four image databases ISIC2018 [19], ISIC2020 [20], COIL100 [16] and YaleB [18] we generated a set of new image databases where the above six VFs are embedded. These image databases are ready to use and have the advantage of enhancing the classification statistics compared to the classification statistics of the original database [1,2,3]. The first repository of image databases with embedded VFs is available with downloads at the URL given below. The web page also contains data to support the claim that embedding VF increases accuracy of classification:

https://www.tamuc.edu/projects/augmented-image-repository/?redirect=none.

On the same web page, we link to the EPLAC software [12] which could be implemented for embedding into any image any of the six VFs described in this paper. An additional advantage of images with embedded VF is for image databases augmentation when there is a shortage of samples or imbalanced classes. However, implementation of this idea requires a deep and rigorous study and support by experimental results. Hence, it will be a subject of study in our future works. Further, the above web page provides the SRWC [1] code which can be used, to classify original and image databases with embedded VFs, for the purpose of comparison with the users’ own classifiers.

The contributions of this study are: 1) Definition of the SPs locations according to the image objects; 2) Generalization of the mappings between the SPs shapes if the six VFs are separately embedded into the same image: 3) Definition of the new type of image and image database named "imprint of an image and imprint of an image database in a VF". These contributions provide the opportunity for the user to choose the VF to be embedded into an image database in accordance to the particular database in order to increase classification statistics.

The rest of this paper is organized as follows: Section 2 is composed of two subsections. The first one describes the six VFs used to augment the image features with SPs shapes, while the second one describes the SPs shapes, the mappings between them, and the SPs locations in an image; Section 3 consists of two main subsections which describe the original, the newly generated image database with embedded VFs, and the software tools available for image features augmentation and image databases classification. The paper ends with a discussion on the contributions and the advantages obtainable by classifying image databases with embedded VFs and/or imprints of image databases in VFs.

2. Vector Fields That Augment the Image Features

2.1. Definition of Vector Fields

The set of VFs for augmenting image features with SPs shapes can be characterized with the help of weighted Laurent polynomials [13], which distinguish 16 kinds (shapes) of SPs. Another way to describe the VFs is through the Eigenvalues of their Jacobian [1,14,15]. This approach defines 7 different kinds (shapes) of SPs. In our studies we adopted the latter classification and distinguish 7 different SPs shapes which we embed into images. Three of the SPs shapes are: springing, sinking and saddle. They correspond to the real Eigenvalues of the Jacobian of the gradient VFs (GVFs). The shapes of the three kinds of SPs are described in [1,11], and are illustrated in Figure 1.

The remaining four SPs shapes are those corresponding to the complex Eigenvalues of the Jacobian of the VFs $v_{\widehat{u}}$, $v_{\widehat{\varphi }}$ and $v_{\widehat{\psi }}$. The SPs shapes are formed according to the directions of the vectors oriented toward the directions of the Eigenvectors and resemble attracting (in) and repealing (out) spirals, as well as clockwise and counterclockwise orbits [3]. The SPs with complex shapes are usually located in homogeneous regions. Examples of SPs with spiral out (repealing) and clockwise orbit shapes are presented in Figure 2.

To generate the seven kinds of SPs shapes described above, we have developed three GVFs with real Eigenvalues [1,11] and three non GVFs with real and complex Eigenvalues [3]. The six VFs are defined with the help of the solution $\widehat{u}(x,y)$ of the Poisson Image equation [1,3,11], where $\partial \Omega$ denotes the image frame and $I(x,y)$ the image function:

$$\Delta u(x,y)=-{\displaystyle \frac{{\left|\nabla I(x,y)\right|}^2}{1+I(x,y)}},u=I(x,y),(x,y)\in \partial \Omega .$$

(1)

The developed GVFs are defined with the following equations [1,11]:

$$\nabla \widehat{u}=u_{x}i+u_{y}j,\nabla \widehat{\varphi }={\widehat{\varphi }}_{x}i+{\widehat{\varphi }}_{y}j,\nabla \widehat{\psi }={\widehat{\psi }}_{x}i+{\widehat{\psi }}_{y}j,$$

(2)

and they generate the sinking, springing and saddle shaped SPs. In Eqs. 2 the function

$$\widehat{\varphi }={∥\nabla \widehat{u}∥}^2,\widehat{\psi }={\displaystyle \frac{-\Delta \widehat{u}+1}{∥\nabla \widehat{u}∥}}={\displaystyle \frac{-\Delta \widehat{u}+1}{\sqrt{\widehat{\varphi }}}}.$$

(3)

The next three VFs are non-GVF and are developed by replacing the summations in Eqs. 2 with subtractions [3]:

$$v_{\widehat{u}}=u_{x}i-u_{y}j,v_{\widehat{\varphi }}={\widehat{\varphi }}_{x}i-{\widehat{\varphi }}_{y}j,v_{\widehat{\psi }}={\widehat{\psi }}_{x}i-{\widehat{\psi }}_{y}j.$$

(4)

The VFs defined with Eq. 4 generate the above three real shaped SPs and the following four complex shaped SPs, named attracting (in) and repealing (out) spirals, as well as clockwise and counterclockwise orbits (known also as centers) [14,15].

The ELPAC software was initially introduced in [12] with the capability of embedding $\nabla \widehat{u}$ to evolve an active contour. The tools to embed $\nabla \widehat{\varphi }$, and $\nabla \widehat{\psi }$ were incorporated to conduct the study in [1], while [3] extended ELPAC with tools to embed $v_{\widehat{u}},v_{\widehat{\varphi }}$ and most recently we added a tool to embed $v_{\widehat{\psi }}$. Hence ELPAC embeds in an image any of the six VFs , as shown in Figure 4, Figure 6, Figure 7. Further, if we remove the objects from the image with embedded VF, we receive an image of the VF generated from the original image (Figure 4 parts (b), (d), (f), (h), (j), (l)). Thus, an image containing only the VF, generated on the original image, we call "imprint of the image in the VF". The imprints of the image from Figure 3, in the six VFs, are shown in Figure 4. Further, we define the notion "imprint of an image database in a VF" as the set of imprints of the images from the original database in the VF.

2.2. Vector Fields SPs for Image Features Augmentation

In the present section we describe the SPs shapes created by each of the six VFs and list approximately where, according to the image objects, the SPs are located. Also, we determine the mappings between the SPs shapes if different VFs are separately embedded into an image. It helps define the similarities and differences between VFs components like SPs, separatrices [1] and architectures (skeletons [21] if embeded into the same image. This comparison facilitates the choice of the VF most appropriate for embedding into an image database to enhance ML and improve classification.

Recall that each of the VFs introduced, in subSection 2.1, is defined with the solution $\widehat{u}(x,y)$ of Eq. 1, which we solve on every image $I(x,y)$ of an image database. This implies that, in every image, we can embed VF features like SPs shapes, edges of SPs shapes, trajectories of vectors, and separatrices [1]. The last entity is created by SPs shapes and trajectories that connect them. A SP edge is a shape composed by a string of SPs shapes generated in very close vicinity of an object’s edge, as shown in Figure 5 (c) and (d). Here one may observe a string of sinking vectors, and in (d) a string of shrinking vectors. Note, the listed VF features are generated from the image geometric features, and the former set is a natural augmentation to the second set.

In [1,11] we proved that $\nabla \widehat{u}$ generates sinking and saddle SPs, while $\nabla \widehat{\varphi }$ and $\nabla \widehat{\psi }$ generate sinking, saddle, and springing SPs. It follows that $\nabla \widehat{u}$ possess less variety and smaller number of SPs if compared with the other two VFs. Further, we showed in [1] that $\nabla \widehat{\varphi }$ and $\nabla \widehat{\psi }$ have same number of SPs such that the springing SPs of $\nabla \widehat{\varphi }$ are mapped into the sinking SPs of the $\nabla \widehat{\varphi }$ VF and vice versa. At the same time the saddle SPs do not change shape, but their vectors have opposite directions in the two VFs. Furthermore, the saddle SPs may appear only in homogeneous regions, while the springing and sinking SPs are located on the boundary. Also, the trajectories and the separatrices of the two VFs have similar architectures, but their vectors have opposite directions [1]. The above-described SPs properties and mappings of the VFs $\nabla \widehat{u}$, $\nabla \widehat{\varphi }$ and $\nabla \widehat{\psi }$ can be observed in Figure 5.

Figure 5. (a) Synthetic object. In the upper row are shown zooms of the lower branch, of the object in (a), with embedded VF; (b) $\nabla \widehat{u}$; (c) $\nabla \widehat{\varphi }$; (d) $\nabla \widehat{\psi }$. In the lower row are shown zooms of the core part, of the object in (a), with embedded VF: (e) $\nabla \widehat{u}$; (f) $\nabla \widehat{\varphi }$; (g) $\nabla \widehat{\psi }$.

Figure 5. (a) Synthetic object. In the upper row are shown zooms of the lower branch, of the object in (a), with embedded VF; (b) $\nabla \widehat{u}$; (c) $\nabla \widehat{\varphi }$; (d) $\nabla \widehat{\psi }$. In the lower row are shown zooms of the core part, of the object in (a), with embedded VF: (e) $\nabla \widehat{u}$; (f) $\nabla \widehat{\varphi }$; (g) $\nabla \widehat{\psi }$.

Figure 6. Overal view of the object in Figure 5 (a) with embedded VF: (a) $\nabla \widehat{u}$; (b) $\nabla \widehat{\varphi }$; (c) $\nabla \widehat{\psi }$.

Figure 6. Overal view of the object in Figure 5 (a) with embedded VF: (a) $\nabla \widehat{u}$; (b) $\nabla \widehat{\varphi }$; (c) $\nabla \widehat{\psi }$.

Figure 5 (b) and Figure 6 (a) show that the GVF $\nabla \widehat{u}$ generates no SPs at convexity vertices, edges and external concavities but has sinking SPs at concavity corners Figure 5 (e). Further, $\nabla \widehat{\varphi }$ generates sinking SPs on the boundary convexity vertices and edges as shown in Figure 5 (c) and Figure 6 (b) and no SP at the boundary concavities’ corners (see Figure 5 (f)). Next, $\nabla \widehat{\psi }$ generates sprinking SPs on the boundary convexity vertices and edges as shown in Figure 5 (d) and Figure 6 (c) and no SP at the boundary concavities’ corners (Figure 5 (g)). Further, $\nabla \widehat{u}$ creates only saddle SP in the object’s core homogeneous regions (see Figure 5 (e)), while $\nabla \widehat{\varphi }$ generates springing SPs at the saddle SPs locations of $\nabla \widehat{u}$, and a saddle SP between the springing (Figure 5 (f)). Also, the VF $\nabla \widehat{\varphi }$ generates saddle SPs at external concavities (Figure 6 (b)). The VF $\nabla \widehat{\psi }$ preserves the saddle SPs of $\nabla \widehat{\varphi }$ , swithing the vectors’ directions, while replaces the springing with sinking SP (Figure 5 (f) and (g), Figure 6 (b) and (c)).

Note, Figure 6 provides an overall view of the 4-ray star from Figure 5 (a), where the three GVFs, with real shaped SPs, are embedded. One may observe that the objects’ external concavity regions in Figure 6 (b) and (c) contain saddle SPs, but the vectors that create their transversal and hyperbolic trajectories have opposite directions. Once again one may observe that the objects with embedded VF $\nabla \widehat{\varphi }$ create edges of sinking SP, while $\nabla \widehat{\psi }$ generates edges of springing SPs. The latter kind of SPs edges are better visually exhibited than the former edges of SPs.

We continue here after with the description of the properties and the locations of the SPs generated by the VFs with real and complex Eigenvalues. Recall that these VFs contain seven kinds of SPs: sinking, springing, saddle, spiral in (attracting), spiral out (repealing), and orbits with clockwise and counterclockwise directions. Examples of their shapes are shown in Figure 1, Figure 2, Figure 5, Figure 7.

In [3] we proved that the CPs of the function $\widehat{u}(x,y)$, which are the solution of Eq. 1, may map to any of the 7 kinds of SP shapes of $v_{\widehat{u}}$. Further, in the same paper we proved that the CP of $\widehat{u}$ and $\widehat{\varphi }$ map to the saddle SPs $v_{\widehat{\varphi }}$. Recently, we validated that the same holds for the CPs of $\widehat{\psi }$ and the SPs of $v_{\widehat{\psi }}$. Also, we validated that the SPs of $v_{\widehat{\varphi }}$ map to the SPs of $v_{\widehat{\psi }}$ such that: springing SP maps to a sinking one and vice versa; saddle SPs do not change shape; spiral in map to spiral out and vice versa; and the clockwise orbits map to counterclockwise orbits and vice versa. These mappings imply that the VFs $v_{\widehat{\varphi }}$ and $v_{\widehat{\psi }}$ possess same number of SPs and trajectory architectures, but the vectors that compose the two kinds of VFs architectures have opposite directions. Moreover, the positions of the SPs generated by the two VFs are in a close vicinity to each other, if projected on one and the same plane. Further, we note that the SPs edges of the VFs $v_{\widehat{\varphi }}$ and $v_{\widehat{\psi }}$ are like each other as structures, but the vectors of these structures are opposite to each other. One may observe in Figure 7 parts (b), (c) that $v_{\widehat{\varphi }}$ possesses sinking SPs edges while $v_{\widehat{\psi }}$ has springing SPs edges. Further $v_{\widehat{\varphi }}$ has saddle SPs points with coming in vertical transversals, while the horizontal are coming out as shown in part (e). On the other hand, the transversal of the saddle SPs of $v_{\widehat{\varphi }}$ at the same positions are build up by vectors with opposite directions as Figure 7 (f) shows.

Figure 7. The object in Figure 5 (a) with embedded VF: $v_{\widehat{u}}$ in the left column, parts (a), (d) and (g); $v_{\widehat{\varphi }}$ in the middle column, parts (b), (e) and (h); $v_{\widehat{\psi }}$ in the right column, parts (c), (f) and (i) .

Figure 7. The object in Figure 5 (a) with embedded VF: $v_{\widehat{u}}$ in the left column, parts (a), (d) and (g); $v_{\widehat{\varphi }}$ in the middle column, parts (b), (e) and (h); $v_{\widehat{\psi }}$ in the right column, parts (c), (f) and (i) .

Next, one may observe that the images with embedded VF $v_{\widehat{u}}$ show no SPs located on the object exterior nor on its boundary. Hence, $v_{\widehat{u}}$ does not create SPs edges, nor are SPs present at: objects boundary; on external concavities; inner convexities; nor at convexity vertices. However, any of the seven SPs may show up in the core part of the object. The above statements are validated in Figure 7 (a), (d), where we present a zoomed in portion of the lower ray and the core part of the four rays star from Figure 5 (a) with embedded VF $v_{\widehat{u}}$. Note, that part (d) exhibits spiral out (the upper left SP), spiral in (the upper right SP), and an springing SP (the lower one). An overall view of the same object and VF is shown in (g) where the SPs are present only in the object core.

VF $v_{\widehat{\varphi }}$ generates spiral in SPs on the exterior concavities of the object (Figure 7 (h)). Also, saddle SPs are located at the convex vertices of the boundary as shown in part (b). Moreover, $v_{\widehat{\varphi }}$ generates edges of sinking SPs (Figure 7 (b)) and springing SPs (see the horizontal rays in part (h)). Further, the core interior of the object may contain several SP shapes from the entire set of seven kinds of SPs. As one may observe from Figure 7 (e), the core of the 4-ray star contains three saddle SPs, and a springing and sinking SPs between them. The upper two saddle SPs have outgoing transversal trajectories which go to the sinking SPs, while the incoming transversal of the lower saddle SP comes from the springing SP.

The last VF to consider is $v_{\widehat{\psi }}$. Recall that, if embedded into an object, it has same number of SPs shapes and similar architecture, of its trajectories, as $v_{\widehat{\varphi }}$ does. Note, the vectors that build up the $v_{\widehat{\psi }}$ trajectories and SPs have opposite directions to the vectors of $v_{\widehat{\varphi }}$. This could be observed in part Figure 7 (c) where at the convex vertex is located a saddle SP whose vectors are opposite to those of the saddle SP in part (b). Also, the four spiral out SPs in the external concavities of the star in part (i) are created with vectors whose directions are opposite to the directions of the vectors which created the spiral out SPs in the external concavities Figure 7 (h). Further, part (f) shows that the VF $v_{\widehat{\psi }}$ generates five SPs in the core of the 4-rays star, as $v_{\widehat{\varphi }}$ does it in part (e). In (f) there are again three saddle SPs, but they have opposite vectors if compared with the saddle SPs in (e). The remaining two SP, in between the three saddles, are springing and sinking, but they switched positions, such that the sinking is below the springing as shown in part (f), while the former is above the latter in part (e) of Figure 7.

Following the above reasoning and observations we develop the diagram in Figure 8 to describe the mappings between the CPs of the functions $\widehat{u}$, $\widehat{\varphi }$, $\widehat{\psi }$ and the SPs of the six VFs if they are separately embedded in one and the same image. Note, the VFs having one to one correspondence have the same trajectory architectures, but vectors with opposite directions, as we described in the above paragraphs. Also, with the notion "partial" we denote that only part of the SPs of $\nabla \widehat{u}$ map to the SPs of $\nabla \widehat{\varphi }$, because $\nabla \widehat{u}$ has sinking SPs ($\widehat{u}$ has maxima as proven in [1]) at concavity vertices (see Figure 5 (e)) while $\nabla \widehat{\varphi }$ does not have SPs at concavity vertices (see part (f)). Also, $\nabla \widehat{\varphi }$ has sinking SPs ($\widehat{\varphi }$ has maxima as proven in [1])) at convexity vertices (see Figure 5 (c)), while $\widehat{u}$ does not (see part (b)).

Table 1 shows the distribution of the SPs of a VF across an image if the VF is embedded into the image. It is a guide designed to help the user select the VF to be embedded into an image database in order to provide the highest classification statistics. Also, sink denotes sinking SP(s), spring denotes springing SP(s), core denotes "core of an object", branches denote branches of objects, and edges denote "boundary edges".

3. Repository of Image Databases with Embedded VFs

3.1. Original Datasets

In this section we detail the technical characteristics of four original datasets, into which we embedded the VFs defined above and used to obtain the results in [1,2,3]. Recently, we added to the repository the image database digit-MNIST [23] with embedded VF $v_{\widehat{\psi }}$.

The ISIC 2018 [19] and 2020 [20] image databases contain skin lesion images along with malignant and benign labels for each skin lesion. For the original ISIC 2018 database, there are 3694 skin lesion images with dimensions ranging from 640x480 to 6670x4440 in JPG format. For the original ISIC 2020 database, there are 33126 skin lesion training images and 10982 skin lesion testing images with dimensions ranging from 640x480 to 6000x4000 in DCM format.

One primary use case for the ISIC2020 databases is to improve ML diagnosis of skin lesions either as malignant or benign. Thus, the database is broken down into two classes of images which are labeled by three dermatologists. ML is tasked with correctly labeling each skin lesion image into one of the two classes. Note that the training images are labeled by dermatologists, but the testing have no labels and classification is shown when results are uploaded to the database website [20]. Examples with embedded VFs from the ISIC skin lesion datasets are found in Figure 9.

The original COIL100 [16] database contains 7200 images. These images were produced by taking 100 common household objects and rotating each object $5^{\circ }$ for a total of 72 rotations. This produced $72\times 100=7200$ images in total, stored in PNG format. The images are sized with dimensions of 128x128 pixels.

One purpose of this database is to help train ML to recognize a variety of objects, and to recognize these objects regardless of the viewing angle. The different classes in this case are the 100 objects presented to the ML algorithm in 72 different rotations.

For embedding VFs to augment the image features of COIL100 database, we resized its images to 420x420. Examples from this database with embedded VFs are found in Figure 10.

The original Yale face database [18] contains 2414 face images taken under different lighting conditions. The original images are stored in the PNG format with dimensions $341x385$. Note that to conduct the experiments in [1] we have resized the original images to $192x168$ and embedded the VFs again. Examples from this database are found in Figure 11.

The Yale face database allows ML to be trained on a set of face images under different lighting conditions in order to produce facial recognition. The classes in this case are each of 39 faces, with a picture of each taken with a different angle of lighting.

3.2. Image Datasets with Augmented Image Features

Our new repository of image databases with embedded VFs, which augment the image features with SPs shapes, currently hosts several sets of image databases with embedded VFs. They are generated from the ISIC2018, ISIC2020, COIL100, and digit-MNIST databases with the help of the software ELPAC [12], which is included in the repository as well.

In particular, from the original ISIC2018 image database we have generated two ISIC2018 databases in PNG format with sizes $500x500$ and $250x250$. Then in each of them we embedded the VFs $\nabla \widehat{\varphi }$ and $\nabla \widehat{\psi }$. Further, we resized the ISIC2020 to $256x192$, in PNG format, and embedded in every training image of the resized database the VF $\nabla \widehat{\varphi }$ or $\nabla \widehat{\varphi }$. Respectively, we named the new image databases as follows:

• ISIC2018$-500x500-\nabla \widehat{\varphi }$ and ISIC2018$-500x500-\nabla \widehat{\psi }$
• ISIC2018$-250x250-\nabla \widehat{\varphi }$ and ISIC2018$-250x250-\nabla \widehat{\psi }$
• ISIC2020$-train-\nabla \widehat{\varphi }$ and ISIC2020$-test-\nabla \widehat{\varphi }$
• ISIC2020$-train-\nabla \widehat{\psi }$ and ISIC2020$-test-\nabla \widehat{\psi }$ .

In addition we made imprints of the ISIC2020 into the VFs $\nabla \widehat{\varphi }$, $\nabla \widehat{\psi }$ and denoted them with $ISIC2020-imprint-\nabla \widehat{\varphi }$ and $ISIC2020-imprint-\nabla \widehat{\psi }$. Note, in the repository we used the names "VF only" to facilitate the users who did not read the paper.

Also, we generated four image databases using the training and testing sets of ISIC2020 and embedding into them the VFs $v_{\widehat{\varphi }}$ and $v_{\widehat{\psi }}$. Analogously, we named the datasets:

• ISIC2020$-train-v_{\widehat{\widehat{ }}u}$ and ISIC2020$-test-v_{\widehat{u}}$
• ISIC2020$-train-v_{\widehat{\varphi }}$ and ISIC2020$-test-v_{\widehat{\varphi }}$

As well, in the resized COIL100 we embedded the VFs $\nabla \widehat{\varphi }$, $\nabla \widehat{\psi }$, $v_{\widehat{u}}$, $v_{\widehat{\varphi }}$, and $v_{\widehat{\psi }}$ generating five new image databases whose image features are augmented with SPs shapes. According to the adopted titles convention we named the generated image databases with: $COIL100-\nabla \widehat{\varphi }$, $COIL100-\nabla \widehat{\psi }$, $COIL100-v_{\widehat{u}}$, $COIL100-v_{\widehat{\varphi }}$, and $COIL100-v_{\widehat{\psi }}$.

To further validate the robustness of the idea that ML classification is boosted by embedding a VF into image database (which augments the image features with VF features), we added Gaussian noise to the original datasets COIL100 and ISIC2020. This Gaussian noise was generated using the built in MatLab function with a 0 mean and variance set to 0.001. The noisy images were embedded with VFs $v_{\widehat{u}}$ and $v_{\widehat{\varphi }}$ respectively. Thus, we created the six new image datasets listed below:

• COIL100$-noise-v_{\widehat{u}}$ and COIL100$-noise-v_{\widehat{\varphi }}$
• ISIC2020$-train-noise-v_{\widehat{u}}$ and ISIC2020$-test-noise-v_{\widehat{u}}$
• ISIC2020$-train-noise-v_{\widehat{\varphi }}$ and ISIC2020$-test-noise-v_{\widehat{\varphi }}$.

In [2] we applied our CNN and classified the image database COIL100$-noise-v_{\widehat{u}}$ with added noise and embedded VF $v_{\widehat{u}}$ with an accuracy of 92.97%. It is 2.9% higher than the accuracy 90.07% of classifying the original database COIL100 with the same CNN. The same held with ISIC2020$-test-noise-v_{\widehat{u}}$ which was classified with an accuracy of 92.09% while the original image database ISIC2020 was classified with 86.67%.

On the other hand, ISIC2020$-test-noise-v_{\widehat{\varphi }}$ was classified with an accuracy of 85.26% which is 1.41% lower than the classification of the original ISIC2020. It holds because the skin lesion images contain many different structures and details. Hence adding noise and embedding VF features simply clutters the image and decreases the classification.

Recently we extended the repository adding the new image database digid-MNIST [23] with embedded VFs $\nabla \widehat{\psi }$ and $v_{\widehat{\psi }}$. The new image databases are named digit-MNIST-$\nabla \widehat{\psi }$ and digit-MNIST-$v_{\widehat{\psi }}$. The original digit-MNIST contains about $60,000$ images of size $28x28$, while the digit-MNIST-$\nabla \widehat{\psi }$ and digit-MNIST-$v_{\widehat{\psi }}$ conain images of sizes $400x400$.

To access the repository that contains the listed above image databases, ELPAC and SRWC software please use the URL:

https://www.tamuc.edu/projects/augmented-image-repository

Files are hosted as single ZIP files for each respective image database and software. Files stored in each respective image database ZIP are saved in the PNG format. New image databases with embedded VFs will be added in the future.

3.3. Software

The image database repository contains links to the software tool ELPAC whose GUI is shown in Figure 12. This software provides sophisticated and mature capabilities for image segmentation based on the work in [12] and VFs embedding based on the works in [1,2,3]. Hence ELPAC incorporates the following features:

• Segmentation via an evolving contour directed by VF flow. To guide the active contour, with parameters customizeable under "Contour Size and Shape", a VF should be selected from the drop-down menu under "Vector Field Generation". The recommended choice is "Norm of Grad in Poisson Eq" ;
• Splitting and tracing contours around multiple objects for full image segmentation. Such options are available under "Splitting Options";
• Selecting any of the VFs from the "Vector Field Generation" drop-down menu. The six VFs described in section 2 are at the top of the list. The first 3 of them have real shaped SPs [1], while the next 3 have real and complex shaped SPs [3]. The selected VF will be embedded into the image file chosen using the Browse option.

In addition to the GUI for ELPAC, the repository provides code for batch VF embedding into an image database. Instructions for how to use the code and software are included in the repository. Furthermore, the code for ML classifier SRWC [1] is given as well.

4. Conclusions

The present paper describes our repository containing a total of twenty nine image databases built from embedding the VFs $\nabla \widehat{\varphi },\nabla \widehat{\psi },v_{\widehat{u}},v_{\widehat{\varphi }}$ and $v_{\widehat{\psi }}$ into four public image databases. Recall that six of the image databases are "injected" with a Gaussian noise and VF are embedded afterward. Also, four of the image databases are imprints of ISIC2020 into $\nabla \widehat{\varphi }$ and $\nabla \widehat{\psi }$. These image databases bring the advantage of enhancing classification statistics if compared with the classifications statistics of the original image databases, as shown in [1,2,3]. The advantage was validated by applying three ML methods, namely the sparse representation wavelet classification (SRWC) [1], the sparse representation classification with quaternions in wavelet domain (SRCQW) [1] and five CNNs [2].

Recall that the contributions of this paper, as formulated in the Introduction and validated through the text, are as follow:

• Definition of the SPs locations according to the image objects, shown in Table 1
• Generalization of the mappings between the SPs shapes, as shown in Figure 8, if the six VFs are separately embedded into the same image
• Definition of the new type of image and image database named "imprint of an image and imprint of an image database in a VF".

The contributions provide the user with the opportunities to choose the VF to be embedded into a particular image database to provide the advantage of increasing classification statistics if compared with the classification statistics of the original image database.

To understand the nature of the VFs and how they correlate to each other we determined in [1] the mappings between the CPs of $u,\widehat{\varphi }$ and $\widehat{\psi }$ and the SPs of the GVFs $\nabla \widehat{u},\nabla \widehat{\varphi },\nabla \widehat{\psi }$. In [3] we investigated the mappings between the CPs of u and the SPs of $v_{\widehat{u}},v_{\widehat{\varphi }}$ as well as between the shapes of the SPs of the latter two VFs. The present paper contributes to the study of SP shapes mapping through the addition of the VF $v_{\widehat{\psi }}$ into the diagram of SPs mappings.

Further the present paper introduced the notion "imprint of an image in a VF". The imprints of the malignant skin lesion from Figure 3 in the six VFs are shown in Figure 4 parts (b), (d), (f), (h), (j) and (l). One may observe that a single image creates different imprints in the different VFs. To validate the usefulness and the efficiency of the new notion we created the imprint of the ISIC2020 image database [20] in the VF $\nabla \widehat{\varphi }$ and named the new database $ISIC2020-imprint-\nabla \widehat{\varphi }$. Then, we validated in [1] that classifying $ISIC2020-imprint-\nabla \widehat{\varphi }$ gives an accuracy of 88.89%, while the accuracy of classifying the original image database is 84.21%.

From the above descriptions, derivations, observations, Figure 8 and Table 1 we derive information regarding the VFs’ SPs shapes, the mappings between them, and the SPs locations in image objects. Hence, we conclude that the VF with the smallest number of SPs is $v_{\widehat{u}}$, which generates SPs only in the core part of the interior (Figure 7 parts (a), (d) and (g)). The next is $\nabla \widehat{u}$, which generates SPs in the core part of the interior and at the concavity corners (Figure 5 parts (b), (e) and Figure 6 (a)). Note, $v_{\widehat{u}}$ may generate the whole set of seven types of SP shapes, while $\nabla \widehat{u}$ generates only saddle and springing SPs as proven in [1,11].

The pair of VFs $\nabla \widehat{\varphi }$ and $\nabla \widehat{\psi }$ generate the three kinds of real shaped SP (springing, singing and saddle) in the objects’ exterior concavities, branches, boundary convexity vertices, and boundary edges. Therefore, $\nabla \widehat{\varphi }$ and $\nabla \widehat{\psi }$ generate more SPs than $v_{\widehat{u}}$ and $\nabla \widehat{u}$ (Figure 5 (b) and (e), Figure 6 (a), Figure 7 (a), (d), and (g)). Recall that $\nabla \widehat{\varphi }$ and $\nabla \widehat{\psi }$ have similar architectures created by the SPs and the trajectories connecting them, but the vectors that build these architectures are inverse to each other, as stated in Section 2.2 and can be observed in Figure 5 c), f) and d), g), as well us Figure 6 b) and c).

The last pair of VFs in our study is $v_{\widehat{\varphi }}$ and $v_{\widehat{\psi }}$, and they generate almost the same number of SP as $\nabla \widehat{\varphi }$ and $\nabla \widehat{\psi }$ in the object core, the exterior concavities, convexity vertices, branches, and boundary edges. Comparing Figure 5 and Figure 6 with Figure 7 one may tell that the real Eigen valued VFs ($\nabla \widehat{\varphi }$ and $\nabla \widehat{\psi }$) have springing and sinking SP at the convexities’ vertices, and saddle SP in the external concave regions and the objects branches. On the other hand the complex Eigen valued VFs ($v_{\widehat{\varphi }}$ and $v_{\widehat{\psi }}$) have saddle SPs on the convexities’ vertices, spiral out SPs in the external concave regions and shrinking, sinking and spiral SPs at objects branches. Regarding the edges of SPs, those generated by $\nabla \widehat{\varphi }$ and $v_{\widehat{\varphi }}$ are similar. The same holds for $\nabla \widehat{\psi }$ and $v_{\widehat{\psi }}$. Finally, the VF architectures of $v_{\widehat{\varphi }}$ and $v_{\widehat{\psi }}$ are alike to each other but have opposite vectors that create them.

The above analysis provides the knowledge to assist users in their choice of VF to be embedded into given image database and/or to make an imprint of the database into a certain VF in order to improve the classification statistics.

For the needs of paper [2] we embedded VFs $v_{\widehat{u}}$ and $v_{\widehat{\varphi }}$ into the image databases ISIC2020 [20] and COIL100 [16]. Then, five contemporary CNN classifiers were applied to classify the original image databases and the image database with embedded VFs. The results show that the maximum accuracy of classifying the original databases ISIC2020 [20] and COIL100 [16] were 91.69% and 90.99 % respectively [2]. Classifying the databases with embedded $v_{\widehat{u}}$ increased the maximum classification accuracies to 93.25% and 91.85% respectively [2]. However, embedding $v_{\widehat{\varphi }}$ into the original databases decreased the accuracies to 78.19% and 90.08% respectively.

Please note that embedding $v_{\widehat{\varphi }}$ into the cluttered skin lesion images led to a drop of accuracy with about 13%, while embedding the same VF into COIL100, which has not a few details, led to a drop of 0.91%, but increased the precision from 70.46% to 71.57%.

Please note in the paper [2] for the image databases $COIL100-v_{\widehat{u}}$, $ISIC2020-test-v_{\widehat{u}}$ and $ISIC2020-test-v_{\widehat{\varphi }}$ we used slightly different notations, namely $v_{\widehat{u}}\left(C\right)$, $v_{\widehat{u}}\left(I\right)$ and $v_{\widehat{\varphi }}\left(I\right)$ respectively.

The advantage for the ML classification of image database with embedded VF is further validated in [1]. In this paper the VFs $\nabla \widehat{\varphi }$ and $\nabla \widehat{\psi }$ where embedded into the ISIC2018 [19], ISIC2020 [20], COIL100 [16] and YaleB [18] creating ten new image databases, including the imprint of ISIC2020 in $\nabla \widehat{\varphi }$ and $\nabla \widehat{\psi }$. We classified the original image databases and their derivatives with embedded VFs by applying the software tools SRWC and SRCQW described in [1]. The accuracy of classifications of the VF embedded image databases increased except in the case of YaleB. For example, COIL100-$\nabla \widehat{\varphi }$ was classified by SRWC with an accuracy of 82.83%, while the original image database exhibited 81.29% [1]. Also, as mentioned above the accuracy of classification of $ISIC2020-imprint-\nabla \widehat{\varphi }$ by SRCQW was 88.89% while the original ISIC2020 was classified with 84.21%. Hence the ISIC2020 imprint classification showed an increase of 4.68%.

There was a drop in accuracy for the database YaleB with embedded $\nabla \widehat{\varphi }$ and $\nabla \widehat{\psi }$ compared to the accuracy of the original YaleB because the objects’ boundaries are not present in the images. Hence the SP edges, which play a role in the classification, were not generated by the VFs.

Our study continues by enlarging our repository with more image databases with embedded VFs and imprints of image databases into VFs. Also, we are developing VFs of the linear combinations of the functions $\widehat{\varphi }$ and $\widehat{\psi }$ and will embed them into the presently used and other contemporary image databases. Further we will create an image database of SPs [22] to train ML classifiers and develop an approach to apply VFs to image data augmentation.