Classification of amber gemstone objects by shape

The article describes a classification solution for amber stones. The problem of classifying amber is known for a long time among jewelers and artisans of amber art. Existing solutions can classify amber pieces according to color, but a need to classify by shape and texture is not satisfied up to now. The proposed solution is capable of classifying the gemstones according to a shape. Amber can be considered as a specific object since the form is difficult to define unambiguously. Data for amber experiments was gathered from amber art craftsmen. In the proposed solution amber form can be classified into 10 different classes (7 classes chosen during the experiment).


Introduction and Related Work
Amber is a natural mineral made of wood resin. Today, amber is still extracted from the Baltic Sea and adjacent mines and is used to create impressive jewelry, souvenirs and mosaics. In order to provide amber art craftsmen with suitable raw materials, amber gemstones are selected and sorted according to their size, shape and shades. Amber can be considered a specific object because there can be distinguished uncountable variety of shades, shapes and sizes.
The aim of this work was to create an image analysis algorithm capable of classifying amber gemstones on a conveyor working in real time conditions. As a result, the submitted object for processing must be assigned to the identified class or "other" class, when it cannot be set to the given object with acceptable accuracy. The main purpose of this work is to ensure high accuracy of identification.
Scientists in this field have achieved good results in obtaining visual properties and using them to classify and sort objects into a small number of categories. The first and second order statistical properties [1] were used for visual surface evaluation. Sorting is often used in the food industry [2] to separate objects in waste recycling operations [3]. These systems are based on the acquisition of optical properties of objects' surfaces using different types of sensors such as CCD cameras, spectroscopy [4], stereo vision, infrared light. The optical properties depend on the lighting conditions, which makes very important isolation of objects from the environment and installing a reliable source of artificial lighting. Such systems have strict operational requirements.
In high-speed automation qualitative classification is important when classifying color images. Using histogram limitation techniques, pixel counting, different types of lighting and removal of contour color tones make it possible to obtain a good classification accuracy [5].
Some authors propose [6,7] amber classification according to the shape, based on contour properties. The algorithm begins with the processing of each photo. Initially, a photo is converted from RGB color palette to a binary form. Later, "empty" spaces in the object are filled up and shadows are removed. The image is normalized so that the longitudinal part of the object is parallel to the X axis. Finally, the object image is resized to fit 256x256 pixels size. Centroid distance function (CDF) has been used to identify object form. This is a one-dimensional function expressed at a distance from the reference points to the centroid of the object. For each photo 64 referrals are selected. The referrals coordinates are found using the Moore-Neighbor tracking algorithm modified by Jacob's stopping criteria [15], which gives an array of object's contour coordinates. The circular shift is applied to eliminate the angle of rotation, resulting in a 64x64 matrix. This resulting matrix is normalized (to eliminate the effect of amber size) and inverted (to eliminate the mirror reflection of amber), which results to the 128x64 matrix which is used for the classification.
The algorithm uses a decision tree ensemble consisting of several decision trees that have been trained with different training and testing portions. The ultimate solution is the class provided of the decision tree ensemble. For each amber photo a 128x64 matrix is created and passed to decision tree ensemble. Due to the relatively large amount of data needed to describe the form of amber, the algorithm is not suitable for real-time operation systems because it requires large computation resources. This algorithm requires a person -an expert who must first select amber forms by hand, train decision trees and then to classify.
In [8,9] authors suggest similar methods to identify forms, where the perimeter and the area ratio values are assigned to the pre-defined classes. The authors [8,9] achieved high (over 90%) results in the classification of standard forms like square, triangle, rectangle, oval, circle, but this method did not produce the desired result in sorting amber gemstones since their form is difficult to evaluate and decide, therefore a new form classification algorithm was adopted.

Proposed Methodology
The practical problem of this research is related to the acquisition of the feature points of an object and its boundaries -the contour of the object for further analysis by use of the segmentation methods. When determining a shape of an object, it is very important to precisely extract the object from the background. When objects are separated from the background it is possible to start extraction of the properties mentioned. At this stage, we have an image in which an amber gemstone is white and the background is in black pixels.
To properly evaluate the object's shape, additional steps of image processing are required, where the long axis of an object is calculated and rotated parallel to the x coordinate axis. The image is subjected to a few more rotation procedures (when necessary), where the narrowest part of the object is aligned to the right with respect to the X axis, and at the top with respect to the Y axis. The algorithm evaluates the length of the X and Y axes, diagonals passing through the center of the object and rotated by 45 degrees from the axes mentioned above and adjacent to the edge of the object, the actual area of the object and the theoretical rectangular area. Figure 1 shows parameters for the object form identification.

Figure 1. Parameters for form identification
Here w1 and w2 (analogous to z1 and z2) are the distance between the center of the object and the contour of corresponding diagonals rotated at 45 degrees. The shape of an amber gemstone can often be described as ambiguous, for example: one side of the stone can be oval-like while the other shaped as triangle, therefore the asymmetric form is included in the list of investigated forms.
The proposed object form recognition algorithm is shown in Code 1 to Code 4 below.

Code 1
Pseudo code for the form identification algorithm.
Here and further tol1 and tol2 -tolerance values (allowed limits of parameters' deviation). P is the theoretical area of the rectangular shape (P = x * y), p is the real area of the investigated form in pixels. A complete list of conditions for each of the "basic" form terms being tested: circle: x/z = 1 IR x/y = 1 IR Z1 = Z2 IR p < P oval: x/z > 1 IR x/y > 1 IR Z1 = Z2 IR p < P rectangle: x/z > 1 IR x/y > 1 IR Z1 = Z2 IR p = P square: x/z < 1 IR x/y = 1 IR Z1 = Z2 IR p = P equilateral triangle: x/z < 1 IR x/y = 1 IR Z1 ≠ Z2 IR p < P isosceles triangle: x/z < 1 IR x/y < 1 IR Z1 ≠ Z2 IR p < P right triangle: x/z > 1 IR x/y > 1 IR Z1 ≠ Z2 IR p < P uneven rhombus: x/z < 1 IR x/y < 1 IR Z1 = Z2 IR p = P The algorithm was implemented by allowing the tolerance of the values by 10% for each condition checking.

Experiments and Results
The pieces of amber fall off the vibrating bowl on a conveyor. The laser fork detects amber that interrupts the laser beam and sends a signal to a digital camera (type FFMV-03MTC, mpg Point Gray, Canada) that captures the image and transmits it for processing. MATLAB 2016b version and computer Intel i5-7260U, 2.2GHz, 8 GB Ram, Win10 x64 were used for algorithm implementation and image processing.
During the experimentation there were 2800 photos produced with the resolution of 640x480 pixels.
It is important to extract amber from the background before starting to calculate amber properties. This is preceded by the evaluation of the histogram and the contrast adjustment, which allows the shadows to be properly removed. If the shadows are not removed properly, the image of the amber gemstone becomes uniform and similar to the oval. Figure 2 shows the pictures of amber gemstones before processing.   Shadow intensity is highly dependent on the type of artificial lighting and amber color, which can vary with the transparency and can be transparent, semi-transparent, and white, have many yellow or brown shades etc. The best results were achieved with the DOME type lighting where the light is evenly distributed. Figure 3 illustrates amber images after shadow removal.
Although the shades of amber gemstones are different, shadows are successfully removed and the exact representation of the object is obtained in white pixels, as shown in Figure 4.
For the purpose of identifying the shape of the object, additional dimensions are calculated for each amber photograph: the length of the X and Y axis passing through the center of the object, the diagonals of the object rotated 45 degrees from the axes mentioned above, the actual and theoretical rectangular area of the object.
Before starting the mentioned calculations, the long axis of a gemstone is rotated parallel to the X axis. The long axis of the object is considered to be the long axis of an ellipse, that has the same second momentum as the white region representing the object. Then, if necessary, the amber photo is rotated so that the narrowest part of the object is on the right with respect to the X axis and the top of the Y axis at the top. In Figure 5 picture of amber before and after turning operations is shown.
Required dimensions are calculated after the initial steps performed to determine the shape of the object (shown in Figure 6). The taken sizes are relative to the image, this allows the form to be evaluated independently of the image scale and height position of the camera.  During the experiment, from 5 to 15% tolerance limit was tested to classify the object by the form when checking the ratio of the corresponding dimensions. Table 1 shows the data of the form classification when using different tolerance values. Here we can see, the classification results depend on the value of the selected tolerance. Using a lower tolerance value, more objects were assigned to the triangle shape, which was not approved by the human expert. Therefore, this tolerance parameter needed to be adopted by performing additional experiments.
Value of the tolerance parameter directly affects a distribution of analyzed objects to the form classes. At a lower values of the tolerance, more objects were assigned to the triangle class due to the algorithm's condition check order, because the specific criteria for that class were met. Analogously at higher tolerance values, analyzed objects were assigned more to the asymmetric form since conditions of this particular class were met. When about 10% tolerance value was chosen, more complex forms were distinguished and this gave the most acceptable results for the human-expert assessment. During the experiment it was determined that different values of tolerance should be used to evaluate the ratio of real and equivalent areas.  (T = 5%)  Circle  13  1  3  Oval  557  725  211  Rectangle  11  95  0  Square  261  423  106  Rhombus  100  312  7  Asymmetrical  1309  1043  1139  Triangle  549  201  1334 Accuracy of the algorithm for classification by form has reached 83.5% of the expert's (human) decisions. There have been cases where the form of amber is difficult to identify unambiguously even for the human eye due to the uncertainty of the shape, for example, when one side of an amber stone is similar to a circle and the other half to a square or the like. The visual results of the classification are shown in Table 2. Using the same set of photographs taken during real-time experiment, methods mentioned in [13, 14] publications (classification by form using centroid distance function CDF) were tested and compared. The results of the method comparison are presented in Table 3. Experiments show that the proposed relative size method better separates "pure" forms, and the CDF method is more responsive to the existing data sample. That is because in CDF the intervention of an expert takes place and samples for each class are picked manually.
One of the main differences between methods is the speed of identification of the object class. Figure 7 shows the average class identification time for those two methods.
Amber classification using CDF method takes an average of 0.33 seconds, when classification using relative size method is made by an average of 0.03 seconds. The relative size method is up to 11 times faster than the CDF method. In addition, tests were carried out on a data set of 4 basic forms. A set of four basic forms of data consists of a circle (set of 3720 samples), a square (set of 3765 shapes), a triangle (set of 3720 shapes) and a star shape (set of 3765 shapes). The size of the analyzed photos is 200x200 pixels. Table 4 shows examples of form data sets. Table 4. Examples of form data sets

Square Star Triangle
The form of the star was not analyzed because the proposed algorithm does not provide for the possibility of classifying star-shaped amber. Each form has been individually tested for classification. Table 5 shows the results of the classification of the main form data set.  Incorrectly identified class  Circle  3484  236  3619  101  Square  3277  488  3651  114  Triangle  3286  434 3610 110 In Figure 8 the accuracy of the relative sizes and the classification of the CDF method is shown. The relative size method showed slightly worse results than the CDF method. This was due to the fact that this data set has photos containing noise, which led to the miscalculation of some relative values. The shape of the amber gemstones is random, so the evaluation of the accuracy has a certain component of "free interpretation", therefore 100% accuracy would only be declarative, but it is hardly measurable. The results gave similar proportions of calculation time for the methods as in previous experiment, with real amber photographs that were shown in Figure 7. This time the CDF method identified gemstone's class by 0.12 seconds in average, while amber class using relative size method (proposed) is identified by an average of 0.01 seconds. Size of the photos taken and the CPU speed influences the average time of calculations, but these parameters influences both methods proportionally, therefore can be omitted, since the datasets and the hardware was used the same.
Publications [10][11][12] also discuss ways of classifying amber by color, but this article does not aimed to discuss the color classification in more detail.

Conclusions
One of the most important steps when classifying the shape of an object is to extract the object from the background as accurately as it is possible.
The form of the amber gemstone is not always unambiguously defined, even the assignments of the expert (human) are ambiguous. To better fit expert-like classification, tolerance values are introduced to the form parameters. When the tolerance value is higher algorithm tends to treat the object more as a trapezoid, and the tolerance value is lower algorithm tends to treat the object form as a triangle.
The accuracy of classification by form is 83.5% compared to human-expert judgment. Proposed method is up to 11 times faster according to the methods described in publications [6] and [7].