Facial Pain Detection Using Multilinear Subspace Learning

1. Introduction and Motivation

Understanding human behavior is complex and has been one of the promising areas since the advent of computing machines. Building a system that can accurately distinguish and understand human behavior has played and still plays a crucial role in research and industry. Our study focuses on a specific property of pain behavior, namely the automatic detection of pain by facial expression. Pain is a very unpleasant sensation caused by an illness or injury, or by mental distress or suffering. It is often referred to as the fifth vital sign in health care because pain, like the other critical signs, is now considered an objective sensation in health care. According to the National Centers for Health Statistics, approximately $76.2$ million people worldwide suffer from pain. In general, medical abnormalities are difficult to assess and manage and are usually measured by patients by self-report. This and similar procedures are accepted because they are simple and easy to understand and often provide information that validates the level of pain experienced by the individual. However, these methods only work when the patient is sufficiently alert and cooperative, which is not always possible in medicine. But beyond that, pain assessment using self-report measures is a significant challenge. It is not always reliable and valid in critically ill adults, including those who are unable to communicate their pain level, such as people with dementia and certain types of neurological disorders, as well as intensive care unit (ICU) patients who require an oxygen mask to breathe. In addition, it cannot be applied to unconscious patients or newborns. To overcome these limitations, observational and psychological measures have become indispensable. Our hypothesis that the human face is a rich source of non-verbal information that delivers semantic cues for understanding emotions. Wang et al. proposes to objectively measure a patient’s pain level using machine learning and pupillary response algorithms in [1]. We confirmed this rich resource information discriminative can be useful in revealing mental status through social cues. The face texture is rich resource information and a free natural password and a means through which people recognize each other, it is used in the early stages of life by an infant to recognize his surrounding people based on facial features in [4]. We were motivated and inspired by the choice of these CNN architectures given their good performance in different vision tasks, as shown in the ImageNet large-scale visual recognition challenge [7]. An overview of analysis on the subject by comparing some GoogleNet, ResNeXt-50, and MobileNet based on CNN in [2]. Theses research recently demonstrates the importance of deep learning and methods conventionnel for pain estimation by [3] is considered a spontaneous reaction to pain. In plus Othman et al.[43] are demonstrates how to recognize pain intensity from faces in videos, while humans are very adept at this task compared to machines. Variation in facial expression often symbolizes the onset of pain. Physicians place great importance on the credibility of these behaviors and consider them to be consistent and convincing signs of pain. Facial activity has been co-opted as a primary or major component of most multidimensional behavioral checklists or pain rating scales. There is a considerable amount of literature in which the Facial Action Coding System (FACS) [5], has been applied to pain expression in [6]. The principal contributions of this article comprise: The distribution of the degraded spectrum by total variation regularization by Liu et al. [23,28,32,33,34,35,36,37]. model capable of calculating with high precision the identity of the person under analysis separately from the incoming angles. Our contributions can be summarized in the following points:

• -We propose a new robust approach that mainly explores a newly designed automatic pain detection using facial expressions.
• -The new framework includes multilinear principal component analysis as a contributing step for face feature extraction and dimension reduction, which increases the detection rate compared to previous studies reported in the literature review to improve the discriminatory power.
• - Our system performance is evaluated by testing on the UNBC-McMaster Shoulder Pain Expression Archive databases confirming its supremacy over those of the state-of-the-art.

Table 1. Proposed Algorithm by Aliradi et Taleb-Ahmed.

Imput: A face in pain from the UNBC-McMaster, learned space projection matrices by Multilinear Whitened Principal Component Analysis (MWPCA), and a fixed threshold. Initialize Type=(A pair of images)
For $i=0$ to 4. $\lambda$= $4\times 2^i$ Calculate $G\times (\lambda ,\theta )$ Compute ${PSPI}_{type}(i+1)$ (equation 1) End.
Output: represent an image that contains five scales, with the convergence parameters.

Table 1. Proposed Algorithm by Aliradi et Taleb-Ahmed.

Imput: A face in pain from the UNBC-McMaster, learned space projection matrices by Multilinear Whitened Principal Component Analysis (MWPCA), and a fixed threshold. Initialize Type=(A pair of images)
For $i=0$ to 4. $\lambda$= $4\times 2^i$ Calculate $G\times (\lambda ,\theta )$ Compute ${PSPI}_{type}(i+1)$ (equation 1) End.
Output: represent an image that contains five scales, with the convergence parameters.

The rest of the paper is organized as follows. Section 2 reviews the current literature in Face Verification Based on Subspace Tensor Transformation. Section 3 presents the framework of the proposed verification system. Section 4 demonstrates the experiments and their settings. Section 5 discusses the obtained results. Finally, the conclusion and future works are introduced in Section 6.

2. Related Works

With the rapid development of computer technology, tensor data increased and extended in many areas such as augmented reality, signal processing, computer vision, medical image analysis, and web data mining. Especially the most famous As the problem of pain detection is not new and much research has been done on this topic, it is very important to have a view of these previous works to have an idea of the approaches used and their limitations as well as the results obtained, so for this, we made a literature review of some articles presented in Table 1. In [8], Ashraf et al. used the Active Appearance Model (AAM) on videos containing pain expressions based on Support Vector Machine (SVM) based machine learning procedure to classify pain and non-pain. With the advantage of representing dynamic alterations in pain-related actions, the best-performing predictive model yielded a success rate of $\mathbf{82}.\mathbf{00}\%$. In [9], Lucey et al. Utilized a classifier SVM to develop a system for detecting image-level pain in two ways on patient images: first directly from facial features, and second by fusing individual action unit (AU) detectors. In addition. extended their work as described to detect pain to use an AAM approach on a frame-by-frame. They showed that fusing all AAM representations using linear logistic regression (LLR) provides notable performance for detecting pain and action units in the image. In [10], Kaltwang et al. proposed to use a different form of facial landmarks and appearance features, namely discrete cosine transform (DCT), relevance vector regression (RVR), and local binary model (LBP), and then merged these features and thus showed that merging these features leads to a better estimation of pain level compared to the specific estimation of pain intensity, and they manage to achieve an accuracy of $\mathbf{92}.\mathbf{00}\%$. In ref [11], authors used AAM to extract the canonical normalized facial appearance. Finally, an SVM classifier is used to detect the pain level on a frame to obtain an accuracy of $\mathbf{73}\%$. In ref [12], Sourav and Mrinal utilized Gabor filtering as a contributing step for face feature extraction and dimension reduction using principal component analysis (PCA), which increases the detection rate according to the comparative study. In [17], authors utilized test datasets to compare the target algorithm. They then use the support vector machine as a classifier and achieve $\mathbf{95}.\mathbf{50}\%$ accuracy on the images, which is the best result in the pain detection problem so far. Still recent research was published in [13] where the authors proposed a fully automatic pain assessment model including a novel convolutional network fusion framework for assessing different pain intensities from raw facial images that achieved an F1-score of $\mathbf{94}.\mathbf{00}\%$ for pain level classification. In ref [14] Fat et al. present a framework for pain detection/classification that uses a combination of KNN and Adaboost classifiers to obtain an accuracy of $\mathbf{91}\%$. Alghamdi et al. in [18], developed a new system expressions-called (FEAPAS) to notify a staff when a patient suffers pain by activating an alarm. Plus the intensive care patients in ref [23]. Pedersen et al. [25] proposed a new approach based on the combination of depth information and descriptors such as RGB values, thermal facial images. Ferroukhi et al. proposed a new coding method video based on bandelet transform algorithm in [26]. In addition, Chen et al. [15] detect a pain event and locating in the video. We used CNN for automatic feature extraction from frames in [24]. Often the pain is expressed verbally, but in some cases, traditional patient self-reporting is not efficient in [16]. Ghosh et al. proposed method in the fourth component which utilize the scores due to the statistical and deep feature analysis are fused to ameliorate the performance in [27]. To utilize the prediction models are based on deep feature analysis and statistical procure scores for the facial region’s pain intensities (low-pain, high-pain, and no-pain). X et al. [20] have extensively used the convolutional neural network (CNN) using a different image classification tasks. Recently authors proposes a new architecture that uses discriminative information, which is based on the exponential discriminant analysis (DIEDA) and the projected histograms for each region are scored using the discriminative metric in refs [21]. Mimouna et al. simplifed a heterogeneous multimodal dataset for advanced environment in [42].[38] Al-Shiha et al. [39]. We see through our article that we have used, several hand-crafted features in the pain estimation task. In addition, many classifiers are used to discriminate these features. They implemented the proposed approaches to a sufficiently detailed level, for instance, in refs. [Aliradi], the relationship between AUs and pain have been considered.

2.0.1. Prkachin and Solomon Pain Intensity Metric (PSPI)

PSPI is a pain assessment metric calculated from pain-related action units (AU). Previous studies where it has been observed that pain-related expressions are characterized by certain facial muscles being activated during pain. We note that the activations of these muscles are encoded by a set of numerical values called AU. During the pain episode, sets of AUs are activated, such as eyebrow lowering (AU4), orbital tightening (AU 6 and AU7), levator raising (AUC9 and AU10), and approximation eyes $(i.e.AU43)$. All of these AUs are measured using a 6-point scale (i.e., 0 and 5 represent absence and maximum, respectively). These studies confirmed that these AUs consist of pain-related information and are defined by the PSPI score. The PSPI score is calculated based on the following equation: By assigning numerical values to those six AUs, pain can be calculated via the Prkachin and Soloman pain intensity (PSPI) equation,

$$PSPI=AU4+\left(AU6orAU7\right)+\left(AU9orAU10\right)+AU43$$

(1)

The dataset also contains pain scores for each frame which are based on the PSPI (Prkachin and Solomon pain intensity scale). The PSPI scale is defined in 17 levels using FAC. The calculation is shown in the equation below: The PSPI score is the sum of AU4, AU6, or AU7 (depending on which is higher in intensity), AU9 or AU10 (depending on which is higher in intensity), and AU43. The AUs are: lowering the eyebrows (AU4), puffing of the cheeks (AU6), tightening of the eyelids (AU7), wrinkling of the nose (AU9), the raising of the upper lip (AU10) and closing of the eyes (AU43). Except for AU43(0 = absent, 1 = present), each AU is coded in 6 levels of intensity (0 = absent, 5 = maximum). Figure 1 shows an example of the same face with different PSPI levels. According to the results of strong pain detection (PSPI = 3) would give a more stable result, therefore, the pain data set is preprocessed into NO-PAIN(PSPI=0) and PAIN(PSPI = 3) subset.

2.0.2. UNBC Databaset

We chose in this work a database called the UNBC-McMaster shoulder pain expression archive[19]. The database consists of 200 videos of 25 subjects with spontaneous facial expressions who suffer from shoulder pain. It has in total 48,398 FACS (Facial Action Coding System) coded frames. The subjects performed the physical exercise from affected and unaffected limbs and all activities were recorded by a video camera. In addition, each frame was labeled by the PSPI score [19]. For the PAIN subset, there are about 3000 images from 25 subjects, for the No-PAIN subset, there are about 40 000 images in total. Advantages: The size of the database is it’s main strength, more than 48,000 images are more than enough to train our model with almost all the algorithms we want to use, and better still, the fact that the labels are not in binary, which means that the pain is expressed in levels for each frame of the video (following the formula above), which helps to greatly improve the accuracy of the model, it also gives us the opportunity to try to predict the level of pain felt by the subject instead of a simple binary guess. Another strong point is that the images have a background this time, which means that testing images outside the database (images we capture) will not confuse the model since it is already used to seeing backgrounds before. Limitations: The main weakness of the dataset is the lack of diversity in the subjects, as there are only 25 subjects who all suffer from the same type of pain (shoulder pain), which makes the 48398 images very similar and repetitive, especially the fact that they are taken from a video with consecutive images. The images are also of significantly lower quality which can cause problems for the face extraction model.

3. Proposed of Our Architecture Method

In this section, we initially present an abbreviation of (faces preprocessing), we explain the details of the proposed sustem for multimodal face verification based on tensor representation. As illustrated in Figure 3, the block diagram of the global architecture of our method consists of four essential components: feature extraction, multilinear subspace transformation tensor, dimensionality reduction, and comparison(Cosine similarity), and in conclusion, we prove our architecture. In this section, we details our work, we introduce a system for pain detection/classification that uses a combination of different techniques in face detection, features extraction, at he theory wavelets Gabor which bases this Frames which invented by Gabor Having looked at the dataset as well as the previously mentioned related works, as shown in Figure 1 the block diagram of the global architecture in general design for the system was essential fixed from the beginning with very little wiggle room; as any facial recognition system works, the main steps are: preprocessing the data, extracting the face, and then training the classification model, except in our case we are working with a Multidimensional Subspace Learning algorithm, meaning the data has to go through another step before being parsed through the model, this step is going to fall between the face extraction and the model training. As for the algorithms and hyperparameters used, we first started with an educated guess based on other articles as well as general Computer Vision classification models’ configurations, then we started experimenting more until we reached the desired performance from the model, the system was created around the UNBC Pain Dataset with over 40 thousand images, except due to overfitting issues as well as time and computing performance constraints we decided to limit the data used to just 20 thousand images (which is generally more than enough for binary classification models), we prove performance our architecture.

3.1. Data Preprocessing

We proceed by several stages of technique preprocessing images as with any computer vision system, the extracted images did not meet our requirements for the model, specifically the image resolution as well as the color channels. Since pain is expressed through changes in our facial expressions, the colors would have no meaning on the pain state in question, hence our decision to transform all images to grayscale, not only will this increase the performance of the model through all its training stages, but it will also get rid of any unwanted or unnecessary features that could cause interference. Since the previous face extraction function will return images of different sizes We also decided to lower the resolution of all images used (for both datasets) to size $(48\times 48)$ pixels, we found that this is the lowest resolution we can use that will still guarantee distinguishable facial features. We must remember that one of the main reasons for using multidimensional subspace learning is not only to increase the accuracy of predictions but especially to reduce learning times. Therefore, we need to make sure that the data we feed to the model is as small as possible (without losing too much detail, of course). The Python library used is OpenCV (via the CV2 interface), which offers easy-to-use and efficient functions for resizing images as well as for changing color channels.

3.2. Feature Extraction

First, we need to extract the faces from each image, this is a crucial step because it not only reduces the size of our images but also removes the background and any unwanted features. Since the facial expressions of pain in humans are expressed by lowering the eyebrows, squeezing the eyes, wrinkling the nose, raising the upper lip, and opening the mouth, these are the only facial features that need to be retained as detailed in [Sri09]. For this, we implemented a function and we decided to use the MTCNN library which allows us to extract the key points we need from the images, namely the right eye, the left eye, and the right corner of the mouth. well as the left corner, the key points are then used to create a rectangular frame around the facial features. The MTCNN library (as the name suggests) uses a CNN classifier to extract landmarks from the face. The model is able to detect faces across different lighting levels and degrees of rotation, with extremely high accuracy of between $\mathbf{94}.\mathbf{80}\%$ and $\mathbf{100}\%$. In that article, a novel technique and scheme of fusion have been proposed that uses high-order Multilinear Whitened Principal Component Analysis (MWPCA).

4. Multilinear Subspace Learning

Over the past decade, most of the data has previously been treated just as a $1D$ vector which required breaking the fabric of the initial data. In order to enhance the subspace tensor approach to use in our painful estimation. Such as the case of the classical PCA approach for dimensionality reduction which has become unsupportable for the presentation of multidimensional space data for very high dimensionality. For this reason, The main objective of our system is to see the effect of multilinear subspace learning on multidimensional imaging. This approach aims at dimensionality reduction on large data tensors containing either vectors or, in our case, matrices. This approach is also a generalization of the familiar linear subspace learning, usually implemented with PCA, ICA, LDA, or CCA, and to solve all problem by using any features with the TXQEDA method for dimensionality reduction in ref [44]. Multilinear subspace learning encompasses multiple dimensionality reduction algorithms. In our case, we decided to use multilinear principal component analysis (MPCA or M-mode PCA), which is actually an N-dimensional extension of principal component analysis, and works by centering the data tensors to find abnormally large (or abnormally small) values over defined intervals to extract and sort them into new lower-dimensional tensors. The MPCA is an extension of PCA. It allows the projection to a lower subspace. The implementation of the MPCA approach is done in 3 main steps, the first one is the centering of the input tensor (the image essentially), then the projection matrices are initialized by finding the eigenvalues and the eigenvectors according to the image in question, then the Q-Based method is applied, this method works by finding the ratio of the eigenvalues after truncating the number of least important eigenvalues on the sum of the other values, the user defines the Q-value in question (in our case we have used $\mathbf{97}\%$) Finally, the MPCA loop is launched, this loop works by applying the Scatter Max method or the Error Minimization method (we chose the first one) on the data tensor with the remaining eigenvalues, which will produce 3 different projection matrices which will then be grouped by the transformation function [18].

5. Training of the Model

Finally, we used SVM as a data classification model, more precisely the Support Vector Classification (SVC) from SkLearn, the data were first divided by $\mathbf{80}\%$ and $\mathbf{20}\%$ for training and validation respectively, the hyper parameters of the SVC model we used were: $[0.1,1,10]$ for C and, $[0.001,0.01,0.1]$ for Gamma and finally we used RBF, Linear and Poly kernels, the best accuracy results for the combinations of these parameters were automatically chosen by the SVC fitting function.

6. Results Analysis

To evaluate the performance, Using the UNBC database, as shown in Table 2 we managed to achieve an accuracy of around $\mathbf{98}.\mathbf{80}\%$ only by using one-half of the data available (20,000 images), and although we were able to use the whole database, there was a fear of falling into overfitting especially considering that this data suffers from the variety and that the images are taken from successive video frames which lead to repetitive images. the trained model gives accurate and precise pain status predictions for both test images from the dataset as well as images outside the set (images we took ourselves), the most notable thing about this model was how short the training time was especially for such a high level of accuracy, this was helped by the smaller tensor size that was used during the training phase, which is where we can definitely see the benefit of using MPCA, although it is important to note that the fitting time for the MPCA model itself, as well as the data preprocessing phase was noticeably longer.

Table 2. Accuracy comparison of the different techniques for detecting painful expressions estimation.

Work/Year	Methods	Dataset	Classifier	Best Result
Rodriguez etal. [12]	Features: VGG-16	UNBC-McMaster Shoulder Pain	LSTM	MSE 0.74 AUC $\mathbf{93}.\mathbf{30}\%$ Accuracy$\mathbf{83}.\mathbf{10}\%$
Tavakolian et al. [13]	Features: CNN	UNBC-McMaster Shoulder Pain	Deep Binary	$MSE0.69$ PCC81%
Bargshady et al. [14]	Features: VGG-face	UNBC-McMaster Shoulder Pain	RNN	MSE $0.95$ $Accuracy\mathbf{75}.\mathbf{20}\%$
Wang et al. [39]	Features: GA-ANN	UNBC-McMaster Shoulder Pain	ANN	MSE $1.57$ $Accuracy\mathbf{81}\%$
Huang et al. [15]	Features: spatiotemporal, and geometric	UNBC-McMaster Shoulder Pain	Hybrid network	MAE $0.40$MSE $0.76$ $Accuracy\mathbf{82}\%$
Fat et al. [43]	Features: Gabor-Adaboost	UNBC-McMaster Shoulder Pain	KNN	$Accuracy\mathbf{91}\%$
El-Morabit et al. [2]	Features DEIT-PNP	UNBC-McMaster Shoulder Pain	SVR	Accuracy $\mathbf{84}.\mathbf{15}\%$
Semwal et al. [11].	Features: Deep-CNN	UNBC-McMaster Shoulder Pain	SVM	$Accuracy\mathbf{92}\%$
Ghosh et al. [6]	Features:Gabor-PCA	UNBC-McMaster Shoulder Pain	SVM	$Accuracy\mathbf{95}.\mathbf{50}\%$
Our system	Features SVM	UNBC-McMaster Shoulder Pain	SVC	MAE $2.20$, and$Accuracy\mathbf{98}.\mathbf{88}\%$

Table 2. Accuracy comparison of the different techniques for detecting painful expressions estimation.

Work/Year	Methods	Dataset	Classifier	Best Result
Rodriguez etal. [12]	Features: VGG-16	UNBC-McMaster Shoulder Pain	LSTM	MSE 0.74 AUC $\mathbf{93}.\mathbf{30}\%$ Accuracy$\mathbf{83}.\mathbf{10}\%$
Tavakolian et al. [13]	Features: CNN	UNBC-McMaster Shoulder Pain	Deep Binary	$MSE0.69$ PCC81%
Bargshady et al. [14]	Features: VGG-face	UNBC-McMaster Shoulder Pain	RNN	MSE $0.95$ $Accuracy\mathbf{75}.\mathbf{20}\%$
Wang et al. [39]	Features: GA-ANN	UNBC-McMaster Shoulder Pain	ANN	MSE $1.57$ $Accuracy\mathbf{81}\%$
Huang et al. [15]	Features: spatiotemporal, and geometric	UNBC-McMaster Shoulder Pain	Hybrid network	MAE $0.40$MSE $0.76$ $Accuracy\mathbf{82}\%$
Fat et al. [43]	Features: Gabor-Adaboost	UNBC-McMaster Shoulder Pain	KNN	$Accuracy\mathbf{91}\%$
El-Morabit et al. [2]	Features DEIT-PNP	UNBC-McMaster Shoulder Pain	SVR	Accuracy $\mathbf{84}.\mathbf{15}\%$
Semwal et al. [11].	Features: Deep-CNN	UNBC-McMaster Shoulder Pain	SVM	$Accuracy\mathbf{92}\%$
Ghosh et al. [6]	Features:Gabor-PCA	UNBC-McMaster Shoulder Pain	SVM	$Accuracy\mathbf{95}.\mathbf{50}\%$
Our system	Features SVM	UNBC-McMaster Shoulder Pain	SVC	MAE $2.20$, and$Accuracy\mathbf{98}.\mathbf{88}\%$

6.1. Defining the Time Cost

which attains a good performance and represents a record time to compare with other methods.

6.2. Comparison with Existing Methods

we compare our method against other state-of-the-art models under the same metrics. Firstly, we conduct a comparison with traditional hand-crafted methods, as shown in Table 1. As can be seen, our method has obvious advantages against these traditional methods under AUC, ICC, PCC, MAE, and MSE. It can be concluded that in the task of pain estimation, our proposed HDN shows a more promising performance than hand-crafted methods. This is reasonable, as the deep learning method could learn target features according to the task. This makes the current pain research tend to deep methods. So, we also list some deep pain methods in Table 2. From the comparison of Tables 1 and 2, it is apparently observed that deep learning methods have better performances, which indicates the superiority of deep models. However, these methods take coarse input rather than considering the locality and relationship characteristics. In this paper, the proposed HDN leverages local and hierarchical branches to involve the locality of pain and region relationships simultaneously. Therefore, our method outperforms these deep learning methods in most cases. Although the results of PCC are not the best, but it is a good enough result. This is because the methods with better PCC performance are devoted to the video segments (e.g. refs. [18], which naturally predict the trend of pain intensity well. On the other side, for single-frame images, their prediction errors are obviously larger than ours. Since our method is targeted to image frames, our training procedure would reduce the distance between the prediction and label of each frame. It is possible to ignore some changes in pain intensity during this process. Therefore we can conclude that our HDN method achieves impressive performance. Besides, there are some other hierarchical methods whose framework is similar to ours, as shown in Table 3. These researches employed a group of hand-crafted features, e.g. refs. [11]. Some combinations of traditional and deep features are used in ref. [23]. These studies tried to provide some solutions from various aspects. However, there is no exploration of the nature of pain. They neglect the locality of pain and region relationship that are significant pain characteristics. Therefore, compared to our method, these methods present disadvantages in terms of performance to some extent

6.3. Discussions

Our principal goal is to offer more credibility to our approach (Facial pain detection using multilinear) in terms of visibility, scalability. We have compared it with other methods. The obtained results confirm the power of the new face pain detection to take into consideration different variability factors for unimodal face pain cases and the one databases used. We have examined the precision performances obtained in tables 2 and 5 with the classification of the various descriptors during our experiences. Individual descriptors and their merging: To compare individual descriptors’ performance, the results consistently show that system is the best. We evaluated the performance of our new architecture, which we chose because it yields the highest accuracy rate (AR) and the lowest error equal rates (EER) based on the findings obtained and summarized in tables 2 and 5. Tables 2 and 5 report the errors EER and the VR at ($0.001$ FAR) on the UMBC Shoulder databases. Findings are recommended for individual functionality and their merger. We notice technique achieved a $\mathbf{98}.\mathbf{80}\%$ of $F1-score$. Moreover, for pain intensity estimation the proposed model has achieved a mean absolute error of $\mathbf{2}.\mathbf{20}\%$ and an accuracy result of $\mathbf{98}.\mathbf{80}\%$ that demonstrate performance outperforms other methods the state-of-the-art in areas under-the-curve of the UNBC-McMaster dataset. We can confirm that the discriminative power of face pain images is higher than that achieved by face images to handle different challenges, such as poses, illuminations, and expressions. In summary, we confronted our results with the state of the art for the (controlled) environments. We compared our obtained results with their last nine methods best cited in the state of the art, including: [12, 13, 14, 39, 43, 2, 11, 6] for the same UNBC-McMaster data sets. The performance of our system is easily proven and visible (tables 2 and 5). The accuracy obtained from our system is better compared to the other systems tested.

Figure 4. The Result of our system proposed, respectively.

Figure 4. The Result of our system proposed, respectively.

Figure 5. Accuracy, respectively.

Figure 5. Accuracy, respectively.

We synthesize the confusion matrix of the ANN classification results using selected functions is shown in Figure 3. The ANN classifier had good performance for data labeled "B" (baseline) and “H” (strong pain). Of class "B", $\mathbf{98}.\mathbf{80}\%$ were classified correctly and $\mathbf{21}\%$ were classified as class "L". Among the “H” class, $\mathbf{95}\%$ were classified correctly, and only $\mathbf{5}\%$ were classified as class “L”. However, the ANN had poor performance for "L" data (low pain). Between "L" classes, only about half were classified correctly and $\mathbf{44}\%$ were classified as "H" class, which means that the classifier was not sensitive to "L" data. It can also be seen in Figure 3. The confusion matrix also suggested that the type 1 error (false positive) and type 2 error (false negative) for class ’B’ were 0 and 3 respectively. Type 1 and type 2 errors for class ’L’ were 33 and 732 respectively. Type 1 and type 2 errors for class ’H’ were respectively 3222 and 13.

6.4. Extensive Experimental Comparison

For the UNBC-McMaster shoulder pain database, we have evaluated the performance of this suggested system with some existing state-of-the-art approaches. We employed Gho et al./2016 with Features:(Gabor+Filters+PCA) in [6] , VGG-16 [12], CNN-Deep Binary [13], VGG-face [14], Deep-CNN [11], Huang et al. 2021a [15], Fat et al. /2021b [14], and DEIT-PNP in [2], methods and obtain the performance of the proposed system under the same training–testing protocol as used in the proposed system. The comparison of the proposed system for UNBC database have been reported in Tables 1. From these performances, it has been observed that the proposed system has obtained better performance than the other competing methods concerning both the employed databases.

7. Conclusion and Future Work

In this article, we presented a new system original (Facial pain detection) based on tensor subspace transformation. Firstly, we propose a new discriminative local face descriptor, Hist-Gabor, based on histograms of basic Gabor filters. The Gabor wavelets images are subdivided into several blocks with the objective to build and encode the face structure exactly. Taking advantage of the histogram features which consider the microstructure information such as plane area, edges, and spots, the histograms of the face blocks are concatenated in order to create a robust feature vector. Secondly, to enhance the discrimination of the proposed tensor face representation, we extended two multilinear subspace methods transformation (MWPCA + TEDA). In addition, we have computed the time cost total of the system, which attains a record time. Experimental results on one publicly available face pain datasets have shown that our system method to comparable or better accuracy, and performance and outperform other existing face descriptors and state-of-the-art face verification methods. For perspective, we are interested in applying the proposed; this calls for the research community to cooperate in efforts to add new and more reliable databases and evaluation protocols to advance face verification research. For a fair comparison, the new databases should include the conditions of the image, and possession and discuss the potential allegation of the limitations of the data sets. We give the code available by request to verify the validity of our experiments.