COVID-19 diagnosis by 5-second facial video processing using vibraimage and artificial intelligence

The Covid-19 pandemic spreads in waves for a year and a half, despite significant worldwide efforts, the development of biochemical diagnostic methods and population vaccination. One of the reasons for the infection spread is the impossibility of early disease detection through biochemical diagnostics, since biochemical processes slowly develop in a body. At the same time, well known that behavioral characteristics of a person, measured based on reflex movements, are capable for inertialess assessment of psychophysiological parameters. Vibraimage technology is the method of head micromovements video processing by inter-frame difference accumulation and converting spatial and temporal characteristics of the inter-frame difference into behavioral and psychophysiological parameters. Here we shown that behavioral parameters measured by vibraimage changed during COVID-19 infection. The identification of changes signs in behavioral parameters detected by AI trained on patients and controls. The best diagnostic accuracy (higher 94%) obtained using instantaneous values of behavioral parameters measured with the following vibraimage settings: 10Hz frequency of basic measurements; 25 inter-frame difference accumulations and averaging the diagnostic results over period of at least 5 seconds. COVID-19 diagnoses by behavioral parameters showed earlier (5-7 days) detection of the disease compared to symptoms and positive results of biochemical RT-PCR testing. Proposed method for COVID-19 diagnosis indicates infected persons within 5 seconds video processing using standard television cameras (web, IP) and computers, allows mass testing/selftesting and will stop the pandemic spread. We assume that head micromovements analysis for diagnosis of various diseases is possible not only with the help of vibraimage technology. Further research of human head micromovement analysis will help stop the COVID-19 pandemic and will contribute to the development of new contactless and environmentally friendly methods for early diagnosis of diseases.

Many outstanding scientists of the past were convinced that reflex movements carried information about the psychophysiological parameters of a person. Charles Darwin 1 argued that facial expressions developed in the process of evolution and determined by the emotional state of a person. Ivan Sechenov 2 claimed that every thought has muscular manifestation. Sigmund Freud 3 wrote that a person does not have random movements, and every movement is informative. Ivan Pavlov 4 assumed that the balance of inhibition and excitation determines physiological processes. Nikolai Bernstein 5 discovered that human movements are discrete and determined by feedback. Norbert Wiener 6 suggested using feedback as the basic principle not only of human movements, but also of any physiological process. Mira y Lopez 7 proposed the method for assessing emotional state and pathological changes based on the registration of muscle movements. Konrad Lorenz 8 calculated the level of aggression using the frequency of reflex movements of animals and humans. To transfer the accumulated knowledge of reflex movement into practical fields of emotions recognition and medical diagnosis, only the technical capabilities of the 20th century were lacking.
The situation changed with vibraimage technology development 9,10 , which made possible to transform reflex micromovements of a human head into behavioral parameters 10, 11 . Human head movement is comprehensive process and information about head movements could be apply in different areas 12,13 . Biometric standard 14 divide all biometric characteristics into biological and behavioral, naturally, referring the characteristics of movement to behavioral parameters. The technical basis of vibraimage technology is accumulation of inter-frame difference 9 . The number of consecutive frames (N) and frequency (f), which are used to accumulate the inter-frame difference, determine the minimum period T=N/f for psychophysiological information integration about a person. For various psychophysiological parameters measuring, N and f settings are changed, correspondently, to detect fast responses, the minimum value of N and the maximum value of f are set. The most frequently used vibraimage settings for emotional state measurement are N=100; f=5Hz, which determines the period of information about the psychophysiological state integration T=20s 11 . These standard settings (N=100; f=5Hz) were used in the first versions of the COVID-19 diagnostic vibraimage programs 15,16 , since head micromovements capturing was done by existing programs 15 . The standard approach on psychophysiological parameters measurement using vibraimage technology focused on the averaged values of behavioral parameters, because this increased the accuracy of emotional states measurements 17 . Averaged values eliminated the influence of chronobiological processes 18,19,20 and increased the stability of behavioral parameters measured as physical quantities based on developed equations 11 . Using vibraimage technology and trained artificial neural networks (ANN) was possible to achieve almost 100% discrimination accuracy in patients and controls database 15,16,21 . However, COVID-19 diagnosis results calculated by averaged vibraimage parameters processing for a random sample showed the accuracy of COVID-19 diagnostics about 80%, which is comparable to the RT-PCR accuracy of COVID-19 diagnostics 22 , but not enough for the mass application of new COVID-19 non-contact diagnostics technology due to the mistrust of specialists and low accuracy.
The purpose of this study was to develop contactless COVID-19 diagnosis method with minimum testing time by video image processing of human head movements (facial video) with at least 90% accuracy. Total BPIV database of preliminary study with the division into groups are in supplementary data (file DB_100).

Materials and methods
The measurement results of controls and patients divided into two groups: learning and testing. The measurement results of the learning group were used to train ANN and the measurement results in the testing group used for testing of diagnosis accuracy. It is the standard approach for ANN leaning and testing, because testing must be dome on independent from learning group results. The size of groups for ANN learning was approximately the same and included 180,753 BPIV for controls and 181,392 BPIV for patients. The remaining measurement results placed, respectively, on testing groups of patients 29,672 BPIV and controls 155,222 BPIV. The structure of used database shown on Fig. 1. More data about groups shown in Table 1. AI learning was carried out by setting 0 to the diagnostic coefficient (DIC) value calculated from the BPIV of controls without COVID-19, and setting 1 to the DIC calculated from the BPIV of the patients with a confirmed diagnosis of COVID-19. AI learning used linear three-layer feedforward ANN structure, described in detail in our earlier article 16 , and used to diagnose COVID-19 based on averaged values of behavioral parameters. AI learning process in 40x80x60x1 ANN was carried out using the standard optimization algorithms ADAM and Nesterov 23,24 by authors code.
BPIV were captured using standard webcams (Microsoft LifeCam Cinema, LifeCam Studio, HD_WebCam from Acer Aspire E5-575G), Windows 10 OS and IntelCore i5; i7 processors. Head resolution in horizontal line was not less 200 pixels. Video Quality 17 captured by vibraimage programs was higher 50%. Camera placing was usually set in front of 0.5 m to sitting person 13 , however some measurements were done by investigation of standing person because behaviors parameters measured by vibraimage programs are not sensitive to body position 11 .
Basic study materials and methods. Basic study used data from recorded video files of a person's face, duration of 180-210 seconds. Then the video files were converted by VibraHT program into digital data of 155,933 BPIV for 40 behavioral parameters 11 with settings (N=25; f=10Hz) for: a) patients with a confirmed diagnosis of COVID-19. The total number of measurement results in the available COVID-19 patient database was 81,343 BPIV; b) controls including healthy and sick people with a confirmed absence of COVID-19 disease. The total number of measurement results in controls was 74,590 BPIV.
BPIV measurement results of controls and patients were divided into two groups: learning and testing analogical to preliminary study. The measurement results of learning group were used to ANN learning, and the measurement results in testing group were used to validate ANN accuracy during testing procedure. The group sizes for ANN learning was set approximately the same and amounted 58,435 for controls and 59,482 for the patients. The remaining measurement results placed for testing of patients and controls, respectively. The structure of the database shown in Fig. 1. Data about measurements in groups during the basic study shown in Table 1.
Total BPIV database of basic study with the division into groups are in supplementary data (file DB_025). The combined data BPIV of 40 behavioral during the preliminary and basic studies shown in Table 1.

Results
Preliminary results. Figure 2 shows probability density function of COVID-19 DIC, calculated by Excel means using DIC values from databases of patients and controls testing for the different periods of DIC averaging. Starting from zero (0) averaging time to 20 seconds DIC averaging. Figure 2. COVID-19 DIC probability density function calculated by trained AI for testing databases of patients and controls on preliminary study. Avg0_0controls DIC without averaging; avg0_1patients DIC without averaging; avg5_0controls DIC with 5 second averaging; avg5_1patient DIC with 5 second averaging; avg20_0controls DIC with 20 second averaging; avg20_1patients DIC with 20 second averaging.
From the graphs presented in Fig. 2, follows that the increase DIC averaging time to 20 second gives short range (0.03-0.19) without false negative errors on avg20_1 curve.
The results of the accuracy, sensitivity and specificity of diagnosing COVID-19 using a trained AI for verification databases of patients and controls for different time DIC averaging (0; 5; 10; 20; 30; 40; 60 seconds) shown in Table 2. ROC sensitivity-specificity dependence 25,26 from the preliminary study shown in Fig. 3. Sensitivity and specificity calculations performed using Excel tools, given at supplementary data. File Sensitivity_specificity_N100 Basic results. Figure 4 shows the probability density function of COVID-19 DIC received on basic study, calculated by Excel means using DIC values from databases of patients and controls testing groups for the different periods of DIC averaging. Starting from 0 averaging period to 20 seconds DIC averaging. The results shown in Fig. 4 differ from the results obtained in the preliminary study (Fig. 2). If the distribution densities of DIC for BPIV (0 averaging) are approximately similar for the preliminary and basic study, however 20-second averaging of DIC of the basic study zeroes out all errors for the ranges 0-0.5 and 0.7-1.0.
The results of the accuracy, sensitivity and specificity of diagnosing COVID-19 using trained AI for testing databases of patients and controls (basic study) for the different times of DIC averaging (0; 5; 10; 20; 30; 40; 60 seconds) shown in Table 3.  Sensitivity and specificity calculations performed using Excel tools, given at supplementary data. File Sensitivity_specificity_N25

Discussion
Given results indicate that COVID-19 diagnosis by the analysis of head micromovements is possible, and the accuracy of the diagnosis depends on the movement analysis settings. Comparative accuracy characteristics of two diagnostic options with different vibraimage settings for the movements analysis (N=100; f=5Hz; T=20s − preliminary study and N=25; f=10Hz; T=2.5s − basic study) shown in Fig. 6. The results obtained on the preliminary study, namely the weak dependence of the diagnostic accuracy on the averaging period, allowed to assume that the decrease in accumulation time of head movements analysis with the higher analysis frequency can increase diagnosis accuracy, that confirmed Fig.6.
It is necessary to clarify that the results shown in Fig. 6 were obtained by various sizes databases processing (Table 1), and this difference cannot be eliminated, since the BPIV measuring by vibraimage with one settings could not be transfer to BPIV with other vibraimage settings. So the results of behavioral parameters measurements obtained in a preliminary study with settings (N=100) cannot be transferred to other settings (N=25), only same video can be used for analysis with different vibraimage settings. But even the same video base of patients and controls, used for processing during the basic study, does not allow achieving identity of databases size for different vibraimage settings, since the size of the resulting databases depends on the integration time of behavioral parameters.
It is interesting to note that the diagnostic accuracy on the left side of the graph (Fig. 6) linelly increases with an increase in the discrimination accuracy of the trained database from 80 to 90%. At the same time, AI overtraining occurs earlier for the longer integration period (N=100, T=20s) at the level of 87% of the discrimination accuracy of the training base. For the shorter period (N=25, T=2,5s) − at the level of 90%. This paradoxical result becomes from the fact that chronobiological processes 20,27 have not changes during COVID-19, and their presence in BPIV only reduces the accuracy of the disease diagnosis. We came to this conclusion because were unable to use the temporal dependences of behavioral parameters for COVID-19 diagnosis, the diagnostic accuracy using temporal data did not exceed 60% with a larger volume of input data. Figure 7 shows COVID-19 diagnosis accuracy dependences for preliminary (N=100) and basic (N=25) studies from DIC averaging time. According to the basic testing, the diagnostic accuracy of COVID-19 is 94.69% with a fivesecond averaging of DIC and then slightly increases to 98.22% with a 40-second averaging of DIC. Averaging gives lower random errors however have not influence to systematic and methodological errors rates.
Comparative characteristics of diagnostic errors shown in Fig. 8. The false negative errors (FNR) of diagnostics in Fig. 9 do not exceed 2.5% after a five-second DIC averaging for said database. In our opinion, false negative errors are the most unpleasant for patients testing with an infectious disease, since it implies the recognition of a sick patient as healthy and the possibility of further infection distribution from a missed patient. The minimum falsenegative error is observed after 20-second DIC averaging is 0.3%, is significantly lower than false negative error for RT-PCR testing of with COVID-19 patients, which could be 20% according to available data 22,28,30 .

Discussion
We foresee the main arguments against the proposed diagnosis method from opponents in the form of the following main remarks, which we have repeatedly heard: 1. The results of the basic study were obtained on a small number of patients with the confirmed diagnosis of COVID-19.
2. The article does not provide a clear relationship between COVID-19 and human head micromovements parameters.
We reply our considerations to the obvious remarks. First, about the amount of analyzed data. Human behavioral parameters are somewhat different in nature from the biological parameters to which most medical and scientific specialists are accustomed. The biological and biochemical characteristics of a person are highly stable, of course, they also change over time, and each subsequent biochemical analysis taken from the same patient will only slightly differ from the previous one. The variability of human behavioral parameters is significantly higher than the variability of biological parameters. The presence of chronobiological processes has a noticeable effect on behavioral parameters, the mood and behavior of each person depends on many external and internal factors 18,20,27 . The behavioral parameters of a person change him every second, it is not for nothing that Heraclitus said that you cannot step into the same river twice 31 . Therefore, several measurements of BPIV for one person in terms of information content are identical to the same number of BPIV measurements of several people. Consequently, the informativeness of given behavioral parameters is determined not by the number of measured patients, but by the total number of BPIV measurements. BPIV data presented in this study exceeds the most part of statistical studies with a confirmed diagnosis of COVID-19 32,33 . Most of the medical studies on COVID-19 diagnosis based on several hundred analysis data 32,33,34 , while the preliminary study includes data from more than 500,000 measurements of 40 behavioral parameters, and the basic study more than 150,000 measurements of 40 behavioral parameters. Of course, an increase in the amount of data (relative to presented results) may lead to some change in the accuracy of diagnostics, but these changes are unlikely to be of a fundamental nature. Futhermore the accuracy for increased BPIV database would be higher than shown results because as we tested decrising database gives lower diagnosis accuracy. Most likely, the accuracy of diagnostics could be affected by the limited use of television cameras when video recording and measurements. To avoid the influence of overtraining and insignificant details, the discrimination accuracy of Learning daatabase was limited to 90% for the development of COVID-19 diagnosis program Covid5s. The graph Fig. 7 shows the correctness of this approach, since overtraining of ANN and tuning for minor details when discriminating against the trained database increases the discrimination accuracy, but decreases the diagnostic (testing) accuracy for an independent test database. Therefore, a remark on the small number of patients studied with a confirmed diagnosis of COVID-19 is not essential precisely for the method with the measurement of BPIV. Large number of measured independent behavioral parameters (40) and hundreds of thousands of BPIV measurement results allow AI to find stable differences in the relationships between the behavioral parameters of patients with COVID-19 and controls represented by healthy and sick people with other diseases (flu, heart-vascular, oncology). The total number of connection coefficients between BPIV after AI learning with the used ANN configuration is 8201. So based on having results we suppose that increased database will only improved diagnostic accuracy.
Next for the connection between the COVID-19 disease and the characteristics of human head micromovements. Since the development of vibraimage technology 9 , we have tried to understand the reason for the dependence of head micromovement parameters on the psychophysiological state of a person. Initially, we assumed thermodynamic equilibrium and human thermodynamics as the main mechanism for the relationship between these phenomena 35 . Then it was hypothesized about the vestibular-emotional reflex linking psychophysiological state and head micromovement 36 . Despite the fact that, in our opinion, it is not so important to find an unambiguous mechanism of this connection, it is more important to statistically reliably confirm its existence, we will make one more assumption substantiating the physical dependence between the described phenomena. One of physical laws, which is a consequence of general relativity theory (the principle of causality), claims that two physical phenomena can be interrelated if one of them happened after the other. Roger Penrose, in a number of publications with physisical approach to conciousnes, has proposed the theory using Orch-OR (orchestrated objective reduction) quantum gravity 37,38 . In the beginning, it seems that general relativity and general gravity are far enough away from the micromovements of a head. However, this is not quite true. The vestibular-emotional reflex consists in maintaining the vertical position of the head due to the contraction of the cervical muscles under the constant influence of the Earth's gravitational field. In space, in the absence of gravity, the vestibular-emotional reflex will not work, since in the absence of gravity, the meaning of muscular regulation of vertical head support is lost. Perhaps, as Penrose suggests, there are gravitational mechanisms of information exchange in the human body and, if this is true, then the vestibular system will be the first to react to gravitational signals. Note that one of the critics of Penrose's Orch-OR gravitational theory, artificial intelligence specialist Marvin Minsky 39 , would have been upset if this AI-assisted study was interpreted as a confirmation of Penrose's Orch-OR gravitational theory. However we do not see contradictions in the approach of Penrose and Minsky. Science is constantly evolving and it is quite possible that later new mechanisms of information exchange in the human body will be discovered. In any case, hardly everyone was wrong − Sechenov, Darwin, Pavlov, Bernstein, Wiener, Mira y Lopez and Lorenz, when they asserted the informativeness of reflex movements. Most likely, opponents of vibraimage technology are mistaken, who do not understand how the micromovements of human head and changes in the psychophysiological state can be connected. Not understanding of process could not delete it.
The human head micromovements are sensitive to any biophysical changes in the human body, they are practically inertia-free respond to changes in the emotional state when detecting a lie 40 or responding to stimuli 41 . The prototype vibraimage system for diagnosing COVID-19 was used at Elsys, St.Petersburg, Russia for pre-shift control of employees since July 2020. During this time, 2 employees with COVID-19 were identified, and COVID-19 detection occurred 5 and 7 days before the onset of symptoms of the disease and a positive RT-PCR test result. Moreover, in the identified cases, on the same day after receiving a positive result of the COVID-19 diagnosis using vibraimaging technology, the employees were sent for RT-PCR analysis, which gave a false negative result. The employees were sent to quarantine and only after 5, 7 days they received a positive RT-PCR test result. This means COVID-19 diagnosis using the analysis of the human head micromovements makes possible to detect the disease much earlier than traditional biochemical testing methods. From physical point of view to diagnosis process, this is quite understandable, since reflex movements are inertialless and, almost instantly, transfer a change in the psychophysiological state to changes in movement. While any biochemical process goes through several stages in its development 42,43 .

Conclusion
It seems to us that the described method of COVID-19 diagnosing, despite its seeming fantasy and simplicity, is based on a scientific approach to the analysis of reflex movements, statistically confirmed and has proven its practical feasibility. Moreover, we admit the possibility of improving the presented results of COVID-19 diagnostics accuracy by increasing the existing database and additional tuning of AI learning and diagnostics algorithms. Video processing of head movement is not limited by PC processing, the next obvious step is transferring this technology to mobile phone platforms.
The source databases used to process the diagnostic results and BPIV publicly available, allows independent developers to develop their algorithms for diagnosing COVID-19 based on different methods.
We also believe that the informativeness of human head reflex micromovements is not limited to the diagnosis of COVID-19. We will be glad to see published results of independent research in this direction. Of course, we are open to cooperation, since the simplicity and availability of video processing method opens broad prospects for its mass application and should prevent the spread of the COVID-19 pandemic. 5-second live video COVID-19 testing could replace covid and vaccine passports and other problems because real time COVID-19 diagnosis gives more health guarantee than formal documents.

Data and code availability
The summarized data generated during the current study are available at https://psymaker.com/downloads/NN2.zip The program Covid5s for COVID-19 diagnosis by behavioral parameters measurement is available at https://psymaker.com/downloads/setupCovid5s.exe The manual for Covid5s program is available at https://psymaker.com/downloads/COVID5S.pdf The custom codes used in this study for the behavioral parameters discrimination and/or activation code for Covid5s program are available from the corresponding author on reasonable request.