Artificial Vision System on Digital Devices for Real-Time Head Tilt Control

2.1. Design and Description of the Algorithm for Detection.

For the design and creation of the new method we need devices that use a CCD detector through which we obtain images in real time.

We developed an algorithm based on image processing techniques [16,17] that allows us to detect and track the subject's pupils in real time through the CCD detector.

The calibration will consist of relating the different interpupillary distances (IPD) of the subjects and with the different distances at which the task is performed. Once the method has been calibrated using the algorithm, we know the pupil coordinates and by means of a simple mathematical calculation we obtain the inclination of the subject's head in real time, showing the degrees of inclination on the screen of the device. Figure 1 shows the flowchart of the algorithm designed to measure head tilt.

2.1.1. Image Tracking and Pre-Processing Algorithm

We capture the first frame and proceed to detect the subject's eyes by selecting a cascade object detector, choosing in this case the Viola & Jones [18] algorithm (Figure 2).

When we have detected the eyes, we go ahead to their segmentation to detect the iris and pupils, we present the two eyes separately so that the algorithm we use will find it easier to detect them.

For the detection of the iris and pupils we choose the iris segmentation program using the Daugman integrodifferential operator [19], finding the iris and giving us the radius and coordinates of the center of the iris radii and coordinates of the iris and pupil center (Figure 3).

Once the iris is found, we go ahead to check and track it by point tracking using the Kanade-Lucas-Tomasi (KLT) feature tracking algorithm [20,21]. The points you must track are the centers of the pupils, searching for them and showing them to us in real time in frames. It is possible that as the video progresses some of the points are lost or not tracked, probably due to a variation of the illumination or loss of fixation of the subject on the text that is presented, in this case the application re-acquires the points again to be able to track them.

Once the centers of each pupil are found, we know the x and y pixel coordinates of the right and left pupil (Figure 4)

2.2. Calibration Process of the Algorithm for Head Tilt Measurement.

For tilt calibration we created a template with a computer design program with 9 interpupillary distances with which we represented almost all subjects [22] IPD (Figure 5) varying between 45 and 85 mm, checked with a graduated ruler with a margin of error of ±1mm.

The detection of the circles is done with the Hough transform algorithm [23], being a technique for shape detection by edge localization, it detects the circle and the coordinates of its center.

To perform the calibration and measurement tests we used a Huawei Matebook laptop computer model MRC-W60 (Figure 6a) which has an integrated camera with which we went ahead to make the video choosing a resolution of 1280 × 720 pixel, an image frequency of 30 frames per second and a refresh rate of 50 Hz.

The template is fixed to a goniometer with a margin of error of 0, 1º, which allows us to tilt and quantify the degrees of inclination of the points with respect to the computer camera.

The system will be leveled with a Fisco model EN 17 inclinometer with a measurement accuracy of 0˚/90˚ = +/-0.05˚ 1˚- 89˚ = +/-0.2˚, at 0º for both the laptop and the template.

The template will be placed at different distances which will be measured with a Bosch GLM40 laser rangefinder with a margin of error of ±1.5mm, and at each distance we will measure different inclinations, taking frames of all the measurements for processing by the mathematical program.

The goniometer on which the template is fixed (Figure 6b) gives us the degrees of rotation, right in positive value and left in negative value, which we will relate to the separation of the pupillary centers in the vertical axis.

All calibration measurements were performed in a light environment between 200 and 400 lux.

To measure the light level, we used an RS Pro ILM 1337 lux meter with a resolution of 0.01 lux and an accuracy of ± 3% + 5dgt.

2.2.1. Adjustment by Linear Regression for the Calibration of the Inclination

We know the coordinates on the vertical (y) axis (Figure 7), so we only need to relate the degrees of tilt of the template to the vertical pixel separation of the two images (y2-y1). Where the result can be negative or positive, depending on whether the subject tilts the head to the left or to the right.

$$\mathrm{d}\mathrm{y}=\mathrm{y}1-\mathrm{y}2$$

(1)

Figure 8a shows the degrees measured by a goniometer (Y axis) and the pixel difference between the centers of the two circles (X axis).

In this case the equation that best relates these variables is that of a straight line, so a linear regression [24] adjustment will be adopted which estimates the values of the X axis (dependent variable) from the values of the Y axis (independent variable) obtaining the following predictive equation:

$$\mathit{y}=\mathbf{53.86}\mathit{x}+(-\mathbf{0.0179})$$

The correlation coefficient R= 0.998 is close to 1, specifically 99.98% of the deviation of the Y variable by respect to its mean. It is explained by the adjusted linear regression model. Finding that this regression model is statistically significant with a p-value= 1.25 e-09, so we can ensure that the linear model is adequate to describe the relationship that exists between these variables and predict the data.

Figure 8. (a). Linear adjustment to relate the degrees and the pixel difference of the Y axis. (b) Error induced by the algorithm in the measurement of template inclination at different distances once calibrated for an IPD of 65 mm.

Figure 8. (a). Linear adjustment to relate the degrees and the pixel difference of the Y axis. (b) Error induced by the algorithm in the measurement of template inclination at different distances once calibrated for an IPD of 65 mm.

Once the system was calibrated, five measurements were made at different inclinations, increasing by 5º degrees at each measurement, until reaching 20º degrees. This process was performed at six different distances, starting from 23.5 cm, increasing the distance up to 70 cm. This process was repeated at different interpupillary distances, with the average measured error being less than 0.30º degrees of inclination (Figure 8b).

2.3. Measurement of Head Tilt in Real Time Evaluated on Subjects.

Once calibration of the system is complete, it is ready to evaluate its performance on subjects by tracking head tilt in real time.

The system was evaluated on seven subjects ranging in age from 12 to 55 years. Two of the subjects, one adult and one child were wearing their usual optical correction in their glasses at the time of testing. The test was performed on a test bench where the subject rested his head on a chin rest (Figure 7a), while being shown a text on the laptop computer. The ambient light environment in which the tests were performed is 300±50 lux.

The video was recorded with the computer camera with a resolution of 720 pixel, an aspect ratio of 16:9 and a speed of 30fps.

Figure 9. a). Volunteer with his head resting on a chin rest for the start of the measurement tests. (b) Laptop with inclination controlled by the inclinometer to conduct the tests.

Figure 9. a). Volunteer with his head resting on a chin rest for the start of the measurement tests. (b) Laptop with inclination controlled by the inclinometer to conduct the tests.

Each subject was videotaped at different distances and at each distance at three different inclinations of the laptop computer (Figure 7b). Between 40 and 50 images were obtained from each video, obtaining the mean head tilt at each distance.

The first measurement will be taken at 0º inclination of the laptop, this will be our starting point since the subjects do not have their eyes perfectly aligned in the horizontal plane. Later measurements will subtract the result obtained at the 0º position of the computer from the rest of the tilt measurements.

4.1. Device Calibration.

Calibration of the device was performed to make the system functional for all subjects with different interpupillary distances, ranging from 45 - 85 mm and for different reading distances ranging from 23.5 cm to 70 cm. For the detection of the circles of the template we chose a program based on the recognition of the edges of the figure, the algorithm used is the Hough transform and we could com-prove that it detected the circles in a quite wide range of light environments, choosing for the calibration a light environment between 200 and 400 lux, which is the light environment we found in the laboratory.

Its high correlation 99.98% to describe the relationship between these variables and the prediction of the data, ensure a good agreement between the measurement made by the system and the measurements made by the laser rangefinder.

The error produced in the calibration was not related to the different distances between the circles (DIP) nor to the distance from the CCD detector to the circle template, so it did not influence the measurement of the tilt angle, obtaining an error margin of 0.30º of tilt, this error is supposed to be produced by human errors when adjusting the system and instrumental errors.

4.2. Measurement in Subjects

The results showed that in most of the variables very good absolute and relative reliability values were found, measuring head inclination with an accuracy of less than 1° within the range of distances between 30 and 70 cm, this being the range of distances most used by users of both mobile phones [28] and computers [29].

The absolute errors achieved by our system in the validation study compared with other systems or published methods are very good. The average error measured in the subjects is 0.744°, with the error increasing as we increase the distance at which the text is presented (Figure 10), this is because the pupil centers detected by the algorithm are not as accurate as at shorter distances, due to the decrease in pixels in the iris and pupil outline.

Figure 12. Absolute uncertainty in the measurement of head tilt at different distances.

Figure 12. Absolute uncertainty in the measurement of head tilt at different distances.

No significant differences were found in the measurement of tilt between the different interpupillary distances of the subjects. The algorithm had no problems in detecting the eyes of the subjects in different light environments and between subjects who wore optical correction (glasses) and those who did not.

The light environment of the laboratory where the measurements were taken is 300±50 lux, and the system was adjusted to quantify in real time the position of the subject's head (degrees) with a high spatiotemporal resolution, detecting the position of the subject 10 times per second. It alerts us when the position of the head exceeds a previously defined inclination by means of a text message and an audio warning.

To our knowledge, we are not aware of any device or application that measures head tilt in real time by monitoring the eyes when the subject is doing an activity in front of a digital device.

Considering its high performance, ease of use and low cost, we believe that this system has great potential to control head tilt and thus be able to correct and avoid visual complications in the future, making it a prevention and treatment system to achieve optimal visual performance.