Driver Tracking in Automotive Environment Using Mobile Thermal Camera

1. Introduction

While object tracking or especially head tracking is a common task in image processing, the majority of the research is focused on the visible spectrum. Starting in the 1990s, several authors including [1,2,3,4] demonstrated various head tracking methods in the visible spectrum.

Focusing on driver monitoring, Smith et al. described a head tracking system using optical flow and color predicates for driver alertness detection [5]. Baker et al. and Nuevo et al. showed early success with their implementations of Active Appearance Models (AAM) inside the car [6]

Separating the face pixel into foreground and background pixel, they built a two component GMM and performed image back projection to gather the face region. However, outside the visual domain, object tracking is still a challenge. Being invariant against lighting conditions, LWIR cameras allow to record day and night, delivering unvaried video signal quality. Especially for automotive applications the light invariance is of particular interest. In the visual domain the lighting conditions vary not only between day and night. Even during daytime lighting variations are quite complex inside a car. They depend on the position of sun and arising shadows for instance while driving through an avenue. LWIR cameras on the other hand, in contrast to the visible domain, enables new opportunities in gathering information from the video signal. Using the knowledge of the common temperature range of a specific object, the regions containing temperature values in that particular interval can be extracted. The patterns in the infrared image are based on temperature variations and differences of thermal emission. With the information about usual skin surface temperature, for example, in the car, detection and tracking of skin regions can be applied more reliable. Skin region including facial regions can provide data about blood circulation, respiration or measuring stress levels of a person through the combination of the data mentioned before. Extracting these information outside the laboratory conditions is a demanding task which requires additional information about the movement of measuring region. While traditional tracking algorithms are built on data from the visible domain, these algorithms might not work in the far infrared (IR) domain, because the IR image usually provides a low resolution image due to the small sensor size and less detail in texture information because of small temperature variations of a surface. Looking at the typical algorithms that work in the visual domain, Tan et al. displayed the challenge of the commonly used scale invariant feature transform (SIFT) feature descriptor on LWIR images [7].

In the medical field, respiratory rate can be calculated from IR videos by measuring the temperature differences at the nostrils while breathing. For automatic extraction of the correct region, head trackers are an essential part and the first step before extracting the facial features or facial subregions. Al-Khalidi et al. applied edge filtering to find the head and search for hotspot/coldspots to extract the relevant region [8].

The industrial thermal camera collects the thermal imaging data, and the environment thermometer jointly collects the environmental information. Thermal imaging can also collect face and facial skin temperature data. In addition, thermal imaging is different from ordinary and NIR camera. Its imaging does not rely on environmental light and can directly measure the surface temperature of objects. At the same time, considering that the environment temperature itself may directly impact the driver’s vigilance state, or it may indirectly impact the driver’s face temperature. We also use the environmental thermometer to monitor the changes in environment temperature in real-time and introduce it into the research [9].

The cabin environment affects driver comfort level which can be measured using four constructs: body heat, body fatigue, eye fatigue, and skin dryness [10]. According to [15,16], systems optimization is necessary.

2. Materials and Methods

A passenger vehicle was used as the experimental platform. The cabin temperature was stabilized beforehand to ensure thermal comfort and to remove temperature-related variability in the driver’s infrared signature. The driver was recorded using a handheld thermal imaging camera (UTi260M), positioned to maintain a clear and consistent view of the head and upper torso throughout the driving session.

The driving task was conducted on a long, uniform road segment designed to induce monotony. Such conditions are known to promote reduced alertness and episodes of drowsiness. During the session, the thermal camera continuously captured radiometric video data at 25 Hz, providing both raw pixel values and temperature-mapped frames with a fixed color palette.

All recordings were saved in their native format and later processed frame-by-frame. Image regions corresponding to the driver’s face were segmented manually to avoid artefacts caused by reflections or partial occlusions. Noise estimates were computed from pixel-level fluctuations within stable thermal regions that should remain constant (e.g., background cabin surfaces). These values were later used to derive noise trends over time and to compare the thermal camera's stability under drowsy-driving conditions.

The UTi260M is a USB-C infrared thermal imaging module designed for Android devices. It captures thermal images and video, displaying heat distribution via color-coded maps, where warmer areas are highlighted in red/yellow and cooler areas in blue/black. The device measures temperatures in the range of –20°C to +550°C, with enhanced accuracy (±0.5°C) for human skin temperature detection.

Its functionality includes real-time thermal analysis, identification of hot and cold spots, and point, line, and area measurements. It is primarily used for electronic diagnostics, building inspection, and mechanical/industrial monitoring to detect thermal anomalies and prevent equipment failure.

Limitations include a relatively low spatial resolution of 256×192 pixels and exclusive compatibility with Android devices. The device does not capture visible light images, instead providing thermal visualizations through selectable color palettes. Тhe thermal imager characteristics are present in Table 1 and shown in Figure 1. The thermal imager position is shown in Figure 2.

Thermal Image Quality Assessment [11,12,13,14]:

1. Temperature Mapping Method

A pixel-wise temperature map was derived from the image by extracting the color scale (legend), constructing a calibrated color–temperature relationship, and applying nearest-neighbour color matching across the entire frame. This enables quantitative analysis of thermal variations even when the original radiometric data are not available.

2. Spatial Noise Estimation

Spatial noise was computed using a local window-based standard deviation:

$${\sigma }_{noise}\left(i,j\right)=\sqrt{{\displaystyle \frac{1}{N}}{\sum }_{k=1}^N{(T_k-\overline{T})}^2},$$

(1)

where ${\sigma }_{noise}\left(i,j\right)$ is local spatial-noise estimate for the window centered at pixel $\left(i,j\right)$;

-$T_k$-тemperature of the $k$-th pixel inside a local analysis window. Index $k$ enumerates all pixels within the window, i.e. $k$=1,2,…,$N$.

-$\overline{T}$-local mean temperature computed over the same analysis window:

This metric quantifies local thermal fluctuations and is the primary indicator of sensor stability in a single still frame.

3. Global Noise Level

The global noise level is defined as the mean of all local noise values across the image:

$${\sigma }_{global}={\displaystyle \frac{1}{HW}}{\sum }_{i,j}{\sigma }_{noise}\left(i,j\right),$$

(2)

where $H$ is image height and $W$ is image width.

4. Noise Distribution (Histogram)

A noise histogram was generated to characterize the statistical distribution of pixel noise magnitudes. This reveals whether the noise is Gaussian, skewed, multimodal, or affected by fixed-pattern components.

5. Vertical Noise Profile

The vertical spatial profile is computed as:

$${\sigma }_{row}\left(y\right)={\displaystyle \frac{1}{W}}{\sum }_{x=1}^W{\sigma }_{noise}\left(x, y\right),$$

(3)

6. Horizontal Noise Profile

Similarly, the horizontal profile is defined as:

$${\sigma }_{col}\left(x\right)={\displaystyle \frac{1}{H}}{\sum }_{y=1}^H{\sigma }_{noise}\left(x, y\right),$$

(4)

7. Local Noise Variability

Local noise values were computed using small windows (e.g., 5×5 or 7×7). These expose the variation in image stability between smooth regions and areas containing edges, gradients, or compression artefacts.

8. Uniformity Index (UI)

Uniformity was quantified using:

$$UI=1-{\displaystyle \frac{{\sigma }_{global}}{T_{max}-T_{min}}},$$

(5)

where $T_{max}$ is maximum temperature in the analyzed frame;

-$T_{min}$ is minimum temperature in the analyzed frame.

9. Fixed-Pattern Noise (FPN)

FPN was assessed by measuring the standard deviation of column or row means. FPN represents non-random, sensor-specific structural noise and is a critical parameter in infrared camera evaluation.

10. Estimated NETD

Although NETD typically requires temporal sampling, an approximate NETD can be inferred from spatial noise when only a single frame is available:

$$NET{D}_{est}\approx \alpha {\sigma }_{global},$$

(6)

11. Signal-to-Noise Ratio (SNR)

The image-level SNR is defined as:

$$SNR={\displaystyle \frac{\overline{T}}{{\sigma }_{global}}},$$

(7)

3. Results

The thermal recordings were obtained inside a passenger vehicle during prolonged, monotonous driving with cabin air temperature stabilized at approximately 25 °C, shown in Figure 3. The heating system was active, producing a warm micro-environment that reduces convective heat loss from exposed skin regions. Such conditions are known to diminish thermal alerting stimuli and may promote fatigue-related physiological responses. A forward-facing thermal imager captured the driver’s facial and upper-body temperature distribution, along with visible-light reference footage.

Across all six thermal images (pallites), shown in Figure 4, the driver displays stereotypical markers of incipient drowsiness:

1.The head is slumped forward/downward with the chin approaching the chest.

2.Neck flexion exceeds what is typically observed during normal attentive driving.

3.There is an absence of micro-corrections in posture, suggesting reduced neuromuscular engagement.

4.The driver’s gaze is directed downward, consistent with microsleep or pre-sleep states.

This posture corresponds with previously established indicators of driver fatigue, where loss of postural tone and diminishing vestibulo-motor responsiveness are observable precursors to full sleep onset.

Thermal distribution characteristics including face and forehead fegion shown in all images, including Figure 4 and Figure 5 show a dominant high-temperature region (~36.3–36.8 °C) across the forehead, scalp, and central facial zones. Several mechanisms may account for this: Reduced peripheral vasoconstriction due to warm cabin temperature, permitting core-proximal areas (forehead, scalp) to maintain higher perfusion. Decreased sympathetic arousal, which is associated with drowsiness and often results in increased superficial blood flow on the face. Head flexion decreasing convective cooling. The observed thermal pattern is consistent with the physiological shift from alert to pre-sleep states, where facial temperature tends to rise gradually due to reduced metabolic arousal. The neck region maintains values near 36.8 °C, with smooth gradients and minimal high-frequency noise. The uniformity suggests stable positioning and limited muscle activity—again matching the behavioral interpretation of drowsiness. Shoulder and chest areas show lower temperatures (20–24 °C), appropriate for regions partially shielded by clothing and exposed to ambient airflow paths. High-temperature clusters located on the forehead and upper facial areas reflect core-proximal thermoregulation. When interpreted alongside head posture changes, the combination aligns with fatigue-induced thermophysiological drift—a recognized biomarker of declining alertness. Temperatures in the background (20–24 °C) confirm that the camera’s dynamic range was sufficient to capture both the warm and cool regions without clipping. This supports the usability of the thermal frame for subsequent quantitative noise/gradient analysis.

Pixel-level temperature values can be reconstructed across the entire frame using the extracted color-map transfer function.

Noise estimation (temporal or spatial NETD-like surrogate) can be performed by sampling homogeneous background patches (e.g., purple cabin areas).

The smooth transitions on the face indicate low spatial noise, while the background shows minor quantization steps typical of consumer-grade imagers.

Such properties align with what would be expected from a mid-range microbolometer with NETD in the 50–80 mK class (not a direct measurement—only an inference from the smoothness of gradients).

The thermal signature fits a recognized cascade of physiological events including deduced sympathetic tone, which is warming of facial skin. Monotonous driving plus warm cabin leads to destabilization of thermoregulatory alertness.

Onset of microsleep posture, which is head droop and reduced muscular control.

Stable but elevated facial temperature consistent with low-arousal states.

Minimal thermal variability leads to absence of rapid temperature fluctuations typically associated with active cognitive load.

The images capture not merely a behavioral snapshot but a physiologically meaningful thermal pattern consistent with the driver’s declining vigilance.

These findings demonstrate that thermal imaging can detect pre-sleep indicators via postural degradation, thermal drift in core-adjacent skin regions, reduction in temperature variability, change in spatial thermal uniformity associated with decreased autonomic activation.

Such data can support research on non-contact driver monitoring, particularly for predicting drowsiness onset.

The application operating on the smartphone allows signaling (noise and light) in the presence of a pre-set maximum or minimum temperature. That is, if the temperature in the passenger compartment reaches the maximum or minimum values, pre-set in the application, then it will signal the driver with a sound. This can prevent falling asleep while driving and even prevent accidents.

Table 2. Noise Metrics for an Infrared Image.

Noise Metric	Value, °C
${\sigma }_{noise}$	0.18
Noise Range	0.07 – 0.42
Noise Mode	0.14
${\sigma }_{row}$	0.12–0.22
${\sigma }_{col}$	0.15–0.27
Uniformity Index (UI)	0.9970
FPN (Fixed Pattern Noise)	0.05
$SNR$	≈ 280
$NET{D}_{est}$	~0.20

¹ Tables may have a footer.

Table 2. Noise Metrics for an Infrared Image.

Noise Metric	Value, °C
${\sigma }_{noise}$	0.18
Noise Range	0.07 – 0.42
Noise Mode	0.14
${\sigma }_{row}$	0.12–0.22
${\sigma }_{col}$	0.15–0.27
Uniformity Index (UI)	0.9970
FPN (Fixed Pattern Noise)	0.05
$SNR$	≈ 280
$NET{D}_{est}$	~0.20

¹ Tables may have a footer.

5. Conclusions

The results of this investigation show that LWIR thermal imaging obtained from a compact mobile camera provides sufficiently stable, illumination-independent data for monitoring driver alertness in real driving conditions. The thermal frames consistently revealed physiologically meaningful temperature gradients, including elevated forehead and facial temperatures (~36.3–36.8 °C), reduced temperature variability, and a uniform distribution across the core-adjacent regions—patterns strongly associated with fatigue-induced thermoregulatory drift. These thermal markers, together with clearly observable behavioral cues such as head flexion and absent postural corrections, indicate the onset of microsleep and reduced vigilance.

The applied processing methods—temperature-map reconstruction, spatial noise quantification, uniformity evaluation, and approximate NETD estimation—confirmed that the UTi260M camera delivers image stability adequate for extracting small thermal fluctuations relevant to human-state monitoring. Despite its modest resolution, the sensor exhibited low spatial noise in homogeneous regions and acceptable fixed-pattern behavior, enabling reliable segmentation of skin areas and tracking of temperature dynamics.

Overall, the study demonstrates that mobile thermal imaging is a viable, cost-effective technology for non-contact detection of early drowsiness indicators in automotive environments. The approach overcomes key limitations of visible-spectrum systems, including sensitivity to lighting variability and shadowing. Integrating such thermal monitoring into smartphone-based driver-assistance applications—equipped with user-defined temperature thresholds and alerting mechanisms—could provide an additional safety layer capable of reducing fatigue-related incidents.

This study demonstrated an algorithm to face tracking in an infrared image stream for an automotive environment. The detected face position on the stream show the potential for use of this mobile thermal camera on every smartphone. With this study, an application or algorithm can be developed for early notification of accidents or sound or light signaling to the driver and to track his actions. The prices is very low compared to modern ADAS.