Road Surface Identification Using Microwave Sensors

1. Introduction

Enhancing road safety is of crucial importance. Human errors have been identified as the leading cause of traffic incidents and advanced automotive sensing systems (ADAS) are to reduce these errors. Furthermore, accurate detection and classification of road surfaces will contribute to improving vehicle safety by enabling automatic speed adjustments based on surface conditions. Moreover, in recent years, substantial research efforts have been directed toward autonomous driving [1, 2], where robust road surface identification is essential for intelligent decision-making and overall system reliability. This review paper aims to present the evolution and current status of road surface classification technology, focusing on microwave sensors mainly, covering both active radar and passive radiometer, along with data analysis techniques. This field falls under the broader category of remote sensing, which involves detecting and monitoring the physical attributes of an area by measuring its reflected and emitted radiation from a distance [3].

In this review, we will use the term "surface identification" to refer to the detailed description of a surface to enable its recognition. On the other hand, "surface classification" will denote the process of sorting surfaces into distinct categories.

Several comprehensive overview papers delve into surface identification, particularly in modeling of backscattering signals. Notably, [4, 5] provide an in-depth and detailed exploration of microwave scattering theory. However, the topic of road surface classification, an important aspect for the successful implementation of autonomous vehicles, especially for those capable of traversing off-road terrain, remains inadequately addressed. The objective of this paper is to provide a comprehensive survey of radar-based road surface classification systems, offering well-balanced coverage including both the technical design of sensors and systems, as well as the algorithms employed for road surface classification. Where possible, we present the development of methods and technologies from a historical perspective.

Based on the design priorities, and underlying signal and systems considerations, surface classification systems can be categorized into two classes:

• Signal Analysis: analysis of specific parameters within the temporal profile of the reflected or emitted signal, followed by classification based on predefined analytical criteria or by subsequent application of statistical classification methods, including machine learning.

• Radar Imaging: the acquisition of radar images followed by the application of image classification techniques using machine learning, including deep learning, to analyze the visual data and identify the road surface.

Due to the substantial differences among these two approaches, the design challenges within these categories are different. In the Signal Analysis class, the primary design and research emphasis typically focuses on acquiring the temporal profile of the reflected or emitted signal and determining the parameters that best represent characteristics of different surfaces. This issue involves two dimensions: time (distance) and signal amplitude. The Radar Imaging class involves three parameters: time (distance), azimuth, and signal amplitude. In addition to the mentioned parameters, polarization (using polarimetric radar or radiometer) as well as velocity (using Doppler radar) can be used, offering the possibility to extract additional signal features. These and other options will be considered in the review.

Although, a direct comparison of the accuracy of the reported surface classification methods is not meaningful, as they were received under different conditions, e.g. varying distances, different sets of surfaces, etc. Nevertheless, we will provide information about the accuracy where it is available to assess the applicability of these methods.

The subsequent sections of this paper are organized as follows. In Section 2, we describe the placement of microwave sensors within the broader landscape of remote sensing methodologies and techniques, emphasizing their importance in surface classification. This section helps define the scope of our review. Moving forward, Section 3 provides a review of the fundamental principles underpinning surface classification. This includes explanation of electromagnetic (EM) signal reflection from surfaces, as well as requirements for microwave sensors and their parameters for the effectiveness of solving the problem under consideration. Section 4, Section 5 and Section 6 provide a comprehensive overview of the literature concerning road surface classification. We start with stationary systems, trace the evolution of radar and radiometer technologies, and conclude with a discussion on radar imaging and sensor fusion techniques. The synthesis of the paper's insights and its concluding remarks are presented in Section 7.

2. Surface Identification Technologies

There is a significant body of research and development focused on remote sensing and intelligent vehicle technologies [6]. Within these technologies, the role of surface identification using microwave sensors holds a prominent position. When discussing surface classification, we can typically divide it into three main categories:

• The first scenario involves detecting and classifying objects on the Earth's surface by measuring the characteristics of reflected or emitted radiation from regions using satellites, aircraft or drones. Such endeavors aim to achieve various objectives, such as detecting forest wildfires, identifying different types of vegetation and crops, determining soil types and moisture content, and monitoring processes like deforestation and urban growth [7]. The methods and algorithms developed for classifying surfaces in the context of remote sensing are often applicable to ground-based sensing as well, which will be the primary focus of this review.
• The second scenario involves the identification and differentiation of the road area and various road objects, such as cars, curbs, potholes, etc., for path planning in tasks related to autonomous driving. Numerous publications and reviews are dedicated to this specific research area [8-10]. Detecting and recognizing various classes of objects presents significant differences. Cars, being dynamic, are typically detected using Doppler methods with clutter suppression. On the other hand, all other objects, including parked cars and stationary road infrastructure, demand a distinct approach, often relying on imagery methods with region-based classification.
• The third scenario considered in this review relates to the classification of road surface types, such as asphalt, gravel, sand, etc., with the primary goal of enhancing driving safety. In this review, we will focus on remote sensing technologies. Therefore, close proximity methodologies involving the analysis of tire noise, heating due to friction, or various parameters of the vehicle's running system [11-16] will be excluded from the scope of this paper.

The position of microwave sensors for classifying surfaces among all road surface classification technologies is shown in Fig. 1.

Figure 1. Road surface classification application domains and technologies.

Figure 1. Road surface classification application domains and technologies.

This review focuses exclusively on microwave sensors, delving into the topic in greater depth. In the area of surface identification, various technologies are employed, such as optical sensors [17-21], including infrared [22], light detection and ranging (LIDAR) [23-25], and acoustic sensors (sonars) [26-28]. LIDAR's use of lasers enables high-resolution detection of small objects in both range and angle. Optical cameras offer cost-effective, lightweight solutions, providing high-resolution images and support for ranges up to hundreds of meters. However, their performance degrades under adverse weather and lighting conditions. Sonar's advantages include compact transceivers, simplicity, and affordability, along with excellent range resolution. However, its range is limited (typically up to 6-8 meters), making it more suitable for parking sensors. In contrast, radar sensors stand out due to their robust performance in harsh weather and lighting conditions [29-32].

Table 1 [33] presents the suitability of competing technological solutions for various aspects of the surface classification task. The table clearly indicates that radar technology adequately addresses most of the essential factors for surface classification. Additionally, combining radar with optical techniques (like video cameras), LIDAR, or sonar, holds the potential to create a highly effective overall solution. It is worth noting that Table 1 refers to sonar and radar without beamforming. However, utilizing widespread multiple-input multiple-output (MIMO) radars or acoustic arrays can significantly enhance angular resolution capabilities.

3. Basic Principles Underlying Surface Classification

3.1. EM Signal Scattering

In this section, we describe the fundamentals of surface classification based on the theory of EM signal reflection from surfaces. We will analyze how EM signal reflection properties depend on the radar or radiometer system’s parameters as well as the characteristics of road surfaces.

The phenomenon of signal scattering from rough surfaces has been a subject of study over an extensive period [35]. The scattering of signals from surfaces is influenced by factors including surface roughness, correlation length, radar frequency, grazing angle, and the effective permittivity of the surface material [4, 5, 34]. These parameters are explained in this section.

Two established solutions exist based on the relationship between wavelength and surface heights: (i) the Kirchhoff approximation, applicable when surface roughness is larger or comparable to the wavelength, and (ii) the small perturbation model, which performs more effectively when the variation of surface heights is small relative to the wavelength [5]. Numerous models have been developed that integrate these two approaches, yet they are not universal and have limitations regarding signal frequency, surface dielectric constant, and roughness characteristics [4]. Additionally, various empirical approaches are extensively employed in backscattering analysis, wherein theoretical models are expanded or modified based on empirical observations [36, 37].

The scattering mechanism from a surface is defined by its degree of roughness, which is defined by the Fraunhofer criterion —the rms of the surface height (h) distribution relative to the wavelength. The surface will be considered perfectly smooth when:

$$h\ll {\displaystyle \frac{\lambda }{32\mathit{cos}\Theta }},$$

(1)

where Θ is the angle of incidence relative to the surface normal, which can be determined by the grazing angle γ as Θ=90°-γ, and λ is the signal wavelength [4]. In this case, only specular reflection is observed. As the surface roughness increases, the proportion of the specular reflection component diminishes, while diffuse reflections, scattering signals in multiple directions, become more pronounced. In the scenario of an extremely rough surface:

$$h\gg {\displaystyle \frac{\lambda }{32\mathit{cos}\Theta }},$$

(2)

diffuse reflection becomes predominant over specular reflection.

The second characteristic of surface roughness is the correlation length, which is linked to the correlation function and the power spectrum of surface noise. In most cases, a short correlation length indicates strong fluctuations and a rough surface, while a long correlation length suggests weaker fluctuations and a smoother surface [5].

The power of backscattered signal from the surface is also used for surface identification. The reflective properties of a surface are characterized by its normalized radar cross-section (RCS) which is dependent on surface roughness, rms height (1), correlation length, as well as surface dielectric properties and grazing angle. The normalized RCS of a distributed target is an ensemble average of the RCS σ per unit area:

$${\sigma }^0=〈\sigma 〉/A,$$

(3)

where A is the illuminated area. The return power when the target is illuminated by a monostatic radar can be calculated using the radar equation [38]:

$$P_R={\displaystyle \frac{P_T{G}^2{\lambda }^2\sigma }{{\left(4\pi \right)}^3{R}_0^4}},$$

(4)

where P_T are transmitted power, G is antenna gain, R₀ is the distance to the target, and σ is RCS. The RCS of a rough surface varies with different signal polarizations. Therefore, the dependence of the backscattered power on signal polarization and grazing angle (distance), surface roughness and dielectric permittivity can be extracted which can serve as a basis for surface identification. Due to the complex patterns that describe signal reflection [36, 37], this problem is not easy to solve [39].

Despite the importance of research in the field of signal scattering, in this review we will consider only those papers that are directly related to the problem of surface classification.

3.2. Considerations for Surface Identification in Automotive Radar Applications

In this section we will focus on the consideration that should take place for surface identification for automotive application. In the case of automotive radars, low grazing angles are expected since it is necessary to classify surfaces tens of meters ahead of the vehicle, and the height of the radar installation is limited by the height of the car. In [40], the case of surface identification at 10 m in front of the car was considered (Fig. 1). At a speed of 40 km/h, the car covers such a distance in about a second so the automatic system should have enough time to determine the optimum terrain response settings for the surface ahead to maintain momentum and vehicle control as one surface transitions to another. If the radar is installed in the mirror on the windshield (height of 1.5 m), the grazing angle will be approximately 8.5° at a distance of 10 m.

As shown in [37], the normalized RCS σ⁰ decreases rapidly with decreasing grazing angle (increasing distance). Therefore, the power of the signal reflected from the road surface, which is proportional to $\sigma /R_0^4$ (4), decreases rapidly with decreasing γ, which leads to a decrease in the signal-to-noise ratio (SNR) and, consequently, difficulty in identifying surfaces.

As can be seen from Fig. 1, the illuminated surface footprint will extend over a wide ground range, sometimes tens of meters. Therefore, when analyzing the backscattered signal, the backscattering at different grazing angles is often considered [41]. Analysis of the dependence of the backscattered signal power on the grazing angle provides additional information about the surface's properties.

Figure 1. [40]. Automotive sensing scenario: H—radar height over ground, α—elevation beam width, R₀—ground range, θ—incident angle, γ—grazing angle.

Figure 1. [40]. Automotive sensing scenario: H—radar height over ground, α—elevation beam width, R₀—ground range, θ—incident angle, γ—grazing angle.

When analyzing methods for classifying road surfaces, we will consider both the technical approach to solving problems (signal parameters, microwave sensor type, number and location of antennas, signal polarization, etc.) and the algorithms used for classification, namely, analysis of the reflected signal parameters according to some criteria or different methods of statistical classification. e.g., k-nearest neighbors, support vector machines, neural networks, etc.

Although our primary focus is on automotive radars and radiometers, we will also consider studies on stationary road condition monitoring systems. It is because at the initial stage of research (1970-1990), the development of automotive microwave sensors for surface classification was a technically challenging task. In addition, the technical approaches and algorithms used in the stationary systems can be successfully extended to the field of automotive radars and radiometers.

3.3. Importance of High Signal Frequencies

The requirements and limitations of automotive radars, as discussed in Section 3.2, emphasize the challenges they face. In this section, we will focus on the benefits that can be achieved by increasing their frequency. Advances in modern electronics have led to the emergence of commercially available radar components with frequencies above 100 GHz [42] and this frequency range is currently of particular interest to automotive radar developers [43, 44]. The potential advantages of radars with higher frequencies include better angular resolution with the same antenna dimensions as lower frequency radars and better range resolution d due to the increased available bandwidth BW:

$$d={\displaystyle \frac{c}{2\bullet BW}},$$

(5)

where c is the speed of light. For example, mid-range radar operating at 24 GHz has a maximum allowable bandwidth of 200 MHz for automotive radar use and range resolution of 75 cm; however, 79 GHz radar has a wider operational bandwidth of 4 GHz [45, 46] which results in much higher range resolution of 3.7 cm. Moreover, the use of smaller antenna sizes at sub-THz frequencies makes it possible to reduce mass and packaging issues which is of high importance for automotive applications.

Another important advantage of higher frequency is higher sensitivity to surface roughness. As can be seen from expression (1), a decrease in the wavelength (an increase in the frequency of the radar) causes surfaces that were smooth at lower frequencies to become relatively rough. The shorter signal wavelength provides increased interaction with the physical surface texture, giving additional information on roughness due to more diffuse scattering [40].

In Section 1.2 all surface classification systems have been categorized into two-dimensional Signal Analysis and three-dimensional Radar Imaging systems. Due to the wide operating bandwidth at sub-THz range, higher range resolution is achievable which, in combination with narrow azimuthal resolution, allows for obtaining a road surface radar image composing of many pixels [47-51]. Surface identification based on radar images is mostly similar to classification based on optical image analysis [21] and potentially can provide higher accuracy than classification based on two-dimensional signal profile.

Although, atmospheric attenuation, primarily due to water vapor absorption, is significant in many spectral regions in the sub-THz band, as shown in [52], especially in adverse weather conditions [53]. However, several transmission windows between water vapor peaks can be exploited: 70–110, 120–170, 200–310, 300–370, 390–440, 625–725, and 780–910 GHz [54, 55]. The results obtained in [30] for frequencies of 77 GHz and 300 GHz show that even in heavy rain the attenuation of the radar signal did not exceed 20 dB/km, and in [31] measured attenuation during snowfall was below 15 dB/km. Therefore, for the automotive radar operational range of up to several hundred meters, atmospheric loss will contribute only a small amount to the power budget required to ensure system efficiency [56].

4. Feature-based Solutions for Road Surface Classification

4.1. Research on Road Surface Classification Using Stationary Systems

In this section, we provide an overview of publications focused on road surface classification using stationary systems, often in laboratory conditions. Most of these publications, though not all, were presented before the 2010s. They generally did not include statistical analysis of the results or numerical estimates of classification accuracy, but they laid the theoretical and practical foundation for the development of automotive surface classification systems. The study of surface classification began in the 1970s, when radars with frequencies above 10 GHz became widely available.

One of the first studies on the dependence of the reflection coefficient of a microwave signal on the grazing angle for different natural and manufactured surfaces was conducted in [34]. Asphalt, concrete, brick versus gravel road and grassy field were explored with 10.5 GHz radar up to 100 meters. It was shown that for small angles of inclination, the exact composition of the road surface has a significantly smaller effect on the power of the reflected signal than surface roughness. The goal of this work [34] as well as [36] were to test the feasibility of the surface identification by analyzing differences in the extracted parameters (features) of the backscattered signal. At that time, practical implementation of such methods and assessment of surface classification accuracy was not posed.

Radar signal depolarization, referring to the change in the orientation of the electric field upon reflection, has been an important parameter for surface identification since early studies in this field. The depolarization of the reflected EM wave is determined largely by the dielectric constant of the surface material [36]. In [36], a fully polarimetric 94 GHz radar system was deployed to characterize the polarimetric backscatter responses of asphalt surfaces under various physical conditions, including ice- and water-covered asphalt surfaces. This radar transmitted a linear chirped signal with a bandwidth of 2 GHz. The network analyzer served as a transceiver of the system with two independent receive channels, enabling the simultaneous reception of vertically and horizontally polarized backscattered signals. This study was continued in [37], where co-polarized and cross-polarized backscattering coefficients of various road surfaces (smooth concrete and asphalt, rough concrete and asphalt, ice, and water over asphalt) were investigated both theoretically and experimentally using the same radar system as in [36]. It has been found that the overall scattering response of road surfaces is composed of volume and surface scattering components. The significance of [36, 37] lies in the development of a complex scattering model based on the vector radiative transfer theory, showing a good agreement between the measured and predicted backscatter response of various road surfaces at incidence angles from 70° to 88°.

In the 1980s and 1990s, radiometers were explored alongside radars for surface identification. In [41] the reflection behavior of ice and water layers on asphalt was examined theoretically. This was followed by measurements using an active bistatic radar and a passive radiometer operating at 35 GHz and 90 GHz. It was shown that these road conditions can be identified by the reflected signals, and it was found that the radiometer was more suitable for this task because it provided better differentiation between surfaces. The brightness temperature and the polarization index of the radiation emitted from the target with dry and wet surfaces were measured as a function of surface temperature in [57] using a dual-polarized radiometer at 10 GHz and 37 GHz. The measurements indicate that a single-polarization microwave radiometer at 37 GHz can identify surface wetness on asphalt and detect a crust of ice of 3 mm.

At the same time, most of the works focused on the use of radars for surface identification rather than radiometers. A fixed 2.45 GHz bistatic road surveillance radar capable of detecting weather-induced road conditions was reported in [58]. The transmitter and receiver consisted of two microstrip antenna arrays for v and h-polarized signal processing. They were mounted at a height of about 3 m above ground on opposite sides of the road to be monitored. The road conditions were evaluated from the ratio of the amplitudes and the difference of the phases of vertically and horizontally polarized components. The system could measure water-layer thickness and water salinity and detect the snow layer on the road.

In [59] a bistatic polarimetric radar operating at 76 GHz was used to detect ice on a road. Measurements of backscattering coefficients of dry asphalt and ice-covered asphalt made in the laboratory, as well as under real traffic conditions, agreed with theoretical predictions.

The developed monostatic continuous-wave 95 GHz radar system was described in [60]. The radar can measure horizontal-horizontal, vertical-vertical and cross-polarized combinations at grazing angles of 5.0° and 7.5°. The narrow beam (beamwidth of 1.7°) antennas were mounted on a turntable to scan the surface. The computed backscatter cumulative distribution function for different signal polarizations was used as a criterion for differentiation between dry asphalt and snow-covered asphalt, achieving an accuracy of over 99%.

Several important works in the 1990s continued to develop the theoretical foundations for the reflection of radar signals from road surfaces. The theory of reflection of microwave signals was verified in numerous experiments with a truck-based polarimetric radar, operating at frequencies of 1.5 GHz, 4.75 GHz, and 9.5 GHz, at multiple incidence angles from 20° to 70° for four bare-soil fields with different surface roughness [61]. The Mueller matrix of the medium, specified by three magnitudes in co-polarizations (vertical and horizontal) and cross-polarizations and the probability density function of the co-polarized phase difference, was computed and compared with the experimental results. It was shown that Mueller matrix parameters provide useful information about the surface, which extends the information given by backscattering coefficients alone and may lead to the development of novel tools for surface sensing.

Understanding the patterns of signal reflection from surfaces led several researchers to the need to apply statistical methods for their classification. In [62], a fully polarimetric, incoherent microwave radar system for Mueller-matrix measurements was designed to work at 76 GHz, consisting of 5 transmitters and 7 receivers with different angles of polarizations. Using this system, 35 different combinations of transmit and receive polarizations can be collected for every illuminated area to calculate the Mueller matrix. The measurement distance was 3 m, and the maximum speed of the system mounted on a moving platform was 2 m/s. According to the work, the system can distinguish between different degrees of roughness and detect surface moisture and ice layers. The implemented classification included three supervised and three unsupervised learning algorithms; however, the authors did not provide data about the used feature extraction and classification algorithms and their accuracy.

The use of multiple frequencies allows for an increased amount of information to be processed, leading to more accurate classification results. In [63], road surface samples were sensed with different frequencies from 26.5 GHz to 40 GHz in laboratory conditions using a radar configuration with transmitting and receiving antennas mounted on both sides of the sample at an incidence angle of 60° and a height of 40 cm. A simple maximum likelihood algorithm was applied to automatically differentiate between dry asphalt, asphalt covered with water, ice, wet and dry snow. The results indicate that the ability to distinguish various surface covers based on reflectivity data is dependent on the frequency used. The paper does not provide quantitative estimates of the accuracy achieved of classification. Despite the potential promise of this method, it has not received significant application. This is due, on the one hand, to the requirements of hardware complication of the system when using several radar frequencies, and, on the other hand, to the manifestation and development of effective methods of statistical classification.

The research of stationary radar system for road conditions monitoring continues in the 2000s. The combining of data from 94 GHz (3 GHz bandwidth) dual channel polarimetric radiometer (vertically and horizontally polarized signals) and automatic weather station measurements were proposed in [64, 65]. The radiometer was mounted at a height of 6 m over the road surface, with a scan of +-30° in both directions. Using 11 variables for classification, the system achieved an accuracy of approximately 94-95% in differentiating between dry, wet, snowy, and icy asphalt with a Bayesian classifier.

A signal model describing the reflectivity of road surfaces was presented in [66]. The experimental system designed to verify this model consisted of a radar sensor and reflector mounted above the ground at a height of 55-75 cm, with 6m between them. Direct and multipath signals arising from reflections off the ground were analyzed separately. The results of measuring the scattering characteristics of asphalt pavements at a frequency of 77 GHz demonstrated the feasibility of determining the effective permittivity of road surfaces. Understanding the scattering and reflection behavior of surfaces is a crucial step in ensuring the safety of autonomous driving.

4.2. Automotive-Based Microwave Sensors for Surface Identification

As hardware evolved and radar sizes decreased, research shifted from stationary systems to automotive-based surface identification technologies. At the same time, the move toward higher-frequency radars introduced advanced capabilities compared to lower-frequency systems, enabling more sophisticated surface classification algorithms, particularly when combined with machine learning. Therefore, in this section, we will separately present radars and radiometers with frequencies up to 100 GHz and sub-THz radars, reflecting both the historical and technological advancements in surface classification methods.

4.2.1. Automotive Microwave Sensors Below 100 GHz

Automotive-based radars for surface identification began to appear in the 1990s. A prototype system capable of detecting road conditions was reported in [67]. A coherent polarization-sensitive millimeter-wave sensor, measuring the complete backscattering matrix (amplitude and phase) at 61 GHz, was constructed, and tested in a standard car under real road conditions. The transmitting and receiving antennas were positioned beneath the vehicle at 0.1 m above the ground in a bistatic configuration, identifying the surface under the car. The system's maximum velocity was limited to 1.4 km/h by its configuration. Road conditions were identified by comparing amplitudes and phases between responses at different polarizations, and the results were compared against reference values. The study demonstrated that, for a limited number of surfaces (such as cobblestone and asphalt), the type of road pavement could be reliably predicted.

Short-range polarimetric automotive radar operating in the 20-24 GHz frequency range was investigated in [68, 69]. The experimental system was able to classify road surfaces at grazing angles of up to 60° and distances of up to 1 meter. The tested surfaces in [68] included wet and dry asphalt, as well as rough and smooth asphalt. The results obtained confirmed the potential of using microwaves for road condition monitoring, suggesting prospects for safety improvements. In [69], the radar installed on the car's bumper was used to study changes in the microwave polarimetric scattering matrix caused by water, ice, and snow on the road. The paper proposed the velocity gating method, wherein prior knowledge of the car's velocity enables filtering only the signal component reflected from the road surface. Preliminary tests indicated that the system could measure the polarimetric matrix of targets moving at different speeds. This capability opens the possibility of measuring road backscattering even in the presence of high echoes from other obstacles on the road. Papers [67,68,69] demonstrated the feasibility of surface classification by analyzing differences in the characteristics of signals with various polarizations reflected from different surfaces. However, no surface classification algorithms suitable for practical implementation on vehicles were proposed.

The automotive radar frequency of 77 GHz is of special interest to researchers. This frequency band extends from 76 GHz to 77 GHz for automotive Long-Range Radar (LRR) and from 77 GHz to 81 GHz for automotive Short-Range Radar (SRR). In the previously referenced work [39], the theoretical exploration of the propagation, reflection, and diffraction of a 77 GHz radar signal was conducted and experimentally verified on various realistic asphalt road surfaces and geometries at distances of up to 100 meters. Through the analysis of signals with both vertical and horizontal polarizations, the study showed how the curvature of the road in the vertical direction influences the properties of the backscattered signals.

Papers [70-71] delve into the discussion of 24 GHz and 77 GHz dual-polarized automotive radars designed for road condition classification. The frequency modulated continuous wave (FMCW) radars operated within the frequency bands of 18-26.5 GHz and 50-79 GHz. The antennas were tilted to capture backscattering with varying incident angles. The research encompassed the examination of reflections from diverse real surfaces in both summer and winter conditions, including dry, wet, snowy, and icy asphalt with different roughness levels. Power budget calculations demonstrated the feasibility of a 24 GHz SRR for road condition identification, with the 24 GHz frequency exhibiting superior results over 77 GHz. However, due to the radar illuminating the road surface at a few meters from the vehicle and the necessity for averaging several measurements for reliable classification, the developed system was deemed unsuitable for providing pre-warning to the driver.

Consequently, an advanced and more potent radar operating within the 75-82 GHz band was developed and presented in [72]. The measurement antennas were positioned 1 m above the road surface, and their main beams were tilted 65° relative to the normal of the road surface. The system demonstrated potential in classifying road surfaces (ice, snow, dry asphalt, water on asphalt) based on differences in backscattered signals with vertical-vertical and horizontal-horizontal polarizations of the transmitting and receiving antennas at distances up to tens of meters, as shown in Fig. 2.

Figure 2. Measured ratio between backscattering coefficients at vertical-vertical (vv) and horizontal-horizontal (hh) polarizations for different road surfaces [72].

Figure 2. Measured ratio between backscattering coefficients at vertical-vertical (vv) and horizontal-horizontal (hh) polarizations for different road surfaces [72].

In [73], a 77GHz FMCW radar with vertical polarization was affixed to the grille of a mobile test vehicle in a monostatic configuration, as shown in Fig. 3, and backscattered signal data were collected. A 75° incidence angle and a height of 0.6 meters above the ground were chosen as a compromise to achieve robust backscatter while ensuring that the road remained in the antenna's far field. This configuration also aimed to minimize contributions from undesired targets, such as other road vehicles. The proposed methodology, centered on the analysis of radar range-velocity maps and the radar range profile, has demonstrated promising capabilities in discerning dry, wet, snowy, and slushy conditions.

Figure 3. Road condition radar [73].

Figure 3. Road condition radar [73].

The objective of the study [74] was to analyze the correlation between the backscatter coefficients of road surfaces and the direction of polarization, grazing angle, and surface type (new, weathered, snow-covered, or wet asphalt, wet and dry concrete, grass). Backscattering coefficients, acquired using a 77 GHz radar with both vertical and horizontal polarized antennas, reveal significant variations in the reflectivity of the surfaces under consideration within the range of elevation angles from 70° to 85°.

In [75], a 77 GHz radar system with one transmitting and four receiving antennas was developed for surface identification. Features were extracted from the backscattered signal using the Mel Frequency Cepstral Coefficients (MFCC) technique. K-Nearest Neighbors (KNN), Decision Tree (DT), and Support Vector Machine (SVM) classifiers were employed to identify five road surface types: dry concrete, dry asphalt, grassy, slush, and sand. The SVM classifier demonstrated superior performance, achieving accuracy ranging from 92% to 99%, depending on the surface type. When the vehicle is moving at 40 km/h, the system has an effective range of about 20 meters in front of it.

Despite growing interest in 77 GHz radars, several studies [76-79] were dedicated to developing a 24 GHz automotive radar featuring a 200 MHz bandwidth. The FMCW radar developed in [76], operating within the frequency band of 24-24.25 GHz, assesses surface scattering across eight incidence angles, reaching up to 85°. This radar is equipped with eight transceivers to measure backscattered power, utilizing a circularly polarized planar array as the transmission antenna and two orthogonal linearly polarized reception arrays. Such an approach holds the potential for achieving higher classification accuracy. The surfaces encompass dry, wet, icy conditions with three different ice layer thicknesses and leaf-covered asphalt. The paper lacks specific surface classification results, but like other reviewed works, it suggests that road surfaces can be recognized based on the measured ratio of received powers at different polarizations.

A notable aspect of the system presented in [77-79] was the integration of a 40 kHz sonar with the radar, and their data fusion. As shown in Fig. 4, the sonar and radar with 2 transmitting and 2 receiving antennas with different polarizations, were installed at the height of the car bumper. Extensive analyses were conducted on numerous real surfaces in both summer and winter conditions, encompassing asphalt, concrete, gravel, grass, sand, mud (both dry and wet), and surfaces covered with snow and ice. The corresponding database was comprehensive, consisting of over 2800 recorded radar and sonar signals from approximately 40 outdoor locations.

For classification purposes, statistical methods, including the Maximum Likelihood Estimator (MLE), Minimum Distance Classifier (MDC), KNN, along with Artificial Neural Network (ANN) based on Multilevel Perceptron (MLP), were employed. The results demonstrated surface classification accuracy of up to 95%, contingent on the chosen method and measurement conditions. However, the incorporation of sonar introduces limitations in the surface sensing range. Solely relying on radar does not yield acceptable accuracy, necessitating the complementary use of sonar in the system.

Figure 4. Sonar (blue box) and radar for surface identification [79].

Figure 4. Sonar (blue box) and radar for surface identification [79].

Gradually, the interest of researchers moved to higher radar frequencies with the aim of developing a practical solution. The objective of [80] was to develop a sensor capable of detecting ice in front of a moving car using a polarimetric radar operating in the range of 87.5-92.5 GHz. The study demonstrated the capability, at short distances (grazing angle of 45°), to confidently distinguish between dry, wet asphalt, and asphalt covered with ice.

In [81], the potential of using a 60 GHz millimeter-wave radar for surface classification was explored, specifically for applications like autonomous lawn mowers or vacuum cleaners. Such applications do not require a large system range, and the speed of such autonomous devices is low, making the classification process simpler. The study revealed that by extracting features that exploit temporal behavior and combining this information with a deep neural network classifier, it is possible to distinguish grass-covered surfaces from other types commonly found adjacent to lawns with satisfactory accuracy ranging from 90% to 99%.

The development of a double-sideband dual-polarized automotive radiometer operating between 82 and 98 GHz for surface classification was considered in [82] and [83]. The brightness temperatures of dry, liquid water, and icy asphalt have been studied in laboratory conditions at observation angles of 50° and 56° and then with a radiometer mounted at an angle of 47° on a slow-moving vehicle. The results showed that the ice patches on asphalt are distinguishable both in terms of the measured H and V brightness temperatures.

In more recent works, the technique of extracting information from the reflected signal has become more sophisticated to broaden the list of available features beyond solely relying on the signal range profile, as was done in earlier works. The method of analyzing road surface scattering as a dynamic process and presenting the road surface signature in the form of a range-Doppler plot was implemented in [84]. The goal was to classify road surfaces and identify road hazards, such as puddles of water or patches of ice, using a 79 GHz automotive radar. This preliminary research aimed to prove the concept, and the authors suggested that the results were promising, though no classification data were provided.

Using MIMO radars provides more accurate azimuth and elevation angle information of targets compared to conventional radars. To enhance surface scattering analysis and enable the use of additional features, a 77 GHz radar system with two transmitting and four receiving antennas was developed in [85]. The antennas were installed at a height of 1 meter with an incident angle of 75 degrees, resulting in a surface distance of approximately 4 meters. Six surface classes, including grass, concrete, asphalt, wet grass, wet concrete, and wet asphalt, were classified using Random Forest (RF), SVM, and KNN methods, with distinct features selected for each. The maximum accuracy of classification achieved was 98.17%.

In [86], a 24 GHz FMCW MIMO radar with a 3 GHz bandwidth, equipped with 2 transmitting and 4 receiving antennas , was mounted on a moving robotic platform to classify surfaces, including carpet, tile, and wood. A fully connected neural network and SVM algorithms were used to analyze the amplitude features of the signal over a range of 1 meter. Classification accuracy improved as the number of radar channels increased, achieving approximately 99%. Interestingly, adding signal phase features did not enhance classification accuracy, suggesting that the classification relies primarily on amplitude features.

In [87], the use of entropy-alpha anisotropy (HαA) decomposition was proposed for recognizing road surface conditions using automotive radar. The HαA decomposition operates on the statistical properties of the polarimetric signatures of an object to extract specific features. A radar operating at 80 GHz with a 10 GHz bandwidth was employed to classify dry asphalt, wet asphalt, and asphalt covered with gravel. Clear differences were observed between surfaces, offering prospects for utilizing this method in surface classification.

4.2.2. Automotive Radars Operating in Sub-THz Range

The practical implementation of an automotive surface classification system imposes several requirements on the installation and operation of radars. As discussed in Section 3.2, the height of the radar installation above the road surface is limited by the car's height; therefore, surface classification is performed at small grazing angles. Consequently, this results in a reduction in the power of the reflected signal and a decrease in the system’s sensitivity to surface roughness (1). While the first issue can be addressed through equipment improvements, such as increased transmitter power and receiver sensitivity, the second issue can be effectively mitigated by increasing the signal frequency, as demonstrated in [40]. For this reason, research on using sub-THz radar for road surface classification, though not yet extensive, has shown promising results.

Several works are exploring the use of radars operating in the sub-THz frequency range for surface classification. Papers [88-90] extend the investigations initiated in studies [36, 37], focusing on the polarimetric radar phenomenology of road surfaces and highway scenes at low grazing angles. A 222 GHz radar with a 2 GHz bandwidth and Gaussian optic lens narrow beam 1° antennas was employed in [88, 90] to measure backscattering coefficients of asphalt and concrete. Measurements were conducted at each angle between 80° and 88°, considering the three polarization combinations vertical-vertical, horizontal-horizontal, and vertical-horizontal. Each radar placement was at a height between 0.7 and 1.5 meters. The papers highlight a significant difference in the properties of radar signals reflected from asphalt and concrete at a frequency of 222 GHz, as depicted in Fig. 5.

Similar experiments were conducted in [89] with frequencies of 230-248 GHz using a dual-polarized transceiver antenna system with a beamwidth of 1°, enabling fully polarimetric measurements with exceptionally fine cross-range resolution. The tests involved various surfaces, including smooth asphalt, smooth asphalt covered with a thin layer of water, smooth asphalt covered with a thin layer of ice, smooth asphalt covered with 5 cm of snow, rough asphalt, concrete, gravel, and gravel covered with 18 cm of snow. A representative set of raw data has been presented, but the final processed data has not been provided.

Figure 5. Comparison between the backscattering coefficient of concrete and asphalt [87].

Figure 5. Comparison between the backscattering coefficient of concrete and asphalt [87].

Studies [40] and [91] delve into the scattering of polarized radar signals within the millimeter and sub-THz frequency range, specifically at frequencies of 150 GHz, 300 GHz, and 670 GHz, each with a 16 GHz bandwidth. These studies aim to explore the advantages of sub-THz radar for surface discrimination in automotive sensing, comparing them with reference results obtained using the standard automotive radar frequency of 79 GHz with a 4 GHz bandwidth. The experiments involved measuring backscattering from sandpaper with varying grit, ranging from smooth to rough, across the entire spectrum of grazing angles from 10° to 90°.

Experimental evidence substantiates that utilizing signals in the sub-THz range enables the acquisition of diffuse reflections from road surfaces, which appear smooth at the conventional frequencies of automobile radars (e.g., 24 GHz or 79 GHz). Thus, of the four surfaces with different roughness studied in [40], the 79 GHz radar was unable to distinguish them, the 150 GHz radar could distinguish one, and the 670 GHz radar confidently distinguished all four surfaces. Additionally, the studies demonstrate the applicability of established models of signal backscattering to the sub-THz frequency region.

The capability of sub-THz frequencies to identify icy road surfaces, attributed to their high sensitivity to slight surface roughness, was investigated in [92]. Studies conducted at frequencies of 140, 220, and 340 GHz, employing different signal polarizations, demonstrated that a system based on a sub-THz frequency radar can effectively recognize the thickness of the ice layer and the surface roughness of a soil road. This research affirms the present and imminent applications of sub-THz frequencies in road surface classification. Moreover, the developed theoretical model is applicable for evaluating surfaces of icy/snowy concrete roads or asphalt roads.

5. Road Surface Classification Based on Radar Imaging

The absolute values of the reflected signals cannot reliably serve as a basis for surface identification, given their dependence on individual parameters of the transmitter and receiver, their installation accuracy, and radome contamination (such as dirt, mud, snow, etc.). A novel approach to surface classification, aimed at eliminating inaccuracies due to reliance on absolute values of reflected signals, is based on the analysis of radar imaging. This became feasible with the advancement of radars offering high resolution in both azimuth and range. The works that have used radar imaging, from mm-wave to sub-THz, for classification are discussed here.

The imaging capabilities of a turntable mounted FMCW radar operating at 150 GHz with a 6 GHz bandwidth were discussed in [47, 48]. The utilized antennas possess a wide E-plane (elevation) beam of 15° to ensure a broad illuminated footprint along the path, and a narrow H-plane (azimuth) beam of 2.2° to achieve fine angular resolution. These antennas were installed on a turntable in a monostatic configuration, covering distances from 3 m to 100 m. Experimental results demonstrate the potential of a sub-THz radar sensor to provide high-resolution images in all terrain scenarios due to its sensitivity even to slight roughness and texture of the objects imaged. As seen in the radar image in Fig. 6, the radar provides effective differentiation between distinct types of surfaces with a sufficiently high level of detail. The radar images proved suitable for visual feature detection and interpretation, resembling optical images.

Building upon these findings, a proof of concept for a method to identify traversable free space for path planning in autonomous vehicles from imagery was presented in [49]. The segmentation method, based on histogram thresholding of super-pixel statistics, demonstrated good segmentation of road and off-road images. It was also shown that radar images can be efficiently used to classify road surface types.

In [50, 51], a mechanically scanned FMCW radar with a bandwidth spanning the full automotive frequency allowance of 5 GHz (76-81 GHz), and azimuthal beamwidth of 1.7°, and elevation beamwidth of 7.2°, was employed to conduct image segmentation and region classification in high-resolution automotive radar imagery within a complex urban environment. This paper presented an algorithm for automatic segmentation and classification of high-resolution radar images in the automotive domain based on supervised classification using statistical parameters of radar image regions. The results demonstrated promising potential for the identification of selected road scene region types, including asphalt, grass, and road objects. The areas of asphalt exhibited the highest classification accuracy at 0.81, while the areas of grass and objects showed the lowest value at 0.64.

The forward-scanning 300 GHz automotive Synthetic Aperture Radar (SAR) with a bandwidth of 6 GHz and a 1.3° beamwidth, as presented in [93], exhibits even greater potential for surface classification. Although the aim of the work is not to recognize road surfaces, the highly detailed radar images obtained can be effectively used for this purpose.

In [94], an aperture synthesis method was employed to investigate the reflective properties of typical road surfaces. An FMCW 80 GHz radar (68-92 GHz) was utilized to generate high-resolution SAR images of asphalt surfaces using different antennas. This approach enables the measurement of surface reflection characteristics with high accuracy.

In the papers previously examined regarding surface identification through radar imaging, various methods were employed to detect differences between surfaces. With the advancement of Convolutional Neural Networks (CNNs), such ANNs have become widely utilized and, at times, the preferred methods for image classification. CNN is a type of ANN that automatically extracts features from a sample and propagates them from a low layer to a high layer through multiple layers of convolutional operations, generating output in the form of classes. CNN has been successfully applied to various applications, including image and speech recognition, object detection, and pattern recognition [95, 96]. CNN employs weight-sharing, reducing the number of weights compared to ANN, resulting in a faster network with lower memory requirements. Unlike ANN, a 2-dimensional image can be fed directly into CNN without the need for conversion to a 1-dimensional image [97]. This offers the advantage of preserving the spatial properties of the image, eliminating the necessity for pre-processing the image data.

In [98], an approach was presented for surface identification based on the analysis of sub-THz radar images using CNN. Promising results were obtained in the classification of surfaces with known roughness in laboratory conditions, employing 150 GHz and 300 GHz mechanically scanning radars with an antenna beamwidth of 2.2°. The sweep bandwidth is 8 GHz for 150 GHz and 16 GHz for 300 GHz radar to achieve sub-centimeter range resolution. The average accuracy of classifying three types of surfaces—relatively smooth, medium rough, and rough—was 95%.

This innovative approach for road surface identification, presented in [98], was applied to classify real road surfaces. A database of images representing various road surfaces (asphalt, mud, gravel, river, rocky road, and uneven countryside road) was collected using a 79 GHz mechanically scanning radar with a 5 GHz bandwidth, 2.2° antenna beamwidth, and 30 mm range resolution [99]. The complete dataset comprised over 2000 cropped images representing these six surface classes. The developed CNN demonstrated a classification accuracy of 98.9%, showcasing the potential of imaging radar in road surface classification based on the surface texture and profile of the road surfaces.

A radar-based road surface classification system for personal mobility devices was presented in [100]. Radar images of five surfaces, including asphalt, brick, tile, sand, and urethane, were obtained using a 60 GHz pulsed radar and processed using the developed DNN. The proposed mechanism demonstrated robustness against ambient noise and the ability to recognize surface types with high accuracy, exceeding 95%, and low computational complexity. However, the system operated at a close range, with distances of 20 cm, which is acceptable for personal mobility devices but too low for automotive applications.

The combination of MIMO beamforming with Doppler beam sharpening, as proposed in [101], and the forward-looking MIMO-SAR (synthetic aperture radar) technique with the Burg algorithm from [102] provides a powerful approach for high-fidelity radar imaging. Using a 77 GHz centre frequency, 2 GHz bandwidth, and a 4TX/16RX MIMO sensor, which forms 61 virtual linear elements in azimuth, it was demonstrated that high-resolution imaging of the stationary environment surrounding a moving vehicle can be achieved with a conventional MIMO sensor, yielding results comparable to those of a mechanically steered radar. Additionally, the Burg algorithm, applied in conjunction with the forward-looking MIMO-SAR technique, enhances the cross-range resolution in boresight and further improves resolution off-boresight by extrapolating data along both Doppler and angular dimensions. This technique provides highly detailed radar images, which can be effectively used for road surface classification, as shown in Fig. 6 [102].

Figure 6. Radar imaging: (a) experimental scene, (b) MIMO-SAR image [102].

Figure 6. Radar imaging: (a) experimental scene, (b) MIMO-SAR image [102].

6. Sensor Fusion for Road Surface Classification

Modern cars are equipped with various sensors [33]. As demonstrated in Section 2, each of these sensors has its strengths and weaknesses. Therefore, it is not surprising that many works propose augmenting data from different sensors for a more robust solution to the challenges faced by machines with varying degrees of autonomy [8, 9]. Typically, this data fusion is employed in tasks related to identifying objects on the road and in its vicinity. Only a limited number of studies focus on sensor fusion for road surface identification. A number of these works have already been cited in this review in sections concerning methods and techniques for surface recognition. In this section we will list these works with an emphasis on sensor fusion technology.

Patent [101] describes a road classification technique that uses echo signal augmentation from a millimeter-wave radar and an infrared laser to classify the surface into predefined condition categories in the direction of the vehicle's travel. The concept of utilizing weather station measurements data (temperature, humidity and pressure) in addition to polarimetric radiometer data to control road conditions was proposed in [64, 65]. This data fusion enables more accurate detection of water, ice, and snow on asphalt.

Studies [77-79] focused on developing a real-time automotive surface classification system using ultrasonic and microwave sensor fusion. The system combined a 24 GHz radar and a 40 kHz ultrasonic sensor, extracting features from separate surface swathes. Principal component analysis and supervised classification were then applied. Despite its short range (up to 10 m) due to sonar limitations, the system achieved high classification accuracy (see Section 4.2). Integrating sonar and radar data improved recognition compared to using each sensor alone, and combining these technologies with advanced classification methods, such as ANN-based MLP, further enhanced accuracy.

Despite the limited number of publications, we consider the sensor fusion approach promising for road surface classification and expect further advancements in this field.

7. Discussion and Conclusions

Over the past decades, significant progress has been made in the development of automotive radars and radiometers for surface classification, driven by several factors:

• The emerging trend towards the introduction of partially autonomous and autonomous vehicles and a growing awareness of the crucial task of classifying surfaces for autonomous driving.
• The development of the theory of signal reflection from surfaces with varying roughness and dielectric properties, providing the theoretical foundations for surface classification.
• The emergence of compact microwave sensors and an increase in their frequency to the range of sub-THz signals.
• Advancements in statistical methods of classification, particularly the utilization of ANNs for surface classification purposes.

The summary of surface identification techniques presented in the reviewed papers is provided in Table 2.

Table 3 and Fig. 7 present a breakdown of publications by decades and frequencies of surface identification radars and radiometers. As indicated in the table, many publications focus on microwave sensors operating in the 24 GHz automotive bands, which is not surprising. However, in recent decades, there has been a shift of interest from 24 GHz to 76-79 GHz frequencies. Additionally, there is a growing interest in radars operating above 100 GHz. As can be seen from Fig. 7, during the first 5 years after 2020, the number of publications on surface identification exceeded the total number from the entire decade of the 2010s.

Figure 7. Breakdown of publications.

Figure 7. Breakdown of publications.

However, the task of developing a microwave surface classification system is still far from practical implementation. We believe that this is due to the complexity of processing the reflected microwave signals in real-time and the challenges in interpreting the obtained information. Most publications on surface classification using radar and radiometer are limited to examining the theory of signal backscattering from various surfaces and verifying it through experimental experiments. They often do not progress to the stage of prototype system development and the verification of classification accuracy. Typically, the surfaces considered are limited to dry, wet, snowy, and icy asphalt; off-road conditions are seldom analyzed.

Nevertheless, given the rapid advances in radar and signal processing, we anticipate the emergence of automotive surface identification systems in the coming years. Among the promising directions for solving the problem of surface classification, the following can be distinguished:

• Increasing the resolution of the radar and the capability of obtaining radar images approaching optical ones.
• Application of deep neural networks for surface classification.
• Sensor fusion.

This task is expected to be gradually addressed with the introduction of autonomous vehicles.