Commodity WiFi-Based Wireless Sensing Advancements over the Past 5 Years

1. Introduction

The demand of ubiquitous internet connection has catalyzed the vast deployment of WiFi infrastructures over the past decades, making WiFi signal available almost everywhere. With the rapid progress of wireless communication and signal processing techniques, researchers have successfully reused WiFi as a sensing platform beyond traditional pure communication medium, which further gives birth to the idea of integrated sensing and communication (ISAC) with WiFi [1,2,3]. After years of persistent research, WiFi sensing is drawing huge attention from both academia and industry [4]. Both communities recognize ISAC as a compelling technology for improving the spectrum efficiency and reducing the hardware cost [5]. It is worth mentioning that, starting from 2020, the IEEE 802.11 working group established an IEEE 802.11bf standardization group for encompassing wireless sensing within the new version of 802.11 standard, greatly pushing Wi-Fi sensing into a reality.

The basic rational behind WiFi sensing is quite straightforward [6]. When wireless signal propagates from the transmitter to the receiver through multiple paths, a phenomenon called multi-path effect, the superimposed receiving signal intrinsically contains the signal component reflected or diffracted by the sensing target. Therefore, by analyzing the target "modulated" receiving signals, researchers can recover the rich information regarding the target, such as location and activity. Compared with classic sensor-based and vision-based sensing paradigms, WiFi wireless sensing has the advantages of low-cost ubiquity, wide coverage, non-intrusive and privacy-protection. Due to its appealing superiority, a plenty of WiFi sensing applications have been developed, ranging from coarse-grained motion detection [7], activity recognition [8] to fine-grained localization [9], breath monitoring [10].

Inspired by existing survey papers [11,12,13,14,15], this paper investigates thrilling achievements made within the last 5 years and presents an in-depth analysis of these sensing systems, aiming to facilitate further research of WiFi sensing area. This paper first divides existing works according to different application scenarios, including localization and tracking, activity recognition, vital sign monitoring and target imaging. For each category, both application-specific problems and solutions are compared and summarized. Then this paper further classifies recent studies based on the methodology adopted, whether it is model-based, handcrafted pattern extraction-based or deep learning-based, pointing out the pros and cons of each method. Furthermore, this paper highlights remaining challenges of current works such as generalization issue and large scale perception. Future research directions requiring further study are discussed in the end. The main contributions of this work are summarized as follows.

• To the best of our knowledge, this is the latest comprehensive survey of WiFi sensing area, covering most recently great progresses made over the past 5 years.
• We categorize existing studies from two distinct perspectives, i.e., application-based and methodology-based, and present in-depth analysis of recent works.
• We highlight the key challenges encountered in existing studies and present a thorough discussion about three promising research directions of WiFi sensing.

The rest of this paper is organized as follows. In Section 2, we briefly introduce the concept of CSI and explain several popular sensing models. In Section 3, we classify state-of-art works with regard to two criteria, i.e., application variety and methodology difference. Practical limitations and challenges are analyzed in Section 4. In Section 5, a detailed discussion about future trends of WiFi sensing is provided. Finally, we conclude this article in Section 6.

2. Preliminary

Before analyzing WiFi sensing, we briefly introduce necessary background of channel state information (CSI) and several general signal sensing models.

2.1. Channel State Information

Serving as a key metric of communication system, CSI depicts how a signal propagates through a wireless channel. Indeed, a wireless communication channel can be defined as:

Y = H * X + N

(1)

where $X$ and $Y$ are the transmitted and received signal, respectively. $H$ is the channel matrix representing CSI and $N$ denotes the channel noise.

In a typical indoor environment shown in Figure 1, a signal sent by the transmitter $\left(Tx\right)$ travels through multiple paths before arriving at the receiver $\left(Rx\right)$, also known as the multi-path effect. Therefore, assuming there are $L$ different paths, the wireless channel $H$ can be mathematically expressed as channel impulse response (CIR) [6]:

$$h\left(t\right)=\sum _{i=1}^L{a}_i{e}^{{-j\theta }_i}\delta \left(\mathrm{t}-{\tau }_i\right)$$

(2)

Where $a_i$, ${\theta }_i$ and ${\tau }_i$ are the complex amplitude attenuation, phase shift and propagation time delay of the $i$-$th$ path, respectively. $\delta \left(t\right)$ is the Dirac delta function. Each impulse in the summation of Equation (2) represents a delayed multi-path component, multiplied by its corresponding amplitude and phase variation.

As shown in Figure 1, when a person moves inside the scenario, the human body will inevitably alter certain propagation path, thus changing the CIR. Hence, the underlying principle of wireless sensing is analyzing human-induced channel variation. However, CIR cannot be precisely measured with commodity WiFi devices, especially given limited bandwidth of WiFi. Fortunately, with the adoption of orthogonal frequency division multiplex (OFDM) technique in present IEEE 802.11 standard, researchers resort to study channel frequency response (CFR), an equivalent channel representation of CIR in frequency domain.

$$CFR\left(f\right)=\left|CFR\left(f\right)\right|e^{j\angle CFR\left(f\right)}$$

(3)

where $\left|CFR\left(f\right)\right|$ and $\angle CFR\left(f\right)$ represent of amplitude-frequency and phase-frequency response of CFR, respectively. With proper driver modification, researchers can obtain an OFDM-based sampling version of CFR with commercial-off-the-shelf (COTS) WiFi network interface card (NIC) since 2010 [16,17], greatly prompting the prosperity of WiFi sensing [12]. To be specific, the extracted CFR depicts the amplitude and phase of different subcarriers:

$$H\left(f_i\right)=\left|H\left(f_i\right)\right|e^{j\angle H\left(f_i\right)}$$

(4)

where $H\left(f_i\right)$ is the CFR sampled at the $i-th$ subcarrier with central frequency of $f_i$. In fact, the CSI data $H=\left\{H\left(f_i\right)|i\in \left[1, N\right]\right\}$ used in most research papers exactly refers to the definition of Equation (4), i.e., a sampled version of CFR at the granularity of subcarrier level.

Generally speaking, this sampled CFR lays the foundation of advanced WiFi sensing, paving the way for the feasibility of various modern applications. CSI data contains rich information of signal propagation and we will simply use CSI to signify the raw WiFi data for brevity in the following part.

2.2. Signal Sensing Models

2.2.1. Fresnel zone-based reflection model

Taking one pair of $Tx-Rx$ link as example, Fresnel zones are concentric ellipses with two foci corresponding to the $Tx$ and $Rx$, as $P_1$ and $P_2$ shown in Figure 2. For a given radio length $\lambda$,the $nth$ Fresnel zone boundary containing $n$ ellipses can be defined as:

$$\left|P_i{Q}_n\right|+\left|Q_n{P}_2\right|-\left|P_1{P}_2\right|=n\lambda /2$$

(5)

Where $Q_n$ is a point on the $nth$ Fresnel zone boundary. The $nth$ Fresnel zone refers to the elliptic annulus between the $(n-1)th$ and $nth$ ellipse boundary, while the innermost ellipse is called the first Fresnel zone (FFZ). Equation (5) indicates that the path length of the signal reflected through the $nth$ Fresnel Zone boundary is $n\lambda /2$ longer than that of the Line-of-Sight (LOS) path, i.e., $\left|P_1{P}_2\right|$.

The Fresnel zone-based reflection model [18] characterizes how the amplitude and phase of CSI change when target moves outside the FFZ. The key property of the reflection sensing model is when a target moves across a series of Fresnel zone boundaries, CSI amplitude and phase will show continuous sinusoidal-like pattern, which can be utilized for sensing applications such as respiration and walking direction detection [19].

2.2.2. Fresnel zone-based diffraction model

Figure 3. Geometry of Fresnel zone diffraction sensing [20].

Figure 3. Geometry of Fresnel zone diffraction sensing [20].

According to RF propagation theory, more than 70% of the signal energy is transferred via the FFZ. Therefore, when a target moves inside the FFZ, signal diffraction becomes more important and dominates the received signal variation. The Fresnel zone-based diffraction model [20] depicts how the amplitude and phase of CSI change when target moves inside the FFT. The key property is when sensing activity inside the FFZ, the CSI amplitude variation will show different shapes, be it either monotonically decrease or non-monotonous “W” according to the target size. Apart from respiration monitoring, diffraction sensing model have also been proved effective for recognizing exercises and daily exercises [8].

2.2.3. Scattering sensing Model

One main limitation of previous models is that the simple reflection or diffraction assumption may not hold true when considering complex target motion, where signals are scattered from multiple human body parts. Different from Fresnel zone-based model, scattering sensing model treats all objects as scatters, taking account of all multipaths together. As shown in Figure 4, intuitively, scattering model considers each scatter as a virtual Tx, e.g., static walls, the arm and leg of moving human. Given numerous multipaths considered, scattering model is in fact a statistical model generally applicable to complex indoor scenarios. Scattering sensing model has been adopted in various speed-oriented tasks [21,22], achieving robust performance even with non-line-of-sight (NLOS) occlusion.

3. WiFi Sensing

Serving as a key property in future wireless system, WiFi sensing has enabled various important applications. In this section, we category recent works from two aspects, i.e., application-oriented and methodology-oriented.

3.1. WiFi sensing applications

Presence detection. Presence detection determines whether target exists or not within the sensing area and serves as the prerequisite for further sensing tasks. Target presence detection could enable many modern applications such as security system and smart home. Although usually included as a detector module in most studies, there have been some new applications based on presence detection. As shown in Table 1, WiCPD [23] studied child presence detection in smart car scenario, preventing potential danger of children if left alone in a vehicle. Hu et al. [24] considered target location relative to the sensing device, supporting more intelligent control system using this area-aware context. Besides, Zhu et al. [25] and WI-MOID [26] further differentiating human from non-human targets to mitigate influence from unwanted objects, avoiding unnecessary false alarm alert.

Gait recognition. Gait, a unique biomarker, refers to the distinctive walking character of different people and has been used for human identification and authentication applications. Early gait sensing works usually required users to walk on fixed trajectories within restricted area, while recent studies, e.g., GaitSense [27], GaitWay [28] and Wi-PIGR [30], aimed for path independent gait recognition where users can waking along arbitrary paths even in through-the-wall scenario. Besides, CAUTION [29], Auto-Fi [31] and GaitFi [32] tried to realize robust gait recognition with limited training data while Wi-Diag [33] further studied more challenging multi-human recognition problem. As depicted in Table 2, all these works greatly contribute to more ubiquitous gait-based sensing applications.

Gesture recognition. Wireless gesture recognition has emerged as an important part of modern human computer interaction, enabling wide applications including smart home control and virtual reality. Previous studies tried to learn the intricate pattern between signal variation and human gesture under the one-to-one mapping assumption. However, this assumption does not hold since the received signal is highly dependent on the relative location and orientation of users, as proved by the Fresnel reflection model [18]. Thus, recent works mainly focused on realizing a position-independent robust gesture recognition system, as illustrated in Table 3. Kang et al. [34], Widar 3.0 [39], UniFi [41], WiTransformer [42] and AirFi [43] leverages various deep learning methods, e.g., adversarial learning, multi-view network and few-shot learning, to realize robust and efficient recognition. On the other hand, WiGesture [35], HandGest [36], DPSense-WiGesture [37], Niu et al. [38] and WiCGesture [44] attempted to extract distinct and consistent feature from a hand-oriented view, realizing reliable and continuous recognition either through more fine-grained signal segmentation or signal quality assessment. Besides, WiFine [40] managed to realize real-time gesture recognition using low-end edge devices, e.g., Raspberry Pi. Overall, these methods bring WiFi gesture recognition one step towards more practical use.

Activity recognition. WiFi-based human activity recognition (HAR) has become the most studied research topic over the past years, covering many applications including people counting [45], fall detection [49,53], door passing detection [56] and daily activities. Table 4 shows the summary of recent HAR works. Most works tried to address the performance degradation due to location, person and environment dynamic, also known as domain-dependent problem [46,47,50,52,54,57,58,59,62,63]. Besides, WiMonitor [48] studied continuous long-term human activity monitoring, capturing user information such as location change, activity intensity and time. Moreover, EfficientFi [51] considered the signal transfer-induced communication problem in large-scale sensing scenario, providing a cloud-enabled solution with efficient CSI compression, while SAT [60] and SecureSense [61] proposed robust sensing schemes under various adversarial attacks. Liu et al. [55] proposed a dynamic Fresnel Zone sensing model using moving receiver such as smartphone, filling the gap of existing fixed-location transceivers.

Localization and tracking. Due to limited channel bandwidth and antenna number of COTS WiFi devices, there have not been much studies for WiFi-based localization and tracking, as shown in Table 5. Recent works tried to improve tracking performance through more accurate target velocity estimation using moving-induced Doppler Frequency Shift (DFS). Niu et al. [64] optimized velocity estimation by devising a dynamic selection scheme, which can choose the optimal set of receivers for tracking. To better track human walking, WiTraj [65] intelligently combined multi-view information provided by different receivers and differentiated walking with in-place activity to avoid tracking error accumulation. FewSense [66] creatively fused phase and information for better DFS estimation, achieving high accuracy even with fewer CSI samples. In addition to these works, Zhang et al. [67,68] achieved sub-centimeter localization accuracy using intelligent reflecting surface (IRS) technique. By constructing IRS, researchers can modulate the spatial distribution of WiFi signal, improving the spatial resolution of WiFi localization. While promising, their current prototype systems are realized using vector network analyzer (VNA), requiring further study with COTS device. Apart from device-free tracking mentioned above, Fan et al. [69] Wi-Drone [70] studied device-based tracking applications. Fan et al. [69] gained accurate moving direction and in-place rotation angle estimation using a single access point, while Wi-Drone [70] realized the first WiFi tracking-based indoor drone flight control system, providing promising candidate solutions for indoor localization and navigation.

Vital sign monitoring. Vital sign plays a crucial role in people’s health and well-being monitoring, providing useful information for early prediction and interference with potential diseases. As shown in Table 6, CSI-based vital sign detection mainly focused on respiration estimation. MultiSense [71] studied multi-person respiration sensing problem, while SMARS [72] and WiFi-Sleep [73] integrated breath monitoring into user’s sleep quality assessment. WiPhone [74] presented a smartphone-based sensing system, achieving robust performance in NLOS scenarios. Xie et al. [76] addressed the motion interference from nearby individuals, bring respiration monitoring closer to practical application.

Pose construction and imaging. WiFi-based pose estimation and target imaging provides a complementary solution to traditional camera-based perception. As listed in Table 7, WiPose [77], GoPose [79], MetaFi++ [81] and PowerSkel [83] proposed different 3D human skeleton construction frameworks, while WiSIA [78], Wiffract [80] and WiProfile [84] further investigated how to recover target images with WiFi signals. Differently, WiMeasure [82] realized millimeter-level high precise target size measurement, making up for a missing piece of WiFi sensing. It should be noted that in order to achieve fine-grained imaging, high sampling rate and even customized antenna deployment are usually required, as shown in later Tables. Therefore, WiFi imaging is only applicable for specific application scenario for the time being.

3.2. WiFi sensing methodologies

Model-based sensing. Since model-based sensing methods have clear advantage of interpretability, researchers have developed several models for describing the physical relationship between CSI variation and target behavior, detailed in Section 2. As shown in Table 8, scattering model has been widely used for velocity and periodic pattern extraction [28,49,72], while diffraction model being adopted in near-the-LOS scenarios, i.e., within FFZ, for fine-grained sensing tasks [56,80,82,84]. Although less mentioned in Table 8 [55], Fresnel zone-based reflection model is in fact the most used model. Reflection model is commonly implicitly incorporated in various sensing systems for quantitatively analyzing signal variations and identifying sensing limitation, thus guiding the implementation of more stable and stable sensing system [85,86,87].

Hand-crafted statistical pattern extraction-based sensing. Derived from feature engineering in traditional machine learning process, researchers have come up with various task-oriented feature extraction schemes, utilizing in-depth analysis of activity characteristics and advanced signal processing techniques. As shown in Table 9, along with signal processing such as signal segmentation and signal energy estimation, statistical features, such as doppler frequency shift and speed estimation, motion navigation primitive (MNP), dynamic phase vector (DPV) and motion rotation variable (MRV), have been derived for various sensing tasks. Albeit promising, since feature extraction and selection plays a key role in system performance, hand-crafted features are usually task-specific, not reusable for new tasks, hindering its usage for ubiquitous sensing.

Automatic deep pattern extraction-based sensing. Since it is challenging to devise effective sensing feature, more and more works began to leverage various deep learning models for better accuracy and robustness, such as Convolution Neural Network (CNN) and Recurrent Neural Network (RNN). As seen in Table 10, the combination of CNN and RNN has been widely adopted in recent works [27,30,32,39,73,77,79] due to its advantage in extracting spatial-temporal feature from CSI signal automatically. Besides, to gain more general representation learning, adversarial learning and few-shot learning have also been used for efficient and robust feature training[29,31,34,43,53,57,60,61]. The end-to-end property of deep learning has made network framework selection and design become the primary factor in sensing system implementation.

Apart from the above differences, we can gain several more findings from Table 1 to Table 10. First, apart from CSI amplitude and phase information, several new base signals, such as BVP of CSI, ACF of CSI and CSI ratio, have been used for alleviating the intrinsic errors of COTS WiFi devices [88]. Among these base signals, CSI ratio is drawing more attention since it can not only remove CSI offset, but also increase the sensing signal-to-noise rate (SNR) [89]. Second, some works have tried to combine pattern-based scheme with model-based scheme to ensure the performance and reliability of complex sensing applications. Third, many systems are developed for single human sensing under constrained deployment, i.e., single room sensing area with LOS condition satisfied.

4. Challenges

Despite of the above endeavors devoted to bring WiFi sensing from laboratory study to real-life applications, either by improving sensing granularity or exploring application scenarios, most of existing works still face great practical challenges. This section presents the challenges and related solution explorations.

Domain dependent issue. As the superposition result of multi-path signals, WiFi is highly sensitive to various factors, such as locations, orientations, targets, environments, also known as the domain-dependence problem [15,18,86]. In order to make WiFi sensing robust in different settings, researchers have explored various methods, as summarized in Table 11. It can be seen from the table that domain-independent feature extraction is most studied, which can be used alone or further integrated with other methods such as transfer learning and data augmentation. To guarantee the robustness and generalization of WiFi sensing, further investigations are needed regarding signal processing techniques and machine learning algorithms.

Sensing range limitation. As declared in last section, existing sensing range is usually just 6-8 m within a single room, while the communication range of WiFi can reach tens of meters. This small sensing range greatly hinders the real-world house environment and several researches have been devoted to push the sensing range limit. FarSense [90] first increased fine-grained sensing range to 8 m using CSI ratio signal, while Zeng et al. [91] and DiverSense [92] further boosted sensing range to 18 m and 40 m by fully utilizing the spatial and frequency diversity. Wang et al. [93] studied the effect of device placement on sensing SNR and doubly expanded the sensing range by properly placing the transmitter and receiver. Sensing range enlargement is still in its infancy and requires further validation in complex real-world scenario.

5. Future research trend discussion

Despite great effort spent on WiFi sensing over the past years, there still exists a great gap for pervasive real-life application. Based on the detailed analysis above, we point out three critical barriers that require further research in this section.

Sensing assessment standardization. One key issue is the lack of standard performance evaluation of various WiFi sensing systems. Unlike widely accepted standard evaluation criterion in computer vision domain, there still lack of effective and consistent testing platform in WiFi sensing. Specifically, the deficiency exists in two aspects, i.e., CSI extraction tool diversity and evaluation dataset scarcity. The diversity of CSI extraction tools is shown in Table 12, with Intel 5300 NIC-based 802.11n CSI Tool being the most popular used. However, sensing techniques developed with old 802.11n protocol have not explored the innovations of newer standards and may even fail on new-generation WiFi cards [108,109]. Besides, as illustrated in Table 13, although there have been some public released datasets, none of them have been widely used. Existing works mostly adopt self-collected dataset collected in different scenarios with different tools, hindering the comparability and replicability of research outcomes. To build comprehensive datasets without labor-intensive and time-consuming efforts, researchers have studied radio signal synthesis [110,111] and physical data augmentation [112], providing promising solutions to the data scarcity problem. We believe a more unified CSI extraction tool compatible with new 802.11 standard and a set of standard datasets for benchmark comparison should be indispensible for the further research cooperation and development of WiFi sensing.

Sensing and communication balance. As illustrated in Table 14, most sensing systems require high sampling rate for reliable performance which will interfere with regular WiFi communication. To be more specific, as shown in Figure 5, the data throughput will undergo great drop when the sampling rate for sensing is higher than 50Hz. SenCom [113] managed to extract CSI from general communication packets, and obtained evenly sampled and sufficient CSI data with detailed signal processing technique. While appealing, SenCom is not yet applicable for COTS clients. Thus, how to enable WiFi sensing while maintaining communication capability, i.e., achieving sensing and communication balance, remains an open problem in current ISAC area.

Sensing generalization and reliability. As noted in Table 12, raw CSI reading is still only accessible with limited hardware, some researchers resorted to sensing with other WiFi signals. For instance, since beamforming feedback matrix (BFM) is readily available with all new-generation MU-MIMO-enabled WiFi cards, researchers have explored generalized WiFi sensing using BFM [114,115]. Besides, to improve the reliability of sensing, multi-modal sensing which integrates WiFi and other sensing modality, e.g., video, is worth studying [32,52,81,116].

6. Conclusion

Owing to the active participation from numerous researchers, notable advances have been made in WiFi sensing techniques in recent years. In an effort to gain insight of future trending, this paper reviews major achievements over the last 5 years and carries out an in-depth analysis of various methods, including limitations and practical challenges faced in existing systems. Moreover, to realize massive real-life applications, this paper highlights three imperative and promising future directions: sensing assessment standardization, sensing and communication balance, sensing generalization and reliability. We hope this review work can help people better understand the progresses and problems within current WiFi sensing research field, inspiring more amazing ideas for the upcoming ubiquitous ISAC.