Passive Radar Sensing for Human Activity Recognition: A Survey

Abstract

Continuous and unobtrusive monitoring of daily human activities in homes can potentially improve the quality of life and prolong independent living for the elderly and people with chronic diseases by recognizing normal daily activities and detecting gradual changes in their conditions. However, existing human activity recognition (HAR) solutions employ wearable and video-based sensors, which either require dedicated devices to be carried by the user or raise privacy concerns. Radar sensors enable non-intrusive long-term monitoring, while they can exploit existing communication systems, e.g., Wi-Fi, as illuminators of opportunity. This survey provides an overview of passive radar system architectures, signal processing techniques, feature extraction, and machine learning's role in HAR applications. Moreover, it points out challenges in wireless human activity sensing research like robustness, privacy, and multiple user activity sensing and suggests possible future directions, including the coexistence of sensing and communications and the construction of open datasets.

I. Introduction

The population of individuals aged 65 years or older has been steadily increasing worldwide [1]. The rapidly aging population is pushing toward novel human-centric healthcare models known as assisted living that leverage modern technologies to continuously monitor patients' health status in their homes. This approach can improve the quality of life of elderly people and those suffering from chronic diseases including Parkinson's, dementia, epilepsy, and multiple sclerosis, extend independent living, and minimize disruptions to daily routine. Moreover, assisted living technologies enable timely identification of deteriorating health conditions, promoting a “proactive care” approach rather than responding to serious medical incidents once they have occurred [1], [2].

To monitor the well-being of vulnerable people, several sensing technologies, including wearable sensors, vision- and sound-based sensors, and pyroelectric infrared (PIR) sensors, have been proposed. However, these technologies require dedicated devices to be carried by the user with potential issues of discomfort and compliance, raise privacy concerns due to the recording of plain images of the monitored subject, or can be easily impacted by environmental conditions [1], [3]. Recently, radar sensing has been considered a suitable technology for assisted living thanks to its contactless and non-intrusive monitoring capabilities. Furthermore, radars are not restricted to unobstructed line-of-sights to the monitored subject and can leverage existing communication systems, such as Wi-Fi, as illuminators of opportunity in passive radar sensing approaches. Radar applications in healthcare include monitoring of vital signs, such as respiration and heartbeat, analysis of gait patterns, classification of activities, and detection of critical events, such as falls [1].

Related literature on human sensing examines both active and passive radars. Active radars employ dedicated transmitters and high-bandwidth waveforms optimized for radar detection. However, they require access to the available radio frequency (RF) spectrum, increasing the possibility of interference with existing systems [1]. In contrast, passive radars exploit signals from third-party transmitters, resulting in lower power consumption and lower hardware costs while avoiding the need for spectrum allocation. Nonetheless, passive radars lack control over transmitted signals, which are typically optimized for efficient data transmission rather than radar detection performance [1], [4], [5]. Moreover, variations in wireless signal characteristics, such as bandwidth, signal strength, and data rate, negatively affect detection accuracy [6], while the limited bandwidth poses challenges for indoor monitoring [7].

In passive operation, the radar sensing system correlates the signal received directly from a third-party transmitter with the echoes reflected from the subject to extract information on their movements [1]. Human motion will alter the channel parameters in terms of frequency shift, propagation paths, and signal attenuation [7] and lead to different patterns within the received signal that can be used for wireless sensing applications. The overall system for passive radar-based sensing consists of five fundamental steps, as shown in Fig. 1.

Fig. 1.

Procedure for human activity recognition with passive radar.

This paper aims to provide an overview of human activity recognition (HAR) with passive radar sensors by summarizing the main techniques reported in the literature from four perspectives: data collection, signal processing, feature extraction, and machine learning. Current surveys discuss passive radar within RF sensing techniques, leaving both theoretical and practical aspects of passive radar-based HAR unaddressed. The core contribution of this survey is as follows:

• 1)It demonstrates the potential of passive radar for HAR and the signal processing, feature extraction, and machine learning techniques that can be employed for sensing.
• 2)It also highlights different research challenges associated with this technology and suggests possible future research directions.

The paper is organized as follows: In Section II, the radar system architecture and available datasets are presented. Section III summarizes signal processing techniques, and Section IV covers feature extraction methodologies. Machine learning and deep learning algorithms for HAR are reviewed in Section V. Section VI outlines some outstanding challenges and future directions, with final conclusions drawn in Section VII.

II. Data Collection

A. System Overview

A typical passive radar system consists of two channels: the reference channel captures the transmitted signal of opportunity, whereas the signal reflected from the target is collected by the surveillance channel [8]. The reference channel is connected to a high-directional antenna steered toward the transmitter of opportunity to avoid multipath and effectively recover the transmitted signal [9], and the surveillance antenna points toward the monitored area, as illustrated in Fig. 2. It is expected that the signal reflected from the target will be masked by the direct signal, as described in Section III. When a digital transmitter is exploited, the original transmitted signal can be recovered directly from the surveillance channel, thus avoiding the use of a reference receiver, and resulting in a significant reduction in system complexity [4]. Moreover, the surveillance receiver can be multi-channel to increase spatial information [7], [9].

Fig. 2.

Typical layout for in-home passive radar sensing. Components include a transmitter of opportunity, e.g., a Wi-Fi access point, a reference antenna, and a surveillance antenna connected to relevant channels. Both surveillance and reference antennas capture signals from the third-party transmitter, with the surveillance signal containing an added Doppler shift due to human movement.

Most of the radar prototypes reported in the literature are built on software defined radio (SDR) platforms. Wi-Fi access points are commonly used as opportunistic transmitters due to their wide availability in residential environments. However, Wi-Fi signals are highly dependent on network usage, and variations in the data rate during the idle state adversely affect HAR accuracy [10]. Energy harvesting (EH) transmitters are considered as an alternative option for passive sensing since they emit constant signals with significantly higher power. In [6], Wi-Fi signals were utilized for occupancy detection (coarse sensing), while EH signals enabled fine-grained activity recognition. Both techniques provide narrow-bandwidth signals, resulting in insufficient range resolution for indoor detection.

Fig. 3 depicts the main subsystems of an experimental passive radar prototype designed for HAR applications, including signal processing, feature extraction, and classification modules. Data sampling and signal processing are performed in real-time, while feature extraction and classification are implemented offline. In [11], [12], a pipeline processing flow was proposed for optimizing data throughput in real-time operations. The radar data processing flow is divided into sub-flows, which are allocated to separate threads. For optimal performance, approximately equal processing times for each subroutine should be preserved.

Fig. 3.

Block diagram of an experimental passive radar system for evaluating the performance of human activity recognition algorithms. Passive radar prototypes for data collection are typically built on software defined radio platforms. Signal processing, including noise reduction and Doppler information extraction, is performed in real time, while feature extraction and activity classification are implemented in the offline phase.

B. Applications and Datasets

Experiments for passive radar sensing systems are typically carried out with real-time data obtained in controlled environments. The number of participants ranges from 1 to 6, and the types of activities evaluated in each study are typically less than 6. Daily living activities, including walking, jumping, sitting down in a chair, and picking up an object, hand gestures (such as waving), and critical events (like falls) are considered. It is envisioned that these simple actions can be combined to detect macro-activities, such as food preparation or getting dressed, and identify anomalies in the usual routine associated with worsening health conditions [1].

To the best of the authors' knowledge, only one dataset for passive radar-based HAR is available. The OPERAnet dataset includes passive Wi-Fi radar (PWR) Doppler spectrograms along with data from Wi-Fi channel state information (CSI), ultra-wideband, and Kinect systems. The dataset was created with the intention of evaluating multimodal data fusion networks for HAR using data from synchronized RF and vision-based sensors [13].

To address the issue of limited available data, the authors of [14] developed SimHumalator, a PWR simulator for generating micro-Doppler spectrograms for daily activities by integrating animation data from motion capture systems with IEEE 802.11 compliant Wi-Fi transmissions. The proposed simulator, though, does not model environmental factors such as noise, multipath, occlusions, and propagation loss, resulting in “clean” spectrograms. In [15], they used generative adversarial networks (GANs) to add noise to the simulated spectrograms.

III. Signal Processing

Passive radar receives two types of signals: a reference signal and a surveillance signal . The signals are processed to extract range and Doppler information, discriminate moving targets from unwanted stationary returns in the presence of a strong direct signal component, and recognize the monitored subject's activity.

A. Range-Doppler Correlation

Range and Doppler information are derived by using Cross Ambiguity Function (CAF) processing [10], given as

where , , , and denote the delay, Doppler shift, integration time, and conjugate operation, respectively. The parameters and provide an estimate of the range and Doppler shift of detected targets. The evaluation of CAF for a passive radar represents one of the most computationally expensive operations due to the long integration time required to achieve high Doppler resolution. An efficient implementation of CAF is obtained by performing the fast Fourier transform (FFT) of cross-correlated reference and surveillance signals [10], [12]. The computational complexity can be further reduced by using suboptimal algorithms for real-time processing. The batch processing method operates by dividing the signals into isometric segments and applying CAF in each batch [11], [12], [35].

In passive operation, Doppler resolution is inversely proportional to integration time as , allowing for potential resolution adjustment to detect activities. Range resolution is defined as . Due to the limited bandwidth of the reviewed wireless signals (20 MHz for Wi-Fi), range resolution is limited to 7.5 m, which is insufficient for in-home monitoring [7], thus Doppler information is typically used. A Doppler spectrogram is generated based on a group of range-Doppler plots by selecting the range column containing the maximum Doppler shift [22]. Examples of Doppler spectrograms obtained through a passive Wi-Fi radar for a human performing common activities such as walking, sitting, and standing are presented in Fig. 4.

Fig. 4.

Examples of PWR Doppler spectrograms from the OPERAnet dataset [13] for a volunteer performing the following activities: (a) walking, (b) sitting on a chair, standing from a chair, walking, lying down, standing from the floor, lying down, standing from the floor, and then walking. Only a 60-second segment of the recordings is considered.

Experimental results [10], [36] revealed that the low frame rates during Wi-Fi idle status negatively impact the generated Doppler signatures and potentially affect HAR accuracy. To address this issue, a modified CAF was proposed in [10], [31]. Their idea involves extracting the beacon signal before the application of CAF to ensure that only useful data is processed. Moreover, [34] presented an algorithm utilizing ESPRIT for estimating range and velocity, while [32] introduced an iterative adaptive algorithm for Doppler resolution enhancement.

B. Direct Signal Cancellation

The strong direct signal between the surveillance antenna and the transmitter of opportunity causes significant interference in passive radar systems, as it masks the desired target's echo, degrading the system's detection sensitivity. Direct signal interference (DSI) along with the signal reflected from stationary objects creates an unwanted peak of high energy in the zero-Doppler bin of the CAF surface. To suppress this interference, most of the studies employ the CLEAN algorithm proposed in [24]. The principle is to iteratively subtract the scaled and phase-corrected self-ambiguity surface of the reference signal from the original CAF surface. The cleaned CAF surface at the -th iteration can be written as [10]

where and are the amplitude and phase shift of the maximum peak of the -th CAF surface amongst the zero-Doppler line.

In [4], the extensive cancellation algorithm (ECA) was used to remove DSI. ECA operates by subtracting from the surveillance signal properly scaled and delayed replicas of the reference signal; however, this approach has high computational load since it is applied in the signal domain rather than the CAF surface. It should be noted that batch processing is applicable to both CLEAN and ECA.

TABLE I. Summary of Available Datasets and Simulation Software.

Reference	System	Configurable parameters	Available data	Application
OPERAnet Dataset [13]	PWR, Wi-Fi CSI, ultra-wideband, Kinect	N/A	Approximately 8 hours of annotated measurements from 4 sensors, PWR data is in the form of Doppler spectrogram	Activity recognition (6 daily living activities) and crowd counting
SimHumalator [14]	PWR	Location and rotation of the target, waveform parameters, monostatic or bistatic radar configuration	CAF map, Doppler spectrogram	Activity recognition (11 daily living activities)

TABLE II. Summary of Signal Processing Techniques.

Category	Examples
Range-Doppler correlation	CAF [4], [5], [6], [7], [10], [11], [12], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], modified CAF with synchronized beacon signal [10], [31]
Direct signal cancellation	CLEAN [6], [7], [10], [11], [15], [16], [17], [18], [21], [23], [24], [26], [27], [31], ECA [4], [5], [32]
Post-processing	CFAR [5], [6], [7], [10], [12], [16], [24], [31], “bad time index” removal [18], beacon signal removal [32], start and end point detection in the micro-Doppler signature [19], [20], motion indicator based on the power of the Doppler spectrogram [21], [22], [23]
Other techniques	Doppler shift extraction method for OFDM signals based on data equalization, re-encoding and FFT [33], range and speed estimation with ESPRIT [34], iterative adaptive approach based on weighted least square for micro-Doppler extraction [32]

TABLE III. Summary of Features Extraction Methods.

Category	Examples
Physical features	activity duration [21], [22], max Doppler shift [21], [22], peak-to-peak bandwidth [21], [22], mean power of Doppler [21], [22], standard deviation of Doppler power [21], [22]
Time-Frequency domain features	Doppler spectrogram [6], [7], [15], [16], [17], [20], [26], [27], [28], [29], [38], Doppler strength [6], Doppler shift of the dominant peak at each time index [18], spectral power along Doppler frequency [18], PCA [19], [20], [21], [25], SVD [21], [23], DCT [25], entropy [25]
Machine/Deep learning-based features	HMM [23], CAE [25], CVAE [25]

TABLE IV. Summary of Machine Learning and Deep Learning Techniques.

Category	Examples
Machine learning	Supervised learning: SVM [18], [20], [21], SRC [19], [20]
	Unsupervised learning: HMM [22], K-means [23], [25], K-medoids [23], [25]
Deep learning	CNN [7], [13], [15], [16], [27], VGG-16 [6], [17], LeNet [10], [26], AlexNet [20], [26], [28], ZFNet [26], ResNet-18 [13], LSTM [6], Transformer [38]
Other techniques	transfer learning [20], [28], [29] fusion techniques [5], [7], [13], [38], data augmentation [14], [15], [17]

C. Post-Processing

After CAF processing and DSI cancellation, the remaining signals contain noise and comprise both motion and non-motion segments. To reduce the noise levels, a constant false-alarm rate (CFAR) algorithm, which is widely used in active radar detection systems, has been employed in several studies [6], [7], [10], [12], [16], [24], [31]. The authors of [18] used an empirically determined threshold of the Doppler power to remove any “bad time index” (i.e., errors caused in the correlation process due to irregular Wi-Fi transmissions) from the Doppler spectrogram.

Precise segmentation, i.e., identifying the start and end timestamps for each action inside the received signal sequence, is significant for accurate feature extraction and activity recognition. The detection of active and inactive intervals is mainly based on thresholds. The works [19], [20] used the weighted standard deviation to detect the start and end points of motion patterns in the micro-Doppler spectrogram, while the authors in [21], [22], [23] implemented a low-complexity motion indicator based on Doppler power to discriminate active and static periods. In [21], the motion indicator was used as a cognitive mechanism, allowing the system to adapt to multiple motion levels with different Doppler resolutions. Moreover, Kruse et al. [37] proposed a segmentation method based on Renyi entropy. Segmentation of sequences into single-activity segments is achieved by detecting rapid changes in the entropy of micro-Doppler spectrograms.

IV. Feature Extraction

Feature extraction is a core step in HAR, because it affects the robustness and accuracy of recognition. It involves identifying a subspace of the original feature space that retains significant information while reducing dimensionality to minimize classifier complexity. Several studies [6], [7], [15], [16], [17], [20], [26], [27], [28], [29], [38] utilized Doppler spectrograms directly in the subsequent classification, while others extract features from the Doppler signatures. Physical features provide physically meaningful information about the Doppler spectrogram (e.g., the duration of an activity or the bandwidth of the Doppler peak), allowing for interpretation of the results. However, they often lack diversity, which degrades classification performance. Dimensionality reduction techniques, such as principal component analysis (PCA) and singular value decomposition (SVD), have been used in radar-based activity recognition applications to remove redundant information [19], [20], [21], [23], [25]. In [21], [23], SVD was shown to provide better performance than PCA and physical features.

The authors of [25] proposed discrete cosine transform (DCT) and entropy-based methods for feature extraction from local areas of pre-processed Doppler spectrograms. Entropy-based analysis exploits different entropy values arising from color intensity variations to quantify the different Doppler patterns of activities. Through the suggested patching strategy, areas of micro-Doppler spectrograms containing slight variations or insignificant information are excluded from the analysis while retaining the significant locally extracted features. For both methods, the optimum patching strategy is determined based on Dunn's index.

Recently, machine learning-based methods have been used for automatic feature extraction from Doppler signatures. The work in [23] employed hidden Markov models (HMM) to generate log-likelihood values as a measure of similarity between Doppler sequences. This approach eliminates the need for a predefined sliding window, which is a limitation of traditional feature extraction methods. Although the log-likelihood values can be used to directly measure the similarity between activities, the authors chose to use them as features. In [25], convolutional autoencoder (CAE) and convolutional variational autoencoder (CVAE) networks were used to extract features from Doppler radar data. The latter achieved great accuracy, but the high computational time needed for it prohibits real-time implementation.

V. Machine Learning

Several machine learning (ML) and deep learning (DL) techniques have been presented for human presence detection and activity recognition. Multiple algorithms for a specific task are often used and compared to identify the best-performing algorithm. Models' performance is typically assessed by metrics such as accuracy, sensitivity, specificity, or computational complexity. However, it is not possible to compare the results of different studies due to the diversity of datasets used.

Supervised learning techniques, including support vector machine (SVM) and sparse representation classifier (SRC), have been explored for HAR using conventional, sliding-window extracted features from Doppler spectrograms [18], [19], [20], [21]. The authors of [21] also evaluated the performance of the SVM classifier in an inter-subject scenario. Classification accuracy was slightly below 80%, suggesting that activities for a new user can be recognized based on existing data. However, activity recognition based on annotated data is challenging in real-world situations where the subject performs a wide range of activities.

The potential of using unsupervised learning techniques to improve recognition accuracy in the absence of labeled data has been investigated by a research group. In [22], an HMM was utilized to recognize five daily activities. The HMM structure includes the activity types as hidden states and the physical features extracted from Doppler spectrograms as observation states. In [23], [25], unsupervised clustering was performed with the K-means and K-medoids algorithms. Since an unsupervised learning framework was adopted, the authors of [25] applied four metrics'Elbow, Silhouette, Davies-Bouldin, and Dunn's index'to automatically detect the number of classes.

Deep learning techniques, including convolutional neural networks (CNNs) and recurrent neural networks (RNNs), have been explored. DL approaches extract high-level features automatically but require more training data. In [6], [7], [10], [13], [15], [16], [17], [20], [26], [27], [28], CNN architectures were evaluated for tasks involving activity recognition and people counting using spectrograms as input data. In [6], the long short-term memory (LSTM) network was used to detect occupancy from Doppler data, leveraging the network's ability to capture data temporal correlation.

Transfer learning (TL) can be utilized to reduce training time and improve a DL model's performance. TL works by using pre-trained weights in the first layer and fine-tuning the subsequent layers. The authors of [20] proposed a deep transfer network based on the AlexNet architecture, pretrained on ImageNet and fine-tuned for micro-Doppler classification. In [28], the network was trained on Wi-Fi data transmission measurements and subsequently fine-tuned on micro-Doppler spectrograms from probe response signals. Tran et al. [29] applied TL using the VGGish model trained on audio and compared its accuracy with TL using the VGG-16 model trained on images. Their method demonstrated higher accuracy compared to image-to-image knowledge transfer.

Recently, multimodal and multi-sensor fusion methods have been studied for HAR. Sensor fusion enables the joint exploitation of information acquired through diverse sensors to better describe human actions and enhance recognition accuracy. [7] presented a probability-level fusion technique using CSI and PWR spectrograms, while [13] introduced a sensor fusion network by concatenating data from RF and vision-based sensors. In [38], a transformer-based model was implemented that can fuse multiple image-based features to recognize six daily activities using the same dataset. [5] presented a sensor fusion method based on the interacting multiple model (IMM) algorithm using PWR and Wi-Fi emission-based measurements. Moreover, the authors of [14], [15], [17] presented data augmentation schemes using synthetic Doppler signatures to address the problem of insufficient training data. Experimental results revealed that this approach can improve classification performance when dealing with limited or unbalanced experimental data.

VI. Challenges and Future Research Directions

Although passive radar sensing has been attracting interest for healthcare applications, several challenges require further exploration.

• •While the reviewed applications exhibit sufficient accuracy, they have been tested in controlled laboratory environments. Therefore, it is necessary to evaluate these algorithms in real-world settings with different clutter and multipath phenomena, sensor placements, and diverse end-users. This approach ensures that classification algorithms can generalize well to unseen conditions.
• •Most studies classify activities using recordings of isolated activities with predetermined duration and clear transitions. However, this approach does not reflect human motion patterns, which are continuous, diverse, and uninterrupted. To address this, classification techniques should transition from analyzing separate movements to analyzing continuous streams. Initial findings in continuous HAR with active radar are documented in [39].
• •Passive radar-based HAR in multi-user environments presents significant challenges, as radars receive a mixture of motion patterns in crowded sensing areas. Therefore, separation techniques are required to isolate independent motion signatures from the combined signal. Recent research on active radar systems demonstrated that blind source separation and direction of arrival estimation techniques could support multi-subject sensing [40].
• •Passive radar technology also raises serious privacy concerns as its non-intrusive nature implies that people may be unaware of the existence of wireless sensing. Hence, security strategies should be included in the passive radar system design.
• •Existing PWR systems often modify the sender's data rate for improved performance. In realistic scenarios, a joint communication and radar sensing framework should be developed so that the same wireless signals are utilized for information transmission and radar detection operations.
• •DL is increasingly used in radar for automatic feature extraction and classification, but it demands significant computational resources, posing challenges for edge device deployment. Moreover, the complexity of DL models makes it hard to comprehend classification decisions. Explainable artificial intelligence techniques, which have been used in fields like medical imaging [41], will allow interpretability of the results.
• •Another issue is the lack of comprehensive open datasets for the benchmarking of different approaches for passive radar-based HAR. Since most of the studies evaluate algorithms' performance using their dataset, systems highly depend on the data collection procedures, which hinders comparison between studies.

VII. Conclusion

This survey provided an overview of passive radar system architectures, signal processing methods, feature extraction, and machine learning-based activity recognition. The findings from various studies suggest that passive radar shows significant promise for assisted living applications. Future research should address several challenges, including multi-subject sensing, continuous activity recognition, efficient DL approaches, advanced radar configurations for enhanced performance and privacy, and the integrated use of wireless signals for communication and sensing.

Supplementary Materials

The supplementary materials offer a thorough summary of the techniques outlined in the reviewed papers.

Funding Statement

This work was supported by the National Recovery and Resilience Plan “Greece 2.0” funded by the European Union -- NextGenerationEU through Hellenic Foundation for Research & Innovation (H.F.R.I.) call “Basic Research Financing (Horizontal support of all Sciences)” under Project 15642.