Time-frequency gated transformer for structural dynamic response reconstruction

Abstract

Structural dynamic response reconstruction is of significant importance because data incompleteness critically undermines the reliability of structural health monitoring. Existing deep-learning reconstruction methods focus solely on the time-domain features of structural dynamic responses and neglect their frequency-domain counterpart. This reduces their reconstruction accuracy because the frequency-domain features directly reflect the inherent dynamic characteristics of structures. To address this problem, we propose a time-frequency gated transformer (TFGFormer) leveraging both the time-domain and frequency-domain features. Specifically, multi-scale dilated transformers and dense blocks are respectively used to extract the time-domain global and local features because the former sparsely encode long-range dependencies and the latter aggregate multi-layer representations. Concurrently, frequency modules are used to extract the frequency-domain features because they are able to adaptively learn the spectral complex-valued weights. Finally, gate modules and a joint time-frequency loss function are used to adaptively fuse these features because the former balance dynamically their contributions and the latter simultaneously penalizes their residuals. Validated on full-scale field data from the Canton Tower and the Hardanger Bridge, TFGFormer outperforms representative baseline methods across various data-missing scenarios and enjoys zero-shot robustness and lightweight design. This reconstruction-oriented time-frequency framework may support more reliable downstream SHM tasks.

Introduction

Structural health monitoring (SHM)^1,2 is fundamental to ensuring the safety and durability of critical infrastructure, such as super-tall buildings and long-span bridges. Given the challenges of aging infrastructure and more frequent extreme loading events, SHM enables real-time risk warnings and facilitates a shift from periodic to data-driven predictive maintenance, thereby significantly enhancing structural safety and operational cost-effectiveness throughout the entire lifecycle of structures^3–5. Reliable SHM decision-making hinges on the completeness of captured structural dynamic responses. However, sensor malfunctions and environmental interference inevitably lead to data loss in real-world applications^6–8. Furthermore, sensors in large-scale structures are often embedded within structural components, rendering them virtually irreparable, which means that a single embedded sensor failure results in the permanent loss of all information at that specific node. More critically, even small-scale random data loss is highly detrimental. Studies^9,10 have shown that data loss as low as 2.5% can degrade downstream tasks as severely as 10% noise. Therefore, high-precision and robust reconstruction of structural dynamic responses is a prerequisite for SHM reliability.

Related research is categorized into model-driven and data-driven approaches. Model-driven ones utilize finite element models (FEMs)^11–13 to recast the reconstruction problem as an unknown-input estimation problem. However, their accuracy is frequently compromised by complex structural configurations and material uncertainties, thereby largely restricting their application. Data-driven ones encompass statistical methods, compressive sensing (CS) and deep learning (DL). Statistical methods^14,15 recover missing data by fitting regression models to measured structural dynamic responses. However, these methods are constrained by rigid assumptions and lack the robustness required for the stochastic data characteristic of engineering environments. CS^16–18 exploits sparsity and Bayesian priors techniques, but it inherently fails to handle the non-ideal sparsity and severe noise levels in real-world data. Driven by powerful nonlinear representation capabilities and end-to-end learning, DL^19–21 has emerged as the dominant method^22–24. The evolution of DL fundamentally centers on effectively modeling the inherent time-domain correlations within structural dynamic responses and has evolved along three main paths. The first path encompasses convolutional neural network (CNN)-based methods-including generative adversarial network (GAN)^9,25,26, fully convolutional neural network (FCNN)^27–29 and U-net^30,31. These methods leverage local connectivity and weight-sharing to extract local short-range transient features. Notably, Fan et al. proposed a densely connected network (DenseNet)³² to establish a benchmark for modeling nonlinear mappings through efficient feature reuse. Jiang et al. proposed a deep fully convolutional network (DFCN)³³ which frames data recovery as conditional probability generation to capture semantic features. However, the restricted receptive field of convolution severely limits their capacity to capture global long-range temporal dependencies-a critical flaw, as wave propagation and modal dynamics rely heavily on such global dependencies. For instance, super-tall buildings like the Canton Tower^34,35 essentially operate as towering cantilever structures whose dominant vibration modes are governed by bending and shear. Given the complex global coupling inherent in such structures between the apex and base, the extraction of global features takes precedence over localized perception. The second path comprises transformer-based methods^36,37 which utilize self-attention^38,39 to dynamically capture global long-range temporal dependencies. Notably, Liang et al. proposed SwinIR⁴⁰ based on shifted window self-attention. Jiao et al. developed DilateFormer⁴¹, employing multi-scale dilated attention (MSDA) to sparsely capture cross-scale global dependencies. However, the absence of inductive bias in transformer hinders the explicit modeling of local continuity, which overlooks critical local short-range transient features. For instance, on long-span structures like the Hardanger Bridge^42,43, the extensive span dictates that structural dynamic responses within localized regions exhibit profound temporal correlations and dependencies between distant extremities are markedly weaker. Therefore, for structures with such pronounced local wave propagation effects, transformer underperforms CNN in reconstruction accuracy. Hence, the third path shifts toward hybrid CNN-Transformer architectures by fusing CNN for localized extraction with transformer for global modeling. Notably, Yang et al. developed a parallel mixed Transformer-CNN (PMTC)⁴⁴ network which decouples the conventional serial dependency between convolutional and transformer layers. By simultaneously extracting both the local short-range transient features and global long-range dependencies in the time domain, PMTC has become a representative advanced model.

In summary, existing deep learning-based structural dynamic response reconstruction methods have made significant progress in time-domain modeling, but they suffer from a widely overlooked limitation: they focus solely on the time-domain features of structural dynamic responses and neglect their frequency-domain counterparts. The frequency domain serves as the intrinsic physical fingerprint of a structure, governing essential dynamic characteristics such as modal frequencies and stiffness. Consequently, methods that focus solely on time-domain features often degenerate into unconstrained curve-fitters, successful at minimizing the training loss to yield visually plausible waveforms in the time domain, yet they suffer from severe spectral distortion in the frequency domain, which fundamentally undermines reconstruction accuracy. Therefore, developing a method that synergistically leverages both time-domain and frequency-domain features is essential for achieving high-accuracy structural dynamic response reconstruction. A related time-frequency method⁴⁵ has recently been reported for electroencephalogram (EEG)-based seizure detection. This method encodes raw EEG signals and continuous-wavelet-transform representations as two parallel streams and fuses high-level attention features through a sigmoid gate before classification. Although this method supports the value of time-frequency fusion, the framework is primarily designed for classification. Its frequency-domain features are generated by predefined wavelet transforms, the dual streams are fused at a high-level feature stage before classification, and the optimization objective targets label prediction rather than signal reconstruction. Consequently, this paradigm does not directly meet the specific requirements of structural dynamic response reconstruction, which requires recovering continuous responses at missing sensor-time locations, preserving available measurements, and explicitly constraining the frequency-domain consistency of the reconstructed waveforms.

Inspired by this, we propose a time-frequency gated transformer (TFGFormer) that leverages both the time-domain and frequency-domain features to shift the structural dynamic response reconstruction paradigm from purely time-domain optimization toward explicit interactions between their representations. Specifically, multi-scale dilated transformers (MSDFs) and dense blocks are respectively used to extract time-domain global and local features because the former employ attention mechanisms with different dilation rates to sparsely encode long-range dependencies and the latter aggregate multi-layer representations to prevent the smoothing of transient features. Concurrently, the frequency modules are used to extract frequency-domain features that are sensitive to the dynamic characteristics of structures because they adaptively learn complex-valued weights to filter frequency components, thereby amplifying critical spectral features while suppressing environmental noise. Finally, gate modules and a joint time-frequency loss function are used to adaptively fuse these dual-domain features because the former dynamically balance the contributions based on signal quality and the latter enforces a multi-dimensional constraint by simultaneously penalizing temporal residuals and complex-valued spectral deviations. TFGFormer ensures that the reconstructed responses remain not merely superficially accurate in the time domain, but spectrally faithful for downstream frequency-dependent SHM tasks. Validated on full-scale field data from the Canton Tower and the Hardanger Bridge, TFGFormer outperforms representative baseline methods across various data-missing scenarios and enjoys zero-shot robustness and lightweight design. An overview of our work is presented in Fig. 1.

Fig. 1. Motivation and framework of the proposed TFGFormer.

The schematic delineates the transition from practical challenges to the proposed solution. a Problem identification. In structural health monitoring (SHM) of critical infrastructure, extensive data missing from sensor networks results in unreliable data, which undermines downstream tasks such as damage detection and modal identification. b Limitations of existing approaches. Prevailing hybrid CNN-Transformer models focus solely on time-domain features, neglecting the frequency-domain counterpart. This leads to spectral distortion where the reconstructed signal deviates from the ground truth in the frequency domain, thereby reducing their reconstruction accuracy. c Proposed TFGFormer. To bridge this gap, TFGFormer employs a dual-domain architecture to concurrently process time-domain and frequency-domain features, integrated via adaptive Gate Modules. A joint time-frequency loss function is proposed to minimize cross-domain reconstruction errors. Consequently, the reconstructed signal achieves high accuracy to the ground truth across both time and frequency domains.

Results

We evaluate TFGFormer on real-world acceleration response datasets from the Canton Tower^34,35 and the Hardanger Bridge^42,43. Detailed dataset descriptions and evaluation metric formulas including the mean absolute error (MAE), root mean square error (RMSE), relative error (RE), coefficient of determination (R²) and signal-to-noise ratio (SNR) are provided in Supplementary Notes 1-3. We benchmark its reconstruction performance against several representative baseline methods: the CNN-based DenseNet³² and DFCN³³, the transformer-based SwinIR⁴⁰ and DilateFormer⁴¹, and the CNN-Transformer hybrid PMTC⁴⁴. To ensure rigorous and fair comparisons, all models were independently trained and evaluated five times under the identical hardware and hyperparameter settings, and the optimal checkpoint for each was selected based on the minimum validation loss. Results are reported as the mean  ± standard deviation to quantify reconstruction results.

Experiment on the super-tall Canton Tower

Across missing rates ranging from 30 to 70%, our method (TFGFormer) consistently outperforms the five evaluated baselines in reconstructing complex data of the super-tall Canton Tower from the perspectives of the evaluation metric (see Fig. 2 and Table 1). Specifically, CNN-based models (DenseNet and DFCN) leading to a rapid deterioration in MAE (exceeding 3.00 × 10⁻³ at a 70% missing rate) because they fail to capture the global structural modes inherent to the super-tall building under massive sensor loss, constrained by localized receptive fields. Conversely, while pure transformer models (SwinIR and DilateFormer) theoretically possess long-range modeling capabilities, they lack the inductive biases requisite for processing randomly missing responses. This deficiency manifests as significant error dispersion-visually evidenced by wide interquartile ranges and extensive whiskers (Fig. 2c–g)-reflecting their heightened vulnerability to environmental noise. Although the hybrid PMTC architecture partially balances accuracy and stability, the absence of explicit frequency-domain constraints leaves its reconstruction error substantially higher than that of TFGFormer. TFGFormer’s core advantage lies in its time-frequency dual-branch fusion mechanism, which accurately anchors fundamental structural frequencies from residual signals. This mechanism enables the model to achieve a low MAE (0.97 × 10⁻³) at a 30% missing rate. Furthermore, TFGFormer maintains an tight error distribution; remarkably, its maximum error boundary (upper whisker) remains consistently lower than the median errors of all baseline models. This statistical superiority firmly establishes its reliability for monitoring super-tall infrastructure. Additional experimental results are detailed in Supplementary Note 4.

Fig. 2. Performance evaluation of TFGFormer on the super-tall Canton Tower.

a, b Schematic and sensor configuration of the Canton Tower. Twenty accelerometers are strategically deployed across eight primary structural levels. c–g Comparative reconstruction RE across different models under missing rates of 30%, 40%, 50%, 60%, and 70%. Data are presented as box plots overlaid with individual data points from n = 5 independent experimental runs. The horizontal line within each box represents the median, and the black point represents the mean. The box limits indicate the 25th and 75th percentiles, and the whiskers extend to the minimum and maximum values. The solid teal line connects the medians across different architectures to visualize the performance trend. TFGFormer consistently achieves the lowest RE and a small variance among the evaluated methods, indicating stable reconstruction performance under the tested Canton Tower missing-data scenarios.

Table 1.

Performance comparison of reconstruction methods under various missing rates (Canton Tower dataset)

Missing Rate	Method	MAE ( × 10−3) ↓	RMSE ( × 10−3) ↓	RE ↓
30%	DenseNet32	1.12 ± 0.02	3.20 ± 0.05	0.150 ± 0.003
	DFCN33	1.06 ± 0.05	2.99 ± 0.09	0.141 ± 0.004
	SwinIR40	1.46 ± 0.12	2.98 ± 0.23	0.141 ± 0.011
	DilateFormer41	1.09 ± 0.01	3.06 ± 0.03	0.144 ± 0.001
	PMTC44	1.06 ± 0.01	2.90 ± 0.01	0.137 ± 0.001
	TFGFormer (ours)	0.97 ± 0.01	2.68 ± 0.04	0.126 ± 0.002
40%	DenseNet32	1.46 ± 0.00	3.54 ± 0.02	0.167 ± 0.001
	DFCN33	1.48 ± 0.06	3.46 ± 0.06	0.163 ± 0.003
	SwinIR40	1.80 ± 0.03	3.26 ± 0.07	0.154 ± 0.003
	DilateFormer41	1.41 ± 0.01	3.38 ± 0.03	0.159 ± 0.001
	PMTC44	1.40 ± 0.01	3.26 ± 0.03	0.154 ± 0.001
	TFGFormer (ours)	1.28 ± 0.01	2.99 ± 0.03	0.141 ± 0.002
50%	DenseNet32	1.91 ± 0.02	4.06 ± 0.04	0.191 ± 0.002
	DFCN33	1.92 ± 0.11	3.95 ± 0.09	0.186 ± 0.004
	SwinIR40	2.08 ± 0.08	3.56 ± 0.07	0.169 ± 0.003
	DilateFormer41	1.79 ± 0.05	3.75 ± 0.10	0.177 ± 0.005
	PMTC44	1.77 ± 0.03	3.62 ± 0.05	0.171 ± 0.002
	TFGFormer (ours)	1.61 ± 0.01	3.34 ± 0.04	0.158 ± 0.002
60%	DenseNet32	2.46 ± 0.02	4.69 ± 0.04	0.222 ± 0.002
	DFCN33	2.30 ± 0.16	4.29 ± 0.17	0.203 ± 0.008
	SwinIR40	2.47 ± 0.14	4.02 ± 0.13	0.191 ± 0.006
	DilateFormer41	2.22 ± 0.03	4.21 ± 0.06	0.199 ± 0.003
	PMTC44	2.18 ± 0.02	4.02 ± 0.03	0.190 ± 0.002
	TFGFormer (ours)	1.98 ± 0.02	3.75 ± 0.06	0.177 ± 0.003
70%	DenseNet32	3.25 ± 0.04	5.63 ± 0.05	0.266 ± 0.002
	DFCN33	3.01 ± 0.08	5.15 ± 0.08	0.243 ± 0.004
	SwinIR40	3.03 ± 0.08	4.68 ± 0.06	0.222 ± 0.003
	DilateFormer41	2.85 ± 0.04	4.92 ± 0.06	0.233 ± 0.003
	PMTC44	2.82 ± 0.02	4.76 ± 0.03	0.225 ± 0.001
	TFGFormer (ours)	2.53 ± 0.03	4.36 ± 0.05	0.206 ± 0.002

Results are reported as mean  ± standard deviation across 5 independent runs.

To elucidate the reconstruction accuracy and underlying physical mechanisms of various methods, Fig. 3 visualizes typical reconstruction results for a randomly selected test sample (Sensor 08) from the Canton Tower dataset under a severe 70% missing rate. In the time-domain acceleration waveforms (Fig. 3, left panels), all baseline models exhibit varying degrees of degradation when approximating complex non-linear vibrations. Specifically, constrained by localized receptive fields, CNN-based DenseNet and DFCN suffer from significant peak attenuation and temporal lags at critical local extrema (indicated by red arrows). Conversely, while transformer-based SwinIR and DilateFormer expand the perceptual range, their lack of priors causes them to similarly fail in reconstructing these high-gradient transient impact features. In stark contrast, TFGFormer’s reconstruction trajectory aligns with the ground truth. The physical origins of these time-domain discrepancies are elucidated in the frequency-domain amplitude spectra (Fig. 3, middle panels). Because existing baselines optimize exclusively in the time domain, they suffer from high-frequency modal energy dissipation and spectral leakage (highlighted by the inset boxes). Empowered by its frequency modules and joint time-frequency loss function, TFGFormer enforces explicit spectral alignment. This mechanism not only anchors the core natural frequencies but also preserves their corresponding modal energies. The scatter plots (Fig. 3, right panels) further quantify this mechanistic advantage. Ultimately, TFGFormer achieves the highest coefficient of determination (R² = 0.9609) and the lowest error (MAE = 0.0023), yielding the best quantitative performance among the evaluated methods for this test example.

Fig. 3. Comparative reconstruction performance of various models on the Canton Tower dataset under a severe 70% missing rate.

Detailed reconstruction results of a randomly selected test sample (Sensor 08) for DenseNet (a), DFCN (b), SwinIR (c), DilateFormer (d), PMTC (e), and the proposed TFGFormer (f). For each subfigure, the left panel compares the time-domain acceleration of the true (blue line) and reconstructed (orange line) signals across 1,024 sampling points. Red arrows highlight critical local extrema where baseline models exhibit peak attenuation or temporal lags. The middle panel displays the corresponding frequency-domain amplitude spectra, with inset boxes magnifying high-frequency modal regions to illustrate spectral leakage and energy dissipation in the baselines. The right panel presents a scatter plot correlating the true and reconstructed values. Notably, in (a–e), TFGFormer’s predictions (blue dots) are overlaid as a reference benchmark against the respective baseline model (orange dots). Four quantitative evaluation metrics---MAE, RMSE, RE, and R²---are provided. The proposed TFGFormer (f) demonstrates superior reconstruction accuracy, visually evidenced by the tightest diagonal-hugging scatter alignment and quantitatively validated by achieving the highest accuracy (R² = 0.9609) and lowest error metrics.

Experiment on the long-span Hardanger Bridge

Generalization experiments on the long-span Hardanger Bridge (Fig. 4 and Table 2) further substantiate the physical consistency of TFGFormer when processing non-stationary environmental excitations. Characterized by low-frequency, large-amplitude fluctuations and wind-induced dynamics typical of suspension bridges, this dataset poses a severe challenge for feature reconstruction. Although DenseNet and DFCN are relatively robust in preserving local features, their RE metrics at a 75% missing rate (0.098) are significantly inferior to those of TFGFormer (0.060). This performance gap exposes the inherent limitations of traditional convolutions in capturing the long-range temporal correlations vital to long-span structures. Conversely, SwinIR and DilateFormer exhibit drastic performance fluctuations. As documented in Fig. 4, their frequent outliers suggest that without explicit frequency constraints, standard self-attention mechanisms tend to misinterpret the genuine low-frequency oscillations of suspension bridges as random drift. Although the hybrid PMTC architecture demonstrates improved stability, it remains inadequate in capturing ultra-weak vibration peaks. Remarkably, at a 25% missing rate, TFGFormer achieves an low MAE of 0.10 × 10⁻³, improving reconstruction accuracy by approximately 50% over the best baseline models. This improvement is consistent with the intended role of the frequency branch, which is designed to reduce spectral leakage and preserve harmonic components. The tight interquartile ranges exhibited by TFGFormer in Fig. 4 demonstrate its robustness and performance advantage over existing CNNs, transformers, and conventional hybrid architectures when monitoring complex, flexible infrastructure. Additional experimental results are detailed in Supplementary Note 5.

Fig. 4. Performance evaluation of TFGFormer for structural response reconstruction on the long-span Hardanger Bridge.

a, b Schematic and sensor configuration of the Hardanger Bridge. Sixteen accelerometers are distributed along the main span to monitor spatially-correlated vibration patterns of the bridge deck. c–e Comparative reconstruction RE across different models under missing rates of 25%, 50%, and 75%. Data are presented as box plots overlaid with individual data points from n = 5 independent experimental runs. The horizontal line within each box represents the median, and the black point represents the mean. The box limits indicate the 25th and 75th percentiles, and the whiskers extend to the minimum and maximum values. The solid teal line connects the medians across different architectures to visualize the performance trend. TFGFormer maintains high stability and accuracy even under extreme data loss (75%), validating its generalization capability for large-scale horizontal infrastructure.

Table 2.

Performance comparison of reconstruction methods under various missing rates (Hardanger Bridge dataset)

Missing Rate	Method	MAE ( × 10−3) ↓	RMSE ( × 10−3) ↓	RE ↓
25%	DenseNet32	0.23 ± 0.03	0.59 ± 0.04	0.029 ± 0.002
	DFCN33	0.18 ± 0.05	0.42 ± 0.06	0.020 ± 0.003
	SwinIR40	0.44 ± 0.13	0.67 ± 0.22	0.032 ± 0.010
	DilateFormer41	0.34 ± 0.06	0.89 ± 0.14	0.043 ± 0.007
	PMTC44	0.20 ± 0.07	0.52 ± 0.13	0.025 ± 0.007
	TFGFormer (ours)	0.10 ± 0.01	0.28 ± 0.02	0.014 ± 0.001
50%	DenseNet32	0.51 ± 0.06	0.99 ± 0.09	0.048 ± 0.004
	DFCN33	0.42 ± 0.05	0.77 ± 0.04	0.037 ± 0.002
	SwinIR40	0.54 ± 0.05	0.84 ± 0.07	0.041 ± 0.003
	DilateFormer41	0.77 ± 0.06	1.42 ± 0.07	0.069 ± 0.003
	PMTC44	0.40 ± 0.02	0.79 ± 0.04	0.038 ± 0.002
	TFGFormer (ours)	0.31 ± 0.01	0.63 ± 0.01	0.030 ± 0.001
75%	DenseNet32	1.27 ± 0.11	2.02 ± 0.13	0.098 ± 0.006
	DFCN33	0.93 ± 0.06	1.45 ± 0.07	0.070 ± 0.003
	SwinIR40	0.94 ± 0.13	1.44 ± 0.19	0.070 ± 0.009
	DilateFormer41	1.56 ± 0.08	2.45 ± 0.12	0.119 ± 0.006
	PMTC44	0.86 ± 0.02	1.42 ± 0.03	0.069 ± 0.001
	TFGFormer (ours)	0.74 ± 0.01	1.24 ± 0.02	0.060 ± 0.001

Results are reported as Mean  ± Standard Deviation across 5 independent runs.

To further validate the model’s generalization capabilities across distinct physical topologies and dynamical characteristics, Fig. 5 illustrates typical reconstruction results for a randomly selected test sample (Sensor 03) from the Hardanger Bridge dataset at a 25% missing rate. Unlike the high-frequency modal responses of the super-tall Canton Tower, the dynamic behavior of a long-span suspension bridge is heavily governed by low-frequency, large-amplitude flexible fluctuations. In the time-domain waveforms (Fig. 5, left panels), pure transformer architectures (e.g., SwinIR and DilateFormer) struggle with these long sequences characterized by strong local correlations. Lacking physical constraints on local continuity priors, they exhibit pronounced peak attenuation and temporal lags when reconstructing low-frequency large displacements (indicated by red arrows). In contrast, TFGFormer, leveraging its underlying MSDFs and feature reuse mechanisms, closely follows the true non-linear drift trajectories of the bridge deck. This physical consistency, tailored for flexible structures, is more profoundly elucidated in the frequency-domain amplitude spectra (Fig. 5, middle panels). The vibrational energy of suspension bridges is highly concentrated within an ultra-narrow, low-frequency band, imposing stringent demands on the model’s frequency resolution. Existing baselines universally suffer from varying degrees of amplitude attenuation and spectral leakage when reconstructing these sharp low-frequency harmonic components (highlighted by the inset boxes). Conversely, guided explicitly by the frequency-branch gating, TFGFormer achieves a accurate recovery of the fundamental excitation frequencies and their higher-order harmonics, thereby effectively preserving ultra-weak physical vibration features. This enhancement in spectral accuracy translates directly into a robust suppression of anomalous predictions, as visually evidenced by the scatter plots (Fig. 5, right panels). For instance, the self-attention mechanisms of SwinIR and DilateFormer misinterpret partial low-frequency oscillations as random drifts, generating numerous severe outliers that deviate significantly from the diagonal (R² = 0.9359 and 0.9039, respectively). In stark contrast, TFGFormer yields an dense, linear scatter convergence, substantially reducing large deviations. Quantitatively, TFGFormer achieves a peak R² of 0.9755 and an low RMSE of 0.0010, marking a leap in accuracy over the best-performing baselines. This performance not only demonstrates TFGFormer’s precise capture of local wave propagation effects but also supports its applicability under the tested condition for the health monitoring of complex, flexible structures.

Fig. 5. Comparative reconstruction performance of various models on the Hardanger Bridge dataset under a 25% missing rate.

Detailed reconstruction results of a randomly selected test sample (Sensor 03) for DenseNet (a), DFCN (b), SwinIR (c), DilateFormer (d), PMTC (e), and the proposed TFGFormer (f). For each subfigure, the left panel compares the time-domain acceleration of the true (blue line) and reconstructed (orange line) signals across 1,024 sampling points. Red arrows highlight regions where baseline models exhibit peak attenuation or temporal lags. The middle panel displays the corresponding frequency-domain amplitude spectra, with inset boxes magnifying the ultra-narrow, low-frequency band to illustrate spectral leakage in the baselines. The right panel presents a scatter plot correlating the true and reconstructed values. Notably, in (a–e), TFGFormer’s precise predictions (blue dots) are overlaid as a reference benchmark against the respective baseline models (orange dots). Four quantitative evaluation metrics---MAE, RMSE, RE, and R²---are provided. The proposed TFGFormer (f) demonstrates superior reconstruction accuracy, visually evidenced by the tightest diagonal-hugging scatter alignment and quantitatively validated by achieving the highest accuracy (R² = 0.9755) and lowest error metrics.

Robustness against out-of-distribution environmental interference

The experimental data acquired from the in-service Canton Tower inherently contains baseline measurement noise alongside complex operational and environmental variations. While TFGFormer successfully adapts to this conventional background noise distribution during training, real-world SHM systems frequently encounter unpredictable, extreme anomalies that fall completely outside the training domain. To rigorously evaluate the model’s zero-shot generalization capability-specifically, its robustness against unseen extreme conditions-we artificially injected escalating Gaussian random noise exclusively into the test set under a severe 70% missing rate. Physically, this high-intensity, out-of-distribution (OOD) noise simulates sudden environmental interferences-such as severe gust loads, transient sensor failures, or unexpected electromagnetic pollution-none of which the model encountered during training. Detailed parameters of the injected noise are provided in Supplementary Note 6.

Figure 6 presents a overview of the model’s response to these unseen extreme perturbations. In the time-domain, as the injected noise intensity escalates from 10 to 50%, the ground-truth signal becomes severely corrupted by high-frequency broadband jitter. However, despite the drastic amplification of local oscillations in the true signal, the reconstructed trajectory exhibits remarkable dynamic stability. TFGFormer effectively isolates the underlying structural dynamic responses from severe background interference, robustly tracking the true dynamic evolution without being misled by OOD random perturbations. This zero-shot robustness is further corroborated in the frequency domain. Although the injected broadband noise significantly elevates the noise floor across the entire spectrum, TFGFormer persistently anchors and reconstructs the dominant low-frequency structural modes (0–4 Hz). As the unseen noise level increased from 10% to 50%, the reconstruction SNR decreased only slightly from 5.4425 dB to 5.3411 dB. This result indicates that TFGFormer retains relatively stable reconstruction performance under the tested OOD Gaussian-noise perturbations. The reconstructed spectra also preserve the dominant low-frequency modal components within 0–4 Hz. This observation is consistent with the frequency-domain modules and gating mechanism contributing to the suppression of broadband high-frequency perturbations. However, the present experiment considers only one type of synthetic noise, and further evaluation under more diverse environmental and sensor disturbances is needed to assess general deployment robustness. Additional results concerning noise robustness are detailed in Supplementary Note 7.

Fig. 6. Zero-shot robustness evaluation of TFGFormer under extreme out-of-distribution environmental noise.

a–i Time-domain acceleration responses and corresponding frequency-domain amplitude spectra comparing the true (blue line) and reconstructed (orange line) signals. The evaluation is conducted on the Canton Tower dataset under a severe 70% missing rate. To simulate unpredictable environmental anomalies (e.g., wind gusts or electromagnetic interference), out-of-distribution Gaussian noise is injected exclusively into the test set, with intensity escalating from 10% (a) to 50% (i). Despite the increasing broadband perturbation, TFGFormer largely preserves the dominant low-frequency structural modes within 0–4 Hz, while the reconstruction SNR decreases only slightly from 5.4425 dB to 5.3411 dB. These results indicate robustness under the tested OOD Gaussian-noise conditions.

Parameter efficiency and reconstruction accuracy trade-off

To evaluate the practical deployment potential of TFGFormer in resource-constrained scenarios, we analyzed the trade-off between reconstruction accuracy and computational overhead, as shown in Fig. 7. Among the evaluated methods, TFGFormer achieves the highest reconstruction accuracy with a relatively compact parameter count of 1.8 M and a moderate computational load of approximately 110 GFLOPs. Compared with the evaluated transformer-based baselines, it requires fewer parameters and lower FLOPs while maintaining lower reconstruction error. Furthermore, TFGFormer not only outperforms traditional CNNs in absolute accuracy, but it also surpasses recent hybrid architectures (e.g., PMTC) in balancing parameter efficiency and computational cost. These results suggest a favorable accuracy-complexity trade-off among the evaluated methods. Further evaluation of inference latency, energy consumption, and hardware-specific deployment performance is needed before drawing conclusions about real-time edge deployment. Detailed numerical results are provided in Supplementary Note 8.

Fig. 7. Three-dimensional efficiency analysis of TFGFormer versus comparative methods.

The 3D scatter plot maps computational cost (x-axis, in GFLOPs), spatial complexity (y-axis, in millions of parameters), and reconstruction accuracy (z-axis, Accuracy defined as $1-\mathrm{RE}$). To enhance visualization, the bubble size is also scaled logarithmically to reflect the parameter quantity. Accuracy values represent the mean performance derived from n = 5 independent experimental runs on the Canton Tower dataset under a 30% missing rate. The proposed TFGFormer (denoted by the red star) lies on a favorable region of the Pareto frontier formed by the evaluated methods, demonstrating a trade-off between high reconstruction accuracy and low resource consumption when compared against CNN-based, transformer-based, and hybrid baselines.

Discussion

TFGFormer extends existing deep learning-based structural response reconstruction methods by incorporating explicit frequency-domain modeling alongside time-domain feature extraction. Existing methods focus solely on the time-domain features of structural dynamic responses and neglect their frequency-domain counterpart. This reduces their reconstruction accuracy because the frequency-domain features directly reflect the inherent dynamic characteristics of structures. To overcome this, we propose the TFGFormer leveraging both the time-domain and frequency-domain features. TFGFormer establishes a robust time-domain perceptive foundation using an MSDFs and dense blocks. Crucially, by integrating frequency modules, gate modules, and a joint time-frequency loss function, the architecture compels the reconstructed signals to not only approximate the ground-truth time-domain waveforms but also encourage the preservation of critical frequency-domain features. Ultimately, this mechanism ensures that the reconstructions are not merely visually accurate but more consistent in both the time and frequency domains, providing a data foundation for downstream damage detection and condition assessment.

Evaluations on two engineering structures with distinct dynamic characteristics demonstrate the consistency of TFGFormer’s performance across the tested building and bridge datasets. For the super-tall Canton Tower-a structure highly dependent on global spatial coupling-TFGFormer exhibits consistently superior performance across missing data rates ranging from 30%-70%. Concurrently, on the Hardanger Bridge-a flexible, long-span structure characterized by strong local correlations and low-frequency, large-amplitude fluctuations-TFGFormer similarly maintains its leading edge across extensive missing scenarios (25%-75%). These results indicate that the time-frequency dual-branch, gate module and joint time-frequency loss function endow the model with high adaptability, enabling it to dynamically match the distinct structural topologies and wave propagation characteristics of diverse physical systems.

Additionally, the OOD noise experiments provide evidence of TFGFormer’s robustness under the tested unseen perturbations. When Gaussian random noise with levels ranging from 10 to 50% was added exclusively to the Canton Tower test set, the reconstruction SNR showed only a small decrease. This limited degradation suggests that the model retains a degree of robustness beyond the background-noise distribution represented in the training data. The frequency-domain results further indicate that the dominant low-frequency modal components within 0–4 Hz are largely preserved under these perturbations. These observations are consistent with the frequency-gating mechanism helping to suppress high-frequency noise while retaining the dominant modal content. Nevertheless, validation under additional noise types and real-world operating conditions is required before broader conclusions regarding deployment reliability can be drawn.

Regarding computational efficiency, TFGFormer shows a favorable trade-off between reconstruction accuracy and model complexity among the evaluated methods. Its relatively compact parameter count suggests potential for resource-constrained SHM applications, although hardware-specific inference latency and energy consumption require further evaluation.

Despite the observed improvement in reconstruction accuracy, the current framework remains dependent on fully sampled historical responses as supervision. Future work may investigate masked modeling or temporal contrastive learning to reduce this dependence and improve applicability to structures for which complete baseline data are unavailable.

Methods

Network architecture

Figure 8 presents the overall architecture of TFGFormer, an encoder-decoder framework engineered to recover missing structural dynamic response. It achieves high-accuracy reconstruction through the parallel execution of time-domain and frequency-domain branches. Initially, a shallow convolutional layer maps the incomplete vibration sequences into a high-dimensional latent space. This design leverages local receptive fields to extract preliminary morphological features, providing stable initial mappings that prevent deep-network instability⁴⁶. The features then enter cascaded time-frequency gated transformer blocks (TFGTBlocks), each designed to facilitate complex cross-domain interactions. Detailed parameters are provided in Supplementary Note 9.

Fig. 8. Detailed architecture of the proposed TFGFormer.

a Overview. The model processes incomplete structural dynamic responses through a shallow feature extraction layer, followed by N cascaded time-frequency gated transformer blocks (TFGTBlocks). Each block consists of local and global processing stages. The process concludes with a mask-guided reconstruction layer. b Gate Module (GM). A dual-path design utilizing a dense block for local detail enhancement and a linear network branch for gating, synergistically integrated via a channel attention block (CAB). The GM performs fine-grained adaptive weighting on time-domain features, thereby indirectly modulating the relative proportion of the subsequently fused frequency-domain features. c Frequency Module (FM). Enforces explicit frequency-domain physical constraints. It transforms time-domain features into the complex frequency domain via 2D FFT, applies dynamic complex weighting to anchor modal energies, and projects the optimized features back via inverse 2D FFT. d Multi-Scale Dilated Transformer (MSDF). The internal topology of the global attention mechanism. Here, multi-scale dilated attention (MSDA) employs varying dilation rates (r = 1, 2, 3) to capture global long-range dependencies and evolutionary patterns across multiple temporal scales.

Within each TFGTBlock, the time-domain branch utilizes an MSDF to capture long-range dependencies, while a dense block enhances local detail recovery. Simultaneously, an FM applies 2D fast Fourier transforms (FFTs) to explicitly constrain structural modal characteristics in the frequency domain. Subsequently, a GM dynamically orchestrates the cross-domain features, adaptively weighting the time-domain and frequency-domain contributions based on response complexity. TFGTBlocks employ residual connections with learnable scaling factors to ensure gradient stability, preserving underlying physical features while iteratively accumulating reconstruction increments. Finally, a convolutional layer reduces the dimensionality. By synergizing time-domain capture, frequency-domain constraints, and gated collaboration, TFGFormer robustly reconstructs structural data across varying missing rates. Mathematically, the forward propagation process can be expressed as:

$$\begin{array}{ccc}\hfill & \hfill \hfill & {\mathit{F}}_0=\text{LReLU}\left(\text{BN}\left(\text{Conv}\left(\mathit{X}\right)\right)\right),\hfill \\ \hfill & \hfill \hfill & {\mathit{F}}_N={\text{TFGTBlock}}_N\left({\text{TFGTBlock}}_{N-1}\left(\cdots {\text{TFGTBlock}}_1\left({\mathit{F}}_0\right)\right)\right),\hfill \\ \hfill & \hfill \hfill & {\mathit{Y}}_{pred}=\text{Conv}\left(\text{LReLU}\left(\text{BN}\left({\mathit{F}}_N\right)\right)\right),\hfill \end{array}$$ 1

where X denotes the input incomplete data, F_i(i = 1, …, N) denotes the features extracted by the i-th TFGTBlock, and Y_pred represents the reconstructed full-sequence signal predicted by the network. Conv, BN, and LReLU denote the convolution operator, batch normalization, and Leaky ReLU activation function, respectively. To make the tensor dimensions explicit, the incomplete input response is represented as $\mathit{X}\in {\mathbb{R}}^{B\times 1\times C_s\times T}$, where B, C_s, and T denote the batch size, the number of sensor channels, and the number of sampling points in each response segment, respectively. In this study, C_s = 20 for the Canton Tower dataset, C_s = 16 for the Hardanger Bridge dataset, and T = 1024. After shallow feature extraction, the latent feature is denoted as ${\mathit{F}}_0\in {\mathbb{R}}^{B\times d\times C_s\times T}$, where d = 32 is the latent feature dimension.

To ensure strict data accuracy, the final output Y_final is obtained by preserving the known sensor measurements and seamlessly filling the missing intervals with the network’s predictions. This is achieved using a binary mask matrix M:

$${\mathit{Y}}_{final}=\mathit{M}\odot \mathit{X}+\left(\mathbf{1}-\mathit{M}\right)\odot {\mathit{Y}}_{pred},$$ 2

where  ⊙ denotes the element-wise multiplication (Hadamard product), and 1 represents an all-ones matrix of the same dimensions as M. The mask matrix $\mathit{M}\in {\left\{0,1\right\}}^{B\times 1\times C_s\times T}$ is defined at each sensor-time coordinate. Specifically, M_b,1,c,t = 0 if the response at sensor c and sampling point t is missing, and M_b,1,c,t = 1 otherwise:

$${\mathit{M}}_{b,1,c,t}=\left\{\begin{array}{cc}0,\hfill & \forall \left(c,t\right)\in {\mathbf{\Omega }}_b,\hfill \\ 1,\hfill & \text{otherwise},\hfill \end{array}\right)$$ 3

where Ω_b denotes the set of missing sensor-time coordinates in the b-th sample. Thus, Eq. (2) retains all available measurements from the input and replaces only the missing locations with network predictions.

Internally, the feature evolution within the i-th TFGTBlock consists of a global stage and a local stage, which can be formulated as:

$$\begin{array}{cc}\hfill & {\mathit{F}}_{global,i}={\alpha }_i\cdot {\mathit{F}}_{i-1}+\text{MSDF}\left({\mathit{F}}_{i-1}\right)+\text{FM}\left({\mathit{F}}_{i-1}\right),\hfill \\ \hfill & \mathrm{\Delta }{\mathit{F}}_i={\beta }_i\cdot {\mathit{F}}_{global,i}+\text{GM}\left({\mathit{F}}_{global,i}\right)+\text{FM}\left({\mathit{F}}_{global,i}\right),\hfill \\ \hfill & {\mathit{F}}_i={\mathit{F}}_{i-1}+\mathrm{\Delta }{\mathit{F}}_i\hfill \end{array}$$ 4

where α_i and β_i denote the learnable scaling factors for the residual connections, and F_global,i represents the intermediate output variable after the global receptive stage within the i-th TFGTBlock. Here, ΔF_i denotes the reconstruction increment produced by the local stage, and the last equation explicitly describes the block-level residual connection. The learnable scaling factors α_i and β_i are channel-wise tensors with size 1 × d × 1 × 1, allowing each feature channel to adaptively control the residual contribution.

Time-domain branch

The time-domain branch restores the temporal characteristics of incomplete structural dynamic responses. At its core, the MSDF and the dense block collaborate to provide global perception and local feature mining, respectively. The MSDF captures long-term evolutionary patterns across missing intervals to reconstruct the global signal envelope leveraging MSDA, which is formulated as:

$$\text{MSDA}\left({\mathit{q}}_{ij},{\mathit{K}}_r,{\mathit{V}}_r\right)=\text{Softmax}\left(\frac{{\mathit{q}}_{ij}{\mathit{K}}_r^T}{\sqrt{d_k}}\right){\mathit{V}}_r,$$ 5

where Q, K, and V denote the query, key, and value matrices generated via linear mapping. For a query vector q_ij at position (i, j) in Q, MSDA performs a sparse retrieval of specific keys K_r and values V_r using dilation operations. This sparse sampling mechanism possesses distinct physical significance for SHM tasks. Given that the dynamic correlations across large-scale structures often exhibit inherent sparsity, blindly computing dense dependencies across all timesteps or sensors incurs substantial computational redundancy. By performing sparse feature selection during global envelope extraction, the model preserves critical evolutionary information while reducing computational complexity.

To prevent the attenuation of high-frequency transient features-such as impulsive shocks from structural damage or stochastic environmental excitations, which are often over-smoothed by standard transformers-the time-domain branch incorporates dense blocks. This module employs a robust feature-reuse mechanism, defined as:

$${\mathit{X}}_l={\text{Conv}}_l\left(\left[{\mathit{X}}_0,{\mathit{X}}_1,\dots ,{\mathit{X}}_{l-1}\right]\right),$$ 6

where Conv_l denotes the l-th convolutional layer within the dense block, and X_l denotes its corresponding output. Rather than relying solely on the preceding layer, the input to the l-th layer is a concatenation of feature maps from all previous layers along the channel dimension. In this study, each dense block contains four convolutional layers with a growth rate of k = 8. Each layer uses a 3 × 3 convolution, and the concatenated output is subsequently compressed back to 32 channels by a 1 × 1 convolution. This architecture facilitates the seamless cross-layer transmission of shallow features, enabling the model to repeatedly leverage high-frequency components from the initial samples. Consequently, it significantly enhances the capture of local transient anomalies, such as acceleration spikes induced by structural degradation. This synergistic integration of global trend prediction and local impulse restoration substantially mitigates the inherent conflict between global envelope shifting and local feature over-smoothing. While the MSDF identifies the underlying backbone of the structural dynamics, the dense blocks capture the transient intricacies of non-stationary signals. Together, they form the technical core for extracting robust time-domain features within TFGFormer, providing a high-dimensional, accurate temporal reference for subsequent frequency-domain alignment.

Frequency-domain branch

To maintain the global coherence of structural dynamic responses amidst intensive environmental noise and substantial data sparsity, the FM enforces critical physical fidelity constraints through explicit spectral modeling. Because structural dynamic responses frequently exhibit non-stationary characteristics-where spectral components undergo dynamic temporal evolution-the FM initially partitions the input time-domain feature maps X into a series of localized patches X_patch. The window partition is performed on the sensor-time feature map while preserving the feature-channel dimension. When the window size is set to the full feature-map size, this operation naturally reduces to a global spectral transformation. Subsequently, a 2D FFT is applied independently to each patch, projecting the sensor-temporal features into a localized frequency domain. This windowing strategy enables the network to precisely capture the time-frequency locality of the vibration signals within specific temporal spans. Physically, this extracts the instantaneous modal signatures of the structure under varying excitation stages or loading conditions, effectively mitigating the over-smoothing of transient information typically incurred by global transformations.

Upon obtaining the localized spectra, the FM performs feature mapping in the complex domain using learnable complex weight matrices W₁ and W₂. These complex weights transform the frequency-domain feature representation within each patch, rather than acting as conventional spatial convolution kernels. Therefore, the FM updates the complex spectral representation while preserving the original frequency-coordinate structure. This spectral optimization process is formulated as:

$${\widehat{\mathit{X}}}_{patch}=\text{IFFT}\left({\mathit{W}}_2\odot \text{GELU}\left({\mathit{W}}_1\odot \text{FFT}\left({\mathit{X}}_{patch}\right)\right)\right),$$ 7

where  ⊙ denotes the complex-valued spectral weighting operation applied to the frequency-domain features. During this process, the real and imaginary components of the spectra are jointly involved, enabling the FM to adjust magnitude- and phase-related information before inverse transformation. By co-adjusting the complex magnitude and phase, the FM adaptively reinforces the dominant structural modal components. Crucially, precise phase reconstruction ensures waveform synchronicity, preventing discontinuities and boundary distortions during the temporal sequence re-stitching. As defined in Eq. (7), these optimized spectral features are subsequently projected back to the time domain via the inverse fast Fourier transform (IFFT), yielding the refined patch ${\widehat{\mathit{X}}}_{patch}$. Finally, a window-reverse operation restores the patches to their original tensor dimensions. Thus, the FM output has the same tensor size as its input, allowing it to be directly integrated with the time-domain features and residual paths in both the global and local stages of each TFGTBlock. This strategy substantially mitigates the risk of spectral aliasing induced by data loss, providing a robust, high-accuracy spectral foundation for downstream structural damage identification and condition assessment.

Gate module

The GM serves as the dynamic orchestration center and decision-making unit within the architecture. Its core objective is to achieve a global balance in the time-frequency composition of the final output signal through precise regulation of the time-domain branch. The GM processes multi-dimensional features via a dual-path gating mechanism. Specifically, a channel attention block (CAB) and a two-layer linear network collaborate to determine the reconstruction focus for each signal segment. Given the global-stage feature F_global,i, the GM first applies a 1 × 1 convolution to expand the channel dimension and then splits the expanded feature into two branches:

$$\left[{\mathit{F}}_{dense,i},{\mathit{F}}_{linear,i}\right]=\text{Split}\left({\text{Conv}}_{1\times 1}\left({\mathit{F}}_{global,i}\right)\right).$$ 8

The first branch, F_dense,i is processed by the dense block and CAB to extract localized time-domain features with channel-wise attention. The second branch, F_linear,i is reshaped along the sensor-time positions and processed by a two-layer linear network to generate a data-dependent modulation feature:

$${\mathit{F}}_{lin,i}=\text{GELU}\left({\mathit{W}}_2\cdot \text{GELU}\left({\mathit{W}}_1\cdot {\mathit{F}}_{linear,i}+{\mathit{b}}_1\right)+{\mathit{b}}_2\right),$$ 9

where F_lin,i denotes the output of the second branch. The final GM output is obtained by element-wise modulation between these two branches:

$${\mathit{F}}_{GM,i}={\mathit{F}}_{lin,i}\odot \text{CAB}\left({\mathit{F}}_{dense,i}\right),$$ 10

where W₁, W₂ and b₁, b₂ denote the learnable weight matrices and bias vectors of the two linear layers, respectively. F_GM,i represents the final GM output of the i-th TFGTBlock. The symbol  ⊙ denotes element-wise multiplication rather than matrix multiplication. In this design, the CAB branch provides local convolutional features and channel attention, while the linear branch provides a data-dependent modulation feature. Their element-wise interaction enables the GM to regulate the contribution of local time-domain features before they are combined with the frequency-domain increment in the local stage.

Because the GM performs fine-grained, adaptive weighting on the time-domain features, it effectively-albeit indirectly-modulates the relative proportion of the frequency-domain information integrated from the FM. This design holds physical significance. For instance, in localized regions suffering from severe signal loss, the GM automatically suppresses the pass-through rate of the corrupted time-domain branch; this allows the physical fidelity constraints provided by the FM to dominate the fused representation, thereby robustly locking onto the structure’s inherent natural frequencies. Conversely, upon detecting impulsive transients (e.g., impact-induced responses), the GM prioritizes the high-frequency time-domain details from the dense block. Facilitated by the synergistic CAB and linear layers, this dual-path gating mechanism dynamically reconciles temporal waveform accuracy with frequency-domain physical consistency.

Residual connection with learnable scaling

To enhance gradient stability and multi-scale feature integration, TFGFormer incorporates residual connections modulated by learnable scaling factors (α_i and β_i). Unlike conventional identity mappings, these data-driven parameters dynamically establish an learned balance between preserving low-level morphological features and absorbing high-level reconstruction increments. This adaptive mechanism effectively mitigates the gradient vanishing and representation degradation frequently encountered by deep networks when processing complex, non-stationary structural responses. Consequently, it ensures that TFGFormer maintains robust convergence and superior reconstruction accuracy across diverse structural damage scenarios and severe environmental excitations.

Joint time-frequency loss function

To guide the network in balancing time-domain waveform accuracy with frequency-domain physical fidelity, TFGFormer incorporates a composite loss function, ${\mathcal{L}}_{total}$, based on joint temporal and spectral constraints. The objective function provides multi-dimensional optimization guidance by computing a weighted combination of time and frequency-domain MSE.

The time-domain loss minimizes the point-to-point discrepancy between the reconstructed responses and the ground-truth signals, ensuring the accuracy of the synthesized signals in terms of temporal morphology and amplitude envelopes. Its mathematical formulation is defined as:

$${\mathcal{L}}_{time}=\frac{1}{N}{\sum }_{i=1}^N{\parallel {\mathit{Y}}_{pred}^{\left(i\right)}-{\mathit{Y}}_{true}^{\left(i\right)}\parallel }_2^2,$$ 11

where N denotes the number of samples in a mini-batch, while ${\mathit{Y}}_{pred}^{\left(i\right)}$ and ${\mathit{Y}}_{true}^{\left(i\right)}$ denote the predicted reconstruction and the fully sampled ground-truth response for the i-th sample, respectively. This loss term penalizes the global residuals across the time series, establishing the fundamental restoration accuracy of the reconstructed outcomes.

Because structural vibration signals exhibit distinct modal features in the frequency domain and are highly sensitive to phase shifts, we introduce a frequency-domain loss, ${\mathcal{L}}_{freq}$, based on the FFT. This module enables the parallel optimization of energy distribution and phase synchronization by applying constraints separately to the real and imaginary components of the complex-valued spectra. First, the time-domain features are projected into the complex frequency domain:

$${\widehat{\mathit{Y}}}_{\mathrm{pred}}^{\left(\mathrm{i}\right)}=\mathrm{FFT}\left({\mathit{Y}}_{\mathrm{pred}}^{\left(\mathrm{i}\right)}\right),{\widehat{\mathit{Y}}}_{\mathrm{pred}}^{\left(\mathrm{i}\right)}=\mathrm{FFT}\left({\mathit{Y}}_{\mathrm{true}}^{\left(\mathrm{i}\right)}\right),$$ 12

where ${\widehat{\mathit{Y}}}_{pred}^{\left(i\right)}$ and ${\widehat{\mathit{Y}}}_{true}^{\left(i\right)}$ denote the complex-valued frequency spectra of the predicted reconstruction and the ground-truth response for the i-th sample, respectively. Subsequently, the frequency-domain loss computes the residual between the predicted and ground-truth spectra on the complex manifold:

$${\mathcal{L}}_{freq}=\frac{1}{N}{\sum }_{i=1}^N\left(\parallel \Re \left({\widehat{\mathit{Y}}}_{pred}^{\left(i\right)}\right)-\Re \left({\widehat{\mathit{Y}}}_{true}^{\left(i\right)}\right){\parallel }_2^2+\parallel \Im \left({\widehat{\mathit{Y}}}_{pred}^{\left(i\right)}\right)-\Im \left({\widehat{\mathit{Y}}}_{true}^{\left(i\right)}\right){\parallel }_2^2\right),$$ 13

where ℜ( ⋅ ) and ℑ( ⋅ ) denote the real and imaginary parts of a complex matrix, respectively.

Within the context of SHM, this loss term possesses profound physical significance. The joint penalization of the real and imaginary parts enables precise anchoring of the structure’s inherent natural frequencies and corrects phase deviations caused by asynchronous sensor sampling or massive data loss. Consequently, it establishes a robust barrier for physical integrity in the frequency domain. The final optimization objective is formulated as a weighted combination of these terms via hyperparameters λ_t and λ_f:

$${\mathcal{L}}_{total}={\lambda }_t{\mathcal{L}}_{time}+{\lambda }_f{\mathcal{L}}_{freq}.$$ 14

Regarding the selection of the scaling factor λ_f, this study adheres to the principle of magnitude alignment. Given that the energy distribution of the frequency spectrum typically exceeds the time-domain signal in numerical magnitude, setting λ_f = 0.5 balances the respective contributions of the two loss terms to the network parameter updates. This proportional allocation prevents the frequency-domain loss from dominating the gradient direction during the initial stages of optimization, ensuring that the network maintains precise tracking of temporal waveform details while concurrently locking onto the structural modal frequencies.

Training setup

All experiments were implemented in Python (v3.10) using the PyTorch framework (v2.3.0). The computational platform was equipped with an Intel Core i7-14700KF CPU (3.40 GHz), 64 GB of RAM, and a single NVIDIA GeForce RTX 4090 GPU (24 GB VRAM). Network parameters were optimized using the Adam optimizer with an initial learning rate of 0.01 to facilitate rapid convergence within the complex, non-linear vibration data space. A mini-batch size of 16 was uniformly applied, and the training process spanned a total of 200 epochs.

Author contributions

X.S. conceived the initial idea and supervised the research. F.Y. designed the model and wrote the manuscript. R.L. collected and preprocessed the data. X.M., Y.W., S.W., Q.D. and S.Z. aided in interpreting the results. All authors revised the manuscript and approved the final version of the manuscript.

Peer review

Peer review information

Communications Engineering thanks Roksana Soleimani-Babakamali and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Primary Handling Editors: [Said Elias Rahimi] and [Philip Coatsworth]. A peer review file is available.

Funding

The authors express their gratitude for financial support from the National Natural Science Foundation of China (Program No. 12571488), Natural Science Basic Research Program of Shaanxi (Program No. 2024JC-ZDXM-23, 2025JC-JCQN-029, 2025JC-YBQN-780) and the Fundamental Research Funds for the Central Universities, CHD (Program No. 300102125740, 300102124501).

Data availability

The Canton Tower data used in this study are available in the official benchmark website http://polyucee.hk/ceyxia/benchmark/index.htm; the Hardanger Bridge data used in this study are available in the official benchmark website https://doi.org/10.21400/5ng8980s. The data generated in this study have been deposited in the Supplementary Information/Source Data file. The source data for Figures are provided in Supplementary Data.

Code availability

The archived version of the code used for this manuscript is available on Zenodo at https://doi.org/10.5281/zenodo.21979714.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at https://doi.org/10.1038/s44172-026-00788-0.