PM_2.5 performance analysis under varying imputation strategies for incomplete sensor data

Abstract

Air pollution remains a primary global health concern, with one of its most hazardous components, fine particulate matter PM_2.5, being recognized as a significant contributor to premature mortality worldwide. Thus, forecasting PM_2.5 concentrations is essential for policymaking and mitigating their harmful effects. Low-cost sensor (LCS) technology has advanced air quality monitoring by providing real-time PM_2.5 measurements at finer spatial and temporal granularity due to their higher deployment density, enabling more comprehensive data for predictive modelling. However, these sensors can experience significant data gaps, necessitating effective imputation methods to improve the accuracy of PM_2.5 predictions. In this study, we focus on reconstructing missing PM_2.5 readings by evaluating the impact of different imputation strategies on PM_2.5 forecasting accuracy using data from eight LCSs deployed across Edmonton, Alberta, from January 2023 to December 2024. The dataset includes 7 features: temperature, relative humidity, wind speed, wind direction, latitude, longitude, and altitude, integrated from both LCS sensors and adjacent air quality monitoring stations. We evaluate the efficacy of Kriging as a spatial imputation method for PM_2.5 forecasting, comparing its performance along with mean imputation and k-nearest neighbours, against a no-imputation benchmark to assess whether imputation techniques provide meaningful improvements in forecasting accuracy. Additionally, we compare its performance across data with varying levels of missingness. Our comparative analysis examines how these imputation techniques affect various prediction models, including Random Forest, Extreme Gradient Boosting, Convolutional Neural Networks, Long Short-Term Memory (LSTM), and LSTM with an Attention mechanism. Our findings indicate that although differences among imputation methods were modest, Kriging consistently yielded the best results, particularly at 90% data availability for the deep learning models. The LSTM-AM architecture produced the best overall results in a short-horizon setting, = 0.9602, MAE = 1.7815μg/m, and RMSE = 4.5208μg/m, suggesting an optimal combination of Kriging imputation with LSTM-AM modeling for near-term PM_2.5 forecasting applications.

Introduction

Fine particulate matter (PM_2.5), with an aerodynamic diameter of 2.5 μ m or smaller, is one of the leading contributors to air pollution-related health issues worldwide¹. Due to their small size, PM_2.5 particles can penetrate deep into the respiratory system and bloodstream, posing significant short and long-term health risks². Exposures to PM_2.5 of up to 24 hours can lead to acute respiratory issues, chronic bronchitis, asthma exacerbation, cardiac-related hospital presentations, and even premature mortality³. Chronic exposure to PM_2.5 can increase mortality rates and has been linked to a range of illnesses such as asthma⁴, chronic obstructive pulmonary disease (COPD)⁵, cardiovascular disorders⁶, diabetes mellitus⁷, and lung cancer⁸. Health Canada estimated that in 2018 alone, approximately 12,500 deaths in Canada were attributable to long-term PM_2.5 exposure⁹. PM_2.5 concentrations can exhibit substantial variation across locations, as they are influenced by regional industrial activities, vehicle emissions, and smokestack emissions³. Therefore, a real-time predictive model to estimate PM_2.5 would empower public health officials and policymakers to implement interventions that help vulnerable populations avoid exposure.

In Canada, the Air Quality Monitoring System is a framework for managing ambient air quality nationwide¹⁰. PM_2.5 concentrations within this system are predominantly monitored through long-term monitoring stations, or Air Quality Monitoring Stations (AQMS). While these stations provide highly accurate measurements, they are expensive to maintain and operate, requiring significant financial and human resources for calibration and upkeep¹¹. Due to constraints imposed by their permanent infrastructure design and extended deployment periods, AQMSs lack spatial coverage, making it difficult to capture fine-scale variations in air quality. Consequently, Low-Cost Sensor (LCS) technology has emerged as a promising alternative due to its affordability, portability, and ability to provide real-time data across larger areas. However, LCSs can exhibit inaccuracies due to the susceptibility of their light-scattering technology to environmental factors such as temperature and relative humidity. Further inaccuracies may also result from the technology’s sensitivity degrading over time¹². Nevertheless, their ability to achieve higher spatiotemporal resolution in PM_2.5 concentration measurements has contributed to their growing adoption in air quality monitoring.

One such low-cost sensor technology supplier is PurpleAir¹³, which provides a unit called the PA-II. This unit uses Plantower PMS5003 particulate matter sensors and can measure different PM concentrations (PM₁, PM_2.5, and PM₁₀). Designed for outdoor use, PA-II units offer a cost-effective alternative to AQMSs but, like other sensor networks, are susceptible to data gaps and inaccuracies due to sensor malfunctions, environmental interference, and power failures¹⁴. Predicting air quality from such incomplete datasets presents challenges for predictive algorithms, as missing data can compromise model reliability. Ensuring the accuracy of PM_2.5 predictions is crucial for public health decision-making, necessitating robust data preprocessing and imputation techniques to enhance model performance. While previous studies^15–17 have explored imputation strategies, their ability to capture the complex spatiotemporal dynamics of PM2.5 remains under question.

In this study, we address these gaps in PM_2.5 sensor data by developing a controlled experiment to evaluate the effectiveness of various imputation techniques for mitigating the impact of missing data on PM_2.5 prediction analysis. We collected hourly PM_2.5 measurements from eight PurpleAir sensors located throughout Edmonton, Alberta, between January 2023 and December 2024. Our dataset includes seven characteristics–temperature (T), relative humidity (RH), wind speed (WS), wind direction (WD), latitude, longitude, and altitude–derived from PurpleAir sensors and nearby AQMSs. Furthermore, as we found the distribution of PM_2.5 values to be highly skewed, we applied the Log transformation during preprocessing to normalize the data, reducing skewness and making it more suitable for modelling. To address the persistent issue of missing data, we propose Kriging as a spatial imputation method that, to the best of our knowledge, has been underutilized for filling missing PM_2.5 values in low-cost sensor networks. We systematically compare Kriging with widely used methods, including K-nearest neighbours (KNN) and mean imputation, and evaluate their effectiveness against a benchmark of non-imputed data. We investigate how these techniques perform under varying levels of missingness (30%, 60%, 90%) and assess the statistical significance of performance differences using non-parametric tests. The impact of these preprocessing techniques is evaluated using multiple models: Random Forest (RF), Extreme Gradient Boosting (XGBoost), Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM), and LSTM with an Attention Mechanism (LSTM-AM). Our study focuses on a one-hour-ahead prediction task, a setting that provides a controlled baseline for evaluating imputation strategies before extending to more challenging multi-step or long-horizon forecasts. This comprehensive evaluation framework allows us to determine optimal approaches for handling missing data while maximizing the prediction accuracy of PM_2.5 concentrations.

The three key research contributions of our work are:

• Design of an integrated dataset combining hourly PM_2.5 measurements from PurpleAir sensors with meteorological data from nearby AQMSs, enabling fine-grained prediction of air quality and capturing key environmental factors affecting PM_2.5 levels.

• Enhancement of data preprocessing by applying the Log transformation to normalize skewed PM_2.5 distributions, and exploring Kriging interpolation as a spatial imputation method for low-cost air quality sensor networks.

• Presentation of a systematic comparison of machine learning and deep learning models (RF, XGBoost, CNN, LSTM, LSTM-AM) across different imputation strategies and varying levels of data missingness, identifying optimal model-imputation combinations for PM_2.5 prediction.

Related works

Given the significant public health threats posed by air pollution, air quality forecasting has become a highly active area of research. In this section, we document the evolution of PM_2.5 prediction methodologies, with particular attention to advanced modelling techniques and data pre-processing strategies.

Traditional machine learning (ML) models have been widely used for PM_2.5 prediction due to their ability to analyze large-scale air quality datasets and capture the complex relationships between PM_2.5 concentrations and factors such as meteorological conditions, industrial emissions, and traffic patterns. ML models, including RF and XGBoost, have frequently been featured in past studies, demonstrating effectiveness in modelling air pollution dynamics^18–20. The predictive accuracy of these ML approaches depends on the features selected, as highlighted by Joharestani et al.²¹, who explored feature importance for PM_2.5 prediction in Tehran using RF, XGBoost, and deep learning (DL) models. They tested 23 features and used Spearman’s correlation coefficient to identify the most relevant ones. Feature importance was estimated using three methods: built-in functions of RF and XGBoost, feature permutation, and recursive feature elimination with XGBoost. Their results indicated that, of 23 candidate features, only 8 to 12 were necessary to achieve acceptable predictive performance, with historical PM_2.5 observations, wind speed, visibility, day of the year, latitude, air pressure, altitude, and dew point as the most significant predictors. The selected features highlight the strong influence of both spatial and temporal factors on PM_2.5 concentrations.

Building on the spatiotemporal dynamic of PM_2.5, Kar et al.²² address the limitations of conventional ML models by implementing several ML-based spatial interpolation techniques for PM_2.5 prediction. They compared RF with hybrid models incorporating geostatistics into their architecture, including Random Forest Spatial Interpolation (RFSI), Space-Time Regression Kriging (STRK), and Random Forest Kriging (RFK). Their findings demonstrated that hybrid spatiotemporal models (e.g., STRK, RFK, RFSI) outperformed non-spatiotemporal models (e.g., RF), further emphasizing the importance of integrating spatial interpolation techniques into ML models.

While traditional ML models have demonstrated effectiveness in PM_2.5 prediction, DL approaches have also been shown to be highly effective, owing to their ability to capture the complex spatiotemporal dependencies inherent in air quality data. In particular, LSTM and CNN are widely used DL models for PM_2.5 forecasting^23–28 due to their ability to model long-term dependencies in sequential air quality data. Furthermore, hybrid deep learning (DL) models have gained significant popularity^29–33 due to their ability to leverage the individual strengths of the constituent algorithms. For example, Pak et al.³⁰ introduce a spatiotemporal CNN-LSTM model to forecast daily average PM_2.5 concentrations in Beijing. They used Mutual Information to construct a Spatiotemporal Feature Vector (STFV) that incorporates historical PM_2.5 data from monitoring stations, along with other highly correlated features. Their results demonstrate that enhancing existing deep learning architectures can improve performance. The proposed CNN-LSTM model automatically learns complex nonlinear relationships and long-term dependencies from high-dimensional spatiotemporal data. By capturing spatial patterns with CNN and temporal dynamics with LSTM, the model significantly improves PM2.5 prediction accuracy, outperforming traditional models such as the Multi-Layer Perceptron (MLP) and LSTM.

However, a significant challenge in air quality prediction is the presence of missing data, which often arises due to sensor failures, communication errors, or external disruptions. Data preprocessing techniques are crucial for ensuring the robustness of deep learning (DL) models. Zhao et al.¹⁵ developed a Long Short-Term Memory - Fully Connected (LSTM-FC) neural network for PM_2.5 concentration prediction but faced issues with missing data. To address this, they used simple linear interpolation, a widely used yet limited approach that can distort data distributions and degrade model accuracy.

To improve the handling of missing data, various advanced data imputation techniques have been explored. A recent survey outlines multiple approaches to this challenge³⁴. List-wise deletion removes entire records with missing values, while pairwise deletion excludes specific variables only when they contain missing data³⁵. Although these methods are simple and computationally efficient, they often result in data loss and biased predictions. More refined approaches include linear interpolation techniques such as mean and median imputation, as well as more sophisticated methods like the Expectation-Maximization (EM) algorithm, which estimates missing values using maximum likelihood estimation³⁶. Additionally, KNN imputation leverages neighbouring data points to estimate missing values, offering improved performance over simple statistical methods^37,38.

Alongside these statistical methods, deep learning has also proven effective for data imputation. Notable examples include Inverse Mapping Generative Adversarial Networks and CNN , both of which have demonstrated superior performance in imputing air quality data compared to traditional statistical methods. Despite these promising results, the computational requirements often make deep learning models impractical for edge computing environments, where processing power and energy consumption are critical constraints –a crucial consideration for low-cost sensor networks that require edge-based processing. This limitation has led researchers to explore lightweight alternatives that balance imputation accuracy with computational efficiency.

Ojagh et al.³⁹ proposed an edge-based preprocessing framework to address missing data issues within these computational constraints. Their method used the Dynamic Time Warping (DTW) distance and the Partitioning Around Medoids (PAM) clustering algorithm to estimate missing values from similar sensor readings and to recalibrate clusters daily. A spatial-based estimation method using K-Nearest Neighbour Euclidean Distance (knned) was also applied. The final missing-value estimation was computed as a weighted combination of temporal and spatial approaches, significantly improving data quality and enabling better predictions. While their method offers an effective strategy for handling missing data, it has been validated only on sensors with less than 5% data loss, limiting its applicability to more severely incomplete datasets.

While traditional imputation methods effectively address missing data, they often fail to preserve the spatial correlation inherent in air pollution patterns. Kriging, a geostatistical interpolation method, has been widely applied in various disciplines, including chemical engineering, traffic volume analysis, and rainfall analysis^40–42. In this work, we evaluate spatial Kriging as a preprocessing technique to impute missing PM_2.5 values prior to applying DL models. This approach enhances conventional imputation methods by maintaining the spatial structure of air pollution data, which is critical for accurate forecasting.

Despite the extensive literature on PM_2.5 forecasting, our study identifies several critical gaps. Building on the limitations identified in existing imputation research, we examine the role of imputation in improving or degrading model performance. We systematically compare imputation strategies against a baseline in which no imputation is performed, allowing us to assess whether these techniques offer meaningful improvements or introduce additional inaccuracies. Importantly, we also evaluate imputation performance across varying levels of data missingness (30%, 60%, and 90%), a dimension underexplored in prior work. Second, while many studies focus on either ML or DL models in isolation, few provide a comprehensive comparative evaluation. In this work, we examine both traditional ML models–RF and XGBoost–and DL models, including CNN, LSTM, and LSTM-AM. Finally, we evaluate Kriging as an imputation technique to address the gap in imputation methods that do not adequately account for spatial correlation.

Materials and methods

The objective of this study is to conduct a controlled evaluation of preprocessing and imputation strategies for reconstructing missing PM_2.5 observations for forecasting. We focus on developing a high-coverage subset of LCS data, reducing variability introduced by sensors with intermittent or sparse recordings and enabling robust comparison of spatial (Kriging) and non-spatial imputation methods across ML and DL architectures. These results are intended as a proof of concept of the method’s efficacy under favourable data availability. In this section, we outline the complete pipeline for evaluating the effectiveness of different imputation strategies for PM_2.5 prediction. We describe the data sources and collection process, then formulate the prediction task, outline preprocessing techniques (including imputation), and present the machine learning and deep learning models employed. Finally, we detail the evaluation metrics used to assess model performance.

Data collection

For this study, we collected hourly PM_2.5 concentrations from outdoor PurpleAir PA-II sensors⁴³ in Edmonton, Alberta (53.5461, −113.4937) between January 1, 2023, and December 31, 2024. Sensor data were retrieved using the PurpleAir bounding box API call⁴⁴, which returns all sensors within a user-defined rectangular boundary defined by its Northwest (NW) and Southeast (SE) corners. We initially collected all sensors within the province of Alberta, using the coordinates (53.7137, −113.7110) for the NW corner and (53.3734, −113.2639) for the SE corner, resulting in 1,191 sensors. Sensor selection then proceeded in two stages. First, we identified sensors that are within a 5 km radius of an AQMS, reducing the dataset to 458 sensors. The AQMS data, obtained from the Alberta Air Data Warehouse⁴⁵, includes hourly measurements of T, RH, WS, and WD. This proximity-based selection enabled us to incorporate local meteorological measurements from nearby AQMS as features for each corresponding PM_2.5 sensor datapoint. Second, to ensure stable model training and fair comparisons of imputation strategies, we retained only sensors with at least 90% hourly data coverage from January 1, 2023, to December 31, 2024. This procedure reduced the pool of 458 candidate sensors to the eight sensors used in the experiments. We emphasize that the need for experimental control drove this decision and that we evaluated imputation quality while minimizing confounding from extreme sensor downtime.

To assess whether the final eight selected sensors are representative of the broader sensor network, we conducted a representativeness analysis using all 458 available sensors. The selected sensors exhibited a mean PM_2.5 concentration of 7.81 μg/m compared to 7.20 μg/m across the full network, and median values of 3.4 and 3.0 μg/m, respectively. Both groups displayed nearly identical variability, with standard deviations of 16.6 (selected) and 16.9 (all), and mean variances of approximately 274 and 182. The interquartile ranges were also comparable (1.6–7.8 μg/m vs. 1.3–7.0 μg/m). The slightly higher variance among the selected sensors suggests that they capture a marginally broader range of PM_2.5 fluctuations, potentially reflecting diverse microenvironments within the study region. Extreme values were consistent across both groups, with maximum concentrations of 297 μg/m³ (selected) and 448 μg/m (all), and nearly identical minima (0.1 vs. 0.0 μg/m). Overall, these results indicate that the eight selected sensors are statistically representative of the larger network with respect to the PM_2.5 range, central tendency, variability, and extremes. These eight sensors are listed in Table 1, and their distribution is further illustrated in the map of sensor locations in Fig. 1.

Table 1.

Final selection of sensor-AQMS pairings their distances.

Sensor ID	AQMS ID	AQMS name	Distance (km)
27709	1722	St. Albert	1.299
52583	1224	Edmonton McIntyre	0.505
24481	1726	Edmonton East	3.980
24491	1724	Edmonton McCauley	3.046
24513	1724	Edmonton McCauley	3.678
53143	1722	St. Albert	2.145
53221	1224	Edmonton McIntyre	2.164
53321	1724	Edmonton McCauley	2.495

Fig. 1.

Spatial distribution of PurpleAir sensors (red dots) and reference monitoring stations (blue dots) across Edmonton used in this study. The map shows the locations of neighbourhood-level sensors and official monitoring sites, including St. Albert, Edmonton McCauley, Edmonton McIntyre, and Edmonton East stations.

Table 2 presents the list of features that were utilized as predictors for the final dataset, including their respective data sources. These variables were selected for their relevance to air quality analysis and for their use in previous studies. Commonly used predictors in PM_2.5 modelling include: T, RH, WS, WD, latitude, longitude, and altitude, which have been widely recognized as significant predictors in prior research^{18,22,31,46,47}.

Table 2.

Environmental parameters and data sources.

Parameter	Unit	Source
Temperature	°C	AQMS (Hourly data)
Relative Humidity	%	AQMS (Hourly data)
Wind Speed	m/s	AQMS (Hourly data)
Wind Direction	degrees	AQMS (Hourly data)
Latitude		PurpleAir Sensor (Hourly data)
Longitude		PurpleAir Sensor (Hourly data)
Altitude	m	PurpleAir Sensor (Hourly data)

Problem formulation

We define the hourly PM_2.5 measurements as from January 1, 2023, to December 31, 2024, where represents the PM_2.5 concentration at sensor i at time t. As described in Eq. 1, represents the PM_2.5 values recorded at time t for all sensors, where the total number of sensors is defined as N = 8.

1

The objective of this study is to estimate a function f that predicts future PM_2.5 concentrations, expressed in Eq. 2.

2

Here, h represents the forecast horizon, e.g. when h = 24 we are predicting the PM_2.5 concentration at sensor i within 24 hours. In this study, we focus on a one-hour-ahead forecast horizon (h = 1) and predict PM_2.5 concentrations for the next hour using the previous 24 hours of observations. denotes historical PM_2.5 measurements from all sensors over a k-hour window. includes meteorological features, which for this study we selected T, RH, WS, and WD. captures spatial features including the latitude, longitude, and elevation of sensor i.

Preprocessing

Preprocessing plays a critical role in training ML models, as data quality can dictate model performance. In this study, we focus on preprocessing, specifically the handling of missing values via data imputation. We aim to investigate whether interpolation techniques meaningfully contribute to capturing air pollution dynamics in the context of imputing missing PM_2.5 data. Furthermore, we explore Kriging as an imputation method for this purpose.

Kriging

Kriging is a geostatistical interpolation technique that utilizes known spatially distributed data points to estimate unknown values, as captured in Eq. 3. Let Z(s) denote the PM_2.5 concentration of the spatial variable at location s, where s = (x,y) represents a spatial coordinate. Given an unknown value at location , denoted as , it can be modelled as a weighted linear combination.

3

We define as the Kriging weights assigned to each known location . These weights reflect the spatial correlation between the sampled and unsampled locations. Kriging assumes that as the distance between points increases, the variability of the measured values increases; therefore, closer points are assigned higher weights. This relationship is modelled using a variogram function as shown in Eq. 4,

4

where is the value at a location separated by distance h. To estimate the unknown value , we follow two main steps: (1) Using the available data, construct an empirical variogram by plotting the semi-variance γ (s,h) against distances between pairs of points. (2) Compute the Kriging weights by solving a system of linear equations derived from the spatial covariance matrix of known points and their covariance with the unknown point.

Computationally, this second step is the most intensive, as it requires inverting the covariance matrix. Consequently, the time complexity of Kriging is , where N is the number of known observation points. While computationally expensive compared with simpler deterministic methods, Kriging accounts for spatial autocorrelation (spatial clustering) and assigns lower weights to densely clustered points, thereby significantly improving the accuracy of spatial estimates.

Benchmark imputation methods for comparison

To evaluate the impact of imputation on PM_2.5 prediction, we establish a benchmark using the original dataset with all missing values removed, thereby creating a complete dataset for training our models without imputation. The performance of each imputation method is assessed relative to this benchmark to achieve comparable or improved results. Specifically, we compare our proposed Kriging-based spatiotemporal interpolation approach against this benchmark and two classical imputation techniques: mean imputation and KNN.

Mean imputation

Mean imputation is a commonly used interpolation method. An unknown value is estimated by taking the mean PM_2.5value at the same hour on the same day across all sensors with available data. As with Kriging, data reliability is ensured by imposing a threshold requiring at least 50% of the sensors to have known values. Since this operation requires only a single linear pass over the available data points to compute the average, it is highly efficient, with time complexity .

K-nearest neighbours imputation

For missing values at a given time step, the Haversine distance between sensors is used to identify the k nearest neighbours. The missing values are then estimated as the weighted average of the k nearest neighbours. The computational cost is primarily driven by calculating the pairwise distances between the target location and all available sensors, followed by sorting to identify the top k. This yields a time complexity of (assuming efficient sorting) or when using a selection algorithm, making it more expensive than mean imputation but generally faster than Kriging for moderate datasets.

Missing data simulation

To further test the robustness and generalizability of the imputation methods evaluated in this study, we simulate additional missingness scenarios. Specifically, we create datasets with 30%, 60%, and 90% missing data, applied sensor-specifically. That is, for each scenario, a corresponding proportion of PM_2.5 values is randomly removed from each sensor’s time series. This approach allows us to assess how well different imputation methods perform as data sparsity increases, reflecting realistic situations such as extended sensor outages or intermittent malfunctions.

Base models

The prediction model for PM_2.5 is a function of T, RH, WS, WD, latitude, longitude, and altitude. To optimize predictions, two traditional ML methods and two DL methods were used: RF, XGBoost, CNN, and LSTM.

RF is a traditional ML approach introduced by Ho⁴⁸ that leverages an ensemble of decision trees, aggregating their outputs to improve predictive accuracy and reduce overfitting. To optimize the RF model, a grid search was conducted to maximize performance. Table 3 shows the RF hyperparameter ranges and the optimized values detected by the grid search.

Table 3.

Random forest grid search hyperparameters.

Hyperparameter	Search range	Best value
n_estimators	[50, 100, 200, 300, 500, 800, 1000, 2000]	300
max_depth	[5, 10, 20, 30, 40, None]	40
min_samples_split	[2, 5, 10, 20]	2
min_samples_leaf	[1, 2, 4, 8, 10]	4

XGBoost, proposed by Chen et al.⁴⁹, is an advanced gradient-boosting algorithm that improves the performance of decision trees by minimizing errors through a sequential training process. Compared with previous gradient-boosting algorithms, it provides greater control over overfitting by employing a regularization technique that improves generalization. As with RF, the hyperparameters of XGBoost require tuning; a grid search was performed, as listed in Table 4.

Table 4.

XGBoost grid search hyperparameters.

Hyperparameter	Search range	Best value
n_estimators	[50, 100, 200, 300, 500, 800, 1000, 2000]	2000
learning_rate	[0.001, 0.005, 0.01, 0.05, 0.1, 0.2]	0.05
max_depth	[3, 6, 9, 12]	12
subsample	[0.6, 0.7, 0.8, 1.0]	0.8
colsample_bytree	[0.6, 0.7, 0.8, 1.0]	1.0
gamma	[0, 0.1, 0.25, 0.5]	0
reg_lambda	[1, 10.0, 20, 50, 100]	1

To avoid optimization bias in the two traditional ML models, we conducted an exhaustive grid search using 5-fold cross-validation on an 80/20 train–test split. Hyperparameters were optimized solely on the training set, using the negative root mean squared error as the scoring metric, with a fixed random seed. The search budget encompassed the full Cartesian product of the parameter grids, evaluating all candidate combinations to ensure thorough coverage of the search space. For XGBoost, early stopping was used.

Since PM_2.5 prediction is inherently a regression task, ML models like RF and XGBoost may be sufficient for capturing the primary relationships between features and PM_2.5 levels. However, if these models fail to fully capture the nonlinear dependencies in air pollution dynamics, we introduce DL models: CNN, LSTM, and LSTM-AM.

Inspired by the layered architecture of the human visual cortex²⁵, CNN is a core architecture in modern DL. Introduced in the 1960 s and operationalized for practical applications by LeCun⁵⁰, CNNs are commonly used for image processing but have also been adapted for time-series prediction. In the context of PM_2.5 prediction, CNNs can extract spatial and temporal patterns by applying convolutional filters to sequences of meteorological data. The initial convolutional filters act as local feature detectors, identifying short-term temporal motifs such as abrupt pollution spikes. Subsequent pooling operations aggregate these responses, enabling the model to form higher-level representations of persistent pollution regimes and dominant temporal trends. The fully connected layers then integrate these learned abstractions to generate the final concentration forecast. In this study, the architecture of our CNN model for PM_2.5 prediction is shown in Table 5. The working flow of the CNN model is presented in Fig. 2.

Table 5.

CNN model architecture for PM_2.5 prediction.

Component	Details
Number of Layers	5 (1 Conv1D, 1 MaxPooling1D, 1 Flatten, 2 Dense)
Neurons	64 (Conv1D)→32 (Dense)→1 (Dense Output)
Kernel Size	5 (Conv1D)
Activation Function	ReLU (for Conv1D and Dense layers)
Optimizer	Adam, Learning Rate: 0.0001
Batch Size	16
Epochs	200 (early stopping with patience = 10)
Sequence Length	24

Fig. 2.

Schematic diagram of the CNN model employed in this study, including the input layer, convolutional and pooling layers, flattening, dense layers, and final output node for PM2.5 prediction.

The CNN architecture consists of a one-dimensional convolutional layer (Conv1D) with 64 filters and a kernel size of 5, followed by a ReLU activation function to introduce nonlinearity. The convolutional layer enables the model to automatically learn localized spatial–temporal patterns in the input sequences. A MaxPooling1D layer with a pool size of 2 was applied to reduce the spatial dimensionality of the feature maps and control overfitting by retaining only the most informative features. This helps the model focus on dominant pollution trends while discarding noise and redundant temporal information. A Dropout layer (rate = 0.3) was then used for additional regularization, further improving generalization to unseen data and ensuring stable performance across different missingness conditions. The flattened output is passed through a Dense layer with 32 neurons (ReLU activation), and a final Dense layer produces the PM_2.5 concentration prediction. This single scalar output corresponds to a one-hour-ahead (t + 1) prediction. The Adam optimizer (learning rate = 0.0001) was used to enable stable convergence and efficient training across diverse temporal patterns, while early stopping (patience = 10 epochs) prevented overfitting and ensured that the model retained generalizable pollution-forecasting behaviour.

LSTM networks are a specialized type of recurrent neural network (RNN) proposed by Hochreiter and Schmidhuber⁵¹. They account for the exploding/vanishing gradient experienced by traditional RNNs through their usage of feedback loops. LSTMs excel at learning long-term dependencies by maintaining information over extended time steps through memory cells and gating mechanisms. Bidirectional LSTM (BiLSTM) is a variation of the LSTM designed by Graves and Schmidhuber⁵². BiLSTM processes input sequences in both forward and backward directions, capturing correlations in data from both the past and the future, potentially increasing prediction accuracy.

In this study, we developed an LSTM model for PM_2.5 prediction, starting with a BiLSTM layer of 64 units that captures long-term temporal dependencies in PM_2.5 concentration sequences by processing the data bidirectionally, thereby leveraging both past and future temporal information. A Dropout layer with a rate of 0.3 follows to mitigate overfitting by randomly deactivating neurons during training. A second Bidirectional LSTM layer with 32 units further refines temporal representations, followed by a Dropout layer (rate = 0.3) to improve generalization. Finally, a Dense output layer with a single neuron produces the PM_2.5 prediction, with L2 regularization (λ = 0.01) applied to penalize large weight magnitudes and improve stability. The model assigned greater weight to influential historical data while downweighting the optimization. A batch size of 16 and early stopping (patience = 10 epochs) were employed to ensure training efficiency and prevent overfitting. The final architecture of our model is further detailed in Table 6, and its workflow is illustrated in Fig. 3.

Table 6.

Bidirectional LSTM model with dropout and L2 regularization.

Component	Details
Number of Layers	4 (Bidirectional LSTM→Dropout→Bidirectional LSTM→Dropout→Dense Output)
Neurons	64 (Bidirectional LSTM)→32 (Bidirectional LSTM)→1 (Dense Output with L2 Regularization)
Activation Function	LSTM layers: Implicit (tanh and sigmoid by default)
	Dense layer: No activation (suitable for regression)
Optimizer	Adam, Learning Rate: 0.001
Batch Size	16
Epochs	200 (early stopping with patience = 10)
Sequence Length	24

Fig. 3.

Schematic diagram of the LSTM model used in this study. The model processes sequential input data through 2 LSTM layers in both forward and backward directions before producing the final output prediction.

We developed a second LSTM model that incorporates an Attention Feature Mechanism, a concept introduced by Bahdanau et al.⁵³, to enhance predictive performance. The initial motivation for incorporating attention is its ability to dynamically assign importance weights across time steps during prediction, giving greater weight to influential historical data while downweighting less significant inputs. However, recent studies^54,55 have shown that attention weights do not necessarily provide reliable or causal explanations of model predictions, and that high attention weights do not correspond to true feature importance or explanatory relevance. Therefore, we treat the attention mechanism as a modelling mechanism that provides flexible temporal weighting within the LSTM architecture. Through this approach, we aim to refine the LSTM model’s predictions and provide a more robust approach to air quality forecasting. The final training parameters and architecture of this enhanced model are presented in Table 7, with its complete workflow depicted in Fig. 4.

Table 7.

Bidirectional LSTM with attention mechanism model architecture.

Component	Details
Number of Layers	5 (Bidirectional LSTM→Attention→Reshape→Bidirectional LSTM→Dropout→Dense Output)
Neurons	64 (Bidirectional LSTM)→Attention Layer→64 (Reshape)→32 (Bidirectional LSTM)→1 (Dense Output)
Activation Function	LSTM layers: Implicit (tanh and sigmoid by default)
	Dense layer: No activation (suitable for regression)
Optimizer	Adam, Learning Rate: 0.001
Batch Size	16
Epochs	200 (early stopping with patience = 10)
Sequence Length	24

Fig. 4.

Schematic diagram of the LSTM-AM model used in this study. The model comprises stacked bidirectional LSTM layers, followed by an attention mechanism, to generate the final output.

All deep learning models were implemented in Python 3.10 using Keras with a TensorFlow 2.14 backend and trained on a Windows 11 workstation equipped with an NVIDIA RTX 4060 (8 GB) GPU. Each model was trained for up to 200 epochs with a sequence length of 24, corresponding to a day’s worth of historical input. The hyperparameters were determined empirically through iterative tuning, balancing computational cost with predictive accuracy. The final configurations, summarized in Tables 5, 6, and 7, represent the optimized architectures that achieved the lowest RMSE on the validation set.

By employing both ML and DL models, we aim to assess their strengths and determine which approach yields the most accurate and reliable predictions of PM_2.5 concentrations.

Evaluation criteria

In this study, three statistical evaluation measures were used to compare both the efficacy of imputation methods and the accuracy of the predictive models: Mean Absolute Error (MAE) 5, Root Mean Square Error (RMSE) 6, and the coefficient of determination () 7.

5

6

7

We define N as the number of sensors, T as the total number of time steps, and as the mean of the observed PM_2.5 values for sensor i.

These measures provide a comprehensive evaluation by encompassing different aspects of model performance. MAE measures the average absolute difference between predicted and actual values; lower MAE values suggest that, on average, predictions are converging toward the exact values. RMSE squares the errors before averaging, capturing the standard deviation of the prediction errors. For both MAE and RMSE, lower values can indicate better accuracy. Finally, values closer to 1 indicate a stronger correlation between predicted and observed values, indicating the model’s ability to capture the data’s variance effectively.

Results

In this section, we present the results of our experiments evaluating the impact of various imputation methods on PM2.5 forecasting accuracy. We begin with an overview of the data characteristics and preprocessing effects, followed by a comparative analysis of model performance under different transformation and imputation strategies.

Descriptive statistics

This section outlines the initial steps undertaken to evaluate the quality and distribution of PM_2.5 data collected from the LCS network. This preliminary data inspection is necessary to identify potential biases that may affect modelling results, such as data gaps and skewness. We first assess data availability across sensors and time periods, then examine the distribution of measurements, and finally discuss the transformations applied to address non-normality.

Figure 5 presents a heatmap of missing PM_2.5 data over time, with the darker red indicating a higher proportion of missing values. The x-axis represents time in months, while the y-axis corresponds to different sensor IDs. Most sensors exhibit relatively low levels of missing data, as indicated by the predominantly light grey shading. However, a few sensors, such as sensor 53221, experience periods of data loss, as seen in the red regions. Notably, missing data appears more concentrated in specific months, such as April 2024, May 2024, and August 2024, for sensor 53221.

Fig. 5.

Heatmap showing the percentage of missing PM_2.5 data for each sensor across monthly intervals from January 2023 to December 2024.

Figure 6 quantifies the percentage of missing PM_2.5 data for each sensor. While many sensors maintain low missing-data rates, sensor 53221 stands out with a significantly higher proportion of missing data than others. This suggests potential sensor malfunctions or external factors affecting data collection. The varying levels of missing values across sensors highlight the need for appropriate imputation techniques prior to further analysis.

Fig. 6.

Percentage of missing PM_2.5 data per sensor. Some sensors exhibit significantly higher missing data rates.

The distribution of PM values sampled by the selected sensors is shown in Fig. 7, and the average, median, and standard deviation are , , and , respectively. The high skewness (6.49) and kurtosis (56.59) indicate that the data are highly skewed and contain extreme outliers, suggesting the need for transformation prior to modelling.

Fig. 7.

Distribution of PM_2.5 values before Log transformation, illustrating significant right skewness.

To address this, we apply a log transformation, which normalizes the data and stabilizes the variance for positively skewed distributions, thereby improving predictive performance. After applying this transformation, the average, median, and standard deviation become 1.83, 1.65, and 0.99, respectively. The skewness is reduced to 0.86, and the kurtosis decreases to 0.69, indicating that the data are now approximately symmetric and more suitable for modelling. The updated distribution of PM can be seen in Fig. 8

Fig. 8.

Distribution of PM_2.5 values following Log transformation, indicating improved symmetry and normalization.

Comparison of skewed and unskewed data

To assess the impact of applying the Log transformation on prediction performance, we compared model performance before and after the transformation, as shown in Table 8. Post-transformation data is reported in the original scale to ensure that the values are interpretable in the original context. To generate these values, model predictions made in the logarithmic space were back-transformed using the exponential function. As inverse transformations can introduce inflation in metrics, Duan’s Smearing Estimator was applied to mitigate retransformation bias.

Table 8.

Comparison of model performance metrics between skewed and unskewed datasets (transformed under Log) without imputation.

Method	Skewed			Unskewed
	RMSE	R	MAE	RMSE	R	MAE
RF	12.8475	0.6607	5.3845	16.5419	0.4761	5.0322
XGBoost	11.8697	0.7104	4.6719	14.8117	0.5800	4.3680
CNN	16.2113	0.4784	8.3429	17.4610	0.4318	5.8269
LSTM	4.8823	0.9527	2.3729	5.0715	0.9521	1.9024
LSTM-AM	5.6123	0.9375	3.1816	5.1499	0.9506	1.9155

Evaluating the results reveals a distinct divergence in performance metrics. For traditional machine learning models, we observe a decline in and an increase in RMSE on the unskewed data. For instance, RF sees a significant drop in (0.6607 to 0.4761) and a rise in RMSE (12.85 μg/m to 16.54μg/m). This phenomenon occurs because the log transformation compresses large values; when optimizing the loss function in the transformed space, the model may tolerate larger relative errors on peak values. Upon back-transformation, these errors are amplified, disproportionately penalizing RMSE, which is sensitive to outliers.

However, the Deep Learning models demonstrated greater resilience to this effect. The LSTM maintained comparable performance, while the LSTM-AM proved to be a notable exception, improving across all metrics (RMSE decreasing from 5.61μg/m to 5.14μg/m and increasing from 0.9375 to 0.9506). Crucially, despite the fluctuations in RMSE and for the traditional models, MAE decreased for every model. The CNN, for example, saw its MAE drop from 8.34 to 5.82. This consistent reduction in MAE indicates that, although the log transformation may yield larger penalties for extreme outliers upon retransformation, it improves the model’s accuracy for most data points and enhances the overall median fit.

Comparison of imputation methods at 90% data availability

To evaluate the impact of different imputation methods on model performance, we conducted experiments using both traditional ML algorithms–RF and XGBoost–and DL models–CNN, LSTM, and LSTM-AM. Table 9 presents the RMSE, , and MAE scores for each model under different imputation strategies: No Imputation, Kriging, Mean, and KNN.

Table 9.

Performance under different imputation methods on 90% available data.

Method	No Imputation			Kriging			Mean			KNN
	RMSE	R	MAE	RMSE	R	MAE	RMSE	R	MAE	RMSE	R	MAE
RF	16.5419	0.4761	5.0322	16.1679	0.4856	4.9451	16.1848	0.4834	4.9592	16.1736	0.4845	4.9513
XGBoost	14.8117	0.5800	4.3680	14.1546	0.6052	4.2470	14.2476	0.5997	4.2654	14.2066	0.6029	4.2465
CNN	17.4610	0.4318	5.8269	16.4483	0.4724	5.4707	16.9970	0.4392	5.8331	16.9714	0.4419	5.7004
LSTM	5.0715	0.9521	1.9024	4.6848	0.9575	1.8074	4.7463	0.9563	1.8047	4.7592	0.9560	1.8347
LSTM-AM	5.1499	0.9506	1.9155	4.5208	0.9602	1.7815	4.7387	0.9564	1.8331	4.7762	0.9558	1.8135

Based on our findings, at 90% data availability, imputation shows consistent but modest improvements across all models. For the traditional ML models, performance before imputation is suboptimal, with RF achieving an of 0.4761 and XGBoost an of 0.58. Applying any type of imputation shows to improve scores, with RF increasing to a maximum of 0.4856 (Kriging), and for XGBoost, 0.6052 (Kriging). Both RMSE and MAE values show further improvement: under mean imputation, RF’s MAE decreases from 5.0322 μg/mto 4.9592 μg/m (a reduction of 0.073 μg/m, while XGBoost’s RMSE decreases from 14.8117 μg/mto 14.2066 μg/m under KNN imputation. While imputation provides marginal gains for traditional ML models, the improvements remain insufficient to achieve strong predictive performance, suggesting these models may be fundamentally limited in capturing the complex spatiotemporal patterns in PM_2.5 data.

In contrast, some DL models exhibit stronger overall performance, with substantially lower RMSE and MAE and higher than ML models. The outlier among the DL models is the CNN, which performs worse than even traditional ML models. Under no imputation, CNN features a of 0.4318, a difference of 0.0443 with RF’s . In comparison, LSTM achieves the best results, with an RMSE of 5.0715μg/m and 0.9521 without imputation.

As with traditional ML models, all DL models improve when imputation is applied. Results show a slight improvement, with the best results being LSTM-AM with Kriging, exhibiting an RMSE of 4.5208μg/m, 0.9602, and MAE of 1.7815μg/m. Among all the models, CNN benefits the most from imputation, with an improvement of 1.0127μg/m in RMSE when using Kriging. In comparison, the largest RMSE improvement for LSTM is 0.3867μg/m with Kriging and 0.6291μg/m for LSTM-AM with Kriging.

Between the three imputation methods, while performance differences are generally small, Kriging consistently performs best. For CNN, Kriging improves from 0.4318 (no imputation) to 0.4724 and reduces RMSE from 17.4610μg/m to 16.4483μg/m, indicating that it moderately enhances the model’s predictive accuracy. Both LSTM and LSTM-AM also benefit from Kriging imputation; specifically, LSTM achieves an RMSE of 5.0715μg/m and 0.9332, while LSTM-AM achieves an RMSE of 0.1387 and = 0.9229.

Comparing Kriging to other imputation methods reveals that it performs marginally better than other imputation methods. For LSTM, Kriging achieves an of 0.9575, outperforming mean imputation (, with a delta of 0.0012) and KNN (, with a delta of 0.0015). Similarly, for XGBoost, Kriging attains an of 0.6052, surpassing mean imputation (, a difference of 0.0055) and KNN (, a difference of 0.0023). RMSE values further support Kriging’s advantage: for LSTM, Kriging achieves 4.6848 μg/m versus 4.7463 μg/m (mean) and 4.7592 μg/m (KNN). This suggests that Kriging more effectively captures the underlying spatial patterns in the missing data than other imputation methods. Overall, the results indicate that Kriging may offer a slight advantage over other imputation methods, particularly for CNN and LSTM. While mean and KNN provide similar benefits, albeit with slightly less impact.

Performance under varying missingness levels

To further evaluate the robustness of our imputation methods, we simulated additional missing-data scenarios beyond the original 90% data availability. Specifically, we artificially reduced the data availability to 30% and 60% by randomly removing observations from each sensor in the original subset. This controlled simulation allowed us to assess how imputation performance varies across different levels of data sparsity.

We once again evaluate the impact of different imputation strategies on model performance using both traditional ML and DL models. Tables 10 and 11 present the evaluation metrics for 30% and 60% data availability respectively. The relative effect of imputation is further illustrated in Fig. 9, which compares changes in RMSE against the no-imputation baseline.

Table 10.

Performance under different imputation methods on 30% available data.

Method	No Imputation			Kriging			Mean			KNN
	RMSE	R	MAE	RMSE	R	MAE	RMSE	R	MAE	RMSE	R	MAE
RF	20.3675	0.5445	6.2612	20.0970	0.5569	5.9796	20.0859	0.5574	5.9751	20.0976	0.5569	5.9860
XGBoost	16.9224	0.6855	5.0637	17.3962	0.6680	4.9332	17.3997	0.6678	4.9461	17.5122	0.6636	4.9775
CNN	21.3578	0.5240	6.6488	18.0303	0.6359	5.7843	18.6175	0.6118	6.0139	19.0458	0.5937	5.9877
LSTM	4.8687	0.9753	2.0008	4.5615	0.9767	1.7366	4.2406	0.9799	1.7067	4.7263	0.9750	1.8966
LSTM-AM	5.0489	0.9734	2.0393	4.6177	0.9761	1.7556	4.4329	0.9780	1.7769	4.8337	0.9738	1.7903

Table 11.

Performance under different imputation methods on 60% available data.

Method	No Imputation			Kriging			Mean			KNN
	RMSE	R	MAE	RMSE	R	MAE	RMSE	R	MAE	RMSE	R	MAE
RF	17.5540	0.5089	5.5566	18.1263	0.5212	5.7289	18.1134	0.5219	5.7238	18.1566	0.5198	5.7281
XGBoost	15.1786	0.6328	4.6818	15.4899	0.6504	4.7543	15.4983	0.6501	4.7383	15.5156	0.6492	4.7643
CNN	16.3608	0.5939	5.7682	16.9252	0.5443	5.5621	16.9755	0.5415	5.8761	17.5008	0.5127	5.8195
LSTM	4.9974	0.9621	1.9931	4.8378	0.9628	1.8600	5.0074	0.9601	1.8928	5.4485	0.9528	1.9529
LSTM-AM	4.8677	0.9641	1.8911	5.3071	0.9552	1.8272	4.9442	0.9611	1.8771	5.7195	0.9480	1.9359

Fig. 9.

Change in RMSE relative to the no-imputation baseline across models and imputation methods for 30% and 60% data availability, where negative values indicate performance improvements.

Performance generally improves with greater data availability for traditional ML models, as evidenced by lower RMSE and MAE scores. For RF, RMSE decrease from 20.3675μg/m at 30% to 17.5540μg/m at 60%. A similar decrease is observed in XGBoost, with a delta of 1.7438 μg/m, indicating that, with more data, accuracy improves for traditional ML models. Despite improvements in RMSE, values decline by 30%- 60% with 30%- 60% data availability. For instance, for XGBoost decreases by 0.0527. This occurs because models can more easily fit the smaller, concentrated data range, suggesting the higher reflects overfitting rather than genuine predictive improvement.

In comparison, most deep learning models exhibit strong, stable performance across data availability levels, with only marginal changes in error metrics and consistently high values. For LSTM, RMSE increases slightly from 4.8687μg/m at 30% data availability to 4.9974μg/m at 60%, while MAE remains nearly unchanged(2.0008 to 1.9931 μg/m. Similarly, LSTM-AM shows minimal variation, with RMSE decreasing from 5.0489 to 4.8677 μg/m and MAE improving from 2.0393 to 1.8911 μg/m. Importantly, both models maintain very high values (above 0.96) across both settings. This stability suggests that DL models are less sensitive to changes in data availability and can effectively capture the underlying temporal structure of PM_2.5, even with reduced data. However, an exception to this trend is the CNN model, which behaves more closely to traditional ML approaches than to recurrent DL models. Although CNN performance improves with increased data availability, RMSE decreases from 21.3578 μg/m at 30% to 16.3608μg/m at 60%, its error remains substantially higher than that of the LSTM-based models. Additionally, CNN exhibits only moderate values (0.5240 at 30% and 0.5939 at 60%), indicating that it does not fit the data as well as LSTM and LSTM-AM.

At 60% data availability, most models exhibit an increase in RMSE alongside an increase in when imputation is applied. For instance, RF RMSE increases from 17.5540 μg/m(no imputation) to 18.11–18.16 μg/m across imputation methods, while increases from 0.5089 to approximately 0.52. A similar pattern is observed for XGBoost and the LSTM-based models, suggesting that, depending on the amount of available data, utilizing imputation methods can introduce additional variance or noise that slightly degrades absolute error metrics while still improving the proportion of variance explained. This indicates that imputed values may help models capture broader trends but at the cost of localized prediction accuracy.

However, this pattern is not observed at 30% data availability. With the exception of XGBoost, performance across all models improves with imputation: RMSE decreases and increases. For example, LSTM’s RMSE decreases from 4.8687 μg/m (no imputation) to 4.2406 μg/m under Mean imputation, with a corresponding increase in. from 0.9753 to 0.9799. Similar improvements are observed for RF and LSTM-AM. This suggests that at higher levels of missingness, adding imputed values can improve model learning by restoring temporal and spatial continuity, enabling both traditional and deep learning models to better capture underlying PM_2.5.

Overall, while imputation improves performance under high levels of missingness, no single imputation method consistently outperforms others across all settings. At higher levels of missingness, simpler approaches such as Mean imputation perform comparably to more complex spatial methods, whereas at lower levels of missingness, the marginal benefit of imputation diminishes and may introduce additional noise.

Statistical significance testing

To assess the statistical significance of performance differences among imputation methods, we focused our analysis on the LSTM model at 90% data availability, which achieved the best performance without imputation. Due to the computational cost of DL training, we limited formal statistical testing to the most informative condition. Using an additional five independent training runs per imputation method with different random seeds. We conducted a Friedman test followed by post-hoc Wilcoxon signed-rank tests. This approach enables us to determine whether the observed performance gains.

Table 12.

Statistical significance results from the Friedman test and Nemenyi post-hoc test comparing imputation methods (lower RMSE is better).

Comparison	p-value	Significant (p<0.05)	Better method
Friedman Test (overall)	0.0189	Yes	–
Baseline vs Kriging	0.0173	Yes	Kriging
Baseline vs Mean	0.2035	No	–
Baseline vs KNN	0.0681	No	–

The Friedman test revealed a statistically significant difference in RMSE across the imputation methods (p=0.0189), suggesting that at least one method outperformed the others. Post hoc analysis indicated that Kriging significantly outperformed the baseline approach (p = 0.0173). These results highlight Kriging’s potential as a more effective imputation strategy in our setting, particularly when compared to no imputation.

Discussion

In the following, we evaluate how the preprocessing methods used in this study affect the performance of ML and DL models. We first examine how applying the Log transformation affects predictive performance across the various models. We then evaluate the efficacy of the proposed imputation methods and conclude by highlighting Kriging’s effectiveness as an imputation technique, discussing its ability to restore values and enhance predictive performance.

Impact of log transformation on predictive performance

The application of the Log transformation influenced model behaviour by stabilizing the learning process and reducing the effect of extreme outliers. The transformation reduces skewness in the PM_2.5 distribution (Fig. 8), which mitigates the disproportionate influence of extreme values during training. The issue with skewed data is that it introduces bias when estimating relationships between variables and can distort statistical analyses, leading to inaccurate conclusions.

As demonstrated in Table 8, the application of the Log transformation combined with Duan Smearing effectively normalizes the distribution for model training while ensuring that the re-transformed predictions remain unbiased and comparable to the original observation context. This approach synthesizes the statistical benefits of a normal distribution with the practical necessity of accurate estimation in the original domain.

Many regression models can perform better on data that follows a normal distribution. For instance, although tree-based methods, like RF and XGBoost, are generally invariant to feature scaling, they are highly sensitive to the distribution of the target variable. By compressing the target range, the log transformation enforces a form of variance stabilization, allowing the trees to focus on learning structural patterns in the majority of the data, resulting in more robust decision boundaries.

In deep learning models, the transformation also facilitates smoother loss landscapes and more stable gradient propagation, indirectly enhancing model generalization within the transformed domain. Furthermore, the previous long-tailed distribution (Fig. 7) can introduce extreme values, which can disproportionately affect model training. By reducing skewness, the transformation minimizes the impact of outliers, which may lead to unstable predictions and poor generalization.

However, when examining the actual results, all models except LSTM-AM show increased and RMSE, although these apparent improvements may be misleading. Recalling that measures a model’s ability to fit the data, the observed improvements indicate that these models generalize better to skewed data, but they do not necessarily imply improved predictive performance. Looking back at Fig. 7, we observe that the original data are highly concentrated within a narrow range. Given this smaller range, models may be able to better fit the dense cluster without generalizing well to the broader distribution, potentially explaining the higher values.

Notably, traditional ML models exhibited more pronounced performance degradation compared to DL models. This divergence is likely due to fundamental differences in how these model types handle transformed data: tree-based models rely on piecewise partitioning, which loses resolution in the compressed tail of the log-distribution, causing large errors upon retransformation. In contrast, DL models approximate continuous functions, enabling them to leverage the smoothed gradient landscape without incurring discretization artifacts. This highlights the importance of considering both multiple performance metrics and model architecture when evaluating effectiveness on transformed data.

Impact of imputation methods on machine learning models

Across all levels of data availability, RF and XGBoost exhibit relatively stable performance, with XGBoost consistently outperforming RF in terms of RMSE, MAE, and . As data availability decreases from 90% to 30%, both models exhibit a gradual performance degradation, reflected by increasing RMSE and MAE and a reduction in . However, this degradation remains moderate, indicating that tree-based ensemble models can leverage the remaining feature information effectively even under substantial missingness.

At 90% data availability, the impact of imputation on RF and XGBoost is minimal. Performance under Kriging, Mean, and KNN imputation remains closely aligned with the no-imputation baseline, with marginal fluctuations in metrics. This suggests that when sufficient observed data are available, imputing missing values does not meaningfully alter the learned feature–target relationships. This nature likely stems from their feature-based regression approach, in contrast to DL models, which rely on sequential dependencies.

As data availability declines to 30%, the impact of imputation varies across models. For both RF and XGBoost, all imputation methods yield modest improvements in RMSE and MAE relative to no imputation, along with small increases in . These gains indicate that, under high missingness, imputation helps RF restore feature completeness, thereby enabling the model to better exploit inter-variable relationships. In contrast, XGBoost exhibits a trade-off; while imputation lowers MAE, it actually increases RMSE and decreases . This suggests that while imputation helps XGBoost reduce the average absolute error, it introduces larger sporadic errors that penalize the model’s overall fit.

A closer examination of 60% data availability indicates that imputation yields mixed results. While generally increases across imputation methods, RMSE and MAE often show slight degradation relative to the no-imputation baseline. This pattern suggests that imputation may help the models capture broader variance in the target variable while simultaneously introducing noise that affects pointwise prediction accuracy. In particular, the rise in RMSE indicates that outliers are becoming more pronounced, as RMSE squares larger errors, giving them greater weight. This pattern suggests that imputation methods may introduce sporadic, high-magnitude errors, thereby influencing the overall error distribution. Thus, our results emphasize the importance of understanding when imputation is beneficial, as desired outcomes may vary with research objectives.

Impact of imputation methods on deep learning models

Based on our results, at 90% data availability, DL models are better able to leverage the imputed dataset than traditional ML models. Missing values can disrupt the temporal structure of time-series data, as shown in Figure 5, thereby reducing the model’s ability to learn meaningful patterns. Imputation fills in these gaps within sequences, providing a more continuous and complete representation of the data and restoring sequential flow, thereby enabling DL models to better use contextual information.

The CNN architecture exhibits the greatest gains from imputation, although it initially underperformed relative to other deep learning models. This sensitivity stems from CNN’s structural reliance on complete, continuous grids for feature extraction. Unlike models that can process inputs more independently, CNNs are highly sensitive to sparsity, as missing values disrupt the local spatial correlations required by convolutional filters. At 90% availability, application of imputation methods exhibit measurable performance gains, most notably in the CNN architecture. For instance, Kriging leads to a notable increase in from 0.4318 to 0.4724 and a decrease in RMSE from 17.4610 μg/m to 16.4483μg/m, suggesting that CNN can improve performance with higher-quality, uninterrupted data. This sensitivity arises because CNNs are designed to learn spatial patterns using convolutional filters that capture local dependencies. Therefore, missing values imputed using methods that preserve spatial structure, such as Kriging, yield inputs that better align with CNN biases, enabling the network to extract meaningful patterns more effectively. This is further exemplified by KNN and Mean, which improve CNN performance but not to the same extent as Kriging, due to their inability to handle outliers effectively. For example, KNN imputes missing values by taking the weighted average of the values from the point’s nearest neighbours. The presence of an outlier may significantly skew the imputed values. Compared with KNN, Kriging is more robust because it accounts for spatial correlations in values, whereas KNN relies solely on local similarity.

On the other hand, LSTM and LSTM-AM exhibit moderate improvement at 90% data availability. The robustness of LSTM models may stem from their internal memory mechanism, which retains information over time and can infer patterns from historical context. Even when values are missing, these models can rely on previous and future time steps to reconstruct the underlying signal, thereby smoothing over gaps without requiring explicit imputation. As a result, the moderate performance gains from applying imputation suggest that LSTMs already have a built-in capacity to compensate for data discontinuities. Interestingly, LSTM outperforms LSTM-AM, implying that the added attention mechanism may not constantly improve predictive accuracy in the presence of missing data. This behaviour may be related to the attention mechanism placing disproportionate weight on less informative or misaligned time steps during sequence processing. Overall, their ability to capture temporal dependencies enables them to better handle missing data than CNNs. While imputation does improve performance, the gains are less pronounced, reinforcing the notion that LSTMs are more resilient to missing values than CNNs.

Kriging performance

Across most models, Kriging achieves the lowest RMSE, demonstrating its effectiveness as an imputation method. This is likely due to Kriging’s ability to handle outliers effectively while preserving spatial and temporal correlations in the data. Unlike mean imputation, which replaces missing values with a constant average, and KNN, which estimates values based on local neighbours, Kriging leverages spatial interpolation techniques to generate smoother, more contextually relevant estimates. Thus, explaining how Kriging consistently improves model performance to a greater extent than other imputation methods. Overall, these findings underscore the importance of selecting an appropriate imputation method when developing these models. While ML models remain robust regardless of imputation, DL models benefit significantly from high-quality imputation, as demonstrated with CNN.

Limitations and future directions

While this study provides valuable insights into PM_2.5 forecasting, we acknowledge that several limitations should be addressed. First, the data used in this research were collected primarily from Edmonton, Alberta. This geographic focus was a deliberate choice, as Alberta’s well-established AQMS networks and comprehensive digital databases provide high-resolution temporal data at hourly intervals, enabling us access to high-quality, large-scale datasets essential for model development. However, this decision may limit the spatial generalizability of our findings, as local climatic conditions, emission sources, and sensor placement characteristics can vary across urban and regional contexts. For example, LCS sensor data can be influenced by local air pollution, including traffic, industrial activity, and episodic events such as wildfires. Incorporating these variables into future research could enhance model performance and provide deeper insights into PM_2.5 dynamics.

Second, although our selection criteria produced a dataset suitable for controlled-method comparison, they impose limits on external generalizability. The sensors selected for this study constitute a high-data-coverage subset. Still, as PurpleAir PA-II sensors remain subject to bias and environmental effects, there is no assurance that results can be generalized to sparser or noisier deployments. We therefore recommend further validation on larger, lower-coverage networks.

Third, while this study reviews the efficacy of Kriging for data imputation, it has drawbacks. Kriging assumes spatial stationarity, meaning data patterns remain relatively constant across space. However, this may not always hold due to variance from local pollution events. This assumption may limit scalability for deployments spanning large areas, although the method remains suitable for local or regional predictions. Another limiting factor for real-time deployability is Kriging’s considerable computational complexity compared to other imputation methods. This complexity stems from the need to build a variogram model to characterize the spatial correlation structure of the data. Despite these limitations, Kriging requires fewer computational resources than deep learning-based imputations, though the trade-offs between computational efficiency and predictive performance remain an open question.

While this study focused primarily on evaluating model accuracy and robustness, an in-depth analysis of computational complexity and runtime efficiency was beyond its scope and is left for future work, particularly for real-time or large-scale deployment scenarios. Future research should investigate when more advanced learning-based imputation techniques, such as autoencoders or Generative Adversarial Imputation Networks (GAIN), outperform traditional spatial methods, especially in handling complex patterns of missing data. Another important direction for future work is to explore hybrid imputation strategies that combine spatial and temporal methods (e.g., Kriging followed by sequence-aware temporal imputation).

While this study focused primarily on evaluating model accuracy and robustness, an in-depth analysis of computational complexity and runtime efficiency was beyond its scope and is left for future work, particularly for real-time or large-scale deployment scenarios. Future research should investigate hybrid imputation strategies that combine spatial and temporal methods, such as Kriging followed by sequence-aware temporal imputation. Further research could also include more advanced learning-based techniques, such as Generative Adversarial Imputation Networks (GAIN) and autoencoders, and test whether they outperform traditional spatial methods, particularly in handling complex patterns of missing data.

Finally, our modelling choices also introduce limitations. Although our study explored several ML and DL architectures, it would be valuable in future work to include additional models, such as transformer-based architectures, which have shown strong performance in sequential data modelling. Integrating a broader range of models and features could improve the robustness and generalizability of PM2.5 forecasting systems. Addressing these limitations offers essential avenues for future research to enhance the practical deployment of air quality prediction tools and to support informed environmental policy decisions.

Conclusions

In this study, we aimed to develop effective PM2.5 prediction models for data collected from LCS networks. Specifically, we collected data from LCSs deployed in Edmonton, Alberta. However, we found that the dataset had a high proportion of missing values, a standard limitation of LCS devices due to sensor malfunctions, environmental interference, and power disruptions. To address this issue, we evaluated the efficacy of various imputation strategies for handling missing data. We evaluated Kriging, an underexplored spatial imputation technique for PM2.5 forecasting, alongside KNN and mean imputation. We compared these techniques against a benchmark with no imputation to determine whether imputation improves overall model performance. Our assessment examined how each technique affected performance across both traditional ML (RF, XGBoost) and DL (CNN, LSTM, LSTM-AM) models. Our findings revealed that while traditional ML models showed no significant change in performance relative to the benchmark, DL models exhibited a more pronounced improvement. This can be explained by the fact that these techniques effectively restore temporal structures disrupted by missing values, enabling DL models to better capture complex patterns in PM_2.5 data. On average, Kriging marginally outperformed other imputation methods, demonstrating its ability to preserve spatial and temporal correlations better when estimating missing values. These results establish Kriging as an effective preprocessing technique for PM_2.5 datasets with missing values in short-horizon forecasting settings. Future work should examine Kriging’s performance relative to other imputation methods, including hybrid and learning-based methods.

Author contributions

RC was responsible for conceptualization, data collection, investigation, methodology, software development, visualization, and writing the original draft. Interpretation of the results was performed by both RC and NE. NE and SC acquired funding, provided resources, supervised the project, and contributed to project administration. Validation was conducted by RC, NE, and SC. Review and editing were carried out by RC, NE, and SC.

Data availability

The low-cost sensor data used in this study are publicly available through the PurpleAir data download tool. Additional meteorological and air quality features can be accessed via the Alberta Air Data Warehouse.

Declarations

Competing interests

The authors declare no competing interests.