Skip to main content
NCBI home page
ACS Omega logoLink to ACS Omega
. 2024 May 16;9(21):23021–23032. doi: 10.1021/acsomega.4c02681

Adaptive Temporal–Spatial Pyramid Variational Autoencoder Model for Multirate Dynamic Chemical Process Soft Sensing Application

Bingbing Shen †, Zeyu Yang ‡,*, Le Yao †,*
PMCID: PMC11137708  PMID: 38826556

Abstract

graphic file with name ao4c02681_0013.jpg

Data-driven soft sensors play an important role in practical processes and have been widely applied. They provide real-time prediction of quality variables and then guide production and improve product quality. In practical chemical production processes, nonlinear dynamic multirate data is widespread and challenging to model. This paper innovatively proposes a temporal–spatial pyramid variational autoencoder (TS-PVAE) model for the nonlinear temporal–spatial feature pyramid extraction from multirate data. This structure not only selectively utilizes multirate data but also handles complex nonlinear time-series data. Based on this, integrated with just-in-time (JIT) learning, an adaptive TS-PVAE (ATS-PVAE) model is developed. In this model, historical data are used for real-time fine-tuning of the model, leading to the development of an adaptive model. Finally, the proposed models are validated by an industrial case of a methanation furnace, demonstrating a superior estimation performance.

Introduction

As chemical processes develop, production processes become increasingly intricate, with a heightened level of integration in production equipment. Real-time monitoring is of great significance for increasing production, reducing energy consumption, and ensuring safety. At the same time, with the development of Internet of Things (IoT) technology, data acquisition has become easier. Therefore, there is a growing focus and challenge in analyzing and modeling process data to develop data-driven process monitoring models.1−4 Quality variables are important indicators that can reflect the status of the production process.5 However, their measurement is usually expensive and has time delays. Establishing data-driven soft sensor models for quality variables can predict them quickly and at low cost and improve production efficiency.

In the collected process data, there are various types of data, such as flow rate, pressure, temperature, and valve data. This data is obtained from different devices with varying sampling rates.6 However, most existing models downsample process data according to the sampling rate of the quality variables. Information from the process data is lost during downsampling. To effectively utilize a large amount of process variable information, some scholars have proposed semisupervised learning methods.7−9 For example, Yao and Ge proposed a hierarchical extreme learning machine based soft sensor for semisupervised process data.10 These methods involve establishing semisupervised learning models by combining a large amount of unlabeled process data with a small amount of labeled process data, further extracting more data information and establishing more effective soft sensing models. However, semisupervised learning can only handle dual-rate data, remaining powerless when it comes to multirate data. To address multirate data modeling problems, some scholars have proposed multirate fault diagnosis models and soft sensing models.11−13 Zhou et al. developed a multirate principal component regression model for soft sensor modeling.14 Shen et al. proposed a pyramid variational autoencoder (PVAE) model for nonlinear multirate industrial data.15

The application of deep learning models in the field of soft sensing has become a highly regarded research direction.16−18 These models can extract complex feature information from large-scale, high-dimensional process data and possess strong nonlinear modeling capabilities. Depending on the specific soft sensing problem and data characteristics, suitable deep learning model architectures can be developed, including deep neural networks (DNN), convolutional neural networks (CNN),19−21 stacked autoencoder (SAE), variational autoencoder (VAE),22−24 etc. Yuan et al. designed a variable-wise weighted SAE for soft sensor modeling.25

In chemical processes, the process data often exhibits dynamic temporal characteristics, requiring the establishment of specific soft sensing models.26−28 As a result, deep time-series soft sensing models based on methods such as recurrent neural networks (RNN), long short-term memory networks (LSTM),29,30 Hidden markov models (HMM),31,32 etc., have been developed. Jiang et al. built a dynamic temporal dependency model based on the similarities between the short-term load forecasting and the transformer architecture.27 Dai et al. developed a time series denoising diffusion probabilistic model to enhance time-series samples for industrial soft sensing.28 Liu et al. proposed a soft sensor based on correntropy long short-term memory for industrial polyethylene process.33 Zheng et al. proposed a supervised hybrid CNN-LSTM network for nonlinear dynamic data soft sensor modeling.34 Geng et al. designed a gated convolutional neural network based transformer regression model for industrial dynamic processes.35

Although dynamic network structures can retain long-term data information, memorable information is limited. In a prolonged dynamic process, it is necessary to combine existing dynamic information with similar historical information to adjust the model in real time to obtain the best adaptive soft sensor model. Just-in-time (JIT) learning26,36 methods can identify similar historical data for test data, establish local models, and then achieve better prediction accuracy. Since JIT models can build models based on the target data, they are suitable for complex nonlinear data and multimode processes. In recent years, soft sensor models based on JIT learning have been widely developed and applied.37 Recently, Xie et al. proposed a novel JIT model by combining with non-Gaussian information on the operation data.38 Guo et al. developed a JIT learning soft sensor based on evolutional multiobjective optimization and VAE.39 In order to solve the modeling problem of nonlinear dynamic multirate data, this paper needs to establish complex deep temporal networks. However, due to the complexity of the model and the abundance of data, JIT learning methods can help select a small amount of data for targeted adjustments and optimizations of existing networks.

In this paper, a temporal–spatial pyramid variational autoencoder (TS-PVAE) model is proposed to deal with the nonlinear dynamic multirate data modeling problem. The existing PVAE model is capable of effectively learning and establishing soft sensor models for multirate nonlinear data. TS-PVAE further dynamically extends multirate data, capturing the dynamic features of the data through LSTM and utilizing them for soft sensor modeling. Since dynamic modeling requires a large amount of data information, leading to the forgetting of historical information, to address this issue the concept of JIT learning is introduced, and an adaptive TS-PVAE (ATS-PVAE) model is proposed. This involves selecting historically similar data to adaptively adjust the existing model, thereby obtaining a more accurate prediction model.

The remainder of this work is organized as follows. In the Preliminaries, a brief introduction to LSTM and a review of the PVAE model are provided. Subsequently, this article presents a detailed introduction to the TS-PVAE model. Furthermore, the ATS-PVAE model is introduced. Finally, an industrial example is used to validate the effectiveness of the model.

Preliminaries

LSTM

Long Short-Term Memory (LSTM) networks, a variant of RNNs, are tailored for processing sequential data by effectively capturing temporal dependencies with strong nonlinear capabilities. Comprising input gates, forget gates, output gates, and memory cells, LSTM architectures address the vanishing or exploding gradient issues inherent in traditional RNNs.

Mathematically, the behavior of an LSTM unit is described as follows:

graphic file with name ao4c02681_m001.jpg 1
graphic file with name ao4c02681_m002.jpg 2
graphic file with name ao4c02681_m003.jpg 3
graphic file with name ao4c02681_m004.jpg 4
graphic file with name ao4c02681_m005.jpg 5
graphic file with name ao4c02681_m006.jpg 6

Here, xt is the input at time t; ht is the hidden state; ct is the cell state; and it, ft, ot, and gt are the input gate, forget gate, output gate, and cell gate activations, respectively. W is weight matrices; b is bias vectors; σ denotes the sigmoid function; and ⊙ denotes element-wise multiplication. The structure of an LSTM is given in Figure 1.

Figure 1.

Figure 1

Open in a new tab

Structure of a LSTM.

In the context of time series data, LSTM networks excel in capturing intricate patterns and dependencies, even in the presence of noise and irregularities. By leveraging their inherent nonlinear processing capabilities and robust memory management, LSTM networks offer a reliable framework for extracting salient features from time-varying signals, while preserving temporal dynamics.

Revisit of PVAE

The PVAE model first utilizes a multirate filter to restructure irregular multirate data into regular multiresolution data. Then, by utilizing VAEs to extract data features from different resolutions, feature pyramids with different resolutions are constructed for regression modeling. Figure 2 illustrates a specific example with four resolutions. In this figure, it is assumed that process variables x1–x4 are sampled every minute; x5–x8 are sampled every 2 min; x9–x12 are sampled every 3 min; and quality variables y1 and y2 are sampled every 6 min. In this example, starting from each sampling rate, variables with higher sampling frequency at that time are found and concatenated to form four multiresolution data sets.

Figure 2.

Figure 2

Open in a new tab

Diagram of the PVAE model.

Afterward, multiresolution features are extracted from these reorganized four multiresolution data sets. Feature pyramids are constructed through these extracted features, along with lower-resolution features, and are used separately for prediction. The detailed model derivation and implementation can be found in ref (15).

Proposed TS-PVAE for Soft Sensor Modeling

To address the modeling problem of dynamic nonlinear multirate data, this paper proposes a TS-PVAE model. This model extends the PVAE model to a dynamic version. LSTM is embedded into encoders to learn and extract nonlinear features from time series data.

First, the multirate data filter of the PVAE model is used to reorganize the data into multiresolution data, and then the data are dynamically expanded to time series multiresolution data. Similar to the PVAE model, establishing a temporal–spatial pyramid model requires extracting each resolution data point separately to integrate low-resolution features into higher resolutions. Taking an example from the PVAE model, after restructuring, there are four resolution data sets. First, it is necessary to construct a temporal–spatial feature extraction model for temporal resolution 4. In this paper, LSTM is employed to process the temporal resolution 4 data; then temporal–spatial features for resolution 4 are extracted through the VAE model. These temporal–spatial features are used for data reconstruction, and a regression model is established using a multilayer perceptron (MLP). The specific training model diagram of TS-PVAE for the resolution 4 (TS-PVAE-r4) data set is shown in Figure 3.

Figure 3.

Figure 3

Open in a new tab

Structure of the TS-PVAE-r4 model.

Detailed derivation of the marginal likelihood for TS-PVAE-r4 in Figure 3 is shown in Appendix 1. The loss function of TS-PVAE-r4 can be given as

graphic file with name ao4c02681_m007.jpg 7

where x(1:t) represents the process time series variables; y(1:t) represents the quality time series variable; z(1:t) represents the latent variables from time series input; subscript r4(1:t) indicates time series resolution 4; and it is assumed that the relationships of x(1:t) and y(1:t) are independent of each other conditionally.

The trained TS-PVAE-r4 model provides temporal–spatial features from the time series resolution 4 data set, serving as the first layer of the feature pyramid. These top-level temporal–spatial features are further introduced into the TS-PVAE for resolution 3 (TS-PVAE-r3) for modeling. The specific model structure is illustrated in Figure 4.

Figure 4.

Figure 4

Open in a new tab

Structure of the TS-PVAE-r3 model.

According to the model structure in Figure 4, the loss function for TS-PVAE-r3 can be derived as

graphic file with name ao4c02681_m008.jpg 8

where subscript r3(1:t) indicates the time series resolution 3 data set and subscript r3,i(1:t) represents time series resolution 3 – i (i = 1, 2). Detailed derivation of the marginal likelihood for TS-PVAE-r3 is given in Appendix 2. To address the data imbalance problem caused by different sampling rates in different networks, a balance coefficient α is introduced into this loss function.

In the TS-PVAE-r3 model, a two-layer temporal–spatial feature pyramid is constructed and used for predicting quality variables. Furthermore, the two-layer temporal–spatial features are introduced into the TS-PVAE-r2 model to extract the third layer temporal–spatial features and build the regression model. The model structure is illustrated in Figure 5.

Figure 5.

Figure 5

Open in a new tab

Structure of the TS-PVAE-r2 model.

Then, the loss function of TS-PVAE for resolution 2 (TS-PVAE-r2) in Figure 5 can be given as

graphic file with name ao4c02681_m009.jpg 9

Here, r2(1:t) indicates the time series resolution 2 data set, and subscript r2,i(1:t) represents time series resolution 2 – i (i = 1, 2, 3). Similar to the above derivation, β is introduced as a balance coefficient. Detailed derivation of the marginal likelihood for TS-PVAE-r2 is given in Appendix 3.

In the TS-PVAE for resolution 1 (TS-PVAE-r1), a three-layer temporal–spatial feature pyramid is introduced. The model structure is shown in Figure 6.

Figure 6.

Figure 6

Open in a new tab

Structure of the TS-PVAE-r1 model.

Further, the loss function of TS-PVAE-r1 can be written as

graphic file with name ao4c02681_m010.jpg 10

where subscript r1(1:t) indicates the time series resolution 1 data set; subscript r1,i(1:t) represents time series resolution 1 – i (i = 1, ..., 6); and γ is also a balance coefficient. Detailed derivation of the marginal likelihood for TS-PVAE-r1 is given in Appendix 4.

Proposed ATS-PVAE for Soft Sensor Modeling

To further utilize historical data for adjusting and optimizing the model, an ATS-PVAE model (Figure 7) based on just-in-time (JIT) learning is proposed in this section. This model adjusts and optimizes the trained model by searching for similar data sets in historical data. For the TS-PVAE-ri model (i represents the resolution, and nri data are selected by comparing the Euclidean distance between the current process data with prediction and the historical process data).

graphic file with name ao4c02681_m011.jpg 11

where q represents the current time-series test data to be queried, while j represents the time-series data from the historical training set.

Figure 7.

Figure 7

Open in a new tab

Flowchart of the ATS-PVAE model.

Case Study

Methanation Furnace

In the production process of synthetic ammonia, nitrogen (N2) and hydrogen (H2) are the main feed gases, typically mixed in a certain ratio and fed into the reactor for the Haber–Bosch process. Hydrogen is usually prepared from natural gas or other hydrocarbons through methods such as steam reforming or partial oxidation, which may produce carbon monoxide and carbon dioxide as byproducts.

Before entering the ammonia synthesis reactor, it is necessary to remove these byproducts through the methanation furnace unit. In the methanation furnace unit, catalyzed by nickel and under high-temperature conditions, the following chemical reactions occur:

graphic file with name ao4c02681_m012.jpg 12

The purpose of this unit is to remove carbon monoxide and carbon dioxide from the feed gases as much as possible, making the residual levels of these gases at the outlet crucial quality variables. In practical production processes, precise measurement of carbon monoxide and carbon dioxide residuals relies on expensive mass spectrometers in laboratory settings, which is not feasible for large-scale synthetic ammonia processes.

Hence, it is possible to establish a soft sensor model linking easily measurable process variables with quality variables representing carbon monoxide and carbon dioxide residuals, enabling timely assessment and adjustment of the methanation furnace unit’s production process and quality conditions. Operators of this facilitation can adjust feed gas proportions effectively based on the soft sensor to prevent production accidents. Figure 8 illustrates the production process of the methanation furnace unit, and Table 1 presents the process variables collected from the methanation furnace unit.

Figure 8.

Figure 8

Open in a new tab

Flowchart of the methanation furnace.

Table 1. Description of the Process Instruments in Methanation Furnace.

Tags Descriptions
U1 Flow rate of process gas at MF’s exit
U2 Pressure of process gas at MF’s entrance
U3 Pressure of process gas into F3’s entrance
U4 Temperature of upper bed at MF
U5 Temperature of middle bed at MF
U6 Temperature of lower bed at MF
U7 Temperature of process gas at MF’s exit
U8 Temperature of process gas at MF’s exit
U9 Temperature of synthesis gas at F2’s exit
U10 Liquid level of F2
Y Content of CO and CO2 at MF’s exit
Open in a new tab

Results and Analysis

In this process, due to the different technical conditions of the measuring instruments and the discrepant nature of the sensors, the process variables are in multiple sampling rates. The time span of the collected data set for modeling is 12000 min. Among the 11 variables, the U1, ···, U5 are sampled every minute; U6, ···, Ux8 are sampled every 2 min; U9, ···, U10 are sampled every 3 min; and the quality variable y is sampled every 30 min. In this case, the data of the first 6000 min are used for training, while the rest of the 6000 data points are used for testing. Then, in total 6000 U1 to U5, 3000 samples of U6 to U8, 2000 samples of U9 to U10, and 200 samples of y are prepared for model training. The remaining 6000 samples of U1 to U5, 3000 samples of U6 to U8, 2000 samples of U9 to U10, and 200 samples of y are taken as the testing data set. The balance coefficients α, β, and γ are also set to 1/2, 1/3, and 1/6, respectively. According to the principle of the multirate data filter in the PVAE model, we selected the least common multiple of the four sampling rates to partition the data. In this vase, every 6 time steps form one group of data. Since y is sampled every 30 min, 5 groups of data can be used to predict y using a dynamic network. The specific composition of the time-series data is shown in Table 2.

Table 2. Reorganized Time Series Multi-Resolution Data.

  Subresolution 1:t = 1:5 Reorganized data
Time Series Resolution 4   T6, T12, ···, T30 U1–U10
Time Series Resolution 3 Resolution 3–1 T3, T9, ···, T27 U1–U5, U9–U10
Resolution 3–2 T6, T12, ···, T30 U1–U10
Time Series Resolution 2 Resolution 2–1 T2, T8, ···, T26 U1–U8
Resolution 2–2 T4, T10, ···, T28 U1–U8
Resolution 2–3 T6, T12, ···, T30 U1–U8
Time Series Resolution 1 Resolution 1–1 T1–T25 U1–U5
Resolution 1–2 T2–T26 U1–U5
Resolution 1–3 T3–T27 U1–U5
Resolution 1–4 T4–T28 U1–U5
Resolution 1–5 T5–T29 U1–U5
Resolution 1–6 T6–T30 U1–U5
Open in a new tab

The prediction performances of the proposed TS-PVAE model and ATS-PVAE model are compared with dimensionality-reduced LSTM and LSTM trained on all multirate data (Mr-LSTM). The detailed model structure and parameters of the proposed models and comparison models are shown in Table 3. The mean absolute error (MAE) and root means square error (RMSE) are taken as indicators to measure the experimental results.

graphic file with name ao4c02681_m013.jpg 13
graphic file with name ao4c02681_m014.jpg 14

where yi is the real value; Inline graphic is the prediction value; N is the length of query samples; L is the length of labeled samples; and Inline graphic is the mean value of the labeled outputs. MAE and RMSE can measure the predicted error. Generally, the best predictions correspond to the lowest MAE and RMSE.

Table 3. Model Structure and Parameters.

  LSTM Encoder Latent variable Decoder Regression Epoch
TS-PVAE-r4 [20,10] [15,10] [8] [10,15] [10,5] 300
TS-PVAE-r3 [20,10] [10,8] × 2 [8] × 2 [8,10] × 2 [10,5] 300
TS-PVAE-r2 [20,10] [8,8] × 3 [8] × 3 [8,8] × 3 [10,5] 400
TS-PVAE-r1 [20,10] [8,8] × 6 [8] × 6 [8,8] × 6 [10,5] 500
LSTM [20,10] / / / / 1000
MR-LSTM [20,10] / / / / 1000
Open in a new tab

The ATS-PVAE model and the TS-PVAE model share the same network structure, but the network parameters are fine-tuned for the TS model based on 50 selected similar data sets. The evaluation indices are listed in Tables 4–Table 6. The prediction results of TS-PVAE-r4, TS-PVAE-r2, ATS-PVAE-r4, and ATS-PVAE-r2 are shown in Figures 9–12.

Table 4. MAE of the Proposed Models in the Methanation Furnace.

  r4 r3 r2 r1
TS-PVAE 0.1218 0.1223 0.1209 0.1202
ATS-PVAE 0.1143 0.1131 0.1035 0.1092
Open in a new tab

Table 6. MAE and RMSE of LSTM and MR-LSTM in the Methanation Furnace.

  MAE RMSE
LSTM 0.1703 0.2178
MR-LSTM 0.1617 0.1990
Open in a new tab

Figure 9.

Figure 9

Open in a new tab

Prediction performance (TS-PVAE-r4).

Figure 12.

Figure 12

Open in a new tab

Prediction performance (ATS-PVAE-r2).

Figure 10.

Figure 10

Open in a new tab

Prediction performance (TS-PVAE-r2).

Figure 11.

Figure 11

Open in a new tab

Prediction performance (ATS-PVAE-r4).

From Figures 9–12 and Table 5, it can be observed that as the temporal–spatial feature pyramid is constructed the TS-PVAE-r2 model achieves the best estimation results when using a three-layer pyramid. The LSTM model only uses down-sampled data, and the model with less data is worse. The MR-LSTM model uses all the multirate data, although the model effect has been improved, but the data has not been reorganized. The temporal–spatial relationship of the data is complicated, so the model effect is not good. Moreover, it can be found that compared to TS-PVAE mdoels ATS-PVAE models have achieved an overall improvement in performance. Additionally, ATS-PVAE-r2 achieves the best prediction performance. In other words, fine-tuning the model with a small amount of similar time series data can enhance the prediction performance of the model.

Table 5. RMSE of the Proposed Models in the Methanation Furnace.

  r4 r3 r2 r1
TS-PVAE 0.1489 0.1480 0.1465 0.1483
ATS-PVAE 0.1389 0.1426 0.1307 0.1329
Open in a new tab

Conclusions

In this paper, a TS-PVAE model is proposed to build a soft sensor for nonlinear dynamic multirate data. The multirate data can be reorganized through the basic PVAE model. Then, the TS-PVAE model extracts nonlinear temporal–spatial features from multiple-time series resolution data. These features construct multiple temporal–spatial feature pyramids for regression modeling. As the feature pyramid expands, the prediction accuracy of the regression model improves. When there is sufficient temporal–spatial feature information, the model achieves an optimal prediction performance. However, when there is information redundancy, the model prediction performance deteriorates. To obtain the best prediction performance of the model, this paper integrated the JIT strategy, utilizing a small amount of similar data for model fine-tuning, and developed the ATS-PVAE model. Finally, a real industrial case has validated the effectiveness and superiority of the ATS-PVAE model. Furthermore, this model can be used for a quality related model. Related feature selection can be carried out through the correlation with quality variables in model building.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China (NSFC) (62303146, 62203169, 62003300), the Natural Science Foundation of Zhejiang Province (LQ23F030004, LQ22F030009), and Huzhou Key Laboratory of Intelligent Sensing and Optimal Control for Industrial Systems (grant 2022-17).

Appendix

1. The detailed derivation of the marginal likelihood for TS-PVAE-r4 can be given as

graphic file with name ao4c02681_m017.jpg 15

2. The detailed derivation of the marginal likelihood for TS-PVAE-r3 can be given as

graphic file with name ao4c02681_m018.jpg 16

3. The detailed derivation of the marginal likelihood for TS-PVAE-r2 can be given as

graphic file with name ao4c02681_m019.jpg 17

4. The detailed derivation of the marginal likelihood for TS-PVAE-r1 can be given as

graphic file with name ao4c02681_m020.jpg 18

The authors declare no competing financial interest.

References

  1. Ding S. X. Data-driven design of monitoring and diagnosis systems for dynamic processes: A review of subspace technique based schemes and some recent results. Journal of Process Control 2014, 24, 431–449. 10.1016/j.jprocont.2013.08.011. [DOI] [Google Scholar]
  2. Jiang Y.; Yin S.; Dong J.; Kaynak O. A review on soft sensors for monitoring, control, and optimization of industrial processes. IEEE Sensors Journal 2021, 21, 12868–12881. 10.1109/JSEN.2020.3033153. [DOI] [Google Scholar]
  3. Yang Z.; Ge Z. On paradigm of industrial big data analytics: From evolution to revolution. IEEE Transactions on Industrial Informatics 2022, 18, 8373–8388. 10.1109/TII.2022.3190394. [DOI] [Google Scholar]
  4. Yao L.; Ge Z. Scalable learning and probabilistic analytics of industrial big data based on parameter server: Framework, methods and applications. Journal of Process Control 2019, 78, 13–33. 10.1016/j.jprocont.2019.03.017. [DOI] [Google Scholar]
  5. Yao L.; Ge Z. Variable selection for nonlinear soft sensor development with enhanced binary differential evolution algorithm. Control Engineering Practice 2018, 72, 68–82. 10.1016/j.conengprac.2017.11.007. [DOI] [Google Scholar]
  6. Yuan X.; Xu N.; Ye L.; Wang K.; Shen F.; Wang Y.; Yang C.; Gui W. Attention-Based Interval Aided Networks for Data Modeling of Heterogeneous Sampling Sequences With Missing Values in Process Industry. IEEE Transactions on Industrial Informatics 2024, 20, 5253. 10.1109/TII.2023.3329684. [DOI] [Google Scholar]
  7. Lu B.; Chiang L. Semi-supervised online soft sensor maintenance experiences in the chemical industry. Journal of Process Control 2018, 67, 23–34. 10.1016/j.jprocont.2017.03.013. [DOI] [Google Scholar]
  8. Shao W.; Li X.; Yao Y.; Chen J.; Zhao D. Semi-supervised local manifold regularization model based on dual representation for industrial soft sensor development. Chemometrics and Intelligent Laboratory Systems 2023, 242, 104937 10.1016/j.chemolab.2023.104937. [DOI] [Google Scholar]
  9. Yao L.; Shen B.; Cui L.; Zheng J.; Ge Z. Semi-supervised deep dynamic probabilistic latent variable model for multimode process soft sensor application. IEEE Transactions on Industrial Informatics 2023, 19, 6056–6068. 10.1109/TII.2022.3183211. [DOI] [Google Scholar]
  10. Yao L.; Ge Z. Deep learning of semisupervised process data with hierarchical extreme learning machine and soft sensor application. IEEE Transactions on Industrial Electronics 2018, 65, 1490–1498. 10.1109/TIE.2017.2733448. [DOI] [Google Scholar]
  11. Lyu Y.; Zhou L.; Cong Y.; Zheng H.; Song Z. Multirate Mixture Probability Principal Component Analysis for Process Monitoring in Multimode Processes. IEEE Transactions on Automation Science and Engineering 2024, 21, 2027. 10.1109/TASE.2023.3253285. [DOI] [Google Scholar]
  12. Song B.; Zhou Y.; Shi H.; Tao Y.; Tan S. A Soft Sensor for Multirate Quality Variables Based on MC-CNN. IEEE Transactions on Neural Networks and Learning Systems 2024, 1. 10.1109/TNNLS.2024.3360030. [DOI] [PubMed] [Google Scholar]
  13. Huang K.; Wu S.; Li Y.; Yang C.; Gui W. A multi-rate sampling data fusion method for fault diagnosis and its industrial applications. Journal of Process Control 2021, 104, 54–61. 10.1016/j.jprocont.2021.06.003. [DOI] [Google Scholar]
  14. Zhou L.; Wang Y.; Ge Z. Multi-rate principal component regression model for soft sensor application in industrial processes. Science China. Information Sciences 2020, 63, 149205 10.1007/s11432-018-9624-8. [DOI] [Google Scholar]
  15. Shen B.; Yao L.; Ge Z. Predictive modeling with multiresolution pyramid VAE and industrial soft sensor applications. IEEE Transactions on Cybernetics 2023, 53, 4867–4879. 10.1109/TCYB.2022.3143613. [DOI] [PubMed] [Google Scholar]
  16. LeCun Y.; Bengio Y.; Hinton G. Deep learning. nature 2015, 521, 436–444. 10.1038/nature14539. [DOI] [PubMed] [Google Scholar]
  17. Shang C.; Yang F.; Huang D.; Lyu W. Data-driven soft sensor development based on deep learning technique. Journal of Process Control 2014, 24, 223–233. 10.1016/j.jprocont.2014.01.012. [DOI] [Google Scholar]
  18. Wang Y.; Yan P. RegGAN: A Virtual Sample Generative Network for Developing Soft Sensors with Small Data. ACS omega 2024, 9, 5954. 10.1021/acsomega.3c09762. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Jiang X.; Ge Z. Augmented multidimensional convolutional neural network for industrial soft sensing. IEEE Transactions on Instrumentation and Measurement 2021, 70, 1–10. 10.1109/TIM.2021.3075515.33776080 [DOI] [Google Scholar]
  20. Yuan X.; Wang Y.; Wang C.; Ye L.; Wang K.; Wang Y.; Yang C.; Gui W.; Shen F. Variable Correlation Analysis-based Convolutional Neural Network for Far Topological Feature Extraction and Industrial Predictive Modeling. IEEE Transactions on Instrumentation and Measurement 2024, 73, 1. 10.1109/TIM.2024.3373085. [DOI] [Google Scholar]
  21. Yuan X.; Huang L.; Ye L.; Wang Y.; Wang K.; Yang C.; Gui W.; Shen F. Quality Prediction Modeling for Industrial Processes Using Multiscale Attention-Based Convolutional Neural Network. IEEE Transactions on Cybernetics 2024, 54, 2696. 10.1109/TCYB.2024.3365068. [DOI] [PubMed] [Google Scholar]
  22. Kingma D. P.; Welling M.. Auto-encoding variational bayes. arXiv 2013. 10.48550/arXiv.1312.6114 [DOI] [Google Scholar]
  23. Doersch C.Tutorial on variational autoencoders. arXiv 2016. 10.48550/arXiv.1606.05908 [DOI] [Google Scholar]
  24. Shen B.; Yao L.; Ge Z. Nonlinear probabilistic latent variable regression models for soft sensor application: From shallow to deep structure. Control Engineering Practice 2020, 94, 104198 10.1016/j.conengprac.2019.104198. [DOI] [Google Scholar]
  25. Yuan X.; Huang B.; Wang Y.; Yang C.; Gui W. Deep learning-based feature representation and its application for soft sensor modeling with variable-wise weighted SAE. IEEE Transactions on Industrial Informatics 2018, 14, 3235–3243. 10.1109/TII.2018.2809730. [DOI] [Google Scholar]
  26. Dai Y.; Yang C.; Zhu J.; Liu Y. Adversarial transferred data-assisted soft sensor for enhanced multigrade quality prediction. ACS omega 2023, 8, 19900–19911. 10.1021/acsomega.3c01832. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Jiang B.; Yang H.; Wang Y.; Liu Y.; Geng H.; Zeng H.; Ding J. Dynamic Temporal Dependency Model for Multiple Steps Ahead Short-term Load Forecasting of Power System. IEEE Trans. Ind. Appl. 2024, 1. 10.1109/TIA.2024.3375802. [DOI] [Google Scholar]
  28. Dai Y.; Yang C.; Liu K.; Liu A.; Liu Y. TimeDDPM: Time series augmentation strategy for industrial soft sensing. IEEE Sensors Journal 2024, 24, 2145. 10.1109/JSEN.2023.3339245. [DOI] [Google Scholar]
  29. Yuan X.; Li L.; Shardt Y. A.; Wang Y.; Yang C. Deep learning with spatiotemporal attention-based LSTM for industrial soft sensor model development. IEEE Transactions on Industrial Electronics 2021, 68, 4404–4414. 10.1109/TIE.2020.2984443. [DOI] [Google Scholar]
  30. Yang Z.; Jia R.; Wang P.; Yao L.; Shen B. Supervised attention-based bidirectional long short-term memory network for nonlinear dynamic soft sensor application. ACS omega 2023, 8, 4196–4208. 10.1021/acsomega.2c07400. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Shao W.; Xiao C.; Wang J.; Zhao D.; Song Z. Real-time estimation of quality-related variable for dynamic and non-Gaussian process based on semisupervised Bayesian HMM. Journal of Process Control 2022, 111, 59–74. 10.1016/j.jprocont.2022.01.007. [DOI] [Google Scholar]
  32. Zhu J.; Ge Z.; Song Z. HMM-driven robust probabilistic principal component analyzer for dynamic process fault classification. IEEE Transactions on Industrial Electronics 2015, 62, 3814–3821. 10.1109/TIE.2015.2396877. [DOI] [Google Scholar]
  33. Liu Q.; Jia M.; Gao Z.; Xu L.; Liu Y. Correntropy long short term memory soft sensor for quality prediction in industrial polyethylene process. Chemometrics and Intelligent Laboratory Systems 2022, 231, 104678 10.1016/j.chemolab.2022.104678. [DOI] [Google Scholar]
  34. Zheng J.; Ma L.; Wu Y.; Ye L.; Shen F. Nonlinear dynamic soft sensor development with a supervised hybrid CNN-LSTM network for industrial processes. ACS omega 2022, 7, 16653–16664. 10.1021/acsomega.2c01108. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Geng Z.; Chen Z.; Meng Q.; Han Y. Novel transformer based on gated convolutional neural network for dynamic soft sensor modeling of industrial processes. IEEE Transactions on Industrial Informatics 2022, 18, 1521–1529. 10.1109/TII.2021.3086798. [DOI] [Google Scholar]
  36. Fujiwara K.; Kano M.; Hasebe S.; Takinami A. Soft-sensor development using correlation-based just-in-time modeling. AIChE J. 2009, 55, 1754–1765. 10.1002/aic.11791. [DOI] [Google Scholar]
  37. Zheng W.; Liu Y.; Gao Z.; Yang J. Just-in-time semi-supervised soft sensor for quality prediction in industrial rubber mixers. Chemometrics and Intelligent Laboratory Systems 2018, 180, 36–41. 10.1016/j.chemolab.2018.07.002. [DOI] [Google Scholar]
  38. Xie L.; Zeng J.; Gao C. Novel just-in-time learning-based soft sensor utilizing non-Gaussian information. IEEE Transactions on Control Systems Technology 2014, 22, 360–368. 10.1109/TCST.2013.2248155. [DOI] [Google Scholar]
  39. Guo F.; Xie R.; Huang B. A deep learning just-in-time modeling approach for soft sensor based on variational autoencoder. Chemometrics and Intelligent Laboratory Systems 2020, 197, 103922 10.1016/j.chemolab.2019.103922. [DOI] [Google Scholar]

Articles from ACS Omega are provided here courtesy of American Chemical Society

RESOURCES