STL-decomposition ensemble deep learning models for daily reservoir inflow forecast for hydroelectricity production

Abstract

Accurate reservoir inflow forecasting is crucial for efficient water management. In this study, different deep learning models, including Dense, Long short-term memory (LSTM), and one-dimensional convolutional neural networks (Conv1D), were used to build ensembles. Seasonal-trend decomposition using loess (STL) was applied to decompose reservoir inflows and precipitations into random, seasonal, and trend components. Seven ensemble models, namely STL-Dense, STL-Conv1D, STL-LSTM, STL-Dense-LSTM-Conv1D, STL-Dense multivariate, STL-LSTM multivariate, and STL-Conv1D multivariate, were proposed and evaluated using daily inflows and precipitation decomposed data from the Lom Pangar reservoir from 2015 to 2020. Evaluation metrics, such as Mean Absolute Error (MAE), Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE), and Nash Sutcliff Efficiency (NSE), were applied to assess model performance. Results showed that the STL-Dense multivariate model was the best ensemble among the thirteen models with MAE of 14.636 m³/s, RMSE of 20.841 m³/s, MAPE of 6.622%, and NSE of 0.988. These findings stress the importance of considering multiple inputs and models for accurate reservoir inflow forecasting and optimal water management. Not all ensemble models were good for Lom pangar inflow forecast as the Dense, Conv1D, and LSTM models performed better than their proposed STL monovariate ensemble models.

Graphical abstract

Highlights

• •Seven ensemble deep learning models are proposed for reservoir inflow forecast.
• •STL-Dense-multivariate model is the best model from evaluation criteria.
• •Dense, Conv1D, and LSTM models perform better than their monovariate ensembles.

1. Introduction

A model for optimal water resource management requires a robust and efficient water management system [1] In the study of efficient water use, statistical and machine learning methods have been employed [[2], [3], [4]] with interesting output results. Due to the random nature of water resources, there is difficulty in modeling these systems using classical methods. To solve this problem, machine learning methods have been applied to model them for efficient sustainability [5]. In an attempt to further increase these models' efficiency, deep learning neural network models have been applied to reservoir inflow forecasting with promising results [[6], [7], [8], [9], [10], [11], [12]] Recurrent Neural Networks (RNN), Gated Recurrent Unit (GRU), and Long-Short Tem Memory (LSTM) models have been used for reservoir inflow forecasting, with LSTM proving to be the most effective [13]. A hybrid framework using machine learning for reservoir inflow forecast has been proposed by Tian et al. [14] with interesting results as an outcome as compared to classical methods. Luo et al. [15] proposed an ensemble model combining the Deep believe network (DBN) and LSTM with the ensemble result better than that of the individual models.

Ensemble modeling has been researched for reservoir inflow prediction due to its improved performance through the combination of the strengths from different base models. An ensemble model consisting of various models has been proposed for forecasting inflow into a reservoir [[16], [17], [18], [19]]. The obtained result for the cited ensembles is better than their composition models. Hong et al. [20] utilized an ensemble of recurrent neural networks (RNN-LSTM) and convolutional neural network (CNN-LSTM) for reservoir inflow prediction, while Zhang et al. [21] proposed a fusion model of CNN, Partial Least Squares (PLS), and Extreme Gradient Boosting (XGBoost) for inflow prediction. Mendes et al. [22] applied an ensemble of High-Resolution Model (HRES) and the Ensemble Prediction System (EPS) products to forecast the inflow of the Aguieira reservoir in Portugal. These proposed ensembles outperform the single models based on evaluation criteria. Wang et al. [23] utilized a hybrid decomposition-based multi-model and multi-parameter ensemble method for stream flow forecasting of the Yalong River, resulting in increased accuracy and reduced uncertainty. Sushanth et al. [24] applied an explainable machine learning model with LSTM for real-time inflow and streamflow forecasting of Konar and Tenughat reservoirs with appreciable accuracy up to a 3-day lead. The shortcoming of the above-cited ensemble models is the proposal of a single ensemble to treat the problem. As a contribution, seven ensembles from different deep-learning models are proposed.

The Lom Pangar reservoir is the largest in the Southern Interconnected Grid (SIG) of Cameroon, serving to regulate the flow rate of the Song bengue river for hydroelectricity production 466 km downstream in the Songloulou hydropower plant. The study of this reservoir inflow rate is crucial for electricity production as it is the largest reservoir in the watershed. Moreover, its outflow regulates the flow rate on the Song bengue river which serves as an inflow in the Songloulou hydropower plant downstream. Songloulou power plant as of now is the largest power plant on SIG, the largest grid in the nation, and supplies energy to six regions out of the ten regions. In this study, seven ensemble deep learning models are proposed from Seasonal-trend decomposition using loess (STL). These models include STL-Dense, STL-Conv1D, STL-LSTM, STL-Dense-LSTM-Conv1D, STL-Dense multivariate, STL-LSTM multivariate, and STL-Conv1D multivariate. STL is used to decompose the reservoir inflow and precipitation into random, seasonal, and trend components. STL was chosen over other decomposition methods such as classical decomposition, and SEATS (seasonal extraction in autoregressive integrated moving average (ARIMA) time series) due to its ability to handle different types of seasonality and user control of smoothness in trend-cycle interaction among others [25,26]. Deep learning models namely the dense model, LSTM model, and Conv1D have been implemented to forecast the STL decomposed components and the resultant reservoir inflow is the sum of the forecasted decomposed inflows of the reservoir. Thirteen deep learning models, namely the dense model, the Conv1D model, the LSTM model, the multivariate dense model, the multivariate Conv1D model, the multivariate LSTM model, the STL-dense model, STL-Conv1D model, STL-LSTM model, STL-Dense_LSTM_Conv1D model, STL-Dense multivariate model, STL-Conv1D multivariate model, and STL-LSTM multivariate model are evaluated using evaluation criteria for the best model. The mean absolute error (MAE), mean square error (MSE), root mean square error (RMSE), mean absolute percentage error (MAPE), and Nash Sutcliff Efficiency (NSE) are used as evaluation criteria.

This manuscript is partitioned as follows: section 1 is the introduction, section 2 presents the study area, section 3 gives the methodology used, and section 4 summarizes the result and discussion. The conclusion of the work is in section 5.

2. Study area

The Lom Pangar reservoir is the largest in the Sanaga watershed. It is located in Bertoua in the East region of Cameroon. Due to the stochastic nature of the hydroelectricity production in the Southern interconnected grid of Cameroon, the reservoir was constructed to regulate the flow rate of water for hydroelectricity production [26]. The reservoir has the following coordinates 5⁰22′16.87″ N, 13 ⁰30′44.67″ E. Its minimum and maximum storage capacity are 100.000.000 m³ and 6280 000 000 m³ respectively. The surface area covered by the reservoir is 540 km² [27]. The reservoir has a 30 MW installed capacity which is still under construction and will serve to supply hydroelectricity to the East region. Daily inflow data from 2015 to 2020 were used in this study to train and test the models. The study site is shown in Fig. 1.

Fig. 1.

Location of the Lom Pangar reservoir.

3. Methods

3.1. Seasonal-trend decomposition using LOESS (STL)

STL is a robust method for decomposing time series data into seasonal, trend, and remainder components. Loess is an algorithm for handling nonlinear correlations. STL is highly desirable for time series decomposition due to its ability to handle any type of seasonality, its ability to be controlled by the user in trend-cycle smoothening, its ability to be robust to outliers, and its ability to allow the seasonal component to change over time [25,26,28]. From Figs. 2 and 3, the magnitude of the daily inflow and precipitation is not increasing with time as they show a seasonal pattern with a period of one year. As such, the additive time series decomposition method was chosen over the multiplicative time series decomposition to decompose the data for this study. Eq. (1) shows the various components of the decomposition reservoir inflow data set. Eq. (1) applies to the precipitation data set.

$$y_i=R_i+T_i+S_i$$ (1)

Fig. 2.

STL decomposition of Lom pangar daily reservoir inflow (a) into random (b), seasonal (c), and trend (d) components.

Fig. 3.

STL decomposition of Lom pangar daily reservoir precipitations (a) into random (b), seasonal (c), and trend (d) components.

$y_i$ is the data, $R_i$ is the remainder or random term, $T_i$ is the trend term, and $S_i$ is the seasonal term of the Lom Pangar daily reservoir inflow and precipitation time series. Fig. 2 (a) represents the time series Lom pangar daily inflow data. STL was applied to decompose the data into random, seasonal, and trend components shown in Fig. 2 (b), Fig. 2 (c), and Fig. 2 (d) respectively. The precipitation time series data is shown in Fig. 3 (a). Its random, seasonal, and trend components as decomposed by STL are shown in Fig. 3 (b–d) respectively.

3.2. Deep learning models

Deep learning models utilize artificial neural networks with multiple hidden layers for computations. They are a type of machine learning that learns from historical data and contains multiple layers of the neural network [29]. In this study, the inflow of the Lom pangar reservoir is forecasted using thirteen deep-learning models. Three models, dense, Conv1D, and LSTM, were used as univariate and multivariate models for inflow forecast. Seven ensemble deep-learning models were proposed to forecast the inflow of the Lom pangar reservoir using train-test data sets from 2015 to 2020. These models include STL-Dense, STL-Conv1D, STL-LSTM, STL-Dense_Conv1D_LSTM, STL-Dense multivariate, STL-LSTM multivariate, and STL-Conv1D multivariate.

3.2.1. Dense model

With the dense model, each neuron is connected to the neuron in the next layer. Information flows from one neuron to the next through matrix multiplication. It is used for time series regression problems [15]. The parameters and hyperparameters of the model are shown in Table 1.

Table 1.

General hyperparameters for the Dense, Conv1D, and LSTM models.

General parameters for Dense model, Conv1D and LSTM
Parameters	Value	Parameters	value
Activation function	Relu	Optimizer	Adam
Number of hidden layers	2	Epoch	100
Loss function	Mean absolute error	Batch size	128
Dense model
The neuron of the first hidden layer	1000	The neuron of the second hidden layer	100
The neuron of the output layer	1
Conv1D model
filters	128	Kernel_size	5
padding	Causal
LSTM
The neuron of the first hidden layer	1000	The neuron of the second hidden layer	100
verbose	0

3.2.2. Conv1D model

Convolutional neural networks are multilayer neural networks capable of handling input data as images for three-dimensional convolutional neural networks or regression problems for one-dimensional neural networks [30]. Table 1 represents the model's parameters and hyperparameters.

3.2.3. LSTM model

The long short-term memory model (LSTM) is a variant of the recurrent neural network (RNN). Each LSTM cell has gates (input gate, output gates, and a forget gate) that can either allow the information to go through or block it through the forget gate. It forecast a time series problem with high efficiency [31,32]. Fig. 3 shows the structure of the three deep learning models respectively. Table 1 gives the general characteristics of the LSTM model. The hyperparameters for the models were chosen empirically from hyperparameter tuning.

Fig. 4 (a–c) represent the basic structure of the dense model, the Conv1D model, and the LSTM cell respectively.

Fig. 4.

Deep learning models: (a) A dense model, (b) the Conv1D model, and (c) the LSTM model.

3.3. Deep learning ensemble models

Combining deep learning models to obtain a single forecasting model is very important as the strength of all the other models is taken into consideration for forecasting and the variance is reduced [19,33]. Due to the seasonality of the case study data, STL-additive decomposition is applied as a decomposition method on the inflows and precipitation time series data and decomposed into three components. Fig. 5, Fig. 6, Fig. 7, Fig. 8, are the schematics for the proposed ensemble models.

Fig. 5.

Multivariate STL-Conv1D ensemble (a) and multivariate STL-Dense ensemble (b).

Fig. 6.

Multivariate STL-LSTM ensemble (a) and STL-Dense ensemble (b).

Fig. 7.

STL-Conv1D ensemble (a) and STL-LSTM ensemble (b).

Fig. 8.

STL-Dense-LSTM-Conv1D ensemble.

In Fig. 5 (a) the seasonal inflows and seasonal precipitations were used as inputs into Conv1D 1 model. The trend inflows and trend precipitations were used as input into the Conv1D 2 model. The random inflows and random precipitations were used as inputs into the Conv1D 3 model. The seasonal, trend, and random inflows components were outputs from the Conv1D 1, Conv1D 2, and Conv1D 3 models respectively. The sum of these decomposed outputs constitutes the final forecast for the Lom pangar reservoir as shown in Eq. (1). The same analogy is applied to the schematics of Fig. 5, Fig. 6, Fig. 7, Fig. 8.

3.4. Evaluation criteria

We used as evaluation criteria the mean absolute error (MAE), the mean square error (MSE), the root mean square error (RMSE), the mean absolute percentage error (MAPE), and Nash-Sutcliffe efficiency (NSE) respectively as shown in Eq. (2), Eq. (3), Eq. (4), Eq. (5), and Eq. (6) [ [34,35]] to evaluate the different models.

$$\text{MAPE}=\frac{1}{\mathrm{N}}{\sum }_{i=1}^N\left|\frac{y_i-{\hat{y}}_i}{y_i}\right|$$ (2)

$$\text{MAE}=\frac{1}{\mathrm{N}}{\sum }_{i=1}^N\left|y_i-{\hat{y}}_i\right|$$ (3)

$$\text{MSE}=\frac{1}{\mathrm{N}}{\sum }_{i=1}^N{\left(y_i-{\hat{y}}_i\right)}^2$$ (4)

$$\text{RMSE}=\sqrt{\frac{1}{\mathrm{N}}{\sum }_{i=1}^N{\left(y_i-{\hat{y}}_i\right)}^2}$$ (5)

$$NSE=1-\frac{{\sum }_{i=1}^N{\left(y_i-{\hat{y}}_i\right)}^2}{{\sum }_{i=1}^N{\left(y_i-\bar{y}\right)}^2}$$ (6)

Where $y_i$ is the observed value, ${\hat{y}}_i$, the forecast value, and $\bar{y}$ the mean of the observed value. N represents the sample sizes.

4. Results and discussion

4.1. Results

The result obtained when the models were applied to forecast a day reservoir inflow of the Lom pangar reservoir using a window size of the past seven days is shown in Fig. 9, Fig. 10, Fig. 11, Fig. 12. The deep learning models were executed in Python 3.8 on a personal computer with AMD A6-3410MX APU Radeon (tm) HD Graphics 1.6 GHz processor. 80% of the data was used to train the models while 20% was used to test the models. This division was done since it provides a better result empirically [36].

Fig. 9.

Lom pangar inflows compared with (a) STL-Conv1D ensemble, (b) STL-Conv1D multivariate ensemble, (c) STL-Conv1D ensemble, and (d) STL-Dense multivariate ensemble.

Fig. 10.

Lom pangar inflows compared with (a) STL-LSTM ensemble, (b) STL-Dense-LSTM-Conv1D ensemble, (c) STL-Dense ensemble, and (d) Multivariate Dense ensemble.

Fig. 11.

Lom pangar inflows compared with (a) LSTM model, (b) Conv1D model, (c) Dense model, and (d) Multivariate LSTM model.

Fig. 12.

Lom pangar inflows compared with (a) multivariate Conv1D model, (b) STL-trend inflow compared with its dense forecast, (c) STL-random inflow compared with its dense forecast, and (d) STL-seasonal inflow compared with its dense forecast.

The elaborate comparative result is shown in Table 2. The best result was obtained with the STL-dense-multivariate model with the least MAE of 14.636 m³/s. With the univariate deep learning models namely the dense model, conv1D model, and LSTM, the least error was obtained with the dense model followed by the LSTM model then conv1D. In general, there is not much variation in the NSE of the models. The values are between 0.986 and 0.988.

Table 2.

Evaluation criteria for the different models.

Deep learning Model	MAE (m3/s)	MSE (m3/s)2	RMSE (m3/s)	MAPE (%)	NSE
Dense model	14.961	474.579	21.785	6.597	0.987
Conv1D model	15.631	521.269	22.831	6.861	0.986
LSTM model	15.575	518.863	22.779	6.881	0.986
Multivariate dense model	15.072	488.127	22.094	6.592	0.987
Multivariate Conv1D model	15.579	516.560	22.728	6.855	0.986
Multivariate LSTM model	15.280	475.594	21.808	6.853	0.987
STL-Dense model	15.178	468.785	21.651	6.757	0.988
STL-Conv1D model	15.783	521.841	22.844	6.921	0.986
STL-LSTM model	16.227	543.184	23.306	7.303	0.986
STL-Dense-LSTM-Conv1D model	15.517	497.051	22.295	6.924	0.987
STL-LSTM multivariate	15.984	506.955	22.516	7.276	0.987
STL-Conv1D multivariate	15.349	489.776	22.131	6.808	0.987
STL-Dense multivariate model	14.636	434.353	20.841	6.622	0.988

Fig. 9, Fig. 10, Fig. 11, Fig. 12 (a), represent the results obtained from forecasting the Lom pangar daily inflows using the thirteen deep learning models. Fig. 9 (a–d) represent the plot of the inflows with STL-Conv1D ensemble model, STL-Conv1D multivariate ensemble model, STL-Conv1D ensemble model, and STL-Dense multivariate ensemble model respectively. Fig. 10(a–d) represent the plot of the inflows with the STL-LSTM ensemble model, STL-Dense-LSTM-Conv1D ensemble model, STL-Dense ensemble model, and Multivariate Dense ensemble model respectively. Fig. 11(a–d) represent the plot of the inflows with the LSTM model, Conv1D model, Dense model, and Multivariate LSTM model. The detailed results from the evaluation criteria for the thirteen models are shown in Table 2. Fig. 1s(a–d) presents the results obtained from forecasting the decomposed components namely the trend, random, and seasonal components using the dense model. Table 3 presents the results obtained from the evaluation criteria.

Table 3.

MAE for the different decomposition terms with Dense, conv1D, and LSTM models.

MAE (m3/s.)	Random inflow Lom pangar	Seasonal inflow Lom pangar	Trend inflow Lom pangar
Dense model	13.871	5.706	0.137
Conv1D model	13.781	8.639	0.129
LSTM model	14.892	6.811	0.135

4.2. Discussion

Unlike Qi et al. [37] which implement Empirical mode decomposition to forecast the reservoir inflow for univariate time series, this work takes into consideration the reservoir precipitation and constitutes a multivariate problem. Again, Qi et al. [37] applied LSTM to forecast the decomposed variable while in addition to LSTM, this work takes into consideration the dense model and Conv1D model to forecast the decomposed variables. The addition of these two deep learning models and the proposal of seven ensembles gives the planner a wide range of methods to choose from. This lack of numerous ensembles is the limitation of Qi et al. [37].

Also, the findings of this work are per Ding et al. [38] who equally use STL decomposition with the application of LSTM to forecast hydroelectricity production. The findings of Ding et al. [38] suggest that the proposed ensemble performed better than the LSTM model. The limitation of [38] concerning this work is the same as that of [37] which includes the proposal of a single ensemble as compared to seven ensembles proposed. Again, the advantage of these ensembles permits us to notice that contrary to Refs. [37,38] in which their proposed methods use LSTM to build the ensembles, this study shows that the LSTM ensemble is good but the best ensemble was obtained with the STL-Dense multivariate ensemble in forecasting the reservoir inflow for Lom pangar reservoir.

Again, the least MAPE of 6.622% was obtained from the multivariate dense model still strengthening the advantage of the multivariate model considering precipitation over the univariate model for reservoir inflow forecast at the Lom Pangar reservoir.

From Table 3, the three models namely Dense, Conv1D, and LSTM forecast the trend term with a very small MAE. Thus, the deep learning models used are suitable to forecast the trend term of the daily reservoir inflow of the Lom Pangar reservoir. For the univariate models, the dense model obtained the best MAE result for forecasting the seasonal inflow of Lom pangar. The Conv1D model forecast the random and trend inflow and obtains the minimum error among the three univariate deep learning models. These show the correlation between the decomposed data sets with the deep learning models, and also the strength of assembling other models to obtain a better fit.

Surprisingly, from Table 2, the Dense, Conv1D, and LSTM models perform better than their monovariate STL decomposed models. The MAE of the models are 14.961 m³/s, 15.631 m³/s, and 15.575 m³/s as compared to 15.178 m³/s, 15.783 m³/s, and 16.227 m³/s respectively. This result shows that not all assemble are better than the individual deep learning model for forecasting the daily reservoir inflow of the Lom pangar reservoir. It gives the application of a new domain of research like using other ensemble methods such as bagging, boosting, and many others on the data set to study the correlations for more efficient ensembles.

5. Conclusion

Due to the stochastic nature of reservoir inflow, there is a need to forecast future inflows as this will help in optimal water management with applications to hydroelectricity production, irrigation, and many others. The limitation of classical methods to handle the study of reservoir inflows due to the stochastic nature of water gives room to advanced methods precisely deep learning ensembles to handle this problem. Seven ensemble deep learning models based on STL were proposed for reservoir inflow forecast. STL was chosen for decomposition over other methods due to its robust nature, its ability to handle all kinds of seasonality, and the parameter tuning properties. STL decomposition was applied to the decomposed reservoir inflows and precipitations data set to forecast a day inflow. The decomposed components namely the random component, the seasonal component, and the trend component of the inflow forecast using deep learning models. The final prediction obtained is the sum of the three decomposed components. The forecasting results obtained with application on the Lom Pangar reservoir with statistical parameters of the ensemble deep learning models were compared with the single deep learning models. Among the thirteen deep learning models used, the STL-Dense multivariate model was the best. The combining effect of using multiple models and the application of multivariate forecasting by considering precipitation as part of the input leads to this model outperforming the other models. Deep learning ensemble models for daily inflow forecast are proposed but we acknowledge their limitations due to uncertainty analysis. Also, better hyperparameter tuning can greatly improve the model's performance. The unavailability of larger data sets in training and testing the models is part of the limitations of the proposed models. Not all ensemble models were better than their single model for the Lom pangar reservoir inflow forecast. The monovariate STL decomposition ensemble models perform poorly as compared to their deep learning models. This gives room for future research for which other ensemble methods like bagging, boosting, random forest, and many others could be taken into consideration for better ensembles. Also, varying hyperparameters to produce an ensemble deep learning model to forecast each of the decomposed components constitute the perspective of this work.

Author contribution statement

Njogho Kenneth Tebong, Théophile Simo, Armand Nzeukou Takougang: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Wrote the paper.

Patrick Herve Ntanguen: Performed the experiments; Analyzed and interpreted the data; Wrote the paper.

Data availability statement

Data will be made available on request.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.