Investigation of Data Size Variability in Wind Speed Prediction Using AI Algorithms

Abstract

Electricity generation from burning fossil fuel is one of the major contributors to global warming. Renewable energy sources are a viable alternative to produce electrical energy and to reduce the emission from power industry. They have unlocked opportunities for consumers to produce electricity locally and use it on-site that reduces dependency on centralized generation. Despite the widespread availability, one of the major challenges is to understand their characteristics in a more informative way. Wind energy is highly dependent on the intermittent wind speed profile. This paper proposes the prediction of wind speed that simplifies wind farm planning and feasibility study. Twelve artificial intelligence algorithms were used for wind speed prediction from collected meteorological parameters. The model performances were compared to determine the wind speed prediction accuracy and model comparison for different sizes of data set. The results show, the most effective algorithm varies based on the data size.

Introduction

Climate change is one of the most challenging questions in the research and non-research communities. Global warming is marked as one of the prime reasons (Rivas and Gonzalo 2019). Electric power generation on the other hand is an integral part of current advancement while being responsible for greenhouse gas emission due to the combustion of fossil fuel. In 2018, 33% of total carbon dioxide emission came from electric power sector (U.S. EIA 2020). Therefore, dependency shift from fossil fuel to renewable energy centered power system is a key. However, among the renewable energy resources, wind has prospect that has not been harnessed in full due to its intermittent nature (variability of wind speed) and capital investment requirement (Office of Energy Efficiency & Renewable Energy 2018a, 2018b). Technological challenges are yet to overcome (Watson et al. 2019).

Wind speed prediction is a regression problem where predictors, in this case, are the meteorological parameters, and the response variable is the wind speed at 80 m height. In general, regression is a classical problem both in statistics and machine learning. Usually, statistical methods are to find the inference while machine learning makes the prediction (Shmueli 2010). Then again, they intersect in some cases and do serve a similar purpose, for example- linear regressio (Matson and Huguenard 2007). However, statistical learning relies on distributions, while machine learning is an empirical process and requires data (Bzdok, Altman, and Krzywinski 2018). The statistical approach, thus, considers how data are collected or generated while machine learning may result in accurate prediction without knowing much about the underlying aspects of data. Another line of the boundary is the shape or volume of data. While the statistical approach is very robust about the number of samples (as it considers the distribution of the data), machine learning is more applicable when the dataset is wide (Bzdok, Altman, and Krzywinski 2018). However, sometimes they are used interchangeably, and statistics are the backbone of machine learning (Brownlee 2018). Besides, some machine learning algorithms use the same bootstrapping methods as statistical models. In addition, researchers are also leveraging deep learning for similar prediction problems (Yen et al. 2019). Artificial Neural network (ANN) based models usually yield benefits in prediction tasks than statistical models due to its robustness toward the nature of data, especially when there are missing values, or the dataset is not well preprocessed, raw, and large data (Chen et al. 2019). Thus, many machine learning regression algorithms use statistical techniques in innovative ways, while deep learning neural network approaches are efficient for analogous tasks. The reason, however, behind the growing popularity of machine learning or deep learning (artificial intelligence, in general) is the availability of computational resources (OpenAI 2018). Therefore, the larger dataset is not a critical issue to work with, which was challenging in the past. In this research, we have considered both approaches, machine learning, and deep learning for our wind speed prediction problem.

The placement of a wind turbine for wind power generation is often a challenging step due to the varying nature of wind speed from a location/height to another location/height (Cetinay, Kuipers, and Guven 2017). Measuring wind speed at the level of turbine hub height is both expensive and requires continuous maintenance. Meteorological parameters also play a vital role in the wind characteristics. Therefore, wind speed profiling with the variation of meteorological parameters has been a research problem that leads to the prediction of wind speed of a certain location based on those parameters (Hoolohan, Tomlin, and Cockerill 2018; Ehsan et al. 2019). Therefore, utilization of easy to access parameters in a low elevation to predict corresponding wind speed at a higher height is a practical approach. Literature show there have been statistical approaches for wind speed application, while artificial intelligence-deep learning and machine learning are being considered recently (Pearre and Swan 2018; Zheng et al. 2011). In addition to machine learning regression algorithms, neural network-based deep learning techniques are getting attention for alike problems due to higher accuracy.

Deep Neural Network (DNN) can map features from raw data to provide regularization, thus minimizes the variance in each layer (Liu et al. 2014). This capability makes DNN suitable for prediction problems. The prediction accuracy is greatly dependent on efficient feature extraction of time series data. Literature (Liu et al. 2018a) shows – Convolutional Neural Network (CNN) creates filters to automate such function that makes it widely applicable for prediction. However, it can represent short term dependence while wind speed comprises both short and long-term dependence fundamentally (Liu et al. 2018b). Therefore, long short-term memory (LSTM) seems more effective for wind speed prediction. LSTM is a form of Recurrent Neural Network (RNN) that is capable of learning long-term dependencies to make a prediction (Pei et al. 2019).

The novelty of this paper is the inclusion of a wider ranged meteorological variables to predict wind speed for a higher elevation of interest using existing regression models, which differentiates this work from previously done research works as described in the literature. The heights where the predictors are measured are also noteworthy. Average wind speed at 2 m is an easy-to-reach height in terms of sensor installation and maintenance compared to 80 m, same goes to temperature, wind direction, and turbulence measuring points’ elevations. Therefore, this paper shows an approach to utilize the information already available or gathered by minimum effort in the prediction of a parameter that is complex and expensive to get.

The rest of this paper is organized as follows. Section “Methodology” describes the methodology for feature selection and prediction application. In Section “Background of Prediction Algorithms,” prediction algorithms, both machine and deep learning approaches are discussed. Performance evaluation metrices are presented in Section “Performance Evaluation.” In Section “Simulations and experiments,” the National Renewable Energy Laboratory (NREL) data set is utilized for simulations and experimental results are demonstrated. In Section “Conclusion,” the conclusions are given.

Methodology

This section describes the theoretical approach taken for this research. Figure 1 illustrates a flow chart detailing every step. The steps include preprocessing of the data, short description of the candidate prediction algorithms, and performance evaluation metrices. These are the formal steps for any prediction modeling. Correlation and collinearity measures help to overcome overfitting in prediction. Then normality test is done to evaluate the data quality. The data is now ready for training the prediction algorithms. There are accuracy measures that are fundamental to evaluate the prediction accuracy. Following subsections contain detailed discussion of data pre-processing step. Then rest of the steps is described in Sections “Background of Prediction Algorithms” and “Performance Evaluation.”

Figure 1.

Wind speed prediction approach.

Correlation Matrix

Correlation is a measure to identify how strongly a pair of variables (X, Y) are related. This score ranges between +1 and −1 based on their degree of relation. Pearson, Kendall, and Spearman are common correlation techniques. We have used Spearman correlation (1) as it does not carry any assumption about the distribution of data (while Pearson’s correlation assumes data are normally distributed, which is not always the case) (Dodge 2008).

$$\rho =\frac{\mathit{\text{cov}}\left(r{g}_X,r{g}_Y\right)}{{\sigma }_{r{g}_X}{\sigma }_{r{g}_Y}}$$ (1)

Where, ρ : correlation coefficient; rg_X, rg_Y : ranks of X and Y; cov : covariance; ${\sigma }_{r{g}_X}{\sigma }_{r{g}_Y}$: the standard deviation of rank variables.

The cross-correlation between the predictors (independent or explanatory variable, X) and the response (dependent variable, Y), is the initial step toward feature selection. If the absolute value of a correlation measure is greater than 0 between two parameters, then they are considered correlated. Perfect correlation is one that usually indicates self-correlation, or a problem called multicollinearity that leads to false predictions. Therefore, features with very high correlation (>0.9) are removed from the dataset as they might mislead the model performance (Vu, Muttaqi, and Agalgaonkar 2015; Alin 2010).

Collinearity Test

Collinearity is a condition when some independent variables are highly correlated. In a regression model, if there are highly correlated predictors, they cannot independently predict the value of the dependent variable. In other words, they explain some of the same variances in the dependent variable, which causes inflated values of coefficients or even reverses the sign. If more than two predictors are associated in that manner, it is called multicollinearity (Tintner 1975).

Variance Inflation Factor (VIF) is applied to investigate if collinearity exists among the features (Marcoulides and Raykov 2019). For a multiple linear regression model with n predictors, VIFs are the diagonal elements of the inverse of the correlation matrix of the predictors. VIF for the nth predictor variable is expressed as (2).

$$VI{F}_n=\frac{1}{1-{R_n}^2}n=1,2,3,\dots$$ (2)

The smallest possible VIF is one representing not collinear at all and higher the value greater the collinearity. However, higher values of VIF (>5) represent the presence of multicollinearity, causing redundancy (more than one predictor describing the same effect on the response variable) (Marcoulides and Raykov 2019). Therefore, one from each collinear subset of predictors is kept, and others are removed from data (James et al. 2013; Bruce and Bruce 2017).

Normality Test

Normality tests are applied to investigate if the dataset is well modeled (likelihood of data to be normally distributed). Some approaches to test for normalization are- graphical method, back-of-the-envelope test, frequentist test, and Bayesian test (Ghasemi and Zahediasl 2012). In this research, we use the graphical test. In this method, the Chi-Square Quantile-Quantile (Q-Q) plot of multivariate distribution is analyzed to see if the features are normally distributed (Liang, Pan, and Yang 2004). If normal, the plot should follow the 45-degree baseline. If not, then normalization is required before fitting the data to any model. Thus, prior normalization is required before feeding it to the models.

Background of Prediction Algorithms

Machine Learning Approach

Machine learning regression algorithms are widely used for prediction applications while they vary in terms of behind the scene mathematics (Stulp and Sigaud 2015). The following algorithms are used in this research.

Multiple Linear Regression

Multiple linear regression relates two or more dependent variables to an independent variable by fitting a linear equation to the data (Hill and Gelman 2007; Uyanık and Güler 2013; Mehryar Mohri and Talwalka 2012). For n independent variables $\left(x_1,x_2,\dots ,x_n\in ℝ\right)$ and m observations, the multiple linear regression model (3) is defined as-

$$\begin{gathered} {\widehat{y}}_i={\beta }_0+{\beta }_1{x}_{i1}+{\beta }_2{x}_{i2}+\dots +{\beta }_n{x}_{in}+{\epsilon }_i \\ (i=1,2,\dots ,m) \end{gathered}$$ (3)

Where, $y_i\in ℝ$ : dependent variable, ${\widehat{y}}_i$ is the estimate of y_i; ε : residual (deviation of ${\widehat{y}}_i$ from its mean value); β : regressor coefficients estimated from least-square estimates (β₀ is known as intercept, and β_n is the slope of the regression line for simple linear regression); m : number of rows in the dataset.

The sum of squares of residuals (SS_residuals) of multiple linear regression is shown in (4), and we would like to minimize this value in the model. The equation for multiple linear regression can also be written as (5), where argmin stands for the argument of minimum. In other words, it finds the β that minimize SS_residuals: β is easy to find from (6), which is also known as the matrix formulation of linear regression, where X and Y represent independent and dependent variables, respectively.

$$S{S}_{\mathit{\text{residuals}}}=\sum _{i=1}^m{\left({\widehat{y}}_i-y_i\right)}^2$$ (4)

$${\widehat{\beta }}^{MLR}=\underset{\beta \in ℝ}{\mathit{\text{argmin}}}\sum _{i=1}^n{\left[y_i-\widehat{y_i}\right]}^2=\underset{\beta \in ℝ}{\mathit{\text{argmin}}}\sum _{i=1}^n[y_i-({\beta }_o+{\beta }_1{x}_{i1}{+{\beta }_2{x}_{i2}+\cdots +{\beta }_n{x}_{in})]}^2$$ (5)

$${\widehat{\beta }}^{MLR}={\left(X^{T}X\right)}^{-1}{X}^{T}Y$$ (6)

Ridge Regression

Ridge regression is a variant of linear regression that uses L2 regularization (Exterkate et al. 2016). The fundamental equation for ridge regression is the same as linear regression with a constraint on it, as described in (7), where C defines the boundaries of ridge regression. The regularization shrinks the parameters and reduces the model complexity by a coefficient of shrinkage. This coefficient is known as the penalty and denoted by λ (hyperparameter). Now we look at the equation for ridge regression and it is clear- first part of (8) is the same as linear regression, and the true difference between linear and ridge is the second term containing the constraint, B, shown in (9) and penalty. Due to the inclusion of the penalty, the residual error is minimized, therefore, ridge regression shall produce better accuracy.

$${{\beta }_0}^2+{{\beta }_1}^2+\cdots +{{\beta }_n}^2\le C^2$$ (7)

$${\widehat{\beta }}^{\mathit{\text{ridge}}}=\underset{\beta \in ℝ}{\mathit{\text{argmin}}}\sum _{i=1}^n{\left[y_i-\widehat{y_i}\right]}^2=\underset{\beta \in ℝ}{\mathit{\text{argmin}}}\mathit{\text{ min}}\left({\Vert y_i-XB\Vert }_2^2+\lambda \Vert B{\Vert }_2^2\right)$$ (8)

$$\Vert B{\Vert }_2=\sqrt{{{\beta }_0}^2+{{\beta }_1}^2+\cdots +{{\beta }_n}^2}$$ (9)

Least Absolute Shrinkage and Selection Operator (Lasso) Regression

Lasso regression uses L1 regularization. It has benefits over ridge regression when there are more features (Illarionov and Khudorozhkov 2018). The equation of lasso regression (10) is quite straight forward from the ridge regression, the only difference is the second term of the equation.

$${\widehat{\beta }}^{\mathit{\text{lasso}}}=\underset{\beta \in ℝ}{\mathit{\text{argmin}}}\sum _{i=1}^n{\left[y_i-{\widehat{y}}_i\right]}^2=\underset{\beta \in ℝ}{\mathit{\text{argmin}}}\mathit{\text{ min}}\left({\Vert y_i-XB\Vert }_2^2+\lambda \Vert \beta {\Vert }_1\right)$$ (10)

Adding regularization is a very important technique in machine learning to prevent overfitting (Giarré and Argenti 2018). The difference between the L1 and L2: L2 is the sum of the square of the weights, while L1 is just the sum of the weights.

Bayesian Ridge Regression

Bayesian view of ridge regression is obtained by noting that the minimizer of (8) can be considered as the posterior mean of a model where ${\beta }_i~N\left(0,\frac{{\sigma }^2}{\lambda }\right)$, for i = 1, 2, …, n (Shi, Abdel-Aty, and Lee 2016).

Huber Regression

In the ridge and lasso, the penalty was a hyperparameter. Instead of considering an estimated constant, Huber function (11) is proposed. This method does not rely on SS_residuals to minimize the error, rather sensitive to outliers $\left(\left|y_i-\widehat{y_i}\right|\le \delta \right)$. Thus, the name, Huber regression (Sun, Zhou, and Fan 2020), came from the loss function is uses. δ is a hyperparameter, and the choice is critical. Residuals larger than δ are minimized with L1 (which is less sensitive to large outliers), while residuals smaller than δ are minimized appropriately with L2.

$$L_{\delta }\left(y_i,\widehat{y_i}\right)=\{\begin{array}{l}\frac{1}{2}{\left(y_i-\widehat{y_i}\right)}^2\mathit{\text{ for }}\left|y_i-\widehat{y_i}\right|\le \delta \hfill \\ \delta \left|y_i-\widehat{y_i}\right|-\frac{1}{2}{\delta }^2\mathit{\text{ otherwise }}\hfill \end{array}$$ (11)

Bagging Regression

Bootstrap aggregating (bagging) prediction models is a general method for fitting multiple versions of a prediction model and then combining them into an aggregated prediction (Breiman 1996; Sutton 2005). It is a straight-forward algorithm in which b bootstrap copies of the original training data are created, the regression is applied to each bootstrap sample, and in the regression context, new predictions are made by averaging the predictions together from the individual base learners. Equation (12) thus demonstrates the formulation by letting x_i as the prior and ${\widehat{y}}_{bag}$ as the bagged prediction. ${\widehat{y}}_{i1},{\widehat{y}}_{i2},\dots ,{\widehat{y}}_{ib}$ are the the predictions from individual base learners for x_i:

$${\widehat{y}}_{bag}={\widehat{y}}_{i1}+{\widehat{y}}_{i2}+\cdots +{\widehat{y}}_{ib}$$ (12)

Random Forest Regression

Random forest regression is also a bootstrap aggregation (bagging) that involves training each decision tree on a different data sample where sampling is done with replacement (Sutton 2005; Pal 2017). Thus, it combines multiple decision trees in determining the model output. A visual illustration of such a model is in Figure 2.

Figure 2.

Visual aid operation of random forest regression.

Adaptive Boosting (AdaBoost) Regression

AdaBoost is stagewise estimation procedures to fit an additive logistic regression model, which minimizes the exponential loss function (Cao et al. 2013). The operational steps of this algorithm start with initializing the observation weights β_i on the original sample (predictors). The classifier then adjusts the weights for residual minimization and re-iterate the process for a defined number of times. Finally, the model becomes an aggregation of classifier function, F_i(x), and weight minimizer, α_i, as shown in Figure 3 (Varmuza 2003).

Figure 3.

Learning steps in AdaBoost regression.

Support Vector Regression (SVR)

Support vector regression is a robust algorithm that can handle both linear and non-linear input regression problems (Smola and Schölkopf 2004). We consider a linear case, where predictors, x and response $\widehat{y_i}$. The formulation for a linear case is shown in (13) where f_i(x) is the transfer function (also known as the kernel), and b is bias. A few of the common transfer functions are linear, non-linear, polynomial, and radial basis functions.

$$\widehat{y_i}=\sum _{i=1}^n{\beta }_i{f}_i(x)+b$$ (13)

Deep Learning Approach

Deep learning algorithms are now applied to solve problems of a diverse nature, including prediction (Filik and Filik 2017). Therefore, we are considering deep learning algorithms for this research. Firstly, we would like to review a few basics of deep learning. The building blocks of deep learning or artificial neural networks are called perceptron, which mimics an equivalent functionality (in computation) as neuron (a biological cell of the nervous system that uniquely communicates with each other) (Goodfellow 2016).

Now, perceptron or artificial neurons receive input signals (x₁, x₂, …, x_m), multiply input by weight (w₁, w₂, …, w_m), add them together with a pre-determined bias, and pass through the activation function, f(x). The signal goes to output as 0 or 1 based on the activation function threshold value. A perceptron with inputs, weights, summation and bias, activation function, and output all together forms a single layer perceptron (Kanal 2001). However, in common neural network diagrams, only input and output layers are shown. In a practical neural network, hidden layers are added between the input and output layers. The number of hidden layers is a hyperparameter and usually determined by evaluating the model performance. If the neural network has a single hidden layer, the model is called a shallow neural network, while a DNN consists of several hidden layers (Goodfellow 2016). In this research, we have considered DNN, CNN, and RNN- in the form of LSTM, all of which will be discussed in the following sections.

Deep Neural Network (DNN)

DNN is composed of three neural network layers, namely- an input layer, hidden layer(s), and an output layer. The (number of hidden layers) is tuned through trial and error (Goodfellow 2016). Figure 4 illustrates such a model structure with two hidden layers consisting of three neurons each, five input neurons, and one output neuron. The number of neurons depends on the number of input and output.

Figure 4.

Simplified architecture of a DNN.

In Figure 4, [x₁, x₂, x₃, x₄, x₅] is input; h represents hidden layer weights, and $\stackrel{⌢}{y}$ is output.

A simplified DNN kernel is formulated in (14) that considers linear modeling. x, W, and c symbolize input, weights, and bias, respectively, while w and b are linear model parameters. The hidden layer parameter h is shown in (15), where g is the activation function. For DNN modeling, ReLu (16) is used as the hidden layer activation function.

$$f(x;W,c,w,b)=w^T\mathit{\text{max}}\left\{0,W^T+c\right\}+b$$ (14)

$$h=g\left(W^{T}x+c\right)$$ (15)

$$f(x)=\mathit{\text{max}}(0,x)$$ (16)

Convolutional Neural Network (CNN)

CNN, also known as ConvNet, is one way to solve the issue with DNN using convolution rather than matrix multiplication (Gulli 2017). In other words, CNN is the regularized version of DNN to ensure model robustness toward overfitting. CNN is very popular for image processing; however, in the prediction problem, it is also utilized (Kiranyaz 2019). In this research, we are using 1 D CNN for wind speed prediction. The characteristics and approaches are the same for all CNNs, regardless of dimensionality (Zhao et al. 2018). The architecture of CNN (Figure 5 shows for 1 D CNN) consists of a convolution layer, pooling layer, and a fully connected neural network layer, thus, incorporates local receptive fields to ponder the spatial information, shared weights, and pooling to consider the summary statistics in the output.

Figure 5.

Architecture of 1 D convolution neural network.

Recurrent Neural Network (RNN) – LSTM

Long Short-Term Memory Networks (LSTM), a form of gated RNN, is proposed to implement. LSTM introduces self-loops to produce paths where the gradient can flow for a long duration; thus, it is capable of learning long-term dependencies (Goodfellow 2016). LSTMs are explicitly designed to avoid the long-term dependency problem, as illustrated in Figure 6. The equations describing the operations are listed below.

$$f(t)={\sigma }_g\left(W_f{x}_t+U_f{h}_{t-1}+b_f\right)$$ (17)

Figure 6.

Block diagram of LSTM operations.

$$i_t={\sigma }_g\left(W_i{x}_t+U_i{h}_{t-1}+b_i\right)$$ (18)

$$o_t={\sigma }_g\left(W_o{x}_t+U_o{h}_{t-1}+b_o\right)$$ (19)

$$c_t=f_t\circ c_{t-1}+i_t\circ {\sigma }_c\left(W_c{x}_t+U_c{h}_{t-1}+b_c\right)$$ (20)

$$h_t=o_t\circ {\sigma }_h\left(c_t\right)$$ (21)

Where, $x_t\in {ℝ}^d$ : Input vector to the LSTM unit; $f_t\in {ℝ}^h$ : Forget states activation vector; $i_t\in {ℝ}^h$: Input/update gate’s activation vector; $o_t\in {ℝ}^h$: Output gate’s activation vector; $h_t\in {ℝ}^h$: Hidden state vector; $c_t\in {ℝ}^h$ : Cell state vector; $W\in {ℝ}^{h\times d}$, $U\in {ℝ}^{h\times h}$, $b\in {ℝ}^h$ : Weight matrices and bias vector parameters which need to be learned during the training; σ_g : Sigmoid function; σ_c, σ_g : hyperbolic tangent function.

Performance Evaluation

Some commonly used accuracy parameters are employed to evaluate how well a model is performing to predict the intended parameter (Joshi 2016). Mean absolute error (MAE), mean square error (MSE), median absolute error (MedAE), and R-square (R2) scores are considered to investigate the model performances on the test set.

MAE is the average of the absolute values of the error (the difference between actual response (y_i) and predicted response $\left(\widehat{y_i}\right)$. As described by (22), n is the number of total input sets. The lower this value is, the better the model performance, while the desired is 0.

$$MAE=\frac{{\sum }_{i=1}^n\left|y_i-\widehat{y_i}\right|}{n}$$ (22)

MSE is the mean of the square of error terms. Similarly, to MAE, it is desired to have 0 or close value for this term. The formula for this measure is in (23).

$$MSE=\frac{{\sum }_{i=1}^n{\left(y_i-{\widehat{y}}_i\right)}^2}{n}$$ (23)

MedAE is the median of all the error terms, defined in (24), thus effective to deal with outliers’ effect in the model performance.

$$\mathit{\text{MedAE}}=\mathit{\text{median }}\left(\left|y_1-\widehat{y_1}\right|,\left|y_2-\widehat{y_2}\right|,\left|y_3-\widehat{y_3}\right|,\dots \dots ,\left|y_n-\widehat{y_n}\right|\right)$$ (24)

R2 score determines how well the model would perform in predicting the response variable as shown in (25) where $\overline{y_i}$ denotes the mean value of all predictions. This value is also known as the coefficients of determination. The best possible value is 1 for this case, and the closer to 1, the better model prediction is.

$$R2\mathit{\text{ score }}=1-\frac{{\sum }_{i=1}^n{\left(y_i-\widehat{y_i}\right)}^2}{{\sum }_{i=1}^n{\left(y_i-{\overline{y}}_i\right)}^2}$$ (25)

Further model fitting is tested using the residual plot by graphing residual (the difference between prediction and actual value) vs. fitted instance.

Simulations and Experiments

Dataset

We collected data from the National Renewable Energy Laboratory (NREL) database available online (Jager and Andreas 1996). The dataset contains three-months hourly samples (May 1 to July 31, 2018) and is used as the reference of comparison here. This paper considers four additional data sets of different sizes: 10 days (March 1–10, 2019), 30 days (January 1–30, 2019), 6 months (January 1 to June 30, 2010), and 1 year (January 1 to December 31, 2015). The raw data entail samples of each minute. It was converted to average hourly instances. Primarily, the dataset had 18 features, among which wind speed in 80 m height is our response variable, and other 17 are predictors- solar radiation [listed as global PSP (Precision Spectral Pyranometer)], temperature (2 m), estimated sea-level pressure, average wind speed (2 m), average wind direction (2 m), average wind shear, turbulence intensity, friction velocity, wind chill temperature, dew point temperature, relative humidity, specific humidity, station pressure, average wind speed (5 m), accumulated precipitation, atmospheric electric field, and estimated surface roughness. Instances inside “()” represents the height where the parameter was measured, “m” stands for meters.

After preprocessing the data following steps from Section “Methodology,” two colinear pairs were found and treated: temperature (2 m) and wind chill temperature, and average wind speeds (2 m and 5 m). Then wind chill temperature and average wind speed (5 m) are removed from the dataset, and VIF applied again to make sure the absence of collinearity. After considering correlation, and collinearity, the selected best features are- temperature (2 m), estimated sea-level pressure, average wind speed (2 m), average wind direction (2 m), average wind shear, turbulence intensity, and friction velocity. The cross-correlations between each model parameters are listed in Table 1, where parameter numbered as “8” is our response, average wind speed at 80 m height. The cross-correlations are also illustrated as a heatmap in Figure 7, lighter being higher and darker being lower correlations. Average wind speed at 2 m height and average wind shear both are highly correlated (+ve) with the response variable. On the other hand, sea-level pressure and turbulence intensity are negatively correlated with the wind speed at 80 m.

Table 1.

Cross-correlation of model parameters.

	Parameters [unit]	1	2	3	4	5	6	7	8
1	Temperature (2 m) [°C]	1.0	−0.02	0.24	−0.22	0.12	0.42	0.05	0.16
2	Sea-level pressure (Est) [mBar]	−0.02	1.0	−0.13	−0.05	−0.09	0.03	−0.07	−0.13
3	Avg wind speed (2 m) [m/s]	0.24	−0.13	1.0	0.08	0.64	0.07	0.23	0.87
4	Avg wind direction (2 m) [°]	−0.22	−0.05	0.08	1.0	0.26	−0.52	0.19	0.25
5	Avg wind shear [1/s]	0.12	−0.09	0.64	0.26	1.0	−0.07	0.52	0.88
6	Turbulence intensity (2 m)	0.42	0.03	0.07	−0.52	−0.07	1.0	−0.01	−0.03
7	Friction velocity (u*) [m/s]	0.05	−0.07	0.23	0.19	0.52	−0.01	1.0	0.41
8	Avg wind speed (80 m) [m/s]	0.16	−0.13	0.87	0.25	0.88	−0.03	0.41	1.0

Figure 7.

Heatmap showing model parameter cross-correlation.

Figure 8 shows the Q-Q plot that tells the dataset is not normalized. Thus, prior normalization is required before feeding it to the models.

Figure 8.

Chi-squared Q-Q plot.

Train-Test Split

The prediction algorithms are trained using training data. However, the performance of a model depends on how well it can predict the response variable when encounters unknown predictors (test data). Therefore, the dataset is usually divided into two sets: training and test sets. The training data set is then used to train the prediction algorithm while the test set is allocated to use them as an unknown predictor to analyze the model performance. The ratio of allocating data for training and test is randomly selected, but literature shows 70–80% for training, and 20–30% for the test is common practice (Illarionov and Khudorozhkov 2018; Ibrahim and Bennett 2014; Shobha and Rangaswamy 2018). In this research, we have separated 80% of the total data to train the models and rest 20% to test the model performance.

Simulation Results

We will discuss the simulation and performances of the state-of-the-art prediction algorithms for wind speed prediction in 80 m height for the NREL dataset. We listed the algorithms as Model-1 to 12 in Table 2 and fitted them on the training data for learning. Once the training is done, we evaluated model performances according to the accuracy measures described in Section “Performance Evaluation” on test data. Following settings are chosen for the reference data set and then used in a generalized manner for the four different test cases.

Table 2.

Comparative model performances (duration: 3 months).

Model	Algorithm	Mean absolute error (MAE)	Mean squared error (MSE)	Median absolute error (MedAE)	R2 score
Model-1	Multiple linear regression	0.421	0.357	0.277	0.923
Model-2	Ridge regression (alpha = 15)	0.579	0.598	0.434	0.872
Model-3	Least absolute shrinkage and selection operator (Lasso) regression (alpha = 0.1)	0.823	1.156	0.704	0.752
Model-4	Bayesian ridge regression	0.428	0.361	0.285	0.922
Model-5	Hubber regression	0.422	0.38	0.259	0.919
Model-6	Bagging regression	0.274	0.171	0.185	0.963
Model-7	Random forest regression	0.275	0.179	0.192	0.962
Model-8	Adaptive boosting (AdaBoost) regression	0.385	0.272	0.297	0.942
Model-9	Support vector regression (SVR)	0.411	0.347	0.261	0.926
Model-10	Multilayer perceptron (MLP)/ DNN (hidden layer = 13, activation = relu)	0.31	0.178	0.234	0.962
Model-11	CNN (filters = 64, kernel size = 2, activation = relu, maxpooling size = 2)	0.634	0.831	0.45	0.82
Model-12	RNN – LSTM (kernel = normal, activation = linear)	0.226	0.107	0.145	0.978

For ridge regression, alpha was considered 15 after a few trial and errors. Similarly, for Lasso, the alpha parameter was set to 0.1. For SVR, the default kernel initializer was applied. Table 2 depicts the accuracy measures for each algorithm. Overall, the considered algorithms were able to predict the average wind speed properly for the refence data with an R2 value greater than 0.9 in most cases, as shown. Among the machine learning algorithms, MAE, MSE, and MedAE are minimum for bagging and random forest regression. Both algorithms show greater accuracy (>96%). On the other hand, Models 3–5 show the lowest accuracy among the machine learning regression algorithms with an R2 Score ≈ 0.92.

Deep learning models- DNN, CNN, and LSTM, are denoted as Models 10–12 in Table 2. Both DNN and CNN use the ReLu activation function. DNN uses 13 hidden layers, while the neural network of CNN consists of 50 neural network layers. Max pooling size for CNN is 2. On the other hand, LSTM uses a linear activation and consists of 50 hidden layers. In terms of accuracy and error parameters, CNN showed the worst performance, while both DNN and LSTM prediction accuracy were high (>96%). However, LSTM (Model-12) showed the best performance in terms of all metrics with the lowest error terms, while the exact accuracy was 97.8%. Overall, we can see from Table 2, plots, and discussion, LSTM performed best for our investigation. Therefore, LSTM is the efficient learning algorithm between 12 test models to predict the wind speed at 80 m height while the temperature at 2 m height, estimated sea-level pressure average wind speed at 2 m height, average wind direction at 2 m height, average wind shear, turbulence intensity at 2 m height, and friction velocity of a certain location are known. Now we shall discuss the model performances for other four test cases and compare them with the reference experiment.

In Table 3, for the 10 days dataset, Bagging regression was able to predict the wind speed with the highest accuracy of 97.9% and CNN with 98.6%. Similar trend is visible for 30 days data set, Bagging regression, CNN and LSTM all were able to predict wind speed with accuracies greater than 98%. For a larger dataset, 6 months, Bagging and random forest regressions outperformed others with >98% accuracy, while deep learning models are comparatively less effective (LSTM with 93.4%). For the largest dataset (1 year), a similar pattern is evident for both Bagging and random forest regressions with 98% accuracy while DNN performs similar.

Table 3.

Comparative model performances (duration: 10 days, 30 days, 6 months, 1 year).

Models	Mean absolute error (MAE)				Mean squared error (MSE)				Median absolute error (MedAE)				R2 score
	10 days	30 days	6 months	1 year	10 days	30 days	6 months	1 year	10 days	30 days	6 months	1 year	10 days	30 days	6 months	1 year
1	0.419	0.475	0.637	1.913	0.332	0.411	0.721	7.021	0.296	0.37	0.489	1.509	0.976	0.979	0.922	0.296
2	1.365	1.152	0.835	1.931	3.917	2.596	1.128	7.243	1.055	0.848	0.696	1.533	0.717	0.87	0.878	0.273
3	0.681	0.72	0.996	2.153	0.753	0.86	1.53	8.817	0.566	0.609	0.895	1.736	0.946	0.957	0.835	0.115
4	0.419	0.475	0.637	1.914	0.332	0.411	0.721	7.031	0.298	0.371	0.489	1.509	0.976	0.979	0.922	0.295
5	0.411	0.476	0.629	1.355	0.343	0.418	0.731	12.302	0.288	0.353	0.462	0.922	0.975	0.979	0.921	−0.23
6	0.401	0.422	0.282	0.299	0.293	0.372	0.16	0.199	0.297	0.296	0.203	0.205	0.979	0.981	0.983	0.98
7	0.429	0.441	0.278	0.295	0.3	0.452	0.156	0.198	0.374	0.313	0.197	0.2	0.978	0.977	0.983	0.98
8	0.503	0.584	0.497	0.629	0.426	0.56	0.391	0.632	0.414	0.44	0.405	0.542	0.969	0.972	0.958	0.937
9	0.693	0.489	0.63	1.832	0.81	0.436	0.742	7.626	0.468	0.391	0.458	1.305	0.942	0.978	0.92	0.235
10	2.694	0.469	0.616	0.290	15.266	0.415	0.696	0.190	2.18	0.332	0.454	0.2	−0.10	0.979	0.925	0.98
11	0.349	0.414	0.617	0.620	0.196	0.304	0.677	0.63	0.311	0.311	0.45	0.54	0.986	0.985	0.927	0.93
12	0.481	0.396	0.572	1.830	0.344	0.314	0.614	0.720	0.413	0.305	0.427	1.3	0.975	0.984	0.934	0.23

In a nutshell, LSTM outperforms other models for three months wind speed data, while it does not hold ground truth for different data sizes. We have analyzed, Bagging regression outperforms other machine learning models for all cases with accuracy measures with accuracies >96%. However, the deep learning models perform differently. Overall, most of the algorithms (Model 1–12) were able to predict wind speed with accuracies greater than 90%.

Conclusion

Wind speed prediction is crucial for understanding the inherent nature of wind energy. Its variation is intricate and the influential weather factors are complex to infer. In this paper, we predicted wind speed at a height that is challenging to reach by using easy to access weather parameters- few of which haven’t been considered before in analogous research. We investigated twelve artificial intelligence algorithms for four different data sizes and compared them to a reference case. This research will be useful for wind farm planning and feasibility study.