An asymptotic and empirical smoothing parameters selection method for smoothing spline ANOVA models in large samples

Summary

Large samples are generated routinely from various sources. Classic statistical models, such as smoothing spline ANOVA models, are not well equipped to analyse such large samples because of high computational costs. In particular, the daunting computational cost of selecting smoothing parameters renders smoothing spline ANOVA models impractical. In this article, we develop an asympirical, i.e., asymptotic and empirical, smoothing parameters selection method for smoothing spline ANOVA models in large samples. The idea of our approach is to use asymptotic analysis to show that the optimal smoothing parameter is a polynomial function of the sample size and an unknown constant. The unknown constant is then estimated through empirical subsample extrapolation. The proposed method significantly reduces the computational burden of selecting smoothing parameters in high-dimensional and large samples. We show that smoothing parameters chosen by the proposed method tend to the optimal smoothing parameters that minimize a specific risk function. In addition, the estimator based on the proposed smoothing parameters achieves the optimal convergence rate. Extensive simulation studies demonstrate the numerical advantage of the proposed method over competing methods in terms of relative efficacy and running time. In an application to molecular dynamics data containing nearly one million observations, the proposed method has the best prediction performance.

1. Introduction

In this article, we consider a nonparametric model of the form

$$y_i=\eta \left(x_i\right)+{ϵ}_i\left(i=1,\mathrm{\dots },n\right),$$ (1)

where $y_i\in \mathbb{R}$ is the response variable for the ith observation, η is a nonparametric function varying in an infinite-dimensional functional space, x_i = (x_i〈1〉, …, x_i〈d〉)^T is a d-dimensional vector of predictors for the ith observation, and the ϵ_i are independent and identically distributed random errors with mean zero and unknown variance σ². We focus on the smoothing spline ANOVA model (Wahba et al., 1995) for a multi-dimensional problem, i.e., with d > 1. In the smoothing spline ANOVA model we decompose the function η as

$$\eta \left(x\right)={\eta }_{\varnothing }+\sum _{j=1}^d{\eta }_j\left(x_{\langle j\rangle }\right)+\sum _{j<k}{\eta }_{j,k}\left(x_{\langle j\rangle },x_{\langle k\rangle }\right)+\cdots +{\eta }_{1,2,\mathrm{\dots },d}\left(x_{\langle 1\rangle ,}{x}_{\langle 2\rangle },\mathrm{\dots },x_{\langle d\rangle }\right),$$ (2)

where η_∅ is a constant, the η_j are main-effect functions, the η_j,k are two-way interaction functions, and η_1,2,…,d(x_〈1〉, x_〈2〉, …, x_〈d〉) is a d-way interaction function. Side conditions on the components are imposed to guarantee a unique decomposition. The nonparametric function η can be estimated by minimizing the penalized least squares

$$\frac{1}{n}\sum _{i=1}^n{\left\{y_i-\eta \left(x_i\right)\right\}}^2+\lambda P\left(\eta \right),$$ (3)

where P(η) = P(η, η) is a quadratic roughness penalty and the smoothing parameter λ controls the trade-off between the lack of fit of η and the roughness of η. Extra smoothing parameters are included in P(η, η) to adjust the strength of the components in (2), but for simplicity we omit them from the notation. The explicit formula for P(η, η) can be found in § 2.1. Since the minimizer of (3), denoted by η_n,λ, is sensitive to the selection of λ, it is crucial to choose an effective and efficient method for selecting the smoothing parameter.

Numerous computational methods for smoothing parameter selection have been proposed. One of the earliest is the C_L method (Mallows, 1973). To circumvent the impracticality of the C_L method due to its dependence on an unknown σ², Craven & Wahba (1978) proposed generalized cross-validation. They showed that the smoothing parameter estimated by generalized cross-validation minimizes a specific risk function asymptotically. Although their method gives a good estimate of λ without prior knowledge of the variance σ², it occasionally has an undersmoothing problem. To overcome this problem, Kim & Gu (2004) developed a modified version of generalized cross-validation by adding a fudge factor. Under the Bayes framework, Wahba (1985) proposed a maximum likelihood estimate for the smoothing parameter. Extensive simulations were performed to demonstrate that the maximum likelihood estimate provides satisfactory estimates. Nonetheless, the minimizer η_n,λ based on the smoothing parameter chosen by maximum likelihood cannot be guaranteed to attain the optimal convergence rate. In contrast to the above methods, the improved Akaike information criterion proposed by Hurvich et al. (1998) aims to avoid the undersmoothing problem of generalized cross-validation. However, the empirical performance of Hurvich et al.’s criterion is not as good as that of other criteria, such as generalized cross-validation, in some situations (Aydin et al., 2013). Moreover, its soundness is hard to justify owing to the lack of theoretical analysis under the smoothing spline ANOVA framework. A more recent line of work for large datasets is the divide-and-recombine method (Shang & Cheng, 2017; Xu & Wang, 2018). In this approach, a large dataset is divided into small subsets to which smoothing spline models are fitted, and the outputs of these models are then recombined. Since the smoothing spline is applied to small subsets, selecting the smoothing parameter is computationally feasible.

For multivariate η, multiple smoothing parameters are used to adjust the strength of the corresponding components in (2). Gu & Wahba (1991) proposed to select multiple smoothing parameters by minimizing the generalized cross-validation function through a modified Newton method. With all smoothing parameters being tunable, the iterative algorithm takes O(Sn³) flops per iteration, where S is the number of smoothing parameters, and needs tens of iterations to converge. The algorithm is quite efficient when S is small. As the number of multi-way interaction components in (2) increases, the number of smoothing parameters grows dramatically. For example, S is 5 for the full two-way model and 19 for the full three-way model. Thus, the algorithm is computationally expensive for multi-dimensional models with interaction terms. Several methods have been proposed to alleviate the heavy computational burden of these models. An obvious option is to provide good prespecified values for multiple smoothing parameters. Gu & Wahba (1991) proposed an algorithm for calculating these values, and showed that the minimizer of (3) based on them usually yields good estimates. Although the algorithm performs well in additive models, it is unreliable when interaction components are present. The unreliable performance may be exacerbated when the model is misspecified. Helwig & Ma (2015) proposed a reparameterization of smoothing parameters in the smoothing spline ANOVA model. For the reparameterization, there is one smoothing parameter for each predictor and the smoothing parameter for an interaction term is the product of the smoothing parameters of the corresponding predictors. This new algorithm has a computational cost comparable to that of generalized additive models (Hastie & Tibshirani, 1986). Nevertheless, the algorithm may produce a biased estimate when the smoothing spline ANOVA model is misspecified. In addition, its theoretical foundation requires further justification.

Parallel to the work under the smoothing spline ANOVA framework, several authors have proposed efficient smoothing parameter selection methods for generalized additive models. For univariate functions, many attempts have been made to estimate the smoothness of functions (Buja et al., 1989; Marx & Eilers, 1998). These algorithms are fast even for large datasets. For multivariate functions, low-rank tensor product methods were developed (Wood, 2006; Lee & Durbán, 2011). To control the smoothness on different predictors within an interaction term, multiple smoothing parameters are associated with the smoothing penalties corresponding to the interaction. For example, for any bivariate interaction η_{j, k}(x_〈j〉, x_〈k〉) there are two smoothing parameters for controlling the smoothness on predictors x_〈j〉 and x_〈k〉, whereas three smoothing parameters are used under the smoothing spline ANOVA framework to adjust the smoothness on x_〈j〉, x_〈k〉, and the interaction of these two predictors separately. The low-rank tensor product methods reduce the number of smoothing parameters and improve the computational efficiency. However, when the bivariate function η_{j, k}(x_〈j〉, x_〈k〉) is not an additive function with respect to the x_〈j〉 and x_〈k〉 directions, the smoothing spline ANOVA models may have numerical advantages since they can model the interaction of these two predictors. Recently, some extensions of the multivariate smoothing approach in generalized additive models have been proposed to estimate the smooth functions (Ruppert et al., 2003; Wand, 2003; Lee et al., 2013; Wood et al., 2013; Rodríguez-Álvarez et al., 2015; Wood & Fasiolo, 2017). Wood et al. (2017) developed an efficient fitting method to estimate generalized additive models in large samples. In particular, a reparameterization is implemented in the fitting iteration, where the smoothing matrix can be computed blockwise. Moreover, instead of fully optimizing the restricted marginal likelihood at each iteration, a single-step Newton update is utilized. To reduce the memory usage for large matrices, a novel covariate discretization scheme is also implemented. While this discretization scheme significantly reduces the computational time of estimation, a rigorous theoretical investigation is still lacking.

The asymptotic behaviour of η_n,λ and the optimal λ has been studied extensively; see Silverman (1982), Rice & Rosenblatt (1983), Cox (1984), Speckman (1985), Cox & O’Sullivan (1990) and Gu & Qiu (1993). The estimator can achieve an optimal convergence rate when the smoothing parameter is of order O{n^−r/(pr+1)} for r > 1 and p ∈ [1, 2]. Lin (2000) further studied the optimal convergence rate of the estimator in tensor product space ANOVA models, and showed that the optimal rate of smoothing parameters depends on the highest order of interactions in (2). One can directly use Cn^−r/(pr+1) with some predefined C, r and p as the smoothing parameter when fitting the model to a sample of size n (Hall, 1990). This method is referred to as the order-based method. However, the numerical performance of the order-based method is unreliable, which is also observed in our simulation studies.

To make the selection of smoothing parameters practical in large samples, we develop an asympirical, i.e., asymptotic and empirical, smoothing parameters selection approach by combining the theoretical properties of smoothing parameters and the aforementioned computational methods in a synergistic manner. In the proposed method, we choose a subsample of size much smaller than the full sample size n, and we select smoothing parameters for the subsample using the generalized cross-validation method. The smoothing parameters for the full sample are extrapolated based on the selected smoothing parameters and the optimal rate O{n^−r/(pr+1)}. The proposed smoothing parameters selection method reduces the computational complexity from tens of O(Sn³) flops, as required by the generalized cross-validation method, to O(B³), where B is the size of the subsamples. The numerical advantage of the proposed algorithm over the other approaches is significant when there are multiple interaction components in the model, as demonstrated through our extensive simulation studies and real data examples. Besides the numerical advantages, the smoothing parameters obtained using our approach share optimal properties with parameters that minimize a specific risk function for full samples. Furthermore, the estimator based on the proposed smoothing parameters attains the optimal convergence rate.

2. Smoothing spline ANOVA models

2.1. Estimation

We review the Kimeldorf–Wahba representer theorem (Kimeldorf & Wahba, 1971; Wahba, 1990; Wang, 2011), which states that the solution of penalized least squares defined in an infinite-dimensional functional space actually resides in a finite-dimensional space. Recall that the minimization of (3) is performed in the tensor product reproducing kernel Hilbert space $\mathcal{H}=\left\{\eta :P\left(\eta ,\eta \right)<\infty \right\}$ with the quadratic roughness penalty $P\left(\eta ,\eta \right)={\sum }_{\delta =1}^S{\theta }_{\delta }^{-1}{\left(\eta ,\eta \right)}_{\delta }$, where the θ_δ are smoothing parameters that adjust the strengths of the corresponding components, (·, ·)_δ is the inner product in ${\mathcal{H}}_{\delta }$ with reproducing kernel R_δ(·, ·), and S is the number of subspaces based on the tensor product decomposition. The space $\mathcal{H}$ has the tensor sum decomposition $\mathcal{H}={\mathcal{N}}_P\oplus {\mathcal{H}}_P$ where ${\mathcal{N}}_P$, the null space of $\mathcal{H}$, is spanned by ${\left\{{\varphi }_{\nu }\right\}}_{\nu =1}^M$ and ${\mathcal{H}}_P={\oplus }_{\delta =1}^S{\mathcal{H}}_{\delta }$ has the reproducing kernel $R\left(·,·\right)={\sum }_{\delta =1}^S{\theta }_{\delta }{R}_{\delta }\left(·,·\right)$.

Theorem 1 (Kimeldorf–Wahba representer theorem). The minimizer of (3) is

$$\eta \left(x\right)=\sum _{\nu =1}^M{d}_{\nu }{\varphi }_{\nu }\left(x\right)+\sum _{i=1}^n{c}_{i}R\left(x_i,x\right),$$

where d = (d₁, …, d_M)^T and c = (c₁, …, c_n)^T are unknown coefficieents.

Theorem 1 facilitates the estimation by reducing an infinite-dimensional optimization problem to a finite-dimensional one. Based on the representer theorem, the minimization in (3) becomes

$${\left(Y-Td-Kc\right)}^{\text{T}}\left(Y-Td-Kc\right)+n\lambda c^{\text{T}}Kc,$$ (4)

where Y = (y₁, …, y_n)^T, T_n×M is a matrix with (i, ν)th entry ϕ_ν(x_i), and K_n×n is a matrix with (i, j) th entry R(x_i, x_j). By differentiating (4) with respect to d and c, and setting the derivatives to zero, one obtains the linear system of equations

$$\left(\begin{array}{cc}{T}^{\text{T}}T& T^{\text{T}}K\\ K^{\text{T}}T& K^{\text{T}}K+n\lambda K\end{array}\right)\left(\begin{array}{c}d\\ c\end{array}\right)=\left(\begin{array}{c}{T}^{\text{T}}Y\\ K^{\text{T}}Y\end{array}\right).$$ (5)

To estimate d and c, one needs to solve the linear system (5). If the smoothing parameters λ and θ_δ are known, the computational cost is typically O(n³).

2.2. Roughness penalties

One can choose different forms of roughness penalties for the estimation. The most popular choice for univariate η on a compact interval $\mathcal{X}$ is

$$P\left(\eta ,\eta \right)=\underset{\chi }{\int }{\left({\eta }^{\left(m\right)}\right)}^2\text{d}x,$$

where η(m) = d^mη/dx^m. Setting m = 2, a cubic spline estimator is obtained by minimizing (3) (Wahba, 1990). One convenient way to define the penalty for multivariate functions that have the form (2) is to construct the tensor product reproducing kernel Hilbert space. The reproducing kernel Hilbert space $\mathcal{H}$ can be decomposed into the space of constants, the spaces of main effects, and the corresponding spaces of interaction terms lying in the tensor product space of the interacting main-effect spaces.

Example 1. For the tensor product cubic spline on [0, 1]², one has the following space decomposition in each variable:

$$\begin{gathered} \left\{f:f^{\left(2\right)}\in {\mathcal{L}}_2\left[0,1\right]\right\}=\left\{f:f\propto 1\right\}\oplus \left\{f:f\propto k_1\right\} \\ \oplus \left\{f:\underset{0}{\overset{1}{\int }}f\text{d}x=\underset{0}{\overset{1}{\int }}{f}^{\left(1\right)}\text{d}x=0,f^{\left(2\right)}\in {\mathcal{L}}_2\left[0,1\right]\right\} \\ ={\mathcal{H}}_{00}\oplus {\mathcal{H}}_{01}\oplus {\mathcal{H}}_1, \end{gathered}$$

where k₁(x) = x − 0.5. The space of constant terms is ${\mathcal{H}}_{00\langle 1\rangle }\otimes {\mathcal{H}}_{00\langle 2\rangle }$; ${\mathcal{H}}_{00\langle 1\rangle }\otimes \left({\mathcal{H}}_{01\langle 2\rangle }\oplus {\mathcal{H}}_{1\langle 2\rangle }\right)$ and ${\mathcal{H}}_{00\langle 2\rangle }\otimes \left({\mathcal{H}}_{01\langle 1\rangle }\oplus {\mathcal{H}}_{1\langle 1\rangle }\right)$ span the space of main effects; and the subspace $\left({\mathcal{H}}_{01\langle 1\rangle }\oplus {\mathcal{H}}_{1\langle 1\rangle }\right)\otimes \left({\mathcal{H}}_{01\langle 2\rangle }\oplus {\mathcal{H}}_{1\langle 2\rangle }\right)$ spans the space of interactions. Let ${\mathcal{H}}_{\nu ,\mu }={\mathcal{H}}_{\nu \langle 1\rangle }\otimes {\mathcal{H}}_{\mu \langle 2\rangle }$ for ν, μ = 00, 01, 1, with inner products (η, η)_ν,μ and reproducing kernels R_{ν, μ} = R_ν〈1〉R_μ〈2〉; see Gu (2013, Theorem 2.6). One may set

$$\begin{gathered} P\left(\eta ,\eta \right)={\theta }_{1,00}^{-1}{\left(\eta ,\eta \right)}_{1,00}+{\theta }_{00,1}^{-1}{\left(\eta ,\eta \right)}_{00,1} \\ \text{+}{\theta }_{1,01}^{-1}{\left(\eta ,\eta \right)}_{1,01}+{\theta }_{01,1}^{-1}{\left(\eta ,\eta \right)}_{01,1}+{\theta }_{1,1}^{-1}{\left(\eta ,\eta \right)}_{1,1}. \end{gathered}$$

The null space of P(η, η) is

$${\mathcal{N}}_P={\mathcal{H}}_{00,00}\oplus {\mathcal{H}}_{01,00}\oplus {\mathcal{H}}_{00,01}\oplus {\mathcal{H}}_{01,01}.$$

As in Example 1, a two-dimensional η can be decomposed into four main terms: one constant term, two main effect terms, and one two-way interaction term. There are five effective smoothing parameters, namely λ/θ_1,00, λ/θ_00,1, λ/θ_1,01, λ/θ_01,1 and λ/θ_1,1. Two of them, λ/θ_1,00 and λ/θ_00,1, are for the main effects, and the rest are for the interaction effects.

Example 2. For the tensor product cubic spline on {1, …, K} × [0, 1], one can use the kernels $R_{0\langle 1\rangle }\left(x_{\langle 1\rangle },{\stackrel{‵}{x}}_{\langle 1\rangle }\right)=1/K$ and $R_{1\langle 1\rangle }\left(x_{\langle 1\rangle },{\stackrel{‵}{x}}_{\langle 1\rangle }\right)=I_{\left(x_{\langle 1\rangle }={\stackrel{‵}{x}}_{\langle 1\rangle }\right)}-1/K$ on {1, …, K} and the kernels $R_{00\langle 2\rangle }\left(x_{\langle 2\rangle },{\stackrel{‵}{x}}_{\langle 2\rangle }\right)=1$, $R_{01\langle 2\rangle }\left(x_{\langle 2\rangle },{\stackrel{‵}{x}}_{\langle 2\rangle }\right)=k_1\left(x_{\langle 2\rangle }\right)k_1\left({\stackrel{‵}{x}}_{\langle 2\rangle }\right)$ and $R_{1\langle 2\rangle }\left(x_{\langle 2\rangle },{\stackrel{‵}{x}}_{\langle 2\rangle }\right)=k_2\left(x_{\langle 2\rangle }\right)k_2\left({\stackrel{‵}{x}}_{\langle 2\rangle }\right)-k_4\left(x_{\langle 2\rangle }-{\stackrel{‵}{x}}_{\langle 2\rangle }\right)$ on [0,1], where the k_u (u = 1, 2, 4) are scaled Bernoulli polynomials. The tensor product space can be constructed in an analogous way to Example 1.

2.3. Generalized cross-validation

When estimating multivariate functions in a tensor product space, multiple smoothing parameters are involved; see Example 1. The multiple smoothing parameters λ/θ control the trade-off between the lack of fit of η and the roughness of η, where θ = (θ₁, …, θ_S)^T. Gu & Wahba (1991) proposed a modified Newton method for minimizing the generalized cross-validation score,

$$\text{G}\left(\lambda /\theta \right)=\frac{n^{-1}{Y}^{\text{T}}{\left\{I-A\left(\lambda /\theta \right)\right\}}^{2}Y}{{\left[n^{-1}\text{tr}\left\{I-A\left(\lambda /\theta \right)\right\}\right]}^2},$$

iteratively for multiple smoothing parameters, where the smoothing matrix A(λ/θ) is given in the Supplementary Material. In particular, the method consists of the following steps: (i) for fixed θ, minimize the generalized cross-validation score with respect to nλ; (ii) update θ based on current information on nλ.

With all smoothing parameters being tunable, the above iterative algorithm takes O(Sn³) flops per iteration and needs tens of iterations to converge. The number of smoothing parameters, S, increases dramatically as the number of multi-way interactions grows. In particular, S = d + 3d(d − 1)/2 or the two-way interaction model which truncates the decomposition in (2) at two-way interactions, so it is impractical to apply smoothing spline ANOVA models to large samples. Even for the additive modelwith d smoothing parameters being tunable, tens ofiterations of O(n³) flops become infeasible in large samples. Since the iterative algorithm depends heavily on the starting values, Gu & Wahba (1991) proposed an algorithm for calculating good starting values of θ. The software developed by Gu (2014) uses these starting values as the final estimate of θ, and the algorithm is called the skip algorithm. With the aid of the skip algorithm, the multiple smoothing parameters selection problem reduces to the single smoothing parameter selection problem, which takes O(n³) flops. The skip algorithm comprises two steps: (i) for θ_δ = {tr(R_δ)}⁻¹, minimize the generalized cross-validation score with respect to nλ, and calculate c; (ii) estimate the starting values ${\theta }_{\delta 0}={\theta }_{\delta }^2{c}^{\text{T}}{R}_{\delta }c$.

3. Asympirical smoothing parameters selection

3.1. The optimal smoothing parameter

We review the optimal smoothing parameters selection method, which motivates the proposed algorithm. The optimality of smoothing parameter selection can be characterized by minimization of the expectation of the loss function, E{L(λ)}, i.e., the risk function, where the loss function is

$$L\left(\lambda \right)=\frac{1}{n}\sum _{i=1}^n{\left\{{\eta }_{n,\lambda }\left(x_i\right)-\eta \left(x_i\right)\right\}}^2.$$ (6)

Wahba (1975) derived the optimal smoothing parameter by minimizing the risk function for smoothing periodic splines in ${\mathcal{H}}^{\left(m\right)}$, defined by

$${\mathcal{H}}^{\left(m\right)}=\{f:f^{\left(\nu \right)}\text{absolutely continuous for}\nu =0,1,\mathrm{\dots },m-1,f^{\left(m\right)}\in {\mathcal{L}}_2\left[0,1\right],f^{\left(\nu \right)}\left(0\right)-f^{\left(\nu \right)}\left(1\right)=0\text{for}\nu =0,1,\dots ,m-1\}.$$

Suppose that $\eta \in {\mathcal{H}}^{\left(2m\right)}$, i.e., η is very smooth, and ‖η^(2m)‖ ≠ 0, where ‖ · ‖ is the ${\mathcal{L}}^2-\text{norm}$. The optimal choice of smoothing parameter, ignoring o(1) terms, is

$${\left\{\frac{{\tilde{k}}_m}{4m}{\frac{{\sigma }^2}{\Vert {\eta }^{\left(2m\right)}\Vert }}^2\right\}}^{2m/\left(4m+1\right)}{n}^{-2m/\left(4m+1\right)},$$ (7)

where ${\tilde{k}}_m=\left(1/\pi \right)\underset{0}{\overset{\infty }{\int }}1/{\left(1+t^{2m}\right)}^{2}dt$ is a constant depending on m. We rewrite the smoothing parameter in (7) as Cn^−2m/(4m+1), since the first term is a constant unrelated to the full sample size n. Likewise, in the subsample of size b → ∞, the asymptotically optimal smoothing parameter λ_RISK(b) is Cb^−2m/(4m+1) for the same C. If we can estimate C in a subsample of size b, then the smoothing parameter λ_RISK(b)(n/b)^−2m/(4m+1) for the full sample of size n is thereby estimated. Under different smoothness conditions, to be defined later, the optimal smoothing parameter that minimizes the risk function has the form Cb^−r/(pr+1) for r > 1 and p ∈ [1, 2] in a subsample of size b (Wahba, 1977, 1985). For instance, we have r = 2m and p = 2 for the above smoothing periodic spline case. Based on the same rationale, the smoothing parameter for the full sample is

$${\lambda }_{\text{RISK}}\left(b\right){\left(n/b\right)}^{-r/\left(pr+1\right)}.$$ (8)

3.2. The asympirical algorithm

It is infeasible to choose the optimal smoothing parameter if the true η and σ² are unknown. Therefore, we replace the optimal smoothing parameter λ_RISK(b) in (8) with λ_GCV(b) chosen by the generalized cross-validation method in a subsample of size b. The detailed procedure is outlined in Algorithm 1.

Algorithm 1. The asympirical smoothing parameters selection algorithm.

Step 1. Take a random subsample of size b from the original data, and apply the generalized cross-validation method to the subsample to estimate the smoothing parameters λ_GCV(b) and θ_GCV(b).

Step 2. Set smoothing parameters λ_ASP(n; b) = λ_GCV(b)(n/b)^−r/(pr+1) and θ_ASP(n; b) = θ_GCV(b) to find the minimizer of (3) for the full sample of size n.

In the first step, the random subsample is selected using uniform sampling. More delicate sampling approaches can be found in Ma et al. (2015) and Meng et al. (2020). To make the estimated smoothing parameters more stable, we usually take multiple subsamples and choose the median of a group of smoothing parameters. In the algorithm, we assume that optimal smoothing parameters share the same rate of decrease as n increases (Gu & Wahba, 1991). Since smoothing parameters θ are used to adjust the roughness penalties imposed on different components, see Example 1, we calculate the optimal θ_GCV(b) for the subsample and perform the minimization based on the estimated θ_GCV(b) for the full sample. Further details on how to choose b, r and p in practice are given in § 4.

4. Theoretical analysis

This section presents the theoretical analysis of the smoothing parameters selected by Algorithm 1. The selected smoothing parameters tend to the parameter values that minimize the risk function. Our theoretical analysis also provides a guide to choosing b, r and p. We then present results on convergence rates of the estimator based on the proposed smoothing parameters. For simplicity, we suppress λ’s dependence on θ and only make λ explicit. All proofs are given in the Supplementary Material.

Let the subsample size be b. The matrix I − A(λ) for the smoothing spline ANOVA model has the representation

$$I-A\left(\lambda \right)=b\lambda Z{\left(D+b\lambda I\right)}^{-1}{Z}^{\text{T}},$$

where the matrix Z satisfies Z^TZ = I_{(b−M)×(b−M)} and D_b−M is a (b − M) × (b − M) diagonal matrix with real-valued entries ζ_νb > 0; more details are given in the Supplementary Material. We derive theoretical results under the following smoothness assumption.

Assumption 1. The function $\eta \in {\mathcal{H}}_p$, where the space ${\mathcal{H}}_p$ is defined as

$${\mathcal{H}}_p=\left\{\eta :P\left(\eta ,\eta \right)>0,\sum _{\nu =1}^{b-M}\frac{h_{\nu b}^2/b}{{\left({\zeta }_{\nu b}/b\right)}^p}⩽J_p+J_{p}o\left(1\right)\right\}.$$

Here the real-valued vector (h_1,b, …, h_{b−M, b})^T = Z^TH with H = {η(x₁), …, η(x_b)}^T, J_p for p ∈ [1, 2] is a real-valued constant independent of the subsample size b, and o(1) → 0 as b → ∞.

Under Assumption 1, we only consider the case where P(η, η) > 0. When P(η, η) = 0, both the risk function and the generalized cross-validation function are minimized for λ = ∞ (Craven & Wahba, 1978).

Theorem 2. Suppose that Assumption 1 holds for some p ∈ [1, 2]. Let r > 1, let λ_GCV(b) be the smoothing parameter chosen by the generalized cross-validation method for the subsample of size b, let λ_RISK(n) be the optimal smoothing parameter minimizing the risk function for the full sample of size n, and let λ_ASP(n; b) be the proposed smoothing parameter for the full sample of size n. Suppose that λ_GCV(b) → 0 and $b{\lambda }_{\text{GCV}}^{1/r}\left(b\right)\to \infty$. Then λ_ASP(n; b) = λ_RISK(n){1 + o(1)}.

Theorem 2 shows that the proposed smoothing parameter λ_ASP(n; b) is an estimate of the minimizer of E{L(·)} asymptotically. We have the following immediate corollary under regularity conditions stated in the Supplementary Material.

Corollary 1. Under regularity conditions in the Supplementary Material, as λ_GCV(b) → 0, $b{\lambda }_{\text{GCV}}^{1/r}\left(b\right)\to \infty$ and n → ∞, we have

$$\frac{E\left[L\left\{{\lambda }_{\text{ASP}}\left(n;b\right)\right\}\right]}{E\left[L\left\{{\lambda }_{\text{RISK}}\left(n\right)\right\}\right]}=1+o(1).$$

This corollary gives the expectation inefficiency of λ_ASP(n; b) relative to λ_RISK(n) as the number of observations n tends to infinity.

In Theorem 2 one needs $b{\lambda }_{\text{GCV}}^{1/r}\left(b\right)\to \infty$. We further assume that λ_GCV(b) achieves the optimal rate n^−r/(pr+1), and it suffices to have b ≍ n^1/(pr+1)+ε for any ε > 0. For $P\left(\eta ,\eta \right)=\underset{0}{\overset{1}{\int }}{\left({\eta }^{\left(2\right)}\right)}^2\text{d}x$ on [0,1], r = 4, and we have p = 1 when η⁽²⁾ is square-integrable and p = 2 when η⁽⁴⁾ is square-integrable. For the tensor product cubic spline, r is typically less than 4 (Wahba, 1990; Lin, 2000), so we set r = 3 empirically. Taking these facts into consideration, we set r = 3, p = 1 and ε = 0, and use b ∝ n^1/4 empirically. In real applications, the subsample size b is set to 50n^1/4. The smoothness of η is indexed by p, which is estimated empirically. We first take a random subsample of size B, and minimize the generalized cross-validation score with respect to p ∈ {1, 2} by replacing λ in the score with λ_GCV(b)(B/b)^−r/(pr+1). We take B = 2b in our simulation studies and real data examples. Thus the computational complexity of the proposed algorithm is of order O(B³). To reduce the computational burden of fitting smoothing spline ANOVA models for large samples, one can implement the fast algorithm proposed by Kim & Gu (2004). In the algorithm, one first randomly selects $\stackrel{⌣}{q}$ basis functions from n and then estimates the minimizer of (3). The algorithm requires $O\left(n{\stackrel{⌣}{q}}^2\right)$ flops to estimate the minimizer for each choice of smoothing parameters. Thus, the corresponding computational complexities of the generalized cross-validation method and the proposed method are also reduced. The complexity of the proposed method is of order $O\left(B{\stackrel{⌣}{q}}^2\right)$ when the fast algorithm is applied.

We now show the convergence rate of the estimator that relies on the proposed smoothing parameters. To study theoretical properties of smoothing spline ANOVA models, one needs the quadratic functional V defined by

$$V\left({\eta }_{n,\lambda }-\eta ,{\eta }_{n,\lambda }-\eta \right)=\underset{\mathcal{X}}{\int }{\left\{{\eta }_{n,\lambda }\left(x\right)-\eta \left(x\right)\right\}}^{2}f\left(x\right)\text{dx},$$

where f(·) is the marginal density of x. The functional represents the mean squared error of the estimator η_n,λ in estimating the function η on a compact domain $\mathcal{X}\subset {\mathbb{R}}^d$.To avoid interpolation, the regularization λP needs to restrict the estimate to an effective model space. To control the bias, the effective model space needs to be increased by letting λ → 0 as the sample size n → ∞. It was shown in Gu (2013, Ch. 9) that (V + λP)(η_n,λ − η, η_n,λ − η) = O(n⁻¹λ^−1/r + λ^p). We establish the following theorem under regularity conditions described in the Supplementary Material.

Theorem 3. Under the regularity conditions in the Supplementary Material and for some p ∈ [1, 2] and r > 1, as λ_RISK(n) → 0 and $n{\lambda }_{\text{RISK}}^{2/r}\left(n\right)\to \infty$, we have

$$\{V+{\lambda }_{\text{RISK}}\left(n\right)P\}\left({\eta }_{n,{\lambda }_{\text{ASP}}\left(n;b\right)}-{\eta }_{n,{\lambda }_{\text{RISK}}\left(n\right)},{\eta }_{n,{\lambda }_{\text{ASP}}\left(n;b\right)}-{\eta }_{n,{\lambda }_{\text{RISK}}\left(n\right)}\right)=O\left(n^{-pr/\left(pr+1\right)}\right).$$

Remark 1. Our result is for the general smoothing spline estimator. If some structures of the underlying function, e.g., shape-restricted, are known a priori, the convergence rate maybe faster, and the estimator may converge in o(·) rather than O(·).

5. Simulation studies

5.1. Simulation settings

Simulation studies, including univariate and multivariate cases, were carried out to assess the performance of the proposed method in terms of mean squared error. For univariate cases, we compared the proposed method with the generalized cross-validation method and the order-based method of Hall (1990). For multivariate cases, the proposed method was compared with the generalized cross-validation method, the skip method, and another three approaches, namely generalized cross-validation, restricted maximum likelihood, and fast restricted maximum like-lihood(Wood et al., 2017) in generalized additive models. For the proposed method we used two sampling schemes to select subsamples: uniform sampling and asymptotic sampling. The former is described in Algorithm 1. The asymptotic sampling strategy is implemented in two steps. First, take random subsamples of size b₁, …, b_N from the original data and apply the generalized cross-validation method to the subsamples to estimate smoothing parameters λ_GCV(b₁), …, λ_GCV(b_N). Second, apply the constrained optimization method to estimate the constant C and rate parameters r and p by minimizing the objective function $\left(1/N\right){\sum }_{k=1}^N{\left\{{\lambda }_{\text{GCV}}\left(b_K\right)-C{b}_k^{-r/\left(pr+1\right)}\right\}}^2$ with constraints p ∈ [1, 2] and r > 1. Compared with the uniform sampling scheme, asymptotic sampling provides empirical estimates of parameters needed for the asympirical smoothing parameters selection without using any prior knowledge on rate parameters. In multivariate cases we set N = 10, and b₁ and b₁₀ were set to 50n^1/4 and 120n^1/4, respectively. In the order-based method, we directly used n^−r/(pr+1) as the smoothing parameter λ for sample size n. The skip method is described in § 2.3. The generalized cross-validation, restricted maximum likelihood, and fast restricted maximum likelihood methods under the generalized additive models framework were implemented in the mgcv package (Wood, 2004,2011; Wood et al., 2017) of R (R Development Core Team, 2020). We used the fast algorithm proposed by Kim & Gu (2004) to reduce the computational burden of fitting smoothing spline ANOVA models. To make a fair comparison, the same number of basis functions was used for all methods. We chose the generalized cross-validation method to be the benchmark and report the log-transformed relative efficacy. This relative efficacy is defined as ${\sum }_{i=1}^n{\left\{\widehat{\eta }\left(x_i\right)-\eta \left(x_i\right)\right\}}^2/{\sum }_{i=1}^n{\left\{\tilde{\eta }\left(x_i\right)-\eta \left(x_i\right)\right\}}^2,$ where $\widehat{\eta }$ is the estimator for the method being compared and $\tilde{\eta }$ is the estimator based on the generalized cross-validation method. A smaller log-transformed relative efficacy indicates better performance. If the log-transformed relative efficacy is zero, the method being compared has the same numerical performance as the generalized cross-validation method. Three univariate and four multivariate functions were evaluated. The full sample size n was set to 20 000, 30 000 and 40 000. Four values, 1, 2, 5 and 7, of the signal-to-noise ratio, defined as snr = sd{η(x)}/σ, were used to generate the data. One hundred replicates were generated for each setting.

5.2. Univariate scenarios

We simulated the data according to (1) using three univariate functions with different orders of smoothness.

Univariate scenario 1:

$${\eta }_{u1}\left(x\right)=\frac{1}{3}{B}_{20,5}\left(x\right)+\frac{1}{3}{B}_{12,12}\left(x\right)+\frac{1}{3}{B}_{7,30}\left(x\right),$$

where

$$B_{\alpha ,\beta }\left(x\right)=\frac{\Gamma \left(\alpha +\beta \right)}{\Gamma \left(\alpha \right)\Gamma \left(\beta \right)}{x}^{\alpha -1}{\left(1-x\right)}^{\beta -1}\left(0⩽x⩽1\right).$$

Univariate scenario 2:

$${\eta }_{u2}\left(x\right)=10{\sin }^2\left(2\pi x\right){\mathbb{1}}_{\left(x⩽\frac{1}{2}\right)},$$

where ${\mathbb{1}}_{\left(x⩽\frac{1}{2}\right)}$ is an indicator function that eqals 1 for $x⩽\frac{1}{2}$ and equals to 0 otherwise .

Univariate scenario 3:

$${\eta }_{u3}\left(x\right)=10\times \left\{-x+2\left(x-\frac{1}{4}\right)\right\}{\mathbb{1}}_{\left(x⩾\frac{1}{4}\right)}+2\left(-x+\frac{3}{4}\right){\mathbb{1}}_{\left(x⩾\frac{3}{4}\right)},$$

where ${\mathbb{1}}_{\left(x⩾\frac{1}{4}\right)}$ and ${\mathbb{1}}_{\left(x⩾\frac{3}{4}\right)}$ are two indicator functions that equal 1 when the conditions in parentheses are satisfied and equal 0 otherwise.

We generated x from a uniform distribution on [0, 1]. The generated data for three univariate functions with snr = 1 and three true function values are shown in Fig. 1. The log-transformed relative efficacies of the proposed method and the order-based method for the three scenarios are shown in Fig. 2. The skip method reduces to the generalized cross-validation method in the single smoothing parameter selection. The performance of the proposed method is comparable to that of the generalized cross-validation method when the signal-to-noise ratio is low, with log-transformed relative efficacies close to zero. The performance of our method is better than that of the generalized cross-validation method as the signal-to-noise ratio increases. Such behaviour may result from unstably estimated smoothing parameters based on subsamples when the signal-to-noise ratio is low. Even though the order-based method performs well in some scenarios, such as univariate scenario 3, it is not reliable because of the large variability in most scenarios.

Fig. 1.

The univariate true functions (solid lines) of (a) η_u1, (b) η_u2, and (c) η_u3, with the data used in the simulation represented by circles.

Fig. 2.

Log-transformed relative efficacies of the proposed method and the order-based method with respect to the generalized cross-validation method for the three univariate scenarios. The vertical axis represents the log-transformed relative efficacies, and the horizontal axis shows the different methods. Different signal-to-noise ratios are indicated by different colours. The results of univariate scenarios 1, 2 and 3 are displayed in the upper, middle and lower panels, respectively, and the results for full sample sizes 20 000, 30 000 and 40 000 are shown in the left, middle and right columns, respectively. asp-u, asympirical method using uniform sampling; order, order-based method; snr, signal-to-noise ratio.

5.3. Multivariate scenarios

We simulated the data according to (1) using four multivariate functions. In these four scenarios, the x values were drawn from the uniform distribution on [0, 1].

Multivariate scenario 1:

$$\begin{gathered} {\eta }_{m1}\left(x\right)=\frac{0.75}{\pi {\sigma }_{x_{\langle 1\rangle }}{\sigma }_{x_{\langle 2\rangle }}}\text{exp}\left\{-\frac{{\left(x_{\langle 1\rangle }-0.2\right)}^2}{{\sigma }_{x_{\langle 1\rangle }}^2}-\frac{{\left(x_{\langle 2\rangle }-0.3\right)}^2}{{\sigma }_{x_{\langle 2\rangle }}^2}\right\} \\ \text{+}\frac{0.45}{\pi {\sigma }_{x_{\langle 1\rangle }}{\sigma }_{x_{\langle 2\rangle }}}\exp \left\{-\frac{{\left(x_{\langle 1\rangle }-0.7\right)}^2}{{\sigma }_{x_{\langle 1\rangle }}^2}-\frac{{\left(x_{\langle 2\rangle }-0.8\right)}^2}{{\sigma }_{x_{\langle 2\rangle }}^2}\right\}, \end{gathered}$$

where σ_x〈1〉 = 0.3 and σ_x〈2〉 = 0.4.

Multivariate scenario 2:

$${\eta }_{m2}\left(x\right)=10\sin \left(\pi x_{\langle 1\rangle }\right)+\exp \left(3x_{\langle 2\rangle }\right)+{10}^6{x}_{\langle 3\rangle }^{11}{\left(1-x_{\langle 3\rangle }\right)}^6+{10}^4{x}_{\langle 3\rangle }^3{\left(1-x_{\langle 3\rangle }\right)}^{10}.$$

Multivariate scenario 3:

$${\eta }_{m3}\left(x\right)=10x_{\langle 2\rangle }+10\sin \left\{\pi \left(x_{\langle 3\rangle }-x_{\langle 2\rangle }\right)\right\}+5\cos \left\{2\pi \left(x_{\langle 1\rangle }-x_{\langle 2\rangle }\right)\right\}.$$

Multivariate scenario 4:

$${\eta }_{m4}\left(x\right)=\sum _{j=1}^{18}{g}_1\left(x_{\langle j\rangle }\right)+\sum _{j=1}^9{g}_2\left(x_{\langle 2j-1\rangle },x_{\langle 2j\rangle }\right)+\sum _{j=1}^6{g}_3\left(x_{\langle 3j-2\rangle },x_{\langle 3j-1\rangle },x_{\langle 3j\rangle }\right),$$

where g₁(x) = 10⁶x¹¹(1 − x)⁶, g₂(x_〈1〉, x_〈2〉) = exp(3x_〈1〉x_〈2〉) and g₃(x_〈1〉, x_〈2〉, x_〈3〉) = 15 sin(2πx_〈1〉)/{2 − sin(2πx_〈2〉x_〈3〉)}.

The full η = η_∅ + η₁ + η₂ + η₁₂ was considered for multivariate scenario 1, and the additive model η = η_∅ + η₁ + η₂ + η₃ was fitted in multivariate scenario 2. In multivariate scenario 3, we considered the partial model η = η_∅ + η₂ + η₂₃ + η₁₂. We further considered the high-dimensional case in multivariate scenario 4. Log-transformed relative efficacies of all methods over the generalized cross-validation method are displayed in Fig. 3.

Fig. 3.

Log-transformed relative efficacies of the methods under comparison in four multivariate scenarios. The vertical axis represents the log-transformed relative efficacies, and the horizontal axis shows the different methods. Different signal-to-noise ratios are indicated by different colours. From top to bottom the panels display the results for multivariate scenarios 1 to 4, and the results for full sample sizes 20 000, 30 000 and 40 000 are shown in the left, middle and right columns, respectively. asp-u, asympirical method using uniform sampling; asp-a, asympirical method using asymptotic sampling; gam-gcv, generalized cross-validation for generalized additive models; reml, restrictedmaximum likelihood for generalized additive models; bam, fast restricted maximum likelihood for generalized additive models; skip, the skip method; snr, signal-to-noise ratio.

All the methods have similar numerical performance in multivariate scenarios 1 and 2. However, the restricted maximum likelihood method has slightly larger relative efficacies in these two scenarios. In multivariate scenario 3, the proposed method based on uniform sampling has slightly larger relative efficacies when the signal-to-noise ratio is small, but its relative efficacies become smaller as the signal-to-noise ratio increases. The proposed method based on asymptotic sampling has smaller relative efficacies than the one based on uniform sampling in this scenario. The median of relative efficacies of the methods under the generalized additive models framework is more than 35, which implies that the mean squared error of these methods is at least 35 times as large as for the generalized cross-validation method. In addition, the relative efficacies of the skip method are around 15. In the high-dimensional setting, to make the generalized cross-validation method feasible, we used the estimated smoothing parameters after the first iteration as the final smoothing parameters. It is expected that the proposed method will perform better than the one-iteration generalized cross-validation method.

Comparing the performances in multivariate scenario 3, we observe a similar phenomenon for the methods under the generalized additive models framework and the skip method in this high-dimensional setting. The median of the relative efficacies of the methods under the generalized additive models framework is about 4, while that for the proposed methods is around 0.7. The methods under the generalized additive models framework construct the bivariate interaction using two smoothing parameters, which control the smoothness on the directions of two predictors. In the smoothing spline ANOVA framework, there are three smoothing parameters associated with the bivariate interaction. The additional smoothing parameter could improve the numerical performance when the interaction is not an additive function, and this may be the reason that the proposed method performs well in the scenarios where multiple interaction components are present. The number of smoothing parameters is different for the methods under the smoothing spline ANOVA models and under the generalized additive models framework. For methods under the smoothing spline ANOVA framework, there are 5, 3, 7 and 87 effective smoothing parameters in multivariate scenarios 1, 2, 3 and 4, respectively, whereas the corresponding numbers of tunable smoothing parameters for methods under the generalized additive models framework are 4, 3, 5 and 54. Although the number of basis functions is the same for all methods, the generalized cross-validation method under the generalized additive models framework is typically faster than the one under the smoothing spline ANOVA models framework, since the method for generalized additive models has fewer tunable smoothing parameters. This is observed in the running time analysis reported in the Supplementary Material.

6. Real data examples

6.1. Superconductivity data

Superconductivity refers to the phenomenon wherein materials can conduct current with zero resistance. Many applications, such as magnetic resonance imaging, are based on superconductivity. Since this phenomenon is only observed at or below a characteristic critical temperature, prediction of the critical temperature of a superconductor is an important problem. In this real data example, we aim to predict the critical temperature by using elemental properties extracted from superconductors. The response is the critical temperature in kelvins. The predictors represent the elemental properties of a superconductor. For instance, one can derive a feature of the superconductor by calculating the average thermal conductivities of the elements in its chemical formula. More details about all the predictors in this example are available in Hamidieh (2018). The dataset contains 21 263 observations. We fit the cubic tensor product smoothing spline ANOVA model to the dataset. Based on the preliminary model diagnostics (Gu, 2004), we consider the following functional ANOVA decomposition:

$$\eta \left(x\right)={\eta }_{\varnothing }+\sum _{j=1}^{42}{\eta }_j\left(x_{\langle j\rangle }\right),$$

where η_∅ is a constant function and η₁(x_〈1〉), …, η₄₂(x_〈42〉) denote the main-effect functions for 42 selected features. Details of the selected features can be downloaded at https://github.com/shawnstat/Asympirical-Smoothing-Parameters-Selection. There are 42 effective smoothing parameters in the decomposition. For a fair comparison, the number of basis functions for all methods is taken to be 10n^2/9 (Kim & Gu, 2004).

Table 1 shows the fitting and prediction statistics for the methods under comparison. To evaluate the prediction performance, we compare the five-fold cross-validated root mean squared errors obtained by dividing the full data into five equal parts. The mean and standard deviation of the five root mean squared errors in predicting the testing data are reported. Compared with the proposed methods and the methods under the generalized additive models framework, the generalized cross-validation method has better performance in terms of fitting and prediction mean squared errors. On the other hand, the proposed methods are much faster in terms of CPU time.

Table 1.

Fitting and prediction statistics of the methods under comparison applied to the superconductivity dataset

Method	R2	Root fitting MSE	Root prediction MSE (mean)	Root prediction MSE (sd)	CPU time (s)
ASP-U	0.786	15.675	15.871	0.239	0.030
ASP-A	0.785	15.719	15.870	0.281	0.790
GAM-GCV	0.765	16.556	16.627	0.249	0.270
REML	0.764	16.625	16.630	0.252	15.200
BAM	0.763	16.625	16.645	0.248	0.062
GCV	0.789	15.363	15.514	0.289	40.560

MSE, mean squared error; sd, standard deviation; ASP-U, asympirical method using uniform sampling; ASP-A, asympirical method using asymptotic sampling; GAM-GCV, generalized cross-validation for generalized additive models; REML, restricted maximum likelihood for generalized additive models; BAM, fast restricted maximum likelihood for generalized additive models; GCV, generalized cross-validation method.

6.2. Molecular dynamics data

With the aid of modern quantum chemistry methods, researchers can conduct systematic simulations of quantum chemical systems, obtaining accurate results on molecular dynamics at the quantum level. Analysis of such molecular dynamics trajectories is crucial for the discovery of new chemicals (Chmiela et al., 2017; Schütt et al., 2017). The molecular dynamics data on malondialdehyde used in this example contain 893 238 observations. The response is the energy in kcal/mol. The predictors encode the molecular structure, which is measured in terms of the reciprocal of the pairwise Euclidean distance between atoms (Montavon et al., 2013). Since there are nine atoms in malondialdehyde, we have a distance vector of length 36 for each trajectory. Therefore, there are 36 predictors for this dataset. We fit the cubic tensor product smoothing spline ANOVA model to the dataset. Based on the preliminary model diagnostics (Gu, 2004), we consider the following functional ANOVA decomposition:

$$\begin{gathered} \eta \left(x\right)={\eta }_{\varnothing }+\sum _{j=1}^{36}{\eta }_j\left(x_{\langle j\rangle }\right) \\ \text{+}{\eta }_{9,10}\left(x_{\langle 9\rangle },x_{\langle 10\rangle }\right)+{\eta }_{9,13}\left(x_{\langle 9\rangle },x_{\langle 13\rangle }\right)+{\eta }_{24,35}\left(x_{\langle 24\rangle },x_{\langle 35\rangle }\right)+{\eta }_{25,36}\left(x_{\langle 25\rangle },x_{\langle 36\rangle }\right), \end{gathered}$$

where x_〈j〉 (j = 1, …, 36) is the jth predictor. The numbers of smoothing parameters for the proposed methods and the methods under the generalized additive models framework are 48 and 44, respectively. Bearing in mind the limits on computational resources, we set the number of basis functions to 4.3n^2/9 for all methods (Kim & Gu, 2004).

We compare the fitting and prediction errors of these smoothing parameter selection methods in Table 2. The mean and standard deviation of the five root mean squared error results for the testing datasets are reported as the prediction error. Since the generalized cross-validation method was infeasible even for one iteration, we compared only the proposed methods and the methods under the generalized additive models framework with the skip method. We also compared the proposed methods with the fast restricted maximum likelihood method for generalized additive models (Wood et al., 2017). The proposed method based on asymptotic sampling has the best performance in terms of fitting and prediction errors. The fast restricted maximum likelihood method for generalized additive models is the fastest in terms of CPU time.

Table 2.

Fitting and prediction statistics of the methods under comparison applied to the molecular dynamics dataset

Method	R2	Root fitting MSE	Root prediction MSE (mean)	Root prediction MSE (sd)	CPU time (s)
ASP-U	0.925	1.130	1.134	0.006	1.596
ASP-A	0.926	1.124	1.134	0.003	1.969
GAM-GCV	0.911	1.229	1.226	0.003	4.891
BAM	0.913	1.219	1.224	0.006	0.490
SKIP	0.918	1.173	1.162	0.010	193.788

MSE, mean squared error; sd, standard deviation; ASP-U, asympirical method using uniform sampling; ASP-A, asympirical method using asymptotic sampling; GAM-GCV, generalized cross-validation for generalized additive models; BAM, fast restricted maximum likelihood for generalized additive models; SKIP, skip method.