Analytic regularity and stochastic collocation of high-dimensional Newton iterates

Abstract

In this paper we introduce concepts from uncertainty quantification (UQ) and numerical analysis for the efficient evaluation of stochastic high dimensional Newton iterates. In particular, we develop complex analytic regularity theory of the solution with respect to the random variables. This justifies the application of sparse grids for the computation of statistical measures. Convergence rates are derived and are shown to be subexponential or algebraic with respect to the number of realizations of random perturbations. Due the accuracy of the method, sparse grids are well suited for computing low probability events with high confidence. We apply our method to the power flow problem. Numerical experiments on the non-trivial 39 bus New England power system model with large stochastic loads are consistent with the theoretical convergence rates. Moreover, compared to the Monte Carlo method our approach is at least 10¹¹ times faster for the same accuracy.

1. Introduction

Newton iteration is a powerful method for solving many scientific and engineering problems, naturally arising in the context of power flow problems (Non-linear networks) [8], non-linear Partial Differential Equations [16], among others.

With the advent of massive computational resources, complex mathematical models are widely used for prediction in many scientific and engineering areas such as finance, weather forecasting, seismology, and semiconductor design. Due to the complex nature of these problems uncertainty naturally arises that can affect the reliability of these forecasts. Mathematically rigorous Uncertainty Quantification (UQ) has become an instrumental approach to judge the reliability of such predictions.

Uncertainty quantification is a mathematical approach that allows the charactizations of uncertainty through out the computational model and for a given Quantity of Interest (QoI). Even with present day computational resources, the application of UQ to large and complex non-linear networks such as electric power grids is a formidable undertaking. In particular, due the highly computational challenges. In this paper we seek to develop a UQ process that is mathematically rigorous, computationally efficient and easy to adapted with currently available power flow solvers [21, 32].

One of the most widely used UQ techniques is the Monte Carlo method [11], which is robust and easy to implement. Indeed, a deep analysis or understanding of the underlying stochastic model is not required, making this an attractive approach for the practicing engineer and scientist. However, convergence rates for iterative approximation methods can be very slow. For the application of UQ to the power flow problem, due to the large numbers of generators, loads and transmission lines one potentially faces, the problem can be high dimensional, non-linear, non-Gaussian, and not feasible with current computational resources. An alternative approach is the use of tensor product methods. However, these methods suffer significantly from the curse of dimensionality, thus making them unattractive even for moderate dimensionalities.

If the regularity of a QoI is relatively high with respect to the fundamental random variables, then application of stochastic collocation with Smolyak sparse grids [24, 25, 29] is a good choice. Indeed, this method has become popular in the field of computational applied mathematics and engineering as a surrogate model of stochastic Partial Differential Equations (sPDEs) [9] where the QoI is composed of moderately large numbers of random variables. The method is easy to implement and non-intrusive, i.e., each collocation point corresponds to uncoupled deterministic problems. The stochastic collocation method can be used with non-linear dependence of the QoI on the random variables. However, such grids still suffer from the curse of dimensionality. Alternative adaptive techniques have been developed, including anisotropic sparse grids [24], dimension adaptive quadrature [12] and quasi-optimal sparse grids [7, 26]. Yet these methods are still not feasible for very high dimensional problems and/or low regularity of the QoI. In addition, although quasi-optimal sparse grids lead to exponential convergence rates with respect to numbers of realizations, there is to our knowledge still no systematic way to construct them.

We have particular interest in the application of the Newton iteration to the solution of the power flow equations of electric grids [8]. In practice many of the generators (wind, solar, etc) and loads are stochastic in nature, and thus a traditional deterministic power flow analysis is insufficient. UQ applied to mathematical/statistical modeling of electrical power grids is still in its infancy. A major 2016 National Academy of Sciences report underscores the importance of this new area of research in its potential to contribute to this and the next generation of electric power grids [22]. The incorporation of uncertainty in the grid has gathered interest in the power system community.

In [15] Hockenberry et al. proposed the probabilistic collocation method (PCM) with applications to power systems. Although the results of this work are good, the uncertainty is computed only with respect to a single parameter. In [27] the authors test a polynomial chaos collocation and Galerkin approach to study the uncertainty of power flow on a small 2-bus power system. The results are good for low stochastic dimensions, but it is not clear how this approach will scale for large electric power grids, with higher associated dimensions. Moreover, there is little mathematical theory on the effectiveness of this approach.

More recently, in [17], Tang et al. proposed a dimension-adaptive sparse grid method [12] by using the off-the-shelf Matlab Sparse Grid Toolbox [18, 19]. The numerical results show the feasibility of this approach. However, there are also some weaknesses. The authors did not analyze the regularity of the power flow with respect to the uncertainty, so that the rate of convergence of the sparse grid is not known. Reduced regularity of the stochastic power flow can choke the accuracy. In contrast to Monte Carlo methods, which are robust, sparse grid methods are sensitive to the regularity of the function at hand. Lack of regularity will lead to erroneous results.

This motivates the application of numerical analysis and UQ theory to the Newton iteration for such problems as the electric power grid. Many of the ideas of this paper originate from the numerical solution of stochastic PDEs [9, 24, 25], where UQ is having a large impact. The goals of our paper is to integrate these methods with the theory and practice of power systems, where we believe that UQ will eventually have a strong impact. Furthermore, from the numerical analysis perspective regularity can be determined by complex analytic extensions of the functions of interest [3, 9, 31]. These lead to sharper convergence rates than regularity in terms of derivatives.

In Section 2 the mathematical background for this paper is introduced. In particular, the Newton-Kantorovich Theorem, sparse grids and convergence rates are discussed. Furthermore, complex analysis from the numerical analysis perspective for polynomial approximation is also treated. In Section 3 the complex analytic regularity of the solution of the Newton iteration is developed with respect to the random perturbations. In Section 4 the theory developed in Section 3 justifies the application of sparse grids to the power flow equations. The sparse grids are applied to the power flow equations of the 39 bus, 10 Generator, New England model. Subexponential or algebraic convergence rates are obtained that are consistent with the sparse grid convergence rates. These convergence rates make the sparse grid method suitable for computing stochastic moments to high accuracy. Furthermore, they will also be well suited for computing small event tail probabilities with high accuracy.

2. Mathematical background

In this section we introduce the general notation and mathematical background that will be used in this paper. We provide a summary of the three important topics: i) Newton-Kantorovich Theorem, ii) Stochastic spaces and iii) Sparse grids (approximation theory).

2.1. Newton-Kantorovich Theorem

Consider the Fréchet differentiable operator f : X → Y that maps a convex open set D of a Banach space X into a Banach space Y . Suppose that we are interested in finding an element x ∈ D such that

$$f(x)=0.$$ (1)

Assuming that a solution of equation (1) exists, a series of successive approximations x^v ∈ D, where $v\in {\mathbb{N}}_0≔\mathbb{N}\cup 0$, can be built. Consider the space of bounded linear operators L(Y,X) from Y into X. Let J(x^v) be the Fréchet derivative of f(x^v). Suppose that J(x^v)⁻¹ ∈ L(Y,X) and consider the sequence

$$x^{v+1}=x^v-J{\left(x^v\right)}^{-1}f\left(x^v\right).$$

Assumption 1

We assume that that J satisfies the Lipschitz condition

$$\Vert J(x)-J(y)\Vert \le \lambda \Vert x-y\Vert$$ (2)

for some constant λ ≥ 0, and all x, y ∈ D. Furthermore, assume that x⁰ ∈ D and there exist positive constants ϰ and δ such that

$$\Vert J{\left(x^0\right)}^{-1}\Vert \le \varkappa ,$$

$$\Vert J{\left(x^0\right)}^{-1}f\left(x^0\right)\Vert \le \delta ,$$ (3)

$$h≔2\varkappa \lambda \delta \le 1,$$

U(x⁰, t*) ⊂ D, with U(x, r) the open ball {y : ‖y − x‖ ≤ r} and $t*=\frac{2}{h}\left(1-\sqrt{1-h}\right)\delta$.

If Assumption 1 above is satisfied, then by the Newton-Kantorovich theorem [2], for all $v\in {\mathbb{N}}_0$,

1. The Newton iterates x^v+1 = x^v − J(x^v)⁻¹f(x^v) and J(x^v)⁻¹ exist.

2. x^v ∈ U(x₀, t*) ⊂ D.

3. x* = lim_v→∞x^v exists, ${\text{x}}^{*}\in \overline{U\left({\text{x}}^0,t^{*}\right)}$, and f(x*) = 0 uniquely.

2.2. Stochastic spaces

Let Ω be the set of outcomes from the complete probability space (Ω, $\mathcal{F}$, $\mathbb{P}$), where $\mathcal{F}$ is a sigma algebra of events and $\mathbb{P}$ is a probability measure. Define $L_{\mathbb{P}}^q(\Omega )$, q ∈ [1, ∞], as the Banach spaces

$$L_{\mathbb{P}}^q(\Omega )≔\left\{u:\Omega \to ℝ|{\int }_{\Omega }{|u\left(\omega \right)|}^{q}d\mathbb{P}\left(\omega \right)<\infty \right\}\text{ and}$$

$$L_{\mathbb{P}}^{\infty }(\Omega )≔\left\{u:\Omega \to ℝ|\mathbb{P}\underset{\omega \in \Omega }{\text{-ess sup}}|u\left(\omega \right)|<\infty \right\}.$$

Let W := [W₁,…,W_N] be an N-component random vector measurable in (Ω, $\mathcal{F}$, $\mathbb{P}$) that takes values on $\Gamma ≔{\Gamma }_1\times \cdots \times {\Gamma }_N\subset {ℝ}^N$, with Γ_n := [−1, 1]. Let $\mathcal{B}(\Gamma )$ be the Borel σ-algebra. Suppose that a induced measure μ_W on (Γ, $\mathcal{B}(\Gamma )$) is defined as ${\mu }_W(A)≔\mathbb{P}\left(W^{-1}(A)\right)$ for all $A\in \mathcal{B}(\Gamma )$. Given that the induced measure is absolutely continuous with respect to Lebesgue measure on Γ, then there exists a density function ρ(q) : Γ → [0, +∞) such that

$$\mathbb{P}(W\in A)≔\mathbb{P}\left(W^{-1}(A)\right)={\int }_A\rho (q)\text{d}q,$$

for any event $A\in \mathcal{B}(\Gamma )$. Furthermore for any measurable function $W\in {\left[L_{\mathbb{P}}^1(\Gamma )\right]}^N$ define the expected value as

$$\mathbb{E}[W]={\int }_{\Gamma }q\rho (q)\text{d}q.$$

Define also the Banach spaces (for q ≥ 1)

$$L_{\rho }^q(\Gamma )≔\left\{u:\Gamma \to ℝ|{\int }_{\Omega }{|u\left(q\right)|}^q\rho \left(q\right)\text{d}q<\infty \right\}\text{ and}$$

$$L_{\rho }^{\infty }(\Gamma )≔\left\{u:\Gamma \to ℝ|\rho \underset{q\in \Gamma }{\text{-ess sup}}|u\left(q\right)|<\infty \right\}.$$

Note that the above ρ-essential supremium is with respect to the measure induced by the density function ρ(·), rather than Lebesgue measure itself.

In general the density ρ(·) will not factorize into independent probability density functions, making higher dimensional manipulations difficult in some cases. In [3] the authors recommend use of an auxiliary probability density function $\widehat{\rho }:\Gamma \to {ℝ}^{+}$ that factorizes into N independent ones, i.e.,

$$\widehat{\rho }(q)=\prod _{n=1}^N\widehat{\rho }\left(q_n\right),q=\left(q_1,\dots ,q_N\right)\in \Gamma ,$$ (4)

where we will assume that ${\Vert \frac{\rho (q)}{\widehat{\rho }(q)}\Vert }_{L^{\infty }(\Gamma )}<\infty$. Note that in contrast L^∞(Γ) (without a subscript) is with respect to the supremium in the Lebesgue measure.

2.3. Sparse grids

Our objective is to efficiently approximate a function $u:\Gamma \to ℝ$ defined on high dimensional domains using global polynomials. The accuracy of the approach will directly depend on the regularity of the function. Let ${\mathcal{P}}_p(\Gamma )\subset L^2\left(\Gamma \right)$ be the span of tensor product polynomials of degree at most p = (p₁, …,p_N); i.e., ${\mathcal{P}}_p(\Gamma )={\otimes }_{n=1}^N{\mathcal{P}}_{p_n}\left({\Gamma }_n\right)$ with ${\mathcal{P}}_{p_n}\left({\Gamma }_n\right)≔\text{span}\left(q_n^m,m=0,\dots ,p_n\right)$, n = 1,…,N. For univariate polynomial approximation, we define a sequence of levels i = 0, 1, 2, 3,… corresponding to increasing degrees $m(i)\in {\mathbb{N}}_0$ polynomial approximation for given coordinates. Here for a given approximation scheme, m(·) is a fixed function. In general our multivariate approximation scheme will assume different levels of approximation i_n for different coordinates n = 1,…,N.

We consider separate univariate Lagrange interpolants in Γ along each dimension n, given as ${\mathcal{I}}_n^{m\left(i_n\right)}:C^0\left({\Gamma }_n\right)\to {\mathcal{P}}_{m\left(i_n\right)-1}\left({\Gamma }_n\right)$. Specifically, let

$${\mathcal{I}}_n^{m\left(i_n\right)}\left(u\left(q_n\right)\right)≔\sum _{j_n=1}^{m\left(i_n\right)}u\left(q_{j_n}^i\right)l_{n,j_n}\left(q_n\right),$$ (5)

where ${\left\{l_{n,j}\right\}}_{j=1}^{m\left(i_n\right)}$ is a Lagrange basis for the space ${\mathcal{P}}_{p_n}\left({\Gamma }_n\right)$, the set ${\left\{q_{j_n}^i\right\}}_{j_n=1}^{m\left(i_n\right)}$, represents m(i_n) discrete locations (the interpolation knots) in Γ_n, the index i_n ≥ 0 is the level of approximation, and $m\left(i_n\right)\in {\mathbb{N}}_{+}$ is the number of collocation nodes at level $i_n\in {\mathbb{N}}_{+}$ where m(0) = 0, m(1) = 1 and m(i_n) ≤ m(i_n +1) if i_n ≥ 1.

Remark 1 Since m(i_n) represents the number of interpolation knots for the Lagrange basis, we have p_n = m(i_n) + 1.

One of the most common approaches to constructing Lagrange interpolants in high dimensions is the formation of tensor products of ${\mathcal{I}}_n^{m\left(i_n\right)}$ along each dimension n. However, as N increases the dimension of ${\mathcal{P}}_p$ increases as ${\prod }_{n=1}^N\left(p_n+1\right)$. Thus even for moderate dimensions N the computational cost of a Lagrange approximation becomes prohibitive. However, in the case of sufficient complex analytic regularity of the function u with respect to the random variables defined on Γ, a better choice is the application of Smolyak sparse grids. In the rest of this section the construction of the classical Smolyak sparse grid (see e.g. [6, 29]) is summarized. More details can be found in [4].

Consider the difference operator along the n^th dimension given by

$${\Delta }_n^{m\left(i_n\right)}≔{\mathcal{I}}_n^{m\left(i_n\right)}-{\mathcal{I}}_n^{m\left(i_n-1\right)}.$$ (6)

Given an integer w ≥ 0, called the approximation level, and a multi-index $i=\left(i_1,\dots ,i_N\right)\in {\mathbb{N}}_0^N$, let $g:{\mathbb{N}}_0^N\to \mathbb{N}$ be a strictly increasing function in each argument and define a sparse grid approximation of function u(q) ∈ C⁰(Γ), restricted in order by g:

$${\mathcal{S}}_w^{m,g}[u(q)]=\sum _{i\in {\mathbb{N}}_0^N:g(i)\le w}\underset{n-1}{\overset{N}{\otimes }}\left({\Delta }_n^{m\left(i_n\right)}\right)(u(q)).$$ (7)

We observe that the sparse grid approximation is constructed from a linear combination of polynomial tensor product interpolations. However, the size of the polynomial space is controlled by g(i) ≤ w in (7).

Define the following N − tuple m(i) := (m(i₁),…,m(i_N)) and consider the set of polynomial multi-degrees

$${\Lambda }^{m,g}(w)=\left\{p\in {\mathbb{N}}^N,g\left(m^{-1}(p+1)\right)\le w\right\},$$

where 1 is an N dimensional vector of ones, and the associated multivariate polynomial space

$${\mathbb{P}}_{{\Lambda }^{m,g}(w)}(\Gamma )=\text{span}\left\{\prod _{n=1}^N{q}_n^{p_n},\text{ with }p\in {\Lambda }^{m,g}(w)\right\}.$$

For a Banach space V let

$$C^0(\Gamma ;V)≔\left\{u:\Gamma \to V\text{ is continuous on }\Gamma \text{ and }\underset{y\in \Gamma }{\text{max}}\Vert u(y){\Vert }_V<\infty \right\}.$$

It can be shown that the approximation formula given by ${\mathcal{S}}_w^{m,g}$ is exact in ${\mathbb{P}}_{{\Lambda }^{m,g}(w)}(\Gamma )$. We state the following proposition that is proved in [4].

Proposition 1

1. For any u ∈ C⁰(Γ; V ), we have ${\mathcal{S}}_w^{m,g}[u]\in {\mathbb{P}}_{{\Lambda }^{m,g}(w)}\otimes V$.

2. Moreover, ${\mathcal{S}}_w^{m,g}[u]=u\forall u\in {\mathbb{P}}_{{\Lambda }^{m,g}(w)}\otimes V$.

Remark 2 The tensor product space ${\mathbb{P}}_{{\Lambda }^{m,g}(w)}\otimes V$ is more easily understood as the space of polynomials with Banach-valued coefficients. Furthermore, ${\mathcal{S}}_w^{m,g}[u]\in {\mathbb{P}}_{{\Lambda }^{m,g}(w)}\otimes V$ is interpreted as a sparse grid approximation of a V -valued continuous function.

A good choice of m and g is given by the Smolyak sparse grid definitions (see [6, 29])

$$m\left(i_n\right)=\{\begin{array}{ll}1\hfill & \text{for }{i}_n=1\hfill \\ 2^{i_n-1}+1\hfill & \text{for }{i}_n>1\hfill \end{array}\text{\hspace{1em} and \hspace{1em}}g(i)=\sum _{n=1}^N\left(i_n-1\right).$$

Furthermore ${\Lambda }^{m,g}(w)≔\left\{p\in {\mathbb{N}}^N:{\sum }_{n}f\left(p_n\right)\le w\right\}$ where

$$f\left(p_n\right)=\{\begin{array}{l}0,p_n=0\hfill \\ 1,p_n=1\hfill \\ \lceil {\text{log}}_2\left(p_n\right)\rceil ,p_n\ge 2\hfill \end{array}.$$

Other common choices are shown in Table 1.

Table 1.

Sparse grid approximations formulas for TD and HC.

Approx. space	sparse grid: m, g	polynomial space: Λ(w)
Total Degree (TD)	m(in) = in g(i) = ∑n(in − 1) ≤ w	$\left\{p\in {\mathbb{N}}_0^N:{\sum }_n{p}_n\le w\right\}$
Hyperbolic Cross (HC)	m(i) = i g(i) = ∏n(in) ≤ w + 1	$\left\{p\in {\mathbb{N}}_0^N:{\Pi }_n\left(p_n+1\right)\le w+1\right\}$

With this choice of (m, g) and the application of Clenshaw-Curtis abscissas (which are locations of interpolation points given as extrema of Chebyshev polynomials) leads to nested sequences of one dimensional interpolation formulae. In consequence, the sparse grid formed by this choice is highly compressed in comparison to the full tensor product grid. Another good choice includes Gaussian abscissas [23]. For any choice of m(i_n) > 1 the Clenshaw-Curtis abscissas are given by

$$q_{j_n}^{i_n}=-\text{cos}\left(\frac{\pi \left(j_n-1\right)}{m\left(i_n\right)-1}\right),j_n=1,\dots ,m\left(i_n\right).$$

In Figure 1 an example of Clenshaw Curtis and Gaussian abscissas are shown for w = 5.

Fig. 1.

Clenshaw-Curtis (left) and Gaussian abscissas (right) for w = 5 levels.

As previously pointed out, the probability density function ρ does not necessarily factorize in higher dimensions. As an alternative we use the auxiliary distribution $\widehat{\rho }$, which factorizes as $\widehat{\rho }(q)={\prod }_{n=1}^N{\widehat{\rho }}_n\left(q_n\right)$ and is close to the original distribution ρ(q). Suppose k is a given global index determined by the set of indices k₁ …,k_N as k = k₁+p₁(k₂−1)+p₁p₂(k₃−1)+p₁p₂p₃(k₄−1)+…. Given a function u : Γ → V , the quadrature scheme ${\mathbb{E}}_{\widehat{\rho }}^p[u]$ that approximates the integral $\mathbb{E}[u(q)]≔{\int }_{\Gamma }u(q)\widehat{\rho }(q)dq$ can now be computed based on the distribution $\widehat{\rho }(q)$ as

$${\mathbb{E}}_{\widehat{\rho }}^p[u]=\sum _{k=1}^{N_P}{\omega }_{k}u\left(q^{(k)}\right),{\omega }_k=\prod _{n=1}^N{\omega }_{k_n}{\omega }_{k_n}={\int }_{{\Gamma }_n}{l}_{n,k_n}^2\left(q_n\right){\widehat{\rho }}_n\left(q_n\right)\text{d}{q}_n,$$

with $q^{(k)}\in {ℝ}^n$ the locations of the quadrature knots, and $N_p\in \mathbb{N}$ the number of Gauss quadrature points controlling the accuracy of the quadrature scheme. Recall that $l_{n,k_n}^2\left(q_n\right)$ define Lagrange polynomials defined in equation (5). The term $\mathbb{E}[u(q)]$ can be approximated as

$$\mathbb{E}\left[{\mathcal{S}}_w^{m,g}[u(q)]\right]\approx {\mathbb{E}}_{\widehat{\rho }}^p\left[{\mathcal{S}}_w^{m,g}[u(q)]\frac{\rho }{\widehat{\rho }}\right],$$

and similarly the variance var[u(q)] is approximated as

$$\text{var}[u(q)]\approx \mathbb{E}\left[{\left({\mathcal{S}}_w^{m,g}[u(q)]\right)}^2\right]-\mathbb{E}{\left[{\mathcal{S}}_w^{m,g}[u(q)]\right]}^2\text{***}\approx {\mathbb{E}}_{\widehat{\rho }}^p\left[{\left({\mathcal{S}}_w^{m,g}[u(q)]\right)}^2\frac{\rho }{\widehat{\rho }}\right]-{\mathbb{E}}_{\widehat{\rho }}^p{\left[{\mathcal{S}}_w^{m,g}[u(q)]\frac{\rho }{\widehat{\rho }}\right]}^2.$$

Remark 3 The weights ${\omega }_{k_n}$ and node locations q^(k) are computed from the auxiliary density $\widehat{\rho }$. For standard distributions of $\widehat{\rho }$ such as uniform and Gaussian, these are already tabulated to full accuracy. Otherwise they must be computed by solving for roots of orthogonal polynomials and using a quadrature scheme. However, the integrals involved are only one dimensional. See [3] (Section 2) for details.

We now develop some rigorous numerical bounds for the accuracy of the sparse grid approximation. Let $C_{\text{mix}}^k(\Gamma ;ℝ)$ denote the space of functions with continuous mixed derivatives up to degree k:

$$C_{\text{mix}}^k(\Gamma ;ℝ)=\left\{u:\Gamma \to ℝ:\frac{{\partial }^{{\alpha }_1,\dots ,{\alpha }_N}u}{{\partial }^{{\alpha }_1}{q}_1\dots {\partial }^{{\alpha }_N}{q}_N}\in C^0(\Gamma ;ℝ),n=1,\dots ,N,{\alpha }_n\le k\right\}$$

and equipped with the following norm:

$$\Vert u{\Vert }_{C_{\text{mix}}^k(\Gamma ;ℝ)}=\left\{u:\Gamma \to \underset{q\in \Gamma }{\text{max}}|\frac{{\partial }^{{\alpha }_1,\dots ,{\alpha }_N}u(q)}{{\partial }^{{\alpha }_1}{q}_1\dots {\partial }^{{\alpha }_N}{q}_N}|<\infty \right\}.$$

Assume that $u\in C_{\text{mix}}^k(\Gamma ;ℝ)$. In [6] the authors show that it must follow that

$${\Vert u-{\mathcal{S}}_w^{m,g}[u]\Vert }_{L^{\infty }(\Gamma )}\le C(k,N)\Vert u{\Vert }_{C_{\text{mix}}^k(\Gamma )}{\eta }^{-k}{(\text{log}\eta )}^{(k+2)(N-1)+1},$$

where η is the number of knots of the sparse grid ${\mathcal{S}}_w^{m,g}$. However, the coefficient C(k,N) is in general not known [6].

If the function u admits a complex analytic extension, a better approach for deriving error bounds for the polynomial approximation arises from exploitation of analysis in the complex plane. In [25] the authors derive $L_{\rho }^{\infty }(\Gamma )$ bounds based on analytic extensions of u on a well defined region $\Psi \subset {ℂ}^N$ with respect to the variables q. These bounds are explicit, and the coefficients can be estimated and depend on the size of the region Ψ.

In [24, 25] error estimates are derived for the isotropic and anisotropic Smolyak sparse grids with the choice of Clenshaw-Curtis and Gaussian abscissas. Let η be the number of collocation knots and ${\mathcal{E}}_{{\widehat{\sigma }}_1,\dots ,{\widehat{\sigma }}_N}≔{\Pi }_{n=1}^N{\mathcal{E}}_{n,{\widehat{\sigma }}_n}\subset \Psi \subset {ℂ}^N$, where

$${\mathcal{E}}_{n,{\widehat{\sigma }}_n}=\left\{z\in ℂ\text{ with Re }z=\frac{e^{{\delta }_n}+e^{-{\delta }_n}}{2}\text{cos}(\theta ),\text{ Im }z=\frac{e^{{\delta }_n}-e^{-{\delta }_n}}{2}\text{sin}(\theta ):\theta \in [0,2\pi ),{\widehat{\sigma }}_n\ge {\delta }_n\ge 0\right\}$$

and ${\widehat{\sigma }}_n>0$ (see the Bernstein ellipse in Figure 2). The authors show that the error ${\Vert u-{\mathcal{S}}_w^{m,g}[u]\Vert }_{L_{\rho }^{\infty }(\Gamma )}$ exhibits algebraic or sub-exponential convergence with respect to η (see Theorems 3.10, 3.11, 3.18 and 3.19 in [25] for more details) whenever $u\in C^0(\Gamma ,ℝ)$ admits an analytic extension on the polyellipse ${\mathcal{E}}_{{\widehat{\sigma }}_1,\dots ,{\widehat{\sigma }}_N}$.

Fig. 2.

Bernstein ellipse along the n^th dimension. The ellipse crosses the real axis at $\frac{e^{{\widehat{\sigma }}_n}+e^{-{\widehat{\sigma }}_n}}{2}$ and the imaginary axis at $\frac{e^{{\widehat{\sigma }}_n}-e^{-{\widehat{\sigma }}_n}}{2}$.

We now recall the definition of Chebyshev polynomials, useful in deriving error estimates on sparse grids. Let $T_k:{\Gamma }_1\to ℝ$, k = 0, 1,…, be a k^th order Chebyshev polynomial over [−1, 1]. These polynomials are defined recursively as:

$$T_0(y)=1,T_1(y)=y,\dots ,T_{k+1}(y)=2y{T}_k(y)-T_{k-1}(y),\dots .$$

The following theorem characterizes approximation of analytic functions using Chebyshev polynomials.

Theorem 1

Let u be analytic and absolutely bounded by M on ${\mathcal{E}}_{\text{log}\zeta }$, $\zeta >1$. Then the expansion

$$u(y)={\alpha }_0+2\sum _{k=1}^{\infty }{\alpha }_k{T}_k(y),$$

holds for all $y\in {\mathcal{E}}_{\text{log}\zeta }$ where

$${\alpha }_k=\frac{1}{\pi }{\int }_{-1}^1\frac{u(y)T_k(y)}{1-y^2}dy$$

and, additionally, |α_k| ≤ M/ζ^k. Furthermore if y ∈ [−1,1] then

$$|u(y)-{\alpha }_0-2\sum _{k=1}^m{\alpha }_k{T}_k(y)|\le \frac{2M}{\zeta -1}{\zeta }^{-m}.$$

Proof See Theorem 8.2 in [31] □

We follow the arguments in [3, 23] and using the fact that the interpolation operator ${\mathcal{I}}_n^{m\left(i_n\right)}$ is exact on the space ${\mathcal{P}}_{p_n-1}$, i.e. for any $v\in {\mathcal{P}}_{p_n-1}$ we have that ${\mathcal{I}}_n^{m\left(i_n\right)}(v)=v$, it can be shown that if u is continuous on [−1,1] and has an analytic extension on ${\mathcal{E}}_{{\sigma }_n}$ we have, from Theorem 1,

$${\Vert \left(I-{\mathcal{I}}_n^{m\left(i_n\right)}\right)u\Vert }_{L_{\rho }^{\infty }\left({\Gamma }_n\right)}\le \left(1+{\Lambda }_{m(i)}\right)\underset{v\in {\mathcal{P}}_{m\left(i_n\right)-1}}{\text{min}}\Vert u-v{\Vert }_{C^0(\Gamma ,ℝ)}\text{****}\le \left(1+{\Lambda }_{m(i)}\right)\frac{2M(u)}{e^{{\sigma }_n}-1}{e}^{-{\sigma }_{n}m\left(i_n\right)},$$

where ${\Lambda }_{m\left(i_n\right)}$ is the Lebesgue constant and is bounded by 2π⁻¹(log(m − 1)+1) (see [3]) and M = M(u) is the maximal value of u on ${\mathcal{E}}_{{\sigma }_n}$. Thus, for n = 1,…,N we have

$${\Vert \left(I-{\mathcal{I}}_n^{m(i)}\right)u\Vert }_{L_{\rho }^{\infty }\left({\Gamma }_n\right)}\le M(u)C\left({\widehat{\sigma }}_n\right)i_n{e}^{-{\sigma }_n{2}^{i_n}},$$ (8)

where ${\sigma }_n=\frac{{\widehat{\sigma }}_n}{2}>0$ and $C\left({\sigma }_n\right)≔\frac{2}{\left(e^{{\sigma }_n}-1\right)}$. Recalling the definition of ${\Delta }_n^{m\left(i_n\right)}$ from equation (6), we have that for all n = 1,…,N

$${\Vert \Delta {(u)}^{m\left(i_n\right)}\Vert }_{L_{\rho }^{\infty }\left({\Gamma }_n\right)}={\Vert \left({\mathcal{I}}_n^{m\left(i_n\right)}-{\mathcal{I}}_n^{m\left(i_n-1\right)}\right)u\Vert }_{L_{\rho }^{\infty }\left({\Gamma }_n\right)}\le {\Vert \left(I-{\mathcal{I}}_n^{m\left(i_n\right)}\right)u\Vert }_{L_{\rho }^{\infty }\left({\Gamma }_n\right)}+{\Vert \left(I-{\mathcal{I}}_n^{m\left(i_n-1\right)}\right)u\Vert }_{L_{\rho }^{\infty }\left({\Gamma }_n\right)}\le 2M(u)C\left({\sigma }_n\right)i_n{e}^{-{\sigma }_n{2}^{i_n-1}}.$$ (9)

By applying equation (9) to Lemma 3.5 in [25], we are now in a position to slightly modify Theorems 3.10 and 3.11 in [25] and restate them into a single theorem given below. However, the following assumptions and definitions are first needed:

• We set $\widehat{\sigma }\equiv {\text{min}}_{n=1,\dots ,N}{\widehat{\sigma }}_n$, i.e. for an isotropic sparse grid the general sub-exponential decay will be restricted by the smallest $\widehat{\sigma }$.

• Let

  $$\tilde{M}(u)=\underset{g\in {\mathcal{E}}_{{\widehat{\sigma }}_1,\dots ,{\widehat{\sigma }}_N}}{\text{sup}}|u(g)|,$$

  $$\sigma =\widehat{\sigma }/2,{\mu }_1=\frac{\sigma }{1+\text{log}(2N)},\text{ and }{\mu }_2(N)=\frac{\text{log}(2)}{N(1+\text{log}(2N))},$$

  $$a(\delta ,\sigma )≔\text{exp}\left(\delta \sigma \left\{\frac{1}{\sigma {\text{ log}}^2(2)}+\frac{1}{\text{log}(2)\sqrt{2\sigma }}+2\left(1+\frac{1}{\text{log}(2)}\sqrt{\frac{\pi }{2\sigma }}\right)\right\}\right),$$

  $${\tilde{C}}_2(\sigma )=1+\frac{1}{\text{log}2}\sqrt{\frac{\pi }{2\sigma }},{\delta }^{*}(\sigma )=\frac{e\text{ log}(2)-1}{{\tilde{C}}_2(\sigma )},$$

  $$C_1(\sigma ,\delta ,\tilde{M}(u))=\frac{4\tilde{M}(u)C(\sigma )a(\delta ,\sigma )}{e\delta \sigma },$$

  $${\mu }_3=\frac{\sigma {\delta }^{*}{\tilde{C}}_2(\sigma )}{1+2\text{ log}(2N)},\text{ and }$$

  $$\mathcal{Q}\left(\sigma ,{\delta }^{*}(\sigma ),N,\tilde{M}(u)\right)=\frac{C_1\left(\sigma ,{\delta }^{*}(\sigma ),\tilde{M}(u)\right)}{\text{exp}\left(\sigma {\delta }^{*}(\sigma ){\tilde{C}}_2(\sigma )\right)}\frac{\text{max}{\left\{1,C_1\left(\sigma ,{\delta }^{*}(\sigma ),\tilde{M}(u)\right)\right\}}^N}{|1-C_1\left(\sigma ,{\delta }^{*}(\sigma ),\tilde{M}(u)\right)|}.$$

Theorem 2

Suppose that $u\in C^0(\Gamma ;ℝ)$ has an analytic extension on ${\mathcal{E}}_{{\widehat{\sigma }}_1,\dots ,{\widehat{\sigma }}_N}$ and is absolutely bounded by $\tilde{M}(u)$. If w > N / log 2 and a sparse grid with Clenshaw-Curtis abscissas is used, then the following bound is valid:

$${\Vert u-{\mathcal{S}}_w^{m,g}u\Vert }_{L_{\rho }^{\infty }(\Gamma )}\le \mathcal{Q}\left(\sigma ,{\delta }^{*}(\sigma ),N,\tilde{M}(u)\right){\eta }^{{\mu }_3\left(\sigma ,{\delta }^{*}(\sigma ),N\right)}\text{ exp}\left(-\frac{N\sigma }{2^{1/N}}{\eta }^{{\mu }_2(N)}\right),$$ (10)

Furthermore, if w ≤ N/log 2 then the following algebraic convergence bound holds:

$${\Vert u-{\mathcal{S}}_w^{m,g}u\Vert }_{L_{\rho }^{\infty }(\Gamma )}\le \frac{C_1\left(\sigma ,{\delta }^{*}(\sigma ),\tilde{M}(u)\right)\text{max}{\left\{1,C_1\left(\sigma ,{\delta }^{*}(\sigma ),\tilde{M}(u)\right)\right\}}^N}{|1-C_1\left(\sigma ,{\delta }^{*}(\sigma ),\tilde{M}(u)\right)|}{\eta }^{-{\mu }_1}.$$ (11)

Proof This is proved by applying the inequality of equation (9) to the proof of Theorems 3.10 and 3.11 in [25]. □

In many practical cases not all dimensions of Γ are equally important. In these cases the dimensionality of the sparse grid can be significantly reduced by means of anisotropic sparse grids. It is not hard to construct associated anisotropic sparse approximation formulae by having the restriction function g depend on the input random variables q_n.

2.4. Analyticity

Throughout this paper we will apply several important complex analysis results. Suppose that $f:U\to ℂ$ is a complex valued function defined on the open set U. Rewrite f into real and complex parts as f(z) := u(z) + iv(z), where z = x + iy. The Cauchy-Riemann equations are then stated as:

$$\frac{\partial u}{\partial x}=\frac{\partial v}{\partial y}\frac{\partial u}{\partial y}=-\frac{\partial v}{\partial x}$$ (12)

From the Cauchy-Riemann criterion (Chap1, p3 in [14]) if f satisfies equation (12) and is continuously differentiable on the real axis then f is analytic.

We state Hartog’s theorem (Chap1, p32 in [20]). This theorem allows us to determine analyticity for multivariate functions a series of single dimensional analytic functions.

Theorem 3 (Hartog’s theorem)

Let $U\subseteq {ℂ}^N$ be an open set and $f:U\to ℂ$. Suppose that for each j = 1,…,N and each fixed z₁,…,z_j−1,z_j+1,…,z_N the function ψ ↦ f(z₁,…,z_j−1,ψ,z_j+1,…,z_N) is holomorphic, in the classical one variable sense, on the set $U\left(z_1,\dots ,z_{j-1},z_{j+1},\dots ,z_N\right)\equiv \left\{\psi \in ℂ:\left(z_1,\dots ,z_{j-1},\psi ,z_{j+1},\dots ,z_N\right)\in U\right\}$. Then f is continuous on U.

From Osgood’s lemma continuity of the function f on U implies analyticity (Chap1, p2 in [14]).

3. Analyticity of the Newton iteration

It is profitable here for purposes of clarification to consider the Newton iteration in a general function space context. Specifically, let X and Y (see section 2) be Banach spaces. Consider the following problem: Find x ∈ D ⊂ X such that

$$f(x,q)=0,$$ (13)

where q ∈ Γ and f : D × Γ → Y . Equation (13) is then solved using the Newton iteration under the conditions of the Newton-Kantorovich Theorem (see section 2).

The convergence rate of the Newton iterates based on a sparse grid approximation as a function of grid size is directly affected by the regularity properties with respect to parameters q ∈ Γ. Regularity is characterized in terms of an analytic extension in ${ℂ}^N$ of the iterates.

In the sequel we will treat q ∈ Γ as a random parameter, which will be suppressed occasionally. Thus for example, below we will write f ≡ f_q, J ≡ J_q, M_v ≡ M_v,q, etc., and we can write f(x, q) ≡ f_q(x). For any q ∈ Γ (which we fix for now) consider the Newton sequence

$$x^v=M_v\left(x^{v-1}\right)\equiv x^{v-1}-J{\left(x^{v-1}\right)}^{-1}f\left(x^{v-1}\right).$$ (14)

where J : X → Y is the Fréchet derivative of f : D → Y . Assume that x₀ ∈ D₀ ⊂ D and for all $v\in \mathbb{N}$ let D_v ⊂ D be the successive images under the map M_v, so that M_v(D_v−1) = D_v. Note at this point the random parameter q ∈ Γ is unchanging throughout the iteration; the iterated domains D_v however depend on q.

Suppose that the parameter q is now extended to a complex parameter g with g ∈ Ψ ⊃ Γ, where $\Psi \subset {ℂ}^N$. We can now form a complex extension of the sequence (14) as follows. We will complexify the pair (x, q) into a pair of complex variables (z, g), with z the complexification of x. Assume that for f(x⁰) ≡ f_q(x⁰) : D₀ → E₀ there exists an analytic extension f(x⁰) ≡ f_g(z⁰) : Θ₀ → Φ₀, where Θ₀ and Φ₀ are contained in a suitable complex Banach spaces, which are the respective complex extensions of X and Y . Given a function f on a real linear domain D, and a function f* on a complex linear domain Θ ⊃ D, we say that f* is an analytic extension of f if f* is analytic on its domain, and the restriction f*|_D = f. When there exists an analytic extension f* we say that f can be analytically extended.

Remark 4 Note that as before we write f ≡ f_g, J ≡ J_g, M_v ≡ M_v,g, etc. It is understood from context that the notational equivalence is over the extension of the variables (x, q) into the complex pair (z, g).

Similarly, assume that J(x⁰) ∈ L(D₀,E₀) can be extended analytically as J(z⁰) ∈ L(Θ₀,Φ₀). Here L(·,·) is the space of bounded operators between two spaces. Through the above complexifications, equation (12) defines a complexification of the mapping M_v. We now repeat the above iteration using the complexified maps defined here. Thus there exists a series of sets Θ₀,…,Θ_v and Φ₀,…,Φ_v, such that for f(x^v) : D_v → E_v and J(x^v) ∈ L(D_v,E_v) the analytic extensions f(z^v) : Θ_v → Φ_v and J(z^v) ∈ L(Θ_v,Φ_v) are onto. Thus the sequence (14) is extended in ${ℂ}^N$ as follows: Let z⁰ = x⁰ and for all $v\in \mathbb{N}$ and g ∈ Ψ (which we also fix) form the sequence

$$z^v=M_v\left(z^{v-1}\right)\equiv z^{v-1}-J{\left(z^{v-1}\right)}^{-1}f\left(z^{v-1}\right),$$ (15)

where M_v : Θ_v−1 → Θ_v.

Remark 5 The domain Θ₀ contains the initial condition z⁰. Under certain assumptions and with a judicious choice of Θ₀ it can be shown that the sequence in (15) converges in a pointwise sense inside Θ₀. This will be explored in detail in section 3.1.

Suppose that z^v is an analytic extension of x_v on $\Psi \subset {ℂ}^N$ (see Figure 3). Then the convergence rates of the sparse grid applied to any entry of interest of z^v can be characterized. The size of the set Ψ determines the regularity properties of the solution. From the sparse grid discussion in section 2.3 we embed a polyellipse ${\mathcal{E}}_{{\widehat{\sigma }}_1,\dots ,{\widehat{\sigma }}_N}≔{\Pi }_{n=1}^N{\mathcal{E}}_{n,{\widehat{\sigma }}_n}$ in Ψ. From Theorem 10 the $L_{\rho }^{\infty }(\Gamma )$ convergence rate of the sparse grid is sub-exponential (or algebraic) with respect to the number of sparse grid knots η. The decay of the sparse grid is dominated by σ = min_n=1,…,N σ_n. Thus, the larger σ is the faster the convergence rate.

Fig. 3.

Analytic extension of the domain Γ. Any vector q ∈ Γ is extended in Ψ by adding a vector $v\in {ℂ}^N$ i.e. g = q + v.

Remark 6 For finite dimensional spaces $X=Y={ℝ}^m$, $m\in {\mathbb{N}}_0$, the Fréchet derivative J corresponds to the Jacobian of f. In the rest of the paper it is assumed that $D\subset {ℝ}^m$ and ‖ · ‖ corresponds to the standard Euclidean norm or the standard matrix norm, depending on context. For the case of the power flow equations m will be simply related to the number of nodes of the power system [8]. We will be using the notion of analytic extensions, which can be defined as follows.

We can now prove an important theorem for our purposes. First, denote f(z^v) : Θ_v → Φ_v as f^v, and J(z^v) ∈ L(Θ_v,Φ_v) as J^v (note that this depends on the generic initial z₀ ∈ Θ₀ at which the Jacobian is computed).

Theorem 4

Assume that for all $v\in {\mathbb{N}}_0:D_v\subset X$ and

1. f^v : D_v → E_v can be analytically extended to f^v : Θ_v → Φ_v.

2. There exists a coefficient c_v > 0 such that

   $${\sigma }_{min}\left(\left[\begin{array}{cc}{J}_R^v& -J_I^v\\ J_I^v& J_R^v\end{array}\right]\right)\ge c_v,$$

   where σ_min(·) refers to the minimum singular value, $J_R^v≔\text{Re }{J}^v$ and $J_I^v≔\text{Im }{J}^v$.

Then for all $v\in {\mathbb{N}}_0$ there exists an analytic extension of x^v on Ψ.

Proof The main strategy for this proof is to use the Cauchy-Riemann equations. This avoids having to explicitly show that the inverse of the complex Jacobian matrix J^v is analytic. The existence of an analytic extension for z^v for each separate complex dimension is shown. The Hartog’s theorem and Osgood’s lemma (Chap1, p2 in [14]) is then used to show analyticity with respect to all the complex dimensions.

For fixed v consider the extension x^v → z^v = x^v + w^v in Θ_v, where $w^v\in {ℂ}^m$ and z^v ∈ Θ_v. In complex form $z^v=z_R^v+i{z}_I^v$, where $z_R^v=\text{Re }{z}^v$, and $z_I^v=\text{Im }{z}^v$. Furthermore, consider the extension of q → g = q + v in Ψ, where $v\in {ℂ}^N$ and g ∈ Ψ. The extension of the iteration (14) on Θ_v ×Ψ leads to the following block form iteration

$$\left[\begin{array}{cc}{J}_R^v& -J_I^v\\ J_I^v& J_R^v\end{array}\right]\left(\left[\begin{array}{l}{z}_R^{v+1}\hfill \\ z_I^{v+1}\hfill \end{array}\right]-\left[\begin{array}{l}{z}_R^v\hfill \\ z_I^v\hfill \end{array}\right]\right)=-\left[\begin{array}{l}{f}_R^v\hfill \\ f_I^v\hfill \end{array}\right],$$ (16)

where $f_R^v≔\text{Re }{f}^v$ and $f_I^v≔\text{Im }{f}^v$. From (ii) it follows that equation (16) is well posed and is a valid extension of equation (14) on Θ_v × Ψ. We now show that z^v+1 is an analytic extension on Θ_v × Ψ.

We focus our attention on the k^th variable of z^v as $z_k^v$ and write it in complex form as $z_k^v=s+iw$. By differentiating equation (16) with respect to s and w we obtain

$$\begin{array}{c}{\partial }_s\left[\begin{array}{cc}{J}_R^v& -J_I^v\\ J_I^v& J_R^v\end{array}\right]\left[\left[\begin{array}{c}{z}_R^{v+1}\\ z_I^{v+1}\end{array}\right]-\left[\begin{array}{c}{z}_R^v\\ z_I^v\end{array}\right]\right]+\left[\begin{array}{cc}{J}_R^v& -J_I^v\\ J_I^v& J_R^v\end{array}\right]{\partial }_s\left[\left[\begin{array}{c}{z}_R^{v+1}\\ z_I^{v+1}\end{array}\right]-\left[\begin{array}{c}{z}_R^v\\ z_I^v\end{array}\right]\right]=-{\partial }_s\left[\begin{array}{c}{f}_R^v\\ f_I^v\end{array}\right]\\ {\partial }_w\left[\begin{array}{cc}{J}_R^v& -J_I^v\\ J_I^v& J_R^v\end{array}\right]\left[\left[\begin{array}{c}{z}_R^{v+1}\\ z_I^{v+1}\end{array}\right]-\left[\begin{array}{c}{z}_R^v\\ z_I^v\end{array}\right]\right]+\left[\begin{array}{cc}{J}_R^v& -J_I^v\\ J_I^v& J_R^v\end{array}\right]{\partial }_w\left[\left[\begin{array}{c}{z}_R^{v+1}\\ z_I^{v+1}\end{array}\right]-\left[\begin{array}{c}{z}_R^v\\ z_I^v\end{array}\right]\right]=-{\partial }_w\left[\begin{array}{c}{f}_R^v\\ f_I^v\end{array}\right]\end{array}.$$ (17)

From assumption (ii) we conclude that ${\partial }_s{z}_R^{v+1}$, ${\partial }_s{z}_I^{v+1}$, ${\partial }_w{z}_R^{v+1}$ and ${\partial }_w{z}_I^{v+1}$ exist on Θ_v × Ψ. The following step is to show that the Cauchy-Riemann equations for z^v+1 are satisfied on Θ_v × Ψ.

Let $P\left(z^v\right)≔{\partial }_s{z}_R^v-{\partial }_w{z}_I^v$ and $Q\left(z^v\right)≔{\partial }_w{z}_R^v+{\partial }_s{z}_I^v$, then from equation (17)

$$\left[\begin{array}{c}\left({\partial }_s{J}_R^v-{\partial }_w{J}_I^v\right)-\left({\partial }_s{J}_I^v+{\partial }_w{J}_R^v\right)\\ \left({\partial }_s{J}_I^v+{\partial }_w{J}_R^v\right)-\left({\partial }_s{J}_R^v-{\partial }_w{J}_I^v\right)\end{array}\right]\left(\left[\begin{array}{c}{z}_R^{v+1}\\ z_I^{v+1}\end{array}\right]-\left[\begin{array}{c}{z}_R^v\\ z_I^v\end{array}\right]\right)+\left[\begin{array}{cc}{J}_R^v& -J_I^v\\ J_I^v& J_R^v\end{array}\right]\left(\left[\begin{array}{l}P\left(z^{v+1}\right)\hfill \\ Q\left(z^{v+1}\right)\hfill \end{array}\right]-\left[\begin{array}{c}P\left(z^v\right)\\ Q\left(z^v\right)\end{array}\right]\right)=-\left[\begin{array}{c}{\partial }_s{f}_R^v-{\partial }_w{f}_I^v\\ {\partial }_s{f}_I^v+{\partial }_w{f}_R^v\end{array}\right].$$

Now, (i) implies that J^v ∈ L(D_v,E_v) can be analytically extended to J^v ∈ L(Θ_v,Φ_v). Since J(z^v, g) and f^v(z^v, g) are analytic on Θ_v ×Ψ then from the Cauchy-Riemann equations

$$\left[\begin{array}{c}\left({\partial }_s{J}_R^v-{\partial }_w{J}_I^v\right)-\left({\partial }_s{J}_I^v+{\partial }_w{J}_R^v\right)\\ \left({\partial }_s{J}_I^v+{\partial }_w{J}_R^v\right)-\left({\partial }_s{J}_R^v+{\partial }_w{J}_I^v\right)\end{array}\right]=0\text{ and }\left[\begin{array}{c}{\partial }_s{f}_R^v-{\partial }_w{f}_I^v\\ {\partial }_s{f}_I^v+{\partial }_w{f}_R^v\end{array}\right]=0.$$

Since $z_k^v$ is a linear polynomial of s + iw then P(z^v) = Q(z^v) = 0 on $C^N$ and thus P(z^v+1) = Q(z^v+1) = 0 on Θ_v × Ψ. We conclude that z^v+1 is analytic for the k^th variable for all z^v ∈ Θ_v and g ∈ Ψ. Following a similar argument we can show that for l = 1,…,N the l^th variable extension of q has leads to an analytic extension of z^v+1 whenever z^v ∈ Θ_v and g ∈ Ψ. We now extend the analyticity of z^v+1 on all of Θ_v × Ψ.

Since $z_k^{v+1}$ is analytic for all k = 1,…,m and the l^th variable of q has an analytic extension for all l = 1,…,N whenever z^v ∈ Θ_v and g ∈ Ψ, then from Hartog’s theorem we conclude that z^v+1 is continuous on Θ_v × Ψ. From Osgood’s lemma it follows that z^v+1 is analytic on Θ_v ×Ψ. From an induction argument and using that fact that the composition of analytic functions is analytic then it follows that z^v+1 is analytic in Ψ, for all $v\in \mathbb{N}$. □

If the assumptions of Theorem 4 are satisfied then z^v is complex analytic in Ψ and it is reasonable to construct a series of sparse grid surrogate models of the entries of the vector x^v. Note that in practice we restrict out attention to a subset of the variables of interest of x^v. With a slight abuse of notation denote ${\mathcal{S}}_w^{m,g}\left[x^v\left(q\right)\right]$ as the sparse grid approximation of the entries of interest of the vector x^v.

From Theorem 2 we observe that the accuracy of the sparse grid approximation is a function of i) the size of the polyellipse ${\mathcal{E}}_{{\widehat{\sigma }}_1,\dots ,{\widehat{\sigma }}_N}\subset \Psi$ and ii)

$$\tilde{M}\left(z^v\right)=\underset{z^v\in {\Theta }_v,k=1,\dots ,m}{\text{sup}}|z_k^v(g)|.$$

Remark 7 It is important to note that if the complex sequence (15) does not converge, then the size of the sets Θ_v can become unbounded. In particular, it is possible that $\tilde{M}\left(z^v\right)\to \infty$ as v → ∞ even if J(z^v)⁻¹ ∈ L(Φ_v,Θ_v) exists for all $v\in {\mathbb{N}}_0$. Thus the sparse grid error bound given by Theorem 2 explodes. A control of the size of the sets Θ_v are need. To this end we shall use the Newton Kantorovich theorem to show that the sets Θ_v are bounded and there exists a constant c_v > 0 such that ${\sigma }_{min}\left(\left[\begin{array}{cc}{J}_R^v& -J_I^v\\ J_I^v& J_R^v\end{array}\right]\right)\ge c_v>0$.

Our objective now is to analyze under what conditions the complex sequence remains bounded. In particular, for all $v\in {\mathbb{N}}_0$, we ask if it is possible to construct bounded regions $U\subset {ℂ}^m$ and $\Psi \subset {ℂ}^N$ such that z^v is contained in U and thus

$$\tilde{M}\left(z^v\right)\le \underset{z^v\in U}{\text{sup}}{\Vert z^v\Vert }_{\infty }.$$

To help answer this question we first show that the complex sequence (15) is itself a Newton sequence.

Remark 8 We have to clarify what we mean by the Fréchet derivative of the complex function f : Θ_v → Φ_v. The algebraic problem of equation (13) can be complexified as follows: Find z ∈ Θ₀ such that f(z, g) = 0 for all g ∈ Ψ. This can be re-written in vector form as: Find z = z_R + iz_I ∈ Θ₀ such that Ref(z_R, z_I, g_R, g_I) = 0 and Im f(z_R, z_I, g_R, g_I) = 0 for all g = g_R +ig_I ∈ Ψ. The corresponding Newton iteration is based on

$$\left[\begin{array}{cc}{\partial }_{z_R}{f}_R^v& {\partial }_{z_I}{f}_R^v\\ {\partial }_{z_R}{f}_I^v& {\partial }_{z_I}{f}_I^v\end{array}\right]\left(\left[\begin{array}{c}{z}_R^{v+1}\\ z_I^{v+1}\end{array}\right]-\left[\begin{array}{c}{z}_R^v\\ z_I^v\end{array}\right]\right)=-\left[\begin{array}{c}{f}_R^v\\ f_I^v\end{array}\right],$$ (18)

where ${\partial }_{z_R}{f}_R^v$ is the Fréchet derivative of $f_R^v$ with respect to the variables z_R and similarly for the rest. We refer to the matrix

$$J_{z^v}≔\left[\begin{array}{cc}{\partial }_{z_R}{f}_R^v& {\partial }_{z_I}{f}_R^v\\ {\partial }_{z_R}{f}_I^v& {\partial }_{z_I}{f}_I^v\end{array}\right]$$

as the Fréchet derivative of f : Θ_v → Φ_v.

Lemma 1

Suppose assumption i) of Theorem 4 are satisfied. Then the complex analytic extension J(z^v) ∈ L(Θ_v, Φ_v) of J(x^v) ∈ L(D_v,E_v) is equivalent to the Fréchet derivative of f^v : Θ_v → Φ_v, i.e.

$$\left[\begin{array}{cc}{J}_R^v& -J_I^v\\ J_I^v& J_R^v\end{array}\right]=\left[\begin{array}{cc}{\partial }_{z_R}{f}_R^v& {\partial }_{z_I}{f}_R^v\\ {\partial }_{z_R}{f}_I^v& {\partial }_{z_I}{f}_I^v\end{array}\right].$$

Proof We first prove this result for m = 1 dimension. Suppose that $f:D\to ℝ$, is a Fréchet differentiable function and let $f:\Xi \to ℂ$ be the analytic continuation on the non-empty open set $\Xi \subset ℂ$. The analytic function $f:\Xi \to ℂ$ can be rewritten as f(x, y) = f_R(x, y) + if_I(x, y) for all x + iy ∈ Ξ. Since f is analytic on Ξ, from the the identity theorem [1] (uniqueness of complex analytic extensions) we have that f_R and f_I are unique in Ξ. Furthermore, since f is analytic the Cauchy-Riemann equations are satisfied. Thus

$$\left[\begin{array}{cc}{\partial }_x{f}_R& -{\partial }_x{f}_I\\ {\partial }_x{f}_I& {\partial }_x{f}_R\end{array}\right]=\left[\begin{array}{cc}{\partial }_x{f}_R& {\partial }_y{f}_R\\ {\partial }_x{f}_I& {\partial }_y{f}_I\end{array}\right]$$ (19)

in Ξ and $f:\Xi \to ℂ$ is Fréchet differentiable. Now, ∂_xf(x, y) = ∂_xf_R(x, y) + i∂_xf_I(x, y) in Ξ, and from the uniqueness property of the Fréchet derivative all the terms are unique. Recall that ∂_xf(x) defined in D is the Fréchet derivative of $f:D\to ℝ$. Write the analytic extension of ∂_xf(x) (defined in D) on Ξ as g(x, y) + ih(x, y), with x + iy ∈ Ξ. Since ∂_xf(x) = ∂_xf(x, y) = ∂_xf_R(x, y) + i∂_xf_I(x, y) for y = 0 and x ∈ D, from the uniqueness of the analytic extension we conclude g = ∂_xf_R and h = ∂_xf_I for all x + iy ∈ Ξ. From equation (19) the conclusion follows.

We can now prove our statement for the general case using a simple extension of the above argument. Since f^v : Θ_v → Φ_v is complex analytic, from the identity theorem [1] it is the unique extension of f^v : D_v → E_v. (Note that the unique extension of the identity theorem applies in multi-variate case, which includes the variables x^v and q in the domains Θ_v and Ψ respectively.) From the Cauchy-Riemann equations the functions f^v : Θ_v → Φ_v are Fréchet differentiable and unique. Now, with a slight abuse of notation, denote $J_{Z_R^v}$ as the Jacobian of f^v : Θ_v → Φ_v, with respect to the real variables $Z_R^v$ only. By using the above one dimensional argument we can show that the analytic extension of each entry of J(x^v) matches $J_{Z}v$ on the real part of Θ_v. From the Cauchy-Riemann equations we conclude that J(z^v) ∈ L(Θ_v, Φ_v) is equivalent to the Fréchet derivative of f^v : Θ_v → Φ_v. □

From Theorem 1 it follows that the complex sequence (15) is a Newton sequence. We can now apply the Newton-Kantorovich Theorem to study the sequence convergence as v → ∞.

3.1. Regions of Analyticity

The size of a polyellipse embedded in the domain Ψ and the magnitude of z^v ∈ Θ_v (for any $v\in {\mathbb{N}}_0$) directly impacts the accuracy of the sparse grid (c.f. Theorem 2). For each $v\in {\mathbb{N}}_0$ the size of the domains Θ_v and Ψ will be characterized by the magnitude of the minimum singular value

$${\sigma }_{min}\left(\left[\begin{array}{cc}{J}_R^v& -J_I^v\\ J_I^v& J_R^v\end{array}\right]\right)\ge c_v>0$$ (20)

for some c_v > 0. However, constructing the domain Ψ would require imposing inequality conditions for each Newton iteration. This leads to a highly complex coupled problem that is hard to solve. Moreover, if the complex extension z^v grows rapidly with respect to v then the size of the domain Ψ will be most likely severely constrained. In contrast, by applying the Newton-Kantorovich Theorem it is sufficient to impose conditions on the initial Jacobian (v = 0) to construct a region of analyticity for $\Psi \subset {ℂ}^N$. Furthermore, the size of the iteration z^v will be controlled.

Consider the iteration

$${\alpha }^{v+1}={\alpha }^v-J{\left({\alpha }^v,g\right)}^{-1}f\left({\alpha }^v,g\right),$$ (21)

where g ∈ Ψ,

$${\alpha }^0≔\left[\begin{array}{c}{x}_0\\ 0\end{array}\right],{\alpha }^v≔\left[\begin{array}{c}{z}_R^v\\ z_I^v\end{array}\right],J\left({\alpha }^v,g\right)≔\left[\begin{array}{cc}{J}_R^v& -J_I^v\\ J_I^v& J_R^v\end{array}\right],\text{ and f}\left({\alpha }^v,g\right)≔\left[\begin{array}{c}{f}_R^v\\ f_I^v\end{array}\right],$$

for all $v\in {\mathbb{N}}_0$.

Remark 9 From Lemma 1 or, alternatively, the Cauchy-Riemann equations, the matrix J(α^v,g) corresponds to the Fréchet derivative of f(α^v,g). Thus the sequence (21) is an Newton iteration and the Newton-Kantorovich Theorem can be used to analyze its convergence properties.

Assumption 2

For all q ∈ Γ Assumption 1 is satisfied.

Assumption 3

Assume that $\tilde{D}$, where $D\subset \tilde{D}$, is an open convex set in ${ℝ}^{2m}$ and the following Lipschitz condition is satisfied:

$$\Vert J(x,g)-J(y,g)\Vert \le {\lambda }_e\Vert x-y\Vert ,$$

for all x, $y\in \tilde{D}$, g ∈ Ψ, and λ_e ≥ 0. Furthermore assume that for all g ∈ Ψ

$$\Vert J{\left({\alpha }^0,g\right)}^{-1}\Vert \le x_e,$$

$$\Vert J{\left({\alpha }^0,g\right)}^{-1}f\left({\alpha }^0,g\right)\Vert \le {\delta }_e,$$

$$h_e=2x_e{\lambda }_e{\delta }_e\le 1,$$

and $U\left({\alpha }^0,t_e^{*}\right)\subset \tilde{D}$, where $t_e^{*}=\frac{2}{h_e}(1-\sqrt{1-h_e}){\delta }_e$.

Theorem 5

If Assumption 3 is satisfied then for all g ∈ Ψ

1. The Newton iterates α^v+1 = α^v + J(α^v, g)⁻¹f(α^v, g) exist and ${\alpha }^v\in U\left({\alpha }_0,t_e^{*}\right)\subset \tilde{D}$.

2. α* := lim_v→∞ α^v exists, ${\alpha }^{*}\in \overline{U\left({\alpha }^0,t_e^{*}\right)}$, and f(α*, g) = 0.

3. J(α^v, g)⁻¹ exists for all $v\in \mathbb{N}$ and equation (20) is satisfied.

Proof From Theorem 1 and Remark 9 we have that the iterates α^v+1 = α^v + J(α^v, g)⁻¹f(α^v, g) are a valid Newton sequence. In particular, J(α^v, g) corresponds to the Fréchet derivative of f(α^v, g). From Assumption 3 and the Newton-Kantorovich theorem [2] the result follows.

Remark 10 Recall from condition (ii) of Theorem 4 for each $v\in \mathbb{N}$ the minimum singular value of the Jacobian matrix J(α^v, g) is bounded by a constant c_v > 0. The constant c_v can be obtained from the proof of the Newton-Kantorovich theorem. See the details of the proof of Theorem 2.2.4 in [2]. In particular, the proof of this theorem shows the existence of a sequence of constants $b_v\in ℝ$ such that σ_max(J(α^v, g)⁻¹) ≤ b_v (Equation (2.2.21) on page 44). Note that the sequence b₀,…,b_v depends on the initial parameters ϰ_e, δ_e and λ_e , i.e. b_v(ϰ_e, δ_e, λ_e).

Remark 11 From Assumptions 2 and 3 and from the fact that extended Newton iteration is a valid extension of the sequence (14) then we have that λ_e ≥ λ, ϰ_e ≥ ϰ, δ_e ≥ δ, h_e ≥ h. This implies that $t_e^{*}\ge t^{*}$ and therefore $U\left(x_0,t^{*}\right)\subseteq U\left({\alpha }_0,t_e^{*}\right)$ for all g ∈ Ψ (See Figure 4). From the Newton-Kantorovich Theorem it follows that that ${\alpha }^v\in U\left({\alpha }_0,t_e^{*}\right)$ for all $v\in {\mathbb{N}}_0$.

Fig. 4.

Region of convergence $U\left({\alpha }_0,t_e^{*}\right)$ for the extended Newton iteration.

We can construct a region Ψ such that for all g ∈ Ψ the extended Newton iteration converges. Let $y=\left[\begin{array}{c}{y}_R\\ y_I\end{array}\right]=\left[\begin{array}{c}\text{Re}g\\ \text{Im}g\end{array}\right]$, and apply the multivariate Taylor theorem for each k = 1,…,n, l = 1,…,n entry of the Jacobian matrix J_R(α₀, g). Evaluating g at q + v, we have that

$${\left[J_R\left({\alpha }_0,q+v_R,0+v_I\right)\right]}^{k,l}={\left[J_R\left(x_0,q,0\right)\right]}^{k,l}+R_{k,l}\left(x_0\right)]\left[\begin{array}{l}{v}_R\hfill \\ v_I\hfill \end{array}\right],$$

v_R := Rev, v_I := Im v, and

$$R_{k,l}\left(x_0\right)=\left[R_{k,l}^1\left(x_0\right),\dots ,R_{k,l}^{2m}\left(x_0\right)\right].$$

The entries of the remainder term R_k,l(x₀) are bounded by

$$|R_{k,l}^{\beta }\left(x_0\right)|\le \underset{t\in (0,1)}{\text{max}}|{\partial }_{y_{\beta }}{\left[J_R\left(x_0,\left[\begin{array}{l}q\hfill \\ 0\hfill \end{array}\right]+t\left[\begin{array}{l}{v}_R\hfill \\ v_I\hfill \end{array}\right]\right)\right]}^{k,l}|,$$

where ∂_yβ refers to the derivative of the β^th variable of the vector y. Form the matrix

$$E≔\left[\begin{array}{cc}0& -J_I\left(x_0,q,v_R,v_I\right)\\ J_I\left(x_0,q,v_R,v_I\right)& 0\end{array}\right]+\left[\begin{array}{cc}Q\left(x_0,q,v_R,v_I\right)& 0\\ 0& Q\left(x_0,q,v_R,v_I\right)\end{array}\right]$$

and let

$$Q_{k,l}\left(x_0,q,v_R,v_I\right)=R_{k,l}\left(x_0\right)\left[\begin{array}{l}{v}_R\hfill \\ v_I\hfill \end{array}\right]$$

be the k = 1,…,n, l = 1,…,n entry of the matrix Q. Then

$$\left[\begin{array}{cc}{J}_R\left(x_0,q\right)& -J_I\left(x_0,q,v_R,v_I\right)\\ J_I\left(x_0,q,v_R,v_I\right)& J_R\left(x_0,q\right)\end{array}\right]=\mathbb{J}+E=\mathbb{J}\left(I+{\mathbb{J}}^{-1}E\right),$$

where $\mathbb{J}≔\left[\begin{array}{cc}{J}_R\left(x_0,q\right)& 0\\ 0& J_R\left(x_0,q\right)\end{array}\right]$.

Theorem 6

Suppose that ϰ_e ≥ ϰ and

$$\Vert E\left({\alpha }^0,g\right)\Vert <\frac{1-\frac{x}{x_e}}{x}$$

whenever g ∈ Ψ then

$$\Vert J{\left({\alpha }^0,g\right)}^{-1}\Vert \le x_e.$$

Proof First note that

$$\Vert J{\left({\alpha }_0,g\right)}^{-1}\Vert \le \Vert {\left(\mathbb{J}\left(I+{\mathbb{J}}^{-1}E\right)\right)}^{-1}\Vert \le \Vert {\mathbb{J}}^{-1}\Vert \Vert {\left(I+{\mathbb{J}}^{-1}E\right)}^{-1}\Vert .$$ (22)

From Lemma 2.2.3 in [13], if ${\Vert {\mathbb{J}}^{-1}E\Vert }_2<1$ then $\left(I+{\mathbb{J}}^{-1}E\right)$ is invertible and

$$\Vert {\left(I+{\mathbb{J}}^{-1}E\right)}^{-1}\Vert <\frac{1}{1-\Vert {\mathbb{J}}^{-1}E\Vert }.$$

Given that $\Vert \mathbb{J}{\left({\text{x}}_0,\text{q}\right)}^{-1}\Vert \le x$ (From Assumption 1) whenever q ∈ Γ, it follows

$$\Vert {\mathbb{J}}^{-1}\Vert \Vert I+{\mathbb{J}}^{-1}E\Vert <\frac{x}{1-\Vert {\mathbb{J}}^{-1}E\Vert }\text{ and }\Vert {\mathbb{J}}^{-1}E\Vert \le \Vert {\mathbb{J}}^{-1}\Vert \Vert E\Vert \le x\Vert E\Vert .$$ (23)

We conclude that if

$$\Vert E\left({\alpha }^0,g\right)\Vert <\frac{1-\frac{x}{x_e}}{x}$$

whenever g ∈ Ψ, then from Equations (22) and (23)

$$\Vert J{\left({\alpha }^0,g\right)}^{-1}\Vert \le x_e.$$

□

Applying the multivariate Taylor’s theorem for each k = 1,…,n, entry of the vector f_R(α₀, g) where g = q + v, we have

$${\left[f_R\left({\alpha }_0,q+v_R,0+v_I\right)\right]}^k={\left[f_R\left(x_0,q,0\right)\right]}^k+S_k\left(x_0,q,0\right)]\left[\begin{array}{l}{v}_R\hfill \\ v_I\hfill \end{array}\right],$$

where v_R := Rev, v_I := Im v, and

$$S_k\left(x_0,q,0\right)=\left[S_k^1\left(x_0,q,0\right),\dots ,S_k^{2m}\left(x_0,q,0\right)\right].$$

The remainder term S_k(x₀, q) is bounded by

$$|S_k^{\beta }\left(x_0,q,0\right)|\le \underset{t\in (0,1)}{\text{max}}|{\partial }_{y_{\beta }}{\left[f_R\left(\left[\begin{array}{l}q\hfill \\ 0\hfill \end{array}\right]+t\left[\begin{array}{l}{v}_R\hfill \\ v_I\hfill \end{array}\right]\right)\right]}^k|,$$

where ∂_yβ refers to the derivative of the β^th variable of y. We can now rewrite the vector f(α₀, g) as

$$f\left({\alpha }_0,g\right)=\mathbb{F}\left(x_0,q\right)+G\left(x_0,q,v_R,v_I\right),$$

where $\mathbb{F}≔\left[\begin{array}{c}{f}_R\left(x_0,q\right)\\ 0\end{array}\right]$, $G≔\left[\begin{array}{l}P\left(x_0,q,v_R,v_I\right)\hfill \\ f_I\left(x_0,q,v_R,v_I\right)\hfill \end{array}\right]$ and

$$P_k\left(x_0,q,v_R,v_I\right)=S_k\left(x_0,q,0\right)\left[\begin{array}{l}{v}_R\hfill \\ v_I\hfill \end{array}\right].$$

Theorem 7

Suppose that ϰ_e ≥ ϰ and δ_e ≥ δ. Then if

$$\Vert G\left({\alpha }^0,g\right)\Vert <\frac{{\delta }_e}{x_e}-\frac{\delta }{x}$$

whenever g ∈ Ψ, it follows

$$\Vert J{\left({\alpha }_0,g\right)}^{-1}f\left({\alpha }_0,g\right)\Vert \le {\delta }_e.$$

Proof For each of the entries k = 1,…,n of the vector P.

$$\Vert J{\left({\alpha }_0,g\right)}^{-1}f\left({\alpha }_0,g\right)\Vert =\Vert {(\mathbb{J}+E)}^{-1}(\mathbb{F}+G)\Vert \le \Vert \left(I+{\mathbb{J}}^{-1}E\right){\mathbb{J}}^{-1}(\mathbb{F}+G)\Vert \text{***}\le \Vert \left(I+{\mathbb{J}}^{-1}E\right){\mathbb{J}}^{-1}\mathbb{F}\Vert +\Vert \left(I+{\mathbb{J}}^{-1}E\right){\mathbb{J}}^{-1}G\Vert \text{***}\le \Vert I+{\mathbb{J}}^{-1}E\Vert \Vert {\mathbb{J}}^{-1}\mathbb{F}\Vert +\Vert I+{\mathbb{J}}^{-1}E\Vert \Vert {\mathbb{J}}^{-1}\Vert \Vert G\Vert \text{***}\le \Vert I+{\mathbb{J}}^{-1}E\Vert \delta +\Vert I+{\mathbb{J}}^{-1}E\Vert \Vert G\Vert x.$$ (24)

Since $\Vert E{\Vert }_2<\frac{1-\frac{x}{x_e}}{x}<x$ (from Theorem 6) from Lemma 2.2.3 in [13] it follows that $\left(I+{\mathbb{J}}^{-1}E\right)$ is invertible and

$$\Vert {\left(I+{\mathbb{J}}^{-1}E\right)}^{-1}\Vert <\frac{1}{1-\Vert {\mathbb{J}}^{-1}E\Vert }\le \frac{x_e}{x}.$$ (25)

Combining equations (24) and (25) we have

$$\Vert J{\left({\alpha }_0,g\right)}^{-1}f\left({\alpha }_0,g\right)\Vert <\frac{x_e}{x}\left(\delta +\Vert G\left(x_0,q,v_R,v_I\right)\Vert x\right).$$

The result follows. □

From the values of ϰ, δ, λ and ϰ_e, δ_e, λ_e and Theorems 6 and 7, the region of analyticity Ψ can be constructed. From this region, convergence rates from Theorem 2 for the sparse grid interpolation can be estimated. If we are interested in forming a sparse grid for each of the entries of the vector $x^v\in {ℝ}^n$, then there are potentially n sparse grids. To estimate the convergence rate of the sequence of sparse grids it is sufficient to embed a polydisk ${\mathcal{E}}_{{\sigma }_1,\dots ,{\sigma }_N}\subset \Psi$ for a suitable set of coefficients {σ₁ …,σ_N}. Furthermore since ${\alpha }^v\in U\left({\alpha }_0,t_e^{*}\right)$, the maximal coefficient $\tilde{M}\left(z^v\right)$ can be bounded as

$$\tilde{M}\left(z^v\right)\le t_e^{*}+{\Vert x_0\Vert }_{l^2\left({ℝ}^{2m}\right)}.$$

4. Application to power flow

The theory developed in Section 3 can be applied to the computation of the statistics of stochastic power flow. In particular we concentrate on the random perturbations of the generators, loads and admittance uncertainty of the transmission lines. Much of the power system network model presented in this section is based on [8].

Consider a network with m+1 mechanical constant power generators. The electrical power injected into the network at each generator is given by

$$P_{G_k}=\sum _{l=0}^m{V}_k{V}_{l}sin\left({\theta }_k-{\theta }_l+{\phi }_{k,l}\right)|Y_{k,l}|,$$ (26)

where the operands of the summation are the power from bus k transmitted to bus l through a line with admittance Y_k,l = G_k,l + iB_k,l, phase shift φ_k,l, and voltage V_k at the buses. These form the algebraic constraints of the power system. The dynamic constraints at generator k are given by

$$M_k{\ddot{\theta }}_i+D_k{\dot{\theta }}_i+P_{G_k}=P_{M_k}+P_{I_k}(\omega )+P_{L_k}(\omega )$$ (27)

where M_k is the moment of inertia of generator i, D_k is the damping factor, $P_{M_k}$ denotes the mechanical power, $P_{L_k}(\omega )$ is the stochastic load and $P_{I_k}(\omega )$ is the intermittent stochastic power applied to bus i. Equations (26) and (27) constitute the swing equation model. Since the intermittent power generators and loads are stochastic, the rotor angle θ_k(ω) and power generation $P_{G_k}(\omega )$ will be stochastic as well. A simple example of a 3 bus power system is shown in Figure 5. From the steady state response the power flow equations are given by

$$P_k(x)=\sum _{l=0}^m{V}_k{V}_l\left[G_{ik}cos\left({\theta }_k-{\theta }_l\right)+B_{ik}sin\left({\theta }_k-{\theta }_l\right)\right]$$

$$Q_k(x)=\sum _{l=0}^m{V}_k{V}_l\left[G_{ik}sin\left({\theta }_k-{\theta }_l\right)+B_{ik}cos\left({\theta }_k-{\theta }_l\right)\right]$$

for k = 0,…,m.

Fig. 5.

2 generators, 3 buses, 1 load simple power system example. This figure is modified from [10]. Bus 1 is the slack bus. Bus 2 contains a stochastic generator. Bus 3 contains the random load. Note that voltages and power flows are in p.u.

It is assumed that at each node the active and reactive power injections (or loads) are given by P₁,…,P_m and Q₁,…,Q_m. The first bus is assumed to be slack bus with known angle θ₀ = 0 and fixed voltage V₀.

Remark 12 According to power system convention, the numbering of the buses (nodes) starts with 1 instead of 0. To simplify the notation in this section we start from 0. However, for the examples and numerical results we revert to the power system standard.

In this paper we limit our discussion of power flow to the case where the power injections P₁,…,P_m and Q₁,…,Q_m are assumed to be known, but could be stochastic. The unknowns are formed by the angles θ₁,…,θ_m and voltages V₁,…,V_m. The power flow equations are solved with a Newton iteration and posed as

$$\theta ≔\left[\begin{array}{c}{\theta }_1\\ ⋮\\ {\theta }_m\end{array}\right],V≔\left[\begin{array}{c}{V}_1\\ ⋮\\ V_m\end{array}\right],x≔\left[\begin{array}{c}\theta \\ V\end{array}\right],f(x)=\left[\begin{array}{c}\Delta P(x)\\ \Delta Q(x)\end{array}\right],$$

where

$$\Delta P(x)≔\left[\begin{array}{c}{P}_1(x)-P_1\\ ⋮\\ P_m(x)-P_m\end{array}\right]\text{ and }\Delta Q(x)≔\left[\begin{array}{c}{Q}_1(x)-Q_1\\ ⋮\\ Q_m(x)-Q_m\end{array}\right].$$

The Jacobian matrix is given in block form as

$$J=\left[\begin{array}{ll}{J}_{11}\hfill & J_{12}\hfill \\ J_{21}\hfill & J_{22}\hfill \end{array}\right],$$

where J₁₁, J₁₂, J₂₁, $J_{22}\in {ℝ}^{n\times n}$. For k, l = 1,…,m let θ_k,l := θ_k − θ_l and if k ≠ l

$$J_{k,l}^{11}=V_k{V}_l{G}_{k,l}sin\left({\theta }_{k,l}\right)-B_{k,l}\left(cos\left({\theta }_{k,l}\right)\right),$$

$$J_{k,l}^{21}=-V_k{V}_l{G}_{k,l}cos\left({\theta }_{k,l}\right)+B_{k,l}\left(sin\left({\theta }_{k,l}\right)\right),$$

$$J_{k,l}^{12}=V_k{G}_{k,l}cos\left({\theta }_{k,l}\right)+B_{k,l}\left(sin\left({\theta }_{k,l}\right)\right),$$

$$J_{k,l}^{22}=V_k{G}_{k,l}sin\left({\theta }_{k,l}\right)-B_{k,l}\left(cos\left({\theta }_{k,l}\right)\right),$$

otherwise

$$\begin{array}{cc}{J}_{k,k}^{11}=-Q_k(x)-B_{k,k}{V}_k^2& J_{k,k}^{21}=P_k(x)-G_{k,k}{V}_k^2\\ J_{k,k}^{12}=\frac{P_k(x)}{V_k}+G_{k,k}{V}_k& J_{k,k}^{22}=\frac{Q_k(x)}{V_k}-B_{k,k}{V}_k\end{array}.$$

It is clear from the structure of f and the Jacobian J that they are analytic everywhere except for V_k = 0, for k = 1,…,m. However, in practice the domain Θ₀ is chosen such that the origin is avoided. Otherwise the analyticity assumptions of Theorem 4 are not satisfied.

There are many forms of uncertainty that can be present in the solution of the power flow equations. We concentrate on the following cases:

• Random loads: The power loads P_k and Q_k, k = 1,…,m, will be a function of the random vector q ∈ Γ:

  $$P_k+i{Q}_k=P_k^0\left(1+c_k{q}_k\right)+Q_k^0\left(1+c_{k+1}{q}_{k+1}\right),$$

  where $P_k^0$ and $Q_k^0$ are the nominal power loads (or generators), q_k ∈ [−1, 1] and $c_k,c_{k+1}\in ℝ$.

• Random admittances: The transmission line admittances Y_k,l will be functions of the random vector q ∈ Γ. Let $\mathcal{A}$ be the set of network index tuples (k, l) such that the admittance is stochastic. Thus for all $k,l\in \mathcal{A}$ let

  $$Y_{k,l}=G_{k,l}+i{B}_{k,l}=G_{k,l}^0\left(1+c_{k,l,1}{q}_{k,l,1}\right)+B_{k,l}^0\left(1+c_{k,l,2}{q}_{k,l,2}\right),$$

  where $G_{k,l}^0$ and $B_{k,l}^0$ are the nominal conductance and susceptance, q_k,l,1, q_k,l,2 ∈ [−1, 1] and $c_{k,l,1},c_{k,l,2}\in ℝ$. Note that with a slight of abuse of notation the vector q consists of all the stochastic random variables ${\left\{q_{k,l,1},q_{k,l,2}\right\}}_{(k,l)\in \mathcal{A}}$.

For sufficiently small coefficients c_k, c_k+1 with k = 1,…,m and c_k,l,1, c_k,l,2 for all tuples $(k,l)\in \mathcal{A}$ the assumptions of Theorems 5, 6 and 7 are satisfied for some initial condition x₀ and thus we can justify the use of the sparse grids. Due to the extent of a detailed analysis of the size of these coefficients, it is left for a future work emphasizing the details of power systems. However, in Appendix A we present the case for random generators and loads.

We test the sparse grid approximation on the New England 39 Bus, 10 Generator, power system model provided from the Matpower 6.0 steady state simulator [21, 32]. In this model buses 1 – 29 are PQ buses, buses 30, 32–39 are generators and bus 31 is the reference (slack).

The following numerical examples indicate that the conclusions of the analyticity theorems we proved are valid i.e. algebraic and sub-exponential convergence of the stochastic norm with respect to the random variables. This is despite the conservative bounds placed on the analyticity region of Ψ from the Newton-Kantorovich Theorem. We expand on this point in the conclusion section.

Two tests are performed. We randomly perturb either the loads or the admittances of the transmission lines. The mean and variance of the voltage V₂₂ at bus 22 are computed. The mean $\mathbb{E}\left[V_{22}\right]$ and variance var[V₂₂] are computed with the Clenshaw-Curtis isotropic Sparse Grid Matlab Kit [4, 30] for N = 2, 4, 12 dimensions and up to the w = 7 level. This last level, w = 7 is taken as the “true” solution. The errors are computed up to level w = 4 with respect to this solution. Two tests are performed:

• Random loads: The loads are considered stochastic and are perturbed by up to ± 50% of their nominal value. For each k = 1,…,N, the so-called k^th PQ bus is stochastically perturbed as

  $$P_k+i{Q}_k=P_k^0\left(1+\frac{q_k}{2}\right)+Q_k^0\left(1+\frac{q_k}{2}\right),$$

  where $P_k^0$ and $Q_k^0$ are the nominal power loads, q_k ∈ [−1, 1], ρ(q_k) has a uniform distribution and the random variables q₁,…,q_N are independent. Note that although the load random perturbations are independent, the power flows will be dependent on all the random variables q₁,…,q_N.

  In Figure 6 (a) & (b) the mean and variance convergence error for the stochastic voltage V₂₂ of bus 22 are shown. A surrogate model based on the sparse grid operator is formed as ${\mathcal{S}}_w^{m,g}\left[V_{22}\right]$ with Clenshaw-Curtis abscissas. Each of the circles corresponds to a sparse grid ${\mathcal{S}}_w^{m,g}$ starting with level w = 1 up to level w = 4. The y-axis corresponds to the error of the mean or variance. The x-axis is the number of sparse grid knots needed to form the grid ${\mathcal{S}}_w^{m,g}$. The dimension of the sparse grid is given by N = 2, 4, 12.

  From Figure 6 (a) & (b) we observe that the error decreases faster than polynomially with respect to the number of knots η. As we increase the number w of levels, sub-exponential convergence is achieved. This is much faster than the ${\eta }^{-\frac{1}{2}}$ convergence rate of the Monte Carlo method. For example, for N = 12, then mean is computed approximately 10¹¹ times faster for the same accuracy. This is the difference between 2 hours of computation on a simple 4 core processor and 20 million years with Monte Carlo. However, as the number of dimensions N increases the convergence rate of the sparse grid decreases, as predicted by Theorem 2. Moreover, if the level w is not large enough then the error bound gives algebraic convergence.

• Random transmission line admittances: The admittances of the network are assumed to be random with

  $$Y_{k,l}=G_{k,l}+i{B}_{k,l}=G_{k,l}^0\left(1+\frac{q_{k,l,1}}{2}\right)+B_{k,l}^0\left(1+\frac{q_{k,l,2}}{2}\right),$$

  where $G_{k,l}^0$ and $B_{k,l}^0$ are the nominal conductance and susceptance. The coefficients q_k,l,1, q_k,l,2 ∈ [−1,1] have a uniform distribution and are all independent. Figure 6 (c) & (d) indicate sub-exponential convergence of the mean and variance of the voltage V₂₂ at bus 22 for a sufficiently large number of knots. However, as the number of stochastic dimensions N increases to 12, the convergence rate decreases and almost approaches polynomial convergence. From Theorem 1 the sufficient condition w > N/log2 leads to subexponential convergence. For N = 12 we have that w has to be larger than 18 to guarantee sub-exponential convergence. In Figures (c) & (d) the largest level for w is 4.

Fig. 6.

Sparse grid convergence rates. (a) & (b) Mean and variance error of the voltage V₂₂ of bus 22 given a stochastic load perturbation with dimension N and the number of knots of the sparse grid. (c) & (d) Mean and variance of error of the voltage V₂₂ of bus 22 given a random admittance with dimension N. Notice that for all 4 cases the convergence rates are faster than polynomial, indicating a sub-exponential convergence rate.

5. Conclusions

In this paper we have introduced ideas from UQ and numerical analysis, typically used in the field of stochastic PDEs, and applied them to non-linear stochastic networks. More specifically, these ideas are applied to the Newton iteration. We have developed a regularity analysis of the solution with respect to the random perturbations. Under sufficient conditions based on the Newton-Kantorovich Theorem there exists analytic extensions of the solution of the Newton iteration. These indicate that the application of sparse grids for the computation of the stochastic moments leads to sub-exponential or algebraic convergence. For a moderate number of dimensions the convergence rates are much faster than traditional Monte Carlo approaches $\left({\eta }^{-\frac{1}{2}}\right)$. In addition, numerical experiments applied to the power flow problem confirm these subexponential and algebraic convergence rates.

A weakness in the application of the Newton-Kantorovich Theorem is that it constricts the size of the region of analyticity Ψ, thus leading to a conservative convergence rate of the sparse grid. This motivates the application of less restrictive methods such as damped Newton iterates [5]. In addition, if we incorporate the assumption that all the Newton iterations converge for each of the knots of the space grid, then by developing an a posteriori method convergence rates can be further improved.

Future work includes the important application of this method to the security constrained problem [28] from the probabilistic perspective. In other words, given stochastic perturbations of the loads and sources what are the optimal power injections into the grid such that the probability of failure is below a tolerance level. Current approaches rely on simplifications of the stochastic perturbations to deal with the high dimensions. However, this can lead to suboptimal results. The high dimensional stochastic quadrature approach developed in this paper will allow more optimal results.

A Analyticity regions for random generators and loads

We examine the case of random generators and loads to obtain convergence rates of the sparse grid. The task is to synthesize an analyticity region Ψ for the extended Newton iteration to converge. We only check that the conditions of Theorem 7 are satisfied and assume that 6 is satisfied. A full analysis will be done in a future work. Thus, we synthesis the region of analyticity for Ψ by checking that

$$\Vert G\left({\alpha }^0,g\right)\Vert <\frac{{\delta }_e}{x_e}-\frac{\delta }{x},$$

whenever g ∈ Ψ. Without loss of generality assume that the first τ ≤ m buses contain stochastic power generators (or loads) with active power P₁(ω) := (q₁ + v_1,R + iv_1,I)c₁ + a₁,,…,P_τ(ω) := (q_τ +v_q,R+iv_q,I)c_τ +a_τ and reactive power Q₁(ω) := (q_τ+1 +v_τ+1,R + iv_τ+1,I)c_τ+1 +a_τ+1,…, Q_τ(ω) := (q_N +v_N,R +iv_N,I)c_N +a_N. The random vector q ∈ Γ is assumed to be stochastic with joint distribution ρ(q) and for all k = 1,...,N v_k := v_k,R + iv_k,I is the complex extension of each q_k ∈ Γ_k. Furthermore, for k = 1,…,N the variables $a_k\in ℝ$ and $c_k\in {ℝ}^{+}$, where a_k + c_k and a_k indicate the maximum and minimum range of the stochastic perturbation. Thus we have

$$f\left({\alpha }_0\right)≔\left[\begin{array}{c}{P}_1\left(x_0\right)-P_1(\omega )\\ ⋮\\ P_{\tau }\left(x_0\right)-P_{\tau }(\omega )\\ P_{\tau +1}\left(x_0\right)-P_{\tau +1}\\ ⋮\\ \frac{P_m\left(x_0\right)-P_m}{Q_1\left(x_0\right)-Q_1(\omega )}\\ ⋮\\ Q_{\tau }\left(x_0\right)-Q_{\tau }(\omega )\\ Q_{\tau +1}\left(x_0\right)-Q_{\tau +1}\\ ⋮\\ Q_m\left(x_0\right)-Q_m\end{array}\right],$$

$$\text{Re }f\left({\alpha }_0\right)=\left[\begin{array}{c}{P}_1\left(x_0\right)-\left(q_1+v_{1,R}\right)c_1+a_1\\ ⋮\\ P_{\tau }\left(x_0\right)-\left(q_{\tau }+v_{q,R}\right)c_{\tau }+a_{\tau }\\ P_{\tau +1}\left(x_0\right)-P_{\tau +1}\\ ⋮\\ \frac{P_m\left(x_0\right)-P_m}{Q_1\left(x_0\right)-\left(q_{\tau +1}+v_{q+1,R}\right)c_{\tau +1}+a_{\tau +1}}\\ ⋮\\ Q_{\tau }\left(x_0\right)-\left(q_N+v_{N,R}\right)c_N+a_N\\ Q_{\tau +1}\left(x_0\right)-Q_{\tau +1}\\ ⋮\\ Q_m\left(x_0\right)-Q_m\end{array}\right],$$

$$\text{Im }f\left({\alpha }_0\right)=-{\left[v_{1,I}{c}_1\dots v_{\tau ,I}{c}_{\tau }0\dots 0|v_{\tau +1,I}{c}_{\tau +1}\dots v_{N,I}{c}_{N}0\dots 0\right]}^T,$$

and therefore

$$P_P=-\left[\begin{array}{c}\left(v_{1,R}+v_{1,I}\right)c_1\\ ⋮\\ \left(v_{\tau ,R}+v_{\tau ,I}\right)c_{\tau }\\ 0\\ ⋮\\ 0\end{array}\right],P_Q=-\left[\begin{array}{c}\left(v_{\tau +1,R}+v_{\tau +1,I}\right)c_{\tau +1}\\ ⋮\\ \left(v_{N,R}+v_{N,I}\right)c_N\\ 0\\ ⋮\\ 0\end{array}\right],$$

$P=\left[\begin{array}{l}{P}_P\hfill \\ Q_Q\hfill \end{array}\right]$ and thus $\Vert G{\Vert }_2^2={{\sum }_{k=1}^N\left(v_{k,R}+v_{k,I}\right)}^2{c}_k^2+{\sum }_{k=1}^N{v}_{k,I}^2{c}_k^2$. The last equality is due to the integral remainder form of Taylor’s Theorem. Let v_k,R = v_R/c_k and v_k,I = v_I/c_k for k = 1,…,2N, where v_R, $v_I\in ℝ$, then for any ϵ > 0

$$\Vert G{\Vert }_2^2=N{v}_R^2+2N{v}_I^2+2N{v}_R{v}_I\text{***}\le v_R^{2}N(1+2ϵ)+v_I^{2}N\left(2+{ϵ}^{-1}/2\right)\text{***}\le {\left(\frac{{\delta }_e}{x_e}-\frac{\delta }{x}\right)}^2.$$

The last inequality is obtained by using Cauchy’s inequality. Let ${\gamma }_e≔\frac{{\delta }_e}{x_e}-\frac{\delta }{x}$, thus the inequality

$$\frac{v_R^2}{{\alpha }^2}+\frac{v_I^2}{{\beta }^2}\le 1,$$ (28)

where ${\alpha }^2≔\frac{{\gamma }_e^2}{N(1+2ϵ)}$ and ${\beta }^2≔\frac{{\gamma }_e^2}{N\left(2+{ϵ}^{-1}/2\right)}$, forms an elliptical region $\Sigma \subset ℂ$ (See Figure 7) such that ‖G‖₂ ≤ γ_e.

Fig. 7.

Embedding of Bernstein ellipse ${\mathcal{E}}_{\sigma }$ in the domain Φ.

Consider the region Φ := {g = q+v_R +iv_I |q ∈ [−1,1], (v_R, v_I) ∈ Σ} where ‖G‖₂ ≤ γ_e is satisfied. Suppose that we want to embed a Bernstein ellipse ${\mathcal{E}}_{\sigma }$ in Φ. To achieve this consider the foci points of ${\mathcal{E}}_{\sigma }$ at −1 and 1. It is not hard to show that $\frac{e^{\sigma }-e^{-\sigma }}{2}\ge \frac{e^{\sigma }+e^{-\sigma }}{2}-1$ for σ > 0. At the foci point ±1 trace the ellipse from Equation (28) and set $\beta =\frac{e^{\sigma }-e^{-\sigma }}{2}$ (See Figure 7).

Choose an ϵ > 0 such that ${\mathcal{E}}_{\sigma }$ is embedded in Φ by solving the following equation

$$\frac{\alpha }{\beta }=\sqrt{\frac{4+{ϵ}^{-1}}{2(1+2ϵ)}}$$

leading to

$$ϵ=\frac{{\left(c^2+4\right)}^{\frac{1}{2}}-c+2}{4c}>0,$$

where c := (α/β)² > 0. Pick ϵ > 0 such that c = (α/β)² = 1. Pick σ > 0 such that $\frac{e^{\sigma }-e^{-\sigma }}{2}=\beta$. This leads to

$$\sigma =\text{log}\left(\beta +{\left({\beta }^2+1\right)}^{\frac{1}{2}}\right)$$

and $\frac{e^{\sigma }+e^{-\sigma }}{2}-1\le \beta =\alpha$. The region bounded by the ellipse ${\mathcal{E}}_{\sigma }$ is therefore embedded in Φ.

A polyellipse in C^N can now be constructed such that such that ‖G‖₂ ≤ γ_e. Recall that v_k,R = v_R/c_k and v_k,I = v_I/c_k for k = 1,…,N and consider the regions Ψ_k := {g = q + v_k,R + iv_k,I |q ∈ [−1, 1], (v_R, v_I) ∈ Σ}. By following the procedure for embedding ${\mathcal{E}}_{\sigma }$ in Φ for each k = 1,…,N an ellipse ${\mathcal{E}}_{\varrho k}$ can be embedded in Ψ_k with

$${\varrho }_k≔\text{log}\left(\frac{\beta }{c_k}+{\left(\frac{{\beta }^2}{c_k^2}+1\right)}^{\frac{1}{2}}\right).$$

The polyellipse ${\epsilon }_{{\varrho }_1,\dots ,{\varrho }_N}≔{\mathcal{E}}_{\varrho 1}\times \cdots \times {\mathcal{E}}_{\varrho N}$ is embedded in Ψ₁ × ⋯ × Ψ_N. Thus for any $g\in {\epsilon }_{{\varrho }_1,\dots ,{\varrho }_N}$

$$\Vert G\left({\alpha }_0,g\right)\Vert \le {\gamma }_e.$$

With the region bounded by the polyellipse ${\epsilon }_{{\varrho }_1,\dots ,{\varrho }_N}$ the convergence rate of the sparse grid can be estimated with respect to the magnitude of the coefficients c_k, for k = 1,...,N.

Remark 13 From this analysis we observe that the size of the analyticity region of Ψ depends directly on

$$\beta =\sqrt{\frac{{\gamma }_e^2}{N\left(2+{ϵ}^{-1}/2\right)}},$$

where $ϵ=\frac{\sqrt{5}+1}{4}$. The size of the region Ψ decays as square root with respect to the number of stochastic dimensions N, thus reducing the convergence rate of the sparse grid.

Example 1 Consider the 3 Bus simple power system with stochastic load and generator based on Example 10.6 in [8] and Figure 5. Bus 1 is the slack bus with . Bus 2 voltage is fixed as V₂ = 1.05 p.u. and contains a stochastic generator P_E = q₁c₁ + a₁, where q₁ ∈ [−1, 1] with a₁ = 0.6661 p.u. and $c_1\in {ℝ}^{+}$, i.e. the generator is random within the range 0.6661±c₁ . Bus 3 contains the random load with P_L+iQ_L = (q₂c₂+a₂)+i(q₃c₃+a₃), where q₂,q₃ ∈ [−1, 1] with a₂ = 2.8653 p.u., a₃ = 1.2244 p.u. and c₂, $c_3\in {ℝ}^{+}$, i.e. the load is random within the range 2.8653 ± c₂ of the active power and 1.2244 ± c₃ for the reactive power. The admittance matrix is to the network is

$$Y_{bus}≔i\left[\begin{array}{rrr}\hfill -20& \hfill 10& \hfill 10\\ \hfill 10& \hfill -20& \hfill 10\\ \hfill 10& \hfill 10& \hfill -20\end{array}\right].$$

The vector of unknowns is x := [θ₂, θ₃, V₃]^T and f(x, q) = [P₁(x)−q₁c₁−a₁,P₂(x)−q₂c₂−a₂,P₃(x)−q₃c₃−a₃]^T for all q ∈ Γ and therefore f(α, g) = [P₁(α)−(q₁+v_1,R+iv_1,I)c₁+a₁,P₂(α)−(q₂+v_2,R+iv_2,I)c₂+a₃, P₃(α)−(q₃+v_3,R+iv_3,I)c₃+a₃]^T for all g ∈ Ψ. With q = 0 the Newton algorithm converges in 20 iterations to x* = [−5.2361×10⁻²,−1.7445×10⁻¹, 0.9500]^T with 10⁻¹⁵ tolerance. Furthermore,

$$J\left(x^{*}\right)=\left(\begin{array}{lll}0.065427\hfill & 0.033847\hfill & 0.0013531\hfill \\ 0.033847\hfill & 0.07112\hfill & 0.011273\hfill \\ 0.001284\hfill & 0.010697\hfill & 0.065493\hfill \end{array}\right)\text{\hspace{1em} and }x={\Vert J^{-1}\left(x^{*}\right)\Vert }_2=\mathrm{0.1043.}$$

Since there is no direct stochastic components (q₁, q₂, q₃) in the Jacobian matrix, then

$$J(x)=\left[\begin{array}{ccc}10.5\left(cos\left({\theta }_2\right)+V_{3}cos\left({\theta }_2-{\theta }_3\right)\right)& -10.5V_{3}cos\left({\theta }_2-{\theta }_3\right)& 10.5sin\left({\theta }_2-{\theta }_3\right)\\ -10.5V_{3}cos\left({\theta }_3-{\theta }_2\right)& 10V_{3}cos\left({\theta }_3\right)+10.5V_{3}cos\left({\theta }_3-{\theta }_2\right)& 10.5sin\left({\theta }_3\right)+10.5sin\left({\theta }_3-{\theta }_2\right)\\ -10.5V_{3}sin\left({\theta }_3-{\theta }_2\right)& 10.5V_3\left(sin\left({\theta }_3\right)+sin\left({\theta }_3-{\theta }_2\right)\right)& -\left(10cos\left({\theta }_3\right)+10.5cos\left({\theta }_3-{\theta }_2\right)-39.96V_3^2\right)\end{array}\right].$$

From the mean value theorem we have that

$$\lambda =\sqrt{\sum _{k,l=1}^m{\Vert \nabla J_{k,l}(x)\Vert }_{L^{\infty }(D\times \Gamma )}}<\infty ,$$

where D is a bounded set, J_k,l(x) is the k^th row and l^th column entry of the Jacobian matrix J(x).

With the initial condition x₀ = x*(0), for small enough coefficients c₁, c₂ and c₃ we have that:

1. ‖J⁻¹(x₀)‖ ≤ ϰ = 0.1043.

2. Furthermore, since f is continuous and f(x₀, 0) = 0 then ‖J⁻¹(x₀)f(x₀, q)‖ ≤ δ < ∞, where δ is arbitrarily small.

3. It follows that h < 1 for all $x\in \overline{B\left(x^{*}(0),t^{*}\right)}\subset D$ where $t^{*}=\frac{2}{h}(1-\sqrt{1-h})\delta$.

4. From the Newton-Kantorovich theorem the iteration converges for all q ∈ Γ and

$${\Vert V_3(q)\Vert }_{L^{\infty }(\Gamma )}=\underset{x\in \overline{B\left(x_0,t^{*}\right)}}{\text{sup}}|x[3]|\le t^{*}+{\Vert x^{*}(0)\Vert }_{l^2\left({ℝ}^m\right)}.$$

Now, pick δ_e > 0 such that δ < δ_e and also pick ϰ_e = ϰ, λ_e = λ such that h < h_e ≤ 1. From the random load analysis we have ${\gamma }_e≔\frac{{\delta }_e^1}{x_e}-\frac{\delta }{x}$ and

$$\Vert G\left({\alpha }_0,g\right)\Vert \le {\gamma }_e$$

for all $g\in {\mathcal{E}}_{\varrho 1}{,}_{\varrho 2}{,}_{\varrho 3}$ where

$$\beta ={\left(\frac{(1+\sqrt{5})}{6(2+\sqrt{5})}\right)}^{\frac{1}{2}}{\gamma }_e$$

and for k = 1,…,3

$${\varrho }_k≔\text{log}\left(-\frac{\beta }{c_k}+{\left(\frac{{\beta }^2}{c_k^2}+1\right)}^{\frac{1}{2}}\right).$$

Assuming that Theorem 6 is satisfied, then from Theorem 7 it follows that whenever g ∈ Ψ then

$$\Vert J{\left(x_0\right)}^{-1}f\left({\alpha }_0,g\right)\Vert \le {\delta }_e,$$

the limit of the Newton iteration converges and is holomorphic in ${\mathcal{E}}_{\varrho 1}{,}_{\varrho 2}{,}_{\varrho 3}\subset \Psi \subset {ℂ}^3$.