Distributed weighted least-squares estimation with fast convergence for large-scale systems

Abstract

In this paper we study a distributed weighted least-squares estimation problem for a large-scale system consisting of a network of interconnected sub-systems. Each sub-system is concerned with a subset of the unknown parameters and has a measurement linear in the unknown parameters with additive noise. The distributed estimation task is for each sub-system to compute the globally optimal estimate of its own parameters using its own measurement and information shared with the network through neighborhood communication. We first provide a fully distributed iterative algorithm to asymptotically compute the global optimal estimate. The convergence rate of the algorithm will be maximized using a scaling parameter and a preconditioning method. This algorithm works for a general network. For a network without loops, we also provide a different iterative algorithm to compute the global optimal estimate which converges in a finite number of steps. We include numerical experiments to illustrate the performances of the proposed methods.

1. Introduction

A sensor network is a web of a large number of intelligent sensing and computing nodes connected via a communication network (Dargie & Poellabauer, 2010). The emergence of sensor networks calls for the development of distributed algorithms for a number of tasks to replace the traditional centralized methods. In particular, the development of distributed algorithms for parameter estimation has recently attracted a great deal of attention (Fang and Li, 2008, Kar et al., 2012, Li and AlRegib, 2007, Li and AlRegib, 2009, Lopes and Sayed, 2008, Ribeiro and Giannakis, 2006a, Ribeiro and Giannakis, 2006b, Ribeiro et al., 2010, Xiao et al., 2006). They find applications in environmental and weather monitoring, industrial process monitoring and control, surveillance, state estimation for smart grid, etc.

Existing distributed estimation methods can be classified in several ways. The first classification of the methods is done according to whether a coordinating node or fusion center (FC) is present. When an FC is present, each node communicates with the FC either directly (via a star topology) or indirectly (via a mesh topology), i.e., there is a directed communication path from either node to the FC. The estimation is carried out at the FC, after some possible pre-processing at each node before transmission. The methods proposed in Fang and Li (2008), Li and AlRegib, 2007, Li and AlRegib, 2009, Ribeiro and Giannakis, 2006a, Ribeiro and Giannakis, 2006b are of this type. On the other hand, when no FC is present, estimation is done by executing a cooperative algorithm among all the nodes of the network. The network is connected in some way and each node communicates with its neighboring nodes only. Representative methods of this type include Conejo, de la Torre, and Canas (2007), Gómez-Expósito, Villa Jaén, Gómez-Quiles, Rousseaux, and Van Cutsem (2011), Kar et al. (2012) and Lopes and Sayed (2008). A second classification is done by whether the estimation method is static or dynamic. In static estimation, a set of parameters is estimated using the measurements of all nodes, which collectively form a snapshot of the system at a fixed time. Examples of this type include Fang and Li (2008), Kar et al. (2012), Li and AlRegib, 2007, Li and AlRegib, 2009, Lopes and Sayed (2008) and Ribeiro and Giannakis, 2006a, Ribeiro and Giannakis, 2006b. On the other hand, in dynamic estimation all the nodes track the evolution of a set of parameters for which a dynamic model is available, as in the Refs. Carli, Chiuso, Schenato, and Zampieri (2008), Hlinka, Sluciak, Hlawatsch, Djuric, and Rupp (2012), Khan and Moura (2008) and Ribeiro et al. (2010). Some “hybrid” methods exist, which permit tracking a time-varying sequence of parameters, without a dynamic model, by updating a static estimate at each time (Lopes & Sayed, 2008). A final classification is done by whether a distributed estimation method is a small-scale one or a large-scale one. In a small-scale method, all nodes estimate a common set of parameters (Fang and Li, 2008, Kar et al., 2012, Li and AlRegib, 2007, Li and AlRegib, 2009, Lopes and Sayed, 2008, Ribeiro and Giannakis, 2006a, Ribeiro and Giannakis, 2006b). But in a large-scale method, each node only reconstructs a subset of the parameters, i.e., the collective knowledge of the parameters is distributed among all the nodes (Conejo et al., 2007, Gómez-Expósito et al., 2011, Huang et al., 2012, Khan and Moura, 2008). Large-scale estimation is in general more challenging than its small-scale counterpart.

In this paper we study distributed static estimation for a large-scale system consisting of a network of interconnected sub-systems. Each sub-system is concerned with a subset of the unknown parameters and has measurements, linear in the unknown parameters, corrupted by additive noise. The distributed estimation task is for each sub-system to estimate its local state using its own measurement and information shared with the network through neighborhood communication. We use the weighted least squares (WLS) criterion for optimal estimation. The goal is that the composite estimate of the whole system, consisting of all local estimates, will become globally optimal in the sense that it is the same as the optimal estimate obtained using all the measurements and a centralized estimation method.

This problem is motivated by many applications involving a large-scale spatially distributed system. For example, the state estimation problem for a large power network is concerned with estimating the voltages and phases of the power supply at each sub-system, consisting of a number of buses or a substation, using measurements (provided by, for example, a phasor measurement unit (PMU) or a supervisory control and data acquisition (SCADA) system) (Conejo et al., 2007, Jiang et al., 2008). Interactions of sub-systems are reflected by the fact that local measurements available at each sub-system typically involve neighboring sub-systems. For example, a current measurement at a conjunction depends on the voltage difference of two neighboring buses. In a smart grid setting, each sub-system is only interested in the local state, i.e., its own voltages and phases, using local measurements and information acquired from neighboring sub-systems via neighborhood communication (Tai, Marelli, Rohr, & Fu, 2013). For a large power network, it is both impractical and unnecessary for every sub-system to estimate the whole state of the system, thus distributed estimation methods for local state estimation are naturally called for. The second example is the localization problem for a wireless sensor network, involving estimating the locations of all sensors using relative measurements (e.g., relative distances or relative positions) between sensors (Diao et al., 2013, Khan et al., 2009). For a small sensor network with a few sensing nodes, it is possible to aggregate all the measurements at an FC to compute a global estimate of all sensor locations. Such an algorithm is not scalable, and would require massive computing resources when the network becomes large. It is also unnecessary for each sensor to localize other nodes. A distributed method is preferred, where each node aims to estimate its own location using local measurements and neighborhood communication. The third example is a traffic network for a city or a highway system, where each node or sub-system wants to estimate its local state (e.g., traffic flow rates, delays, etc.). Due to the spatial correlations of the traffic flows in different sub-systems, neighboring traffic information is certainly useful in predicting the traffic conditions at each sub-system. Again, distributed estimation methods are preferred over centralized methods. Many other examples in sensor networks can be found in, e.g., Ribeiro et al. (2010), Kar et al. (2012) and Yang et al. (2010).

We first provide a fully distributed iterative algorithm for each node to compute its own local estimate. This algorithm works for a general connected network. Contrary to the method proposed in Conejo et al. (2007), our algorithm guarantees that the composite estimate will converge asymptotically to the global WLS estimate. We then focus on the convergence rate of the algorithm. Since our method is based on Richardson’s method for solving systems of linear equations (Bertsekas & Tsitsiklis, 1997), its convergence rate depends on certain scaling parameter and a preconditioning matrix. Choosing the optimum scaling parameter requires the knowledge of the largest and the smallest eigenvalues of certain positive definite matrix (the estimation error covariance). A distributed algorithm for estimating these values can be obtained using the power method (Bertsekas & Tsitsiklis, 1997). However, to prevent numerical instability, this approach requires periodically executing a normalization step, which needs to be carried out in a distributed manner. In Yang et al. (2010), this is done using average consensus. A drawback of this approach is that consensus itself converges asymptotically. This significantly slows down the convergence of the eigenvalue estimation. To avoid this, we propose a different method in which normalization is done locally, at each node, without inter-node communication. In this way, the optimal scaling parameter can be obtained using a distributed method. We then address the problem of designing the preconditioning matrix. Our distributed scenario constrains us to use a block diagonal matrix. Choosing the optimal matrix under this constraint, and using only distributed processing, is a very challenging problem for which we are not able to provide a solution. Nevertheless, we are able to bound the difference between the convergence rate achieved using the optimal matrix, and the convergence rate resulting using a practically feasible matrix design. This bound turns out to have a simple expression which depends on the network connectivity. A shortened version of this method appears in the conference paper (Marelli & Fu, 2013).

For an acyclic network (i.e., its communication graph contains no loops), we provide a different iterative algorithm for distributed estimation. As opposed to the previous algorithm, in this one, the composite estimate is guaranteed to converge to the globally optimal estimate in a finite number of steps. Indeed, we show that the convergence time equals the diameter of the aforementioned graph. Numerical experiments show that this method offers a major reduction in convergence time.

The rest of paper is organized as follows. In Section  2 we describe the distributed WLS estimation the problem. In Section  3 we derive the first distributed WLS method, which converges asymptotically. In Section  3.1 we describe distributed methods for finding the value of the scaling parameter which yields the fastest convergence rate. In Section  3.2 we describe a sub-optimal choice for preconditioning matrix, and we bound its sub-optimality. In Section  4 we introduce the second distributed WLS method which converges in finite time. Numerical experiments are presented in Section  5, and concluding remarks are given in Section  6. For the ease of readability, all proofs are contained in the Appendix A, Appendix B.

Notation 1

For a vector$x$,$\Vert x\Vert$denotes its  2-norm, and ${\left[x\right]}_i$ denotes its $i\mathrm{th}$ entry. For a matrix $X$, $\Vert X\Vert$ denotes its operator (induced) norm. For square symmetric matrices $X$ and $Y$, $X<Y$ means that matrix $Y-X$ is positive-definite. For vectors and matrices, the superscript ${}^T$ denotes its transpose, and ${}^{\ast }$ denotes its transpose conjugate.

2. Problem description

Consider a network formed by $I$ nodes. For each $i=1,\dots ,I$, Node $i$ has an associated parameter vector $x_i\in {\mathbb{C}}^{d_i}$, and measures the vector $y_i\in {\mathbb{C}}^{p_i}$, which is given by

$$y_i=\sum _{j=1}^I{A}_{i,j}{x}_j+v_i\text{,}$$ (1)

where $v_i\sim \mathcal{N}(0,R_i)$ denotes the measurement noise. The noises $v_i$ and $v_j$ are statistically independent, whenever $i\ne j$.

Let $x^T=[x_1^T,\dots ,x_I^T]$, $y^T=[y_1^T,\dots ,y_I^T]$, $v^T=[v_1^T,\dots ,v_I^T]$, $R=\mathrm{diag}\{R_1,\dots ,R_I\}$ and $A={\left[A_{i,j}\right]}_{i,j=1,\dots ,I}$. Then, we can write the measurement model of the whole network as

$$y=Ax+v\text{,}$$ (2)

with $v\sim \mathcal{N}(0,R)$. The WLS estimator $\hat{x}$ of $x$ is defined by Kay (1993, Eq. (8.14))

$$\hat{x}=arg\underset{x}{min}{(y-Ax)}^{\ast }{R}^{-1}(y-Ax)\text{.}$$

Its solution is given by

$$\hat{x}={\Psi }^{-1}\alpha$$ (3)

with

$$\alpha =A^{\ast }{R}^{-1}y\text{and}\Psi =A^{\ast }{R}^{-1}A\text{.}$$

For the WLS problem to be well defined, we make the following assumption:

Assumption 2

Matrix $A$ has full column rank and $R$ is non-singular.

Computing (3) requires global information, i.e., all the measurements and the information about $A$ and $R$ need to be made available together. Our goal is to derive distributed methods in which Node $i$ computes the component ${\hat{x}}_i$ of the estimate $\hat{x}$, corresponding to $x_i$, using only the local measurement $y_i$ and information received from its neighbors (a formal definition of neighborhood will be given later).

In the rest of the paper we use the following notation:

Notation 3

Let${\mathcal{I}}_i=\{j:A_{j,i}\ne 0\}$denote the set of nodes whose measurements involve the parameters of Node$i$, and${\mathcal{O}}_i=\{j:A_{i,j}\ne 0\}$denote the set of nodes whose parameters are involved in the measurements of Node$i$. Let${\mathcal{N}}_i={\mathcal{I}}_i\cup {\mathcal{O}}_i$be the neighborhood of Node$i$. We also define${\mathcal{B}}_i=\{j:{\mathcal{N}}_i\cap {\mathcal{N}}_j\ne 0̸\}$to be the set of nodes whose neighborhood have a non-empty intersection with that of Node$i$. Notice that${\mathcal{B}}_i=\{j:{\Psi }_{i,j}\ne 0\}$.

3. Asymptotic method for WLS estimation

The distributed WLS method derived in this section uses the following definition of neighbor node.

Definition 4

Node $j$ is a neighbor of Node $i$ if $j\in {\mathcal{N}}_i$.

Also, the proposed method requires the following connectivity assumption:

Assumption 5

For each $i=1,\dots ,I$, Node $i$ can send/receive information to/from all its neighbors. Also, $A_{i,j}$, for all $j\in {\mathcal{O}}_i$, and $R_i$ are available at Node $i$.

Although our method works regardless of whether the network is sparse or not, it works most efficiently for sparse networks. A network is called sparse if the cardinality of ${\mathcal{N}}_i$ is small for all $i=1,2,\dots ,I$.

Consider any block diagonal positive definite matrix

$$\Pi =\mathrm{diag}\{{\Pi }_1,\dots ,{\Pi }_I\}$$ (4)

with ${\Pi }_i\in {\mathbb{C}}^{d_i\times d_i},i=1,\dots ,I$. Then define

$${\rm Y}={\Pi }^{1/2}\Psi {\Pi }^{1/2}$$ (5)

and choose

$$0<\gamma <2{\Vert {\rm Y}\Vert }^{-1}\text{.}$$ (6)

Let $\tilde{\alpha }={\left(\gamma \Pi \right)}^{1/2}\alpha$ and $\widetilde{\stackrel{ˆ}{x}}={\left(\gamma \Pi \right)}^{-1/2}\hat{x}$. From (3) we have

$$\widetilde{\stackrel{ˆ}{x}}={\left(\gamma {\rm Y}\right)}^{-1}\tilde{\alpha }\text{.}$$

From (6), $0<\gamma {\rm Y}<2I$. Hence, $-I<\gamma {\rm Y}-I<I$ and therefore $\Vert I-\gamma {\rm Y}\Vert <1$. In view of this, we can use Richardson’s method (Bertsekas & Tsitsiklis, 1997) to compute $\widetilde{\stackrel{ˆ}{x}}$ recursively. This leads to

$$\widetilde{\stackrel{ˆ}{x}}(t+1)=(I-\gamma {\rm Y})\widetilde{\stackrel{ˆ}{x}}\left(t\right)+\tilde{\alpha }\text{.}$$ (7)

Then, by substituting the expressions of $\tilde{\alpha }$ and $\widetilde{\stackrel{ˆ}{x}}$, we obtain straightforwardly

$$\hat{x}(t+1)=(I-\gamma \Pi \Psi )\hat{x}\left(t\right)+\gamma \Pi \alpha \text{.}$$ (8)

We call $\Pi$ the preconditioning matrix, because, as it will be explained in Section  3.2, it is used to increase the convergence rate of the recursions (8).

Let

$${\alpha }_i=\sum _{k\in {\mathcal{I}}_i}\underset{i}{\overset{\left(k\right)}{\alpha }}\text{,}$$ (9)

with ${\alpha }_i^{\left(k\right)}=A_{k,i}^{\ast }{R}_k^{-1}{y}_k$, for $k=1,\dots ,I$, so that ${\alpha }^T=[{\alpha }_1^T,\dots ,{\alpha }_I^T]$. Also, for $i,j=1,\dots ,I$, let

$${\Psi }_{i,j}=\sum _{k:i,j\in {\mathcal{O}}_k}\underset{i,j}{\overset{\left(k\right)}{\Psi }}\text{,}$$ (10)

where ${\Psi }_{i,j}^{\left(k\right)}=A_{k,i}^{\ast }{R}_k^{-1}{A}_{k,j}$, for all $k=1,\dots ,I$, so that $\Psi ={\left[{\Psi }_{i,j}\right]}_{i,j=1,\dots ,I}$. We have that

$${\left[\Psi \hat{x}\left(t\right)\right]}_i=\sum _{j=1}^I{\Psi }_{i,j}{\hat{x}}_j\left(t\right)=\sum _{j=1}^I\sum _{k:i,j\in {\mathcal{O}}_k}\underset{i,j}{\overset{\left(k\right)}{\Psi }}{\hat{x}}_j\left(t\right)=\sum _{k:i\in {\mathcal{O}}_k}\sum _{j\in {\mathcal{O}}_k}\underset{i,j}{\overset{\left(k\right)}{\Psi }}{\hat{x}}_j\left(t\right)=\sum _{k\in {\mathcal{I}}_i}\sum _{j\in {\mathcal{O}}_k}\underset{i,j}{\overset{\left(k\right)}{\Psi }}{\hat{x}}_j\left(t\right)\text{.}$$ (11)

Then, from (8), (11), (9), we obtain

$${\hat{x}}_i(t+1)={\hat{x}}_i\left(t\right)-\gamma {\Pi }_i\sum _{j=1}^I{\Psi }_{i,j}{\hat{x}}_j\left(t\right)+\gamma {\Pi }_i{\alpha }_i={\hat{x}}_i\left(t\right)-\gamma {\Pi }_i(\sum _{k\in {\mathcal{I}}_i}\sum _{j\in {\mathcal{O}}_k}\underset{i,j}{\overset{\left(k\right)}{\Psi }}{\hat{x}}_j\left(t\right)-\sum _{k\in {\mathcal{I}}_i}\underset{i}{\overset{\left(k\right)}{\alpha }})\text{.}$$ (12)

Notice that the matrices ${\Psi }_{i,j}^{\left(k\right)}$ are only available at Node $k$. That is, Node $k$ acts as an intermediary between Node $j$, which transmits ${\hat{x}}_j\left(t\right)$, and Node $i$, which receives ${\sum }_{j\in {\mathcal{O}}_k}{\Psi }_{i,j}^{\left(k\right)}{\hat{x}}_j\left(t\right)$. This means that node  $j$ should transmit ${\hat{x}}_j\left(t\right)$ to all nodes $k$ with $j\in {\mathcal{O}}_k$, or equivalently, to all nodes in ${\mathcal{I}}_j$. However, Node $j$ does not know which nodes are in ${\mathcal{I}}_j$. Thus, Node $j$ simply transmits ${\hat{x}}_j\left(t\right)$ to all nodes in ${\mathcal{N}}_j$, and it is up to the receiving Node $k$ to detect whether Node $j\in {\mathcal{O}}_k$ or not.

Following the discussion above, we obtain the following algorithm to implement (12).

Algorithm 1 - distributed WLS estimation:

Initialization:

• (1)For each $k=1,\cdots ,I$ and $i\in {\mathcal{O}}_k$, Node  $k$ computes ${\alpha }_i^{\left(k\right)}$ and sends it to Node  $i$.
• (2)On reception, Node  $i$ computes ${\alpha }_i={\sum }_{k\in {\mathcal{I}}_i}{\alpha }_i^{\left(k\right)}$.
• (3)For each $i=1,\cdots ,I$, Node  $i$ sets ${\hat{x}}_i\left(1\right)=0$.

Main loop:  At time $t\in \mathbb{N}$:

• (1)For each $j=1,\cdots ,I$ and $k\in {\mathcal{N}}_j$, Node  $j$ sends its current estimate ${\hat{x}}_j\left(t\right)$ to Node  $k$.
• (2)
  On reception, for each $k=1,\cdots ,I$ and $i\in {\mathcal{O}}_k$, Node  $k$ sends to Node  $i$

  $${\check{x}}_{i,k}\left(t\right)=\sum _{j\in {\mathcal{O}}_k}\underset{i,j}{\overset{\left(k\right)}{\Psi }}{\hat{x}}_j\left(t\right)\text{.}$$
• (3)
  On reception, for each $i=1,\cdots ,I$, Node  $i$ computes

  $${\hat{x}}_i(t+1)={\hat{x}}_i\left(t\right)-\gamma {\Pi }_i(\sum _{k\in {\mathcal{I}}_i}{\check{x}}_{i,k}\left(t\right)-{\alpha }_i)\text{.}$$

To implement Algorithm 1, we need to design the scaling factor $\gamma$ and the preconditioning matrices ${\Pi }_i$, for all $i=1,\dots ,I$. We address these two tasks in Sections  3.1, 3.2, respectively.

3.1. Distributed design of the scaling factor $\gamma$

In this section we study two approaches for designing the scaling factor $\gamma$. In Section  3.1.1 we describe a distributed algorithm which converges asymptotically to the optimal value of $\gamma$, i.e., the resulting $\gamma$ will achieve the maximum convergence speed. In Section  3.1.2, we give another distributed algorithm which converges in finite time to a sub-optimal value of $\gamma$.

3.1.1. Asymptotic algorithm for $\gamma$

In view of (7), the value of $\gamma$ that maximizes the convergence rate is

$$\gamma =\frac{2}{\Vert {\rm Y}\Vert +{\Vert {{\rm Y}}^{-1}\Vert }^{-1}}\text{,}$$ (13)

because this is the value that minimizes $\Vert I-\gamma {\rm Y}\Vert$. In order for each node to find the value of $\gamma$ given in (13), we need distributed methods for finding $\Vert {\rm Y}\Vert$ and ${\Vert {{\rm Y}}^{-1}\Vert }^{-1}$. We give these methods below. These methods yield, at Node $i$ and time step $t$, estimates ${\overline{{\rm Y}}}_i\left(t\right)$ and ${\underset{¯}{{\rm Y}}}_i\left(t\right)$, of $\Vert {\rm Y}\Vert$ and ${\Vert {{\rm Y}}^{-1}\Vert }^{-1}$, respectively. Then, at the same node and time step, the estimate ${\gamma }_i\left(t\right)$ of $\gamma$ is obtained by

$${\gamma }_i\left(t\right)=\frac{2}{{\overline{{\rm Y}}}_i\left(t\right)+{\underset{¯}{{\rm Y}}}_i\left(t\right)}\text{.}$$

Distributed method for finding$\Vert {\rm Y}\Vert$:

Choose any vector $b\left(0\right)\ne 0$ and let $b\left(t\right)={{\rm Y}}^{t}b\left(0\right)$. Then, using (11) with ${\Pi }^{1/2}b\left(t\right)$ in place of $\hat{x}\left(t\right)$, we obtain at Node $i$,

$$b_i(t+1)={\left[{\rm Y}b\left(t\right)\right]}_i={\Pi }_i^{1/2}{\left[\Psi {\Pi }^{1/2}b\left(t\right)\right]}_i={\Pi }_i^{1/2}\sum _{k\in {\mathcal{I}}_i}\sum _{j\in {\mathcal{O}}_k}\underset{i,j}{\overset{\left(k\right)}{\Psi }}\underset{j}{\overset{1/2}{\Pi }}{b}_j\left(t\right)\text{,}$$ (14)

where $b_i\left(t\right)$ denotes the $i\mathrm{th}$ block component of $b\left(t\right)$. Then, using the power method (Bertsekas & Tsitsiklis, 1997), Node $i$ can asymptotically compute $\Vert {\rm Y}\Vert$ as follows

$$\Vert {\rm Y}\Vert =\underset{t\to \infty }{lim}\frac{\Vert b_i\left(t\right)\Vert }{\Vert b_i(t-1)\Vert }\text{.}$$ (15)

An inconvenience with the approach above is that $b\left(t\right)$ either increases or decreases indefinitely. To avoid this, the vector $b\left(t\right)$ can be periodically normalized. In Yang et al. (2010), this was done using average consensus (in the continuous-time case). As we mentioned in Section  1, we avoid the drawbacks of that method by providing an alternative approach in which $b\left(t\right)$ is normalized at each node, and each iteration, without inter-node communication. This algorithm is given below:

Algorithm 2 - distributed estimation of $\Vert {\rm Y}\Vert$: For each $k=1,\cdots ,I$, Node  $k$, chooses ${\overline{b}}_k\left(1\right)$, with $\Vert {\overline{b}}_k\left(1\right)\Vert =1$ and sets ${\varsigma }_k\left(1\right)=1$ and ${\upsilon }_{i,j}^{\left(k\right)}\left(1\right)=1$, for all $i,j\in {\mathcal{N}}_k$. Then, at time $t\in \mathbb{N}$:

• (1)For each $j=1,\cdots ,I$ and $k\in {\mathcal{N}}_j$, Node  $j$ sends $({\Pi }_j^{1/2}{\overline{b}}_j\left(t\right),{\varsigma }_j\left(t\right))$ to Node  $k$.
• (2)
  On reception, for each $k=1,\cdots ,I$ and $i\in {\mathcal{O}}_k$, Node  $k$ sends $({\check{b}}_i^{\left(k\right)}\left(t\right),{\overline{\varsigma }}_i^{\left(k\right)}\left(t\right))$ to Node  $i$, where

  $${\check{b}}_i^{\left(k\right)}\left(t\right)=\sum _{j\in {\mathcal{O}}_k}\underset{i,j}{\overset{\left(k\right)}{\upsilon }}\left(t\right){\Psi }_{i,j}^{\left(k\right)}{\Pi }_j^{1/2}{\overline{b}}_j\left(t\right),{\overline{\varsigma }}_i^{\left(k\right)}\left(t\right)=\underset{j\in {\mathcal{N}}_k}{max}\underset{i,j}{\overset{\left(k\right)}{\upsilon }}\left(t\right)\text{,}$$

  and

  $${\upsilon }_{i,j}^{\left(k\right)}\left(t\right)=\frac{{\varsigma }_i\left(t\right)}{{\varsigma }_j\left(t\right)}{\upsilon }_{i,j}^{\left(k\right)}(t-1)\text{.}$$
• (3)
  On reception, for each $i=1,\cdots ,I$, Node  $i$ computes

  $${\overline{b}}_i(t+1)={\varsigma }_i(t+1){\tilde{b}}_i(t+1),{\varsigma }_i(t+1)=max{\{\Vert {\tilde{b}}_i(t+1)\Vert ,\underset{i}{\overset{\left(k\right)}{\overline{\varsigma }}}\left(t\right),k\in {\mathcal{I}}_i\}}^{-1}\text{,}$$

  with

  $${\tilde{b}}_i(t+1)={\Pi }_i^{1/2}\sum _{k\in {\mathcal{I}}_i}\underset{i}{\overset{\left(k\right)}{\check{b}}}\left(t\right)\text{.}$$ (16)

  Also, the estimate ${\overline{{\rm Y}}}_i\left(t\right)$ of $\Vert {\rm Y}\Vert$ is

  $${\overline{{\rm Y}}}_i\left(t\right)={\varsigma }_i{(t+1)}^{-1}\text{.}$$ (17)

The convergence of Algorithm 2 to $\Vert {\rm Y}\Vert$ is guaranteed by the next theorem.

Theorem 6

Consider the network   (1)   together with   Assumption 2, Assumption 5. Then, for each $i\in \{1,\dots ,I\}$,

$$\underset{t\to \infty }{lim}{\overline{{\rm Y}}}_i\left(t\right)=\Vert {\rm Y}\Vert \text{.}$$ (18)

Distributed method for finding${\Vert {{\rm Y}}^{-1}\Vert }^{-1}$:

Let $c\ge \Vert {\rm Y}\Vert$ and $\Phi =cI-{\rm Y}$. It follows that

$${\Vert {{\rm Y}}^{-1}\Vert }^{-1}=\underset{¯}{\mathrm{eig}}\left({\rm Y}\right)=c-\overline{\mathrm{eig}}\left(\Phi \right)=c-\Vert \Phi \Vert \text{,}$$

where, for a symmetric matrix $X$, $\underset{¯}{\mathrm{eig}}\left(X\right)$ and $\overline{\mathrm{eig}}\left(X\right)$ denote the smallest and largest eigenvalues of $X$, respectively. Hence, we can find ${\Vert {{\rm Y}}^{-1}\Vert }^{-1}$ by applying Algorithm 2 on $\Phi$, to find $\Vert \Phi \Vert$. To this end, at Node $i$ and time $t$, we choose $c={\overline{{\rm Y}}}_i\left(t\right)$. This leads to the following algorithm:

Algorithm 3 - distributed estimation of ${\Vert {{\rm Y}}^{-1}\Vert }^{-1}$: Apply Algorithm 2 with (16) replaced by

$${\tilde{b}}_i(t+1)={\overline{{\rm Y}}}_i\left(t\right){\overline{b}}_i\left(t\right)-{\Pi }_i^{1/2}\sum _{k\in {\mathcal{I}}_i}{\check{b}}_{i,k}\left(t\right)\text{,}$$

and  (17) replaced by

$${\underset{¯}{{\rm Y}}}_i\left(t\right)={\overline{{\rm Y}}}_i\left(t\right)-{\overline{\Phi }}_i\left(t\right),{\overline{\Phi }}_i\left(t\right)={\varsigma }_i{(t+1)}^{-1}\text{.}$$

3.1.2. Finite-time algorithm for $\gamma$

A sub-optimal design of $\gamma$ can be achieved using the following result.

Theorem 7

Condition   (6)   is satisfied by choosing $\gamma$ so that

$$0<\gamma <\frac{2}{\underset{i}{max}{\varphi }_i}\text{,}$$

where

$${\varphi }_i=\sum _{k\in {\mathcal{I}}_i}{\upsilon }_{i,k}\text{,}$$

$${\upsilon }_{i,k}=\sum _{j\in {\mathcal{O}}_k}\Vert \underset{i}{\overset{1/2}{\Pi }}{\Psi }_{i,j}^{\left(k\right)}{\Pi }_j^{1/2}\Vert \text{.}$$

The design of $\gamma$ using Theorem 7 requires the global information ${max}_i{\varphi }_i$. For each, $i=1,\dots ,I$, Node $i$ can obtain ${\varphi }_i$ from an initialization stage in which it receives ${\upsilon }_{i,k}$, from each Node $k$, with $k\in {\mathcal{I}}_i$. Then, ${max}_i{\varphi }_i$ can be obtained by running the max-consensus algorithm (Olfati-Saber & Murray, 2004), in parallel with the estimation Algorithm 1. Notice that the max-consensus algorithm converges in finite time.

3.2. Design of the preconditioning matrix $\Pi$

As mentioned above, for a given choice of ${\rm Y}$, the fastest convergence rate of Algorithm 1 is achieved when $\gamma$ is chosen as in (13). Under this choice of $\gamma$, we have that

$$\Vert I-\gamma {\rm Y}\Vert =\gamma \Vert {\rm Y}\Vert -1=\frac{2\Vert {\rm Y}\Vert }{\Vert {\rm Y}\Vert +{\Vert {{\rm Y}}^{-1}\Vert }^{-1}}-1=\frac{\Vert {\rm Y}\Vert -{\Vert {{\rm Y}}^{-1}\Vert }^{-1}}{\Vert {\rm Y}\Vert +{\Vert {{\rm Y}}^{-1}\Vert }^{-1}}=\frac{\kappa \left({\rm Y}\right)-1}{\kappa \left({\rm Y}\right)+1}\text{,}$$

where $\kappa \left({\rm Y}\right)=\Vert {\rm Y}\Vert \Vert {{\rm Y}}^{-1}\Vert$ denotes the condition number of ${\rm Y}$. Then, from (7), there exists $K\ge 0$, such that

$$\Vert \hat{x}-\hat{x}\left(t\right)\Vert \le K{\Vert I-\gamma {\rm Y}\Vert }^t=Kexp\{tlog\frac{\kappa \left({\rm Y}\right)-1}{\kappa \left({\rm Y}\right)+1}\}\text{,}$$

where we recall that $\hat{x}$ denotes the global estimate of $x$, given by (3). Then, we define the time constant $\tau \left({\rm Y}\right)$ of the distributed WLS algorithm by

$$\tau \left({\rm Y}\right)=\frac{1}{log\frac{\kappa \left({\rm Y}\right)+1}{\kappa \left({\rm Y}\right)-1}}\text{.}$$ (19)

Hence, a natural question is whether the preconditioning matrices ${\Pi }_i$, $i=1,\dots ,I$, can be chosen so that $\tau \left({\rm Y}\right)$ is minimized. While we are not able to answer this question, we have the following result, which follows using an argument similar to the one in Demmel (1983, Theorem 2).

Theorem 8

If${\Pi }_i={\Psi }_{i,i}^{-1}$, for all$i=1,\dots ,I$, then

$$\kappa \left({\rm Y}\right)\le \beta {\kappa }_{\star }\text{,}$$

where

$$\beta =\underset{i}{max}\left|{\mathcal{B}}_i\right|\text{,}$$

$${\kappa }_{\star }=\underset{\tilde{\Pi }\in \mathcal{P}}{min}\kappa \left({\tilde{\Pi }}^{1/2}\Psi {\tilde{\Pi }}^{1/2}\right)\text{,}$$

 with  $\mathcal{P}$ denoting the set of positive definite block diagonal matrices of the form   (4).

Theorem 8 states that, if the preconditioning matrices ${\Pi }_i$, $i=1,\dots ,I$, are chosen as

$${\Pi }_i={\Psi }_{i,i}^{-1}$$ (20)

then $\kappa \left({\rm Y}\right)$ is at most $\beta$ times bigger than the smallest possible value ${\kappa }_{\star }$ achievable using block diagonal preconditioning matrices. Notice that ${\mathcal{B}}_i=\{j:{\mathcal{I}}_i\cap {\mathcal{I}}_j\ne 0̸\}\subseteq \{j:{\mathcal{N}}_i\cap {\mathcal{N}}_j\ne 0̸\}$. Hence, $\beta$ is bounded by the maximum number of two-hop neighbors over the whole network. Hence, it does not necessarily grow with the network size.

Now, we have

$$\underset{\kappa \to \infty }{lim}\kappa log\left(\frac{\kappa +1}{\kappa -1}\right)=2\text{.}$$

Hence, from Theorem 8, for large $\kappa \left({\rm Y}\right)$, we have

$$\tau \left({\rm Y}\right)\simeq \frac{\kappa \left({\rm Y}\right)}{2}\le \frac{\beta }{2}\underset{\tilde{\Pi }}{min}\kappa \left({\tilde{\Pi }}^{1/2}\Psi {\tilde{\Pi }}^{1/2}\right)\simeq \beta {\tau }_{\star }\text{,}$$ (21)

where

$${\tau }_{\star }=\underset{\tilde{\Pi }\in \mathcal{P}}{min}\tau \left({\tilde{\Pi }}^{1/2}\Psi {\tilde{\Pi }}^{1/2}\right)\text{.}$$

Hence, if ${\Pi }_i$, $i=1,\dots ,I$, are chosen as in (20), and $\kappa \left({\rm Y}\right)$ is large, then the time constant $\tau \left({\rm Y}\right)$ is at most $\beta$ times bigger than the minimum value ${\tau }_{\star }$.

Remark 9

In view of (20), (10),

$${\Pi }_i={\left(\sum _{k\in {\mathcal{I}}_i}\underset{i,i}{\overset{\left(k\right)}{\Psi }}\right)}^{-1}\text{.}$$

Hence, its computation requires the matrices ${\Psi }_{i,i}^{\left(k\right)}$, $k\in {\mathcal{I}}_i$, to be transmitted from Node $k$ to Node $i$ during an initialization stage.

4. Finite-time method for WLS estimation

In this method we replace the definition of neighborhood by the following one:

Definition 10

Node $j$ is a neighbor of Node $i$ if $j\in {\mathcal{B}}_i$.

Consequently, we replace the connectivity Assumption 5 by the following one:

Assumption 11

For each $i=1,\dots ,I$, Node $i$ can send/receive information to/from all its neighbors. Also, ${\Psi }_{j,i}$, for all $j\in {\mathcal{B}}_i$, and ${\alpha }_i$ are available at Node $i$.

To illustrate the idea behind the proposed algorithm, we consider a network with two nodes. The next lemma states how to obtain the global optimal solution, at each node, in this simple case.

Lemma 12

Consider the network   (1)   together with   Assumption  2. If there are only two nodes, labeled by $a$ and $b$, then ${\Psi }_{a,a}-{\Psi }_{a,b}{\check{\Sigma }}_b{\Psi }_{b,a}$ is an invertible matrix and the global estimate ${\hat{x}}_a$ of the components $x_a$ associated to Node $a$ is given by

$${\hat{x}}_a={\Sigma }_a({\alpha }_a-{\Psi }_{a,b}{\check{x}}_b)\text{,}$$

$${\Sigma }_a={({\Psi }_{a,a}-{\Psi }_{a,b}{\check{\Sigma }}_b{\Psi }_{b,a})}^{-1}\text{,}$$

where

$${\check{x}}_b={\check{\Sigma }}_b{\alpha }_b\text{,}$$

$${\check{\Sigma }}_b={\Psi }_{b,b}^{-1}\text{.}$$

Our next result is an immediate generalization of the one above, to a network with a star topology, i.e., in which all nodes are only possibly connected to a single one.

Lemma 13

Consider the network   (1)   together with   Assumption  2. Suppose that ${\Psi }_{j,k}=0$, for all $j,k\in \{1,\dots ,I\}\setminus \left\{i\right\}$ and $j\ne k$ (i.e., all nodes are only possibly connected to Node $i$ ). Then ${\Psi }_{i,i}-{\sum }_{j\in {\mathcal{B}}_i\setminus \left\{i\right\}}{\Psi }_{i,j}{\check{\Sigma }}_j{\Psi }_{j,i}$ is an invertible matrix and ${\hat{x}}_i$ is given by

$${\hat{x}}_i={\Sigma }_i({\alpha }_i-\sum _{j\in {\mathcal{B}}_i\setminus \left\{i\right\}}{\Psi }_{i,j}{\check{x}}_j)\text{,}$$

$${\Sigma }_i={({\Psi }_{i,i}-\sum _{j\in {\mathcal{B}}_i\setminus \left\{i\right\}}{\Psi }_{i,j}{\check{\Sigma }}_j{\Psi }_{j,i})}^{-1}\text{,}$$

where

$${\check{x}}_j={\check{\Sigma }}_j{\alpha }_j\text{,}$$

$${\check{\Sigma }}_j={\Psi }_{j,j}^{-1}\text{.}$$

Then, using (11), (9) and Lemma 13, we obtain the following algorithm:

Algorithm 4 - distributed WLS estimation:

Initialization: For each $i=1,\cdots ,I$,

• (1)
  Node  $i$ computes

  $${\check{x}}_i\left(0\right)={\check{\Sigma }}_i\left(0\right){\alpha }_i,{\check{\Sigma }}_i\left(0\right)={\Psi }_{i,i}^{-1}\text{.}$$

  and for each $j\in {\mathcal{B}}_i\setminus \left\{i\right\}$,

  $${\check{x}}_{i,j}\left(0\right)={\check{x}}_i\left(0\right),{\check{\Sigma }}_{i,j}\left(0\right)={\check{\Sigma }}_i\left(0\right)\text{.}$$

Main loop: For each $i=1,\cdots ,I$, and time $t\in \mathbb{N}$

• (1)
  Node  $i$ computes, for each $j\in {\mathcal{B}}_i\setminus \left\{i\right\}$,

  $${\gamma }_{i,j}\left(t\right)={\Psi }_{j,i}{\check{x}}_i(t-1),{\Gamma }_{i,j}\left(t\right)={\Psi }_{j,i}{\check{\Sigma }}_i(t-1){\Psi }_{i,j}\text{,}$$

  and sends $({\gamma }_{i,j}\left(t\right),{\Gamma }_{i,j}\left(t\right))$ to Node  $j$.
• (2)
  Node  $i$ computes

  $${\check{x}}_i\left(t\right)={\check{\Sigma }}_i\left(t\right)({\alpha }_i-\sum _{j\in {\mathcal{B}}_i\setminus \left\{i\right\}}{\gamma }_{j,i}(t-1)),{\check{\Sigma }}_i\left(t\right)={({\Psi }_{i,i}-\sum _{j\in {\mathcal{B}}_i\setminus \left\{i\right\}}{\Gamma }_{j,i}(t-1))}^{-1}\text{,}$$

  and, for each $j\in {\mathcal{B}}_i\setminus \left\{i\right\}$,

  $${\check{x}}_{i,j}\left(t\right)={\check{\Sigma }}_{i,j}\left(t\right)({\alpha }_i-\sum _{j\in {\mathcal{B}}_i\setminus \{i,j\}}{\gamma }_{j,i}(t-1)),{\check{\Sigma }}_{i,j}\left(t\right)={({\Psi }_{i,i}-\sum _{j\in {\mathcal{B}}_i\setminus \{i,j\}}{\Gamma }_{j,i}(t-1))}^{-1}\text{.}$$

Our next step is to show that Algorithm 4 converges in finite time to the global WLS solution.

Definition 14

Each pair $(i,j)$, $i,j\in \{1,\dots ,I\}$, is called an edge if ${\Psi }_{i,j}\ne 0$. A path is a concatenation of contiguous edges, and its length is the number of edges forming it. For each $i,j\in \{1,\dots ,I\}$, the distance $d_{i,j}$ between Nodes $i$ and $j$ is defined as the minimum length of a path joining these two nodes. The radius ${\rho }_i$ of Node $i$ is defined as the maximum distance between Node $i$ and any other node in the network. The diameter of the network is the maximum radius between all its nodes. A network is called acyclic if it does not contain a path forming a cycle.

The next theorem states that, if the network is acyclic, then the algorithm above yields the global estimate at each node in finite time.

Theorem 15

Consider the network   (1)   together with   Assumption 2, Assumption 11. If the network is acyclic, then, for each $i\in \{1,\dots ,I\}$, $j\in {\mathcal{B}}_i\setminus \left\{i\right\}$ and $t\in \mathbb{N}$, the matrices ${\Psi }_{i,i}-{\sum }_{j\in {\mathcal{B}}_i\setminus \left\{i\right\}}{\Gamma }_{j,i}(t-1)$ and ${\Psi }_{i,i}-{\sum }_{j\in {\mathcal{B}}_i\setminus \{i,j\}}{\Gamma }_{j,i}(t-1)$ are invertible, and for all $t\ge {\rho }_i$,

$${\check{x}}_i\left(t\right)={\hat{x}}_i\text{.}$$ (22)

5. Simulations

5.1. State estimation in power systems

In the first simulation we use the proposed distributed methods for state estimation in smart electricity networks, involving multi-area interconnected power systems (Huang et al., 2012). To this end, we use the IEEE 118-bus test system, whose specifications are given in Christie (1993). The system’s diagram is shown in Fig. 1, where buses are represented by circles and lines by edges. Some buses have a phasor measurement unit (PMU) installed. These buses are shown in gray. Each PMU measures the voltage of the bus where it is installed, as well as the currents of the lines attached to that bus. The goal is to estimate the state vector $x$, containing voltage (a complex phasor) at each bus. For the purposes of state estimation, the buses are clustered in nodes. Two clustering examples as shown in Table 1, Table 3.

Fig. 1.

Diagram of the IEEE 118-bus test system.

Table 1.

Nodes forming a cyclic network topology.

Node	Buses
1	1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 16, 117
2	13, 15, 17, 18, 19, 20, 21, 22, 23, 25, 26, 27, 28, 29, 30, 31, 32, 33, 113, 114, 115
3	24, 38, 70, 71, 72, 73, 74
4	34, 35, 36, 37, 39, 40, 41, 42, 43
5	44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 77, 80, 81, 100, 116
6	75, 76, 78, 79, 82, 95, 96, 97, 98, 118
7	83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94
8	99, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112

Table 3.

Nodes forming an acyclic network topology.

Node	Buses
1	1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 117
2	23, 25, 26, 27, 28, 29, 31, 32, 113, 114, 115
3	5, 16, 17, 18, 19, 20, 21, 22, 24, 30, 33, 34, 35, 36, 37, 39, 40, 71, 72, 73
4	38, 41, 42, 43, 44, 45, 46, 47, 48, 69, 70, 74, 75, 76, 77, 118
5	49, 50, 51, 54, 65, 66, 68, 78, 79, 80, 81, 82, 95, 96, 97, 98, 99, 116
6	52, 53, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 67
7	83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112

Let $P$ denote the number of PMUs in the whole system. For each $p=1,\dots ,P$, let $L_p$ denote the number of lines attached to the bus where PMU  $p$ is installed. Let also $B_p^T=[e_p^T,y_{p,1}^T,\dots ,y_{p,L_p}^T]$, where the vectors $e_p$ and $y_{p,l}$, $l=1,\dots ,L_p$, are defined such that $e_{p}x$ is the voltage of the installation bus, and $y_{p,l}x$ is the current of the $p$-th attached line (the value of $y_{p,l}$ is taken from Christie, 1993). Then, matrix $A$ in (2) is given by $A^T=[A_1^T,\dots ,A_I^T]$, where, for each $i=1,\dots ,I$, the block of rows $A_i$ corresponding to Node $i$ is formed by stacking the matrices $B_p$ corresponding to all PMUs contained in Node $i$, i.e., $A_i^T=[B_{p_1}^T,\dots ,B_{p_{P_i}}^T]$, where $p_1,\dots ,p_{P_i}$ denote the indexes of those PMUs.

We place the PMUs using the method in Tai et al. (2013). This guarantees that matrix $A$ has full column rank. We also assume that the noise covariance is $R={\sigma }^{2}I$, with $\sigma =0.05$. Notice that voltage and current values in the test system are per unit values, i.e., they appear divided by the nominal voltage $V_0$ and the nominal current $I_0$, respectively. Hence, $\sigma =0.05$ means that voltage measurements have a standard deviation of $0.05\times V_0$ volts, and current measurements have one of $0.05\times I_0$ amperes. This leads to a global estimate $\hat{x}$ having a relative estimation error of

$$e=20{log}_{10}\frac{\Vert x-\hat{x}\Vert }{\Vert x\Vert }=-17.45\text{dB .}$$ (23)

In the simulations below, we use

$$r\left(t\right)=20{log}_{10}\frac{\Vert \hat{x}-\hat{x}\left(t\right)\Vert }{\Vert \hat{x}\Vert }\text{,}$$ (24)

to measure the relative difference between the global estimate $\hat{x}$ and the one yielded, at time step $t$, by the proposed distributed algorithms.

5.1.1. Cyclic network topology

In the first simulation we cluster the buses into eight nodes, as shown in Table 1. From the definition of ${\mathcal{N}}_i$, it follows that $j\in {\mathcal{N}}_i$ if there is a bus in either, Node $i$ or $j$, with a PMU installed, having an attached line coming from a bus inside the other node. Fig. 2 shows the topology of the communication network induced by the clustering given in Table 1.

Fig. 2.

Cyclic network topology induced by the nodes in Table 1.

Fig. 3 shows the convergence of the asymptotic method without preconditioning. To this end, we show the modulus of the estimated voltage of each bus at each step. We see that the convergence is very slow, with a relative difference of $r\left({10}^6\right)=-37\mathrm{dB}$, between the global estimate and the one obtained by the distributed algorithm at $t={10}^6$. The reason for the slow convergence is that the condition number of $\Psi$ is 478 972. The preconditioning matrix in (20) gives a condition number of 700, which leads to a much faster convergence. This is shown in Fig. 4, where $r(2\times {10}^3)=-52.47\mathrm{dB}$. Fig. 5 shows that the convergence of the estimation of $\Vert {\rm Y}\Vert$ and ${\Vert {{\rm Y}}^{-1}\Vert }^{-1}$, at each node, is much faster than that of the WLS estimation algorithm. Finally, Table 2 shows the complexity at each node. To this end, we measure the number of multiplications in a whole cycle of Algorithms 1–3.

Fig. 3.

Convergence of the asymptotic method, without preconditioning, in a cyclic network.

Fig. 4.

Convergence of the asymptotic method, with preconditioning, in a cyclic network.

Fig. 5.

Convergence of the distributed eigenvalue estimation algorithm.

Table 2.

Complexity at each node, in number of multiplications per iteration, for the cyclic network topology.

Node	1	2	3	4	5	6	7	8
Complexity	5202	8613	10 469	14 848	26 054	11 810	14 352	10 628

5.1.2. Acyclic network topology

In the second simulation we do the clustering such that the induced topology is acyclic. From the definition of ${\mathcal{B}}_i$, it follows that $j\in {\mathcal{B}}_i$ if there is a bus (possibly neither in Node $i$ nor in  $j$), with a PMU installed, having one neighbor bus (i.e., a bus connected to it via an attached line), including possibly itself, in each node, $i$ and $j$. The clustering and its induced topology are shown in Table 3 and Fig. 6, respectively.

Fig. 6.

Acyclic network topology induced by the nodes in Table 3.

The convergence of the asymptotic method (with preconditioning) is shown in Fig. 7, with a final relative difference with the global estimate of $r(20\times {10}^3)=-69.67\mathrm{dB}$. In this case, we wait for 100 steps before starting with the distributed estimation. The gap caused by this delay can be seen at the beginning of the graph. We introduced this late start so as to give time for Algorithm 2 and 3 to obtain reasonable approximations of $\Vert {\rm Y}\Vert$ and ${\Vert {{\rm Y}}^{-1}\Vert }^{-1}$, respectively. We see that the asymptotic method presents an oscillating behavior between time steps 1500 and 3500. This is because the transients in the estimation of the scaling factor $\gamma \left(t\right)$ cause the recursions (8) to become temporarily unstable. We also see that the asymptotic method requires about 20×10³ steps to converge. This is because in this case, preconditioning leads to a condition number of 5264. On the other hand, the convergence of the finite-time method does not depend on the condition number, but on the network diameter, which in this case is four. Fig. 8 shows the convergence of this method in four steps, with a final error of $r\left(4\right)=-223.7\mathrm{dB}$, caused by numerical inaccuracy.

Fig. 7.

Convergence of the asymptotic method (with preconditioning) in an acyclic network.

Fig. 8.

Convergence of the finite-time method (in an acyclic network).

Table 4 shows the complexity at each node. To this end, we consider that solving the positive-definite linear system for computing ${\check{x}}_{i,j}\left(t\right)$, using the Cholesky decomposition, requires $n^3/3+2n^2$ multiplications ($n^3/3$ for the decomposition and $2n^2$ for solving two triangular linear systems). Also, computing the inverse of the matrix ${\check{\Sigma }}_i\left(t\right)$, using also the Cholesky Decomposition, requires $n^3/2$ multiplications (Krishnamoorthy & Menon, 2011).

Table 4.

Complexity at each node, in a number of multiplications per iteration, for the acyclic network topology.

Node	Asymptotic method	Finite-time method
1	4985	1912
2	3645	786
3	14 088	4400
4	10 400	2304
5	17 802	3240
6	3891	1267
7	8919	8437

5.2. Sensor localization

Sensor localization refers to the problem of obtaining the locations of each node in a network, based on the knowledge of the locations of a few anchor nodes, a well as the mutual distances between neighbor nodes. A distributed method for carrying out this task is proposed in Khan et al. (2009). This method requires that, for each $i=1,\dots ,I$, Node $i$ lies inside of at least one triangle defined by three of its neighbors ${\mathcal{N}}_i=\{j,k,l\}$. Then, the coordinates $x_i$ of Node $i$ can be written as

$$x_i=\sum _{j\in {\mathcal{N}}_i}{c}_{i,j}{x}_{i,j}\text{,}$$ (25)

where the barycentric coordinates $c_{i,j}$ are given by

$$c_{i,j}=\frac{S(i\cup {\mathcal{N}}_i\setminus j)}{S\left({\mathcal{N}}_i\right)}\text{,}$$

with $S(i,j,k)$ denoting the area of the triangle formed by Nodes $i$, $j$ and $k$. The latter can be computed using the Cayley–Menger determinant as follows

$$S^2(i,j,k)=-\frac{1}{16}\left|\begin{array}{cccc}\hfill 0\hfill & \hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill \\ \hfill 1\hfill & \hfill 0\hfill & \hfill d_{i,j}^2\hfill & \hfill d_{i,k}^2\hfill \\ \hfill 1\hfill & \hfill d_{i,j}^2\hfill & \hfill 0\hfill & \hfill d_{j,k}^2\hfill \\ \hfill 1\hfill & \hfill d_{i,k}^2\hfill & \hfill d_{j,k}^2\hfill & \hfill 0\hfill \end{array}\right|\text{,}$$

where $d_{i,j}=\Vert x_i-x_j\Vert$ denotes the distance between Nodes $i$and  $j$.

For each $i=1,\dots ,I$, we have one equation of the form (25), for each triangle containing Node $i$. We assume that $N$ such triangles exist for each node. Hence, we have $N\times I$ equations. Let $x_i\in {\mathbb{R}}^2$, $i=1,\dots ,I$, denote the node coordinates and $a_j\in {\mathbb{R}}^2$, $j=1,\dots ,J$, denote those of the anchor nodes. Let also $x^T=[x_1^T,\dots ,x_I^T]$ and $a^T=[a_1^T,\dots ,a_J^T]$. Then, the aforementioned $N\times I$ equations can be written as

$$x=(C\otimes I_2)x+(D\otimes I_2)a\text{,}$$

or equivalently,

$$y=Ax\text{,}$$ (26)

with $y=(D\otimes I_2)a$ and $A=I-C\otimes I_2$. Due to inaccuracy in distance measurements, (26) can be approximately expressed as in (2). In that case, we can use our proposed distributed method to obtain, at each node, a WLS estimation of its coordinates.

The experiment setup is shown in Fig. 9. It includes three anchor nodes, defining a triangle containing $I=20$ randomly placed nodes. We use a noise covariance matrix $R={\sigma }^2{I}_d$, where $I_d$ denotes the identity matrix, and $\sigma =\sqrt{10^{-3}}\simeq 31.62$ centimeters. With this setup, the global estimate $\hat{x}$ yields a relative localization error of $e=-33.39\mathrm{dB}$, defined as in (23). The convergence of the coordinate estimates at each node, using the proposed method, with preconditioning, is shown in Fig. 10. As before, we wait for 10 steps before starting the iterations, to give time for Algorithms 2 and 3 to obtain reasonable approximations of $\Vert {\rm Y}\Vert$ and ${\Vert {{\rm Y}}^{-1}\Vert }^{-1}$, respectively. The convergences of these estimates are shown in Fig. 11. Finally, the complexity at each node is shown in Table 5.

Fig. 9.

Node positions and estimates.

Fig. 10.

Convergence of the node coordinate estimates.

Fig. 11.

Convergence of the estimated eigenvalues.

Table 5.

Complexity at each sensor node, in a number of multiplications per iteration.

Node	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20
Complexity	170	282	282	426	170	282	602	602	282	426	282	426	282	170	170	426	426	426	426	426

For comparison, we also consider the distributed iterative localization algorithm (DILOC) proposed in Khan et al. (2009). This method solves (26) using Richardson’s recursions to invert matrix $A$. This requires that $N=1$, i.e., only one equation of the form (25) is considered for each node. In this case, the recursions are guaranteed to converge because, as the authors show, $\Vert I-A\Vert <1$ holds in this problem. Fig. 12 shows the evolution of the relative difference $r\left(t\right)$ (defined as in (24)) between the estimates of each method, and the global estimate $\hat{x}$. We see that the DILOC method has a faster convergence. This is because the condition number of $A$ is smaller than that of $A^{\ast }{R}^{-1}A$, which is the matrix inverted by our proposed method. However, at $t=500$, the DILOC method yields $r\left(500\right)=-29.44\mathrm{dB}$, while the proposed one gives $r\left(500\right)=-72.71\mathrm{dB}$. This difference results from the fact that the DILOC method does not produce the WLS solution on the limit.¹

Fig. 12.

Relative difference with the global estimate $\hat{x}$ vs. iteration $t$.

6. Conclusion

We proposed two methods for weighted least squares estimation in large-scale systems. Both methods converge to the global solution and aim to maximize the convergence speed. The first method converges asymptotically and involves a distributed estimation of the scaling parameter upon which the convergence speed depends. To further speed up the convergence, we also use a practically feasible preconditioning method, for which we bounded the speed difference with respect to the fastest theoretically achievable. The second proposed method has an even faster convergence, as it achieves the global optimal in finite time. However, it is only suitable for applications where the graph produced by the communication network contains no loops.

Biographies

Damián Edgardo Marelli received his Bachelors Degree in Electronics Engineering from the Universidad Nacional de Rosario, Argentina, in 1995, a Ph.D. degree in Electrical Engineering and a Bachelor (Honous) degree in Mathematics from the University of Newcastle, Australia, in 2003. From 2004 to 2005 he held a postdoctoral position at the Laboratoire d’Analyse Topologie et Probabilités, CNRS/Université de Provence, France. Since 2006 he is Research Academic at the School of Electrical Engineering and Computer Science at the University of Newcastle, Australia. In 2007 he received a Marie Curie Postdoctoral Fellowship, hosted at the Faculty of Mathematics, University of Vienna, Austria, and in 2010 he received a Lise Meitner Senior Fellowship, hosted at the Acoustics Research Institute of the Austrian Academy of Sciences. His main research interests include signal processing and communications.

Minyue Fu received his Bachelor’s Degree in Electrical Engineering from the University of Science and Technology of China, Hefei, China, in 1982, and M.S. and Ph.D. degrees in Electrical Engineering from the University of Wisconsin-Madison in 1983 and 1987, respectively. From 1983 to 1987, he held a teaching assistantship and a research assistantship at the University of Wisconsin-Madison. He worked as a Computer Engineering Consultant at Nicolet Instruments, Inc., Madison, Wisconsin, during 1987. From 1987 to 1989, he served as an Assistant Professor in the Department of Electrical and Computer Engineering, Wayne State University, Detroit, Michigan. He joined the Department of Electrical and Computer Engineering, the University of Newcastle, Australia, in 1989. Currently, he is a Chair Professor in Electrical Engineering and Head of School of Electrical Engineering and Computer Science. In addition, he was a Visiting Associate Professor at University of Iowa in 1995–1996, and a Senior Fellow/Visiting Professor at Nanyang Technological University, Singapore, 2002. He holds a Qian-ren Professorship at Zhejiang University, China. He is a Fellow of IEEE. His main research interests include control systems, signal processing and communications. He has been an Associate Editor for the IEEE Transactions on Automatic Control, Automatica and Journal of Optimization and Engineering.

Appendix A. Proofs of Section  3

A.1. Proof of Theorem 6

Fix $t\in \mathbb{N}$. Let $k_i\left(t\right)\in \mathbb{R}$ and

$${\overline{b}}_i\left(t\right)=k_i\left(t\right)b_i\left(t\right)\text{.}$$ (A.1)

From (14), we have

$${\overline{b}}_i(t+1)=\frac{k_i(t+1)}{k_i\left(t\right)}{\Pi }_i^{1/2}\sum _{k\in {\mathcal{I}}_i}\sum _{j\in {\mathcal{O}}_k}\frac{k_i\left(t\right)}{k_j\left(t\right)}\underset{i,j}{\overset{\left(k\right)}{\Psi }}\underset{j}{\overset{1/2}{\Pi }}{\overline{b}}_j\left(t\right)\text{.}$$

Let $k_i\left(0\right)=1$ and

$${\varsigma }_i\left(t\right)=\frac{k_i\left(t\right)}{k_i(t-1)}\text{,}$$

so that

$$k_i\left(t\right)=\prod _{\tau =1}^t{\varsigma }_i\left(\tau \right)\text{.}$$

Then,

$${\overline{b}}_i(t+1)={\varsigma }_i(t+1){\tilde{b}}_i(t+1)\text{,}$$ (A.2)

with

$${\tilde{b}}_i(t+1)={\Pi }_i^{1/2}\sum _{k\in {\mathcal{I}}_i}\sum _{j\in {\mathcal{O}}_k}{\upsilon }_{i,j}\left(t\right)\underset{i,j}{\overset{\left(k\right)}{\Psi }}\underset{j}{\overset{1/2}{\Pi }}{\overline{b}}_j\left(t\right)\text{,}$$

$${\upsilon }_{i,j}\left(t\right)=\prod _{\tau =1}^t\frac{{\varsigma }_i\left(\tau \right)}{{\varsigma }_j\left(\tau \right)}\text{.}$$

We need to design $k_i(t+1)$, or equivalently ${\varsigma }_i(t+1)$, to avoid the indefinite increase or decrease of $b\left(t\right)$. In principle, this could be achieved by choosing

$$k_i\left(t\right)={\Vert b_i\left(t\right)\Vert }^{-1}\text{,}$$

so that $\Vert {\overline{b}}_i\left(t\right)\Vert =1$, for all $t\in \mathbb{N}$. From (A.2), this would lead to

$${\varsigma }_i(t+1)={\Vert {\tilde{b}}_i(t+1)\Vert }^{-1}\text{.}$$

However, the question then arises as to whether some of the scalars ${\upsilon }_{i,j}\left(t\right)$ would grow to infinity. Notice that

$${\upsilon }_{i,j}\left(t\right)=\frac{k_i\left(t\right)}{k_j\left(t\right)}\text{.}$$

Hence, this could only happen if some vector in the eigenspace associated to the largest eigenvalue of ${\rm Y}$ has zero components in the entries corresponding to $b_j\left(t\right)$. We call a matrix satisfying this property, ill-posed. Although the set of ill-posed matrices is nowhere dense, (i.e., it is unlikely to have an ill-posed matrix ${\rm Y}$), we can avoid the indefinite growth of ${\upsilon }_{i,j}\left(t\right)$ by choosing ${\varsigma }_i(t+1)$ so that $\Vert {\overline{b}}_i(t+1)\Vert \le 1$ and, for all $j\in {\mathcal{B}}_i=\{j:{\Psi }_{i,j}\ne 0\}$,

$${\varsigma }_i(t+1){\upsilon }_{i,j}\left(t\right)\le 1\text{.}$$

This leads to

$${\varsigma }_i(t+1)=min\{{\Vert {\tilde{b}}_i(t+1)\Vert }^{-1},\underset{i,j}{\overset{-1}{\upsilon }}\left(t\right),j\in {\mathcal{B}}_i\}=max{\{\Vert {\tilde{b}}_i(t+1)\Vert ,{\upsilon }_{i,j}\left(t\right),j\in {\mathcal{B}}_i\}}^{-1}\text{.}$$

From (15), (A.1), the estimate ${\overline{{\rm Y}}}_i\left(t\right)$ of $\Vert {\rm Y}\Vert$ at $t$ is

$${\overline{{\rm Y}}}_i\left(t\right)=\frac{\Vert b_i\left(t\right)\Vert }{\Vert b_i(t-1)\Vert }={\varsigma }_i^{-1}\left(t\right)\frac{\Vert {\overline{b}}_i\left(t\right)\Vert }{\Vert {\overline{b}}_i(t-1)\Vert }\text{.}$$

However, if ${\rm Y}$ is ill-posed, $\Vert {\overline{b}}_i\left(t\right)\Vert$ will tend to zero. In such case, ${\overline{{\rm Y}}}_i\left(t\right)$ can be computed by

$${\overline{{\rm Y}}}_i\left(t\right)={\varsigma }_j^{-1}\left(t\right)\frac{\Vert {\overline{b}}_j\left(t\right)\Vert }{\Vert {\overline{b}}_j(t-1)\Vert }\text{,}$$

for some neighbor node $j$ for which $\Vert {\overline{b}}_j\left(t\right)\Vert$ does not tend to zero. Notice that such a neighbor always exists, for otherwise Node $i$ would be isolated from all other nodes.

A.2. Proof of Theorem 7

We need use the following result.

Lemma 16

Let$M={\left[M_{i,j}\right]}_{i,j=1,\dots ,I}$be a block symmetric matrix. Then

$$\Vert M\Vert \le \underset{i}{max}{\psi }_i\text{,}$$

where

$${\psi }_i=\sum _{j=1}^I\Vert M_{i,j}\Vert \text{.}$$

Proof

Let $y=Mx$, with $x=[x_1,\dots ,x_I]$ and $y=[y_1,\dots ,y_I]$. We have

$$\Vert y_i\Vert \le \sum _{j=1}^I\Vert M_{i,j}\Vert \Vert x_j\Vert =\sum _{j=1}^I{\Vert M_{i,j}\Vert }^{1/2}{\left(\Vert M_{i,j}\Vert {\Vert x_j\Vert }^2\right)}^{1/2}\le {\left(\sum _{j=1}^I\Vert M_{i,j}\Vert \right)}^{1/2}{\left(\sum _{j=1}^I\Vert M_{i,j}\Vert {\Vert x_j\Vert }^2\right)}^{1/2}\text{.}$$

Let $\psi ={max}_i{\psi }_i$. Then,

$${\Vert y\Vert }^2=\sum _{i=1}^I{\Vert y_i\Vert }^2\le \sum _{i=1}^I\left(\sum _{j=1}^I\Vert M_{i,j}\Vert \right)\left(\sum _{j=1}^I\Vert M_{i,j}\Vert {\Vert x_j\Vert }^2\right)\le \psi \sum _{i=1}^I\sum _{j=1}^I\Vert M_{i,j}\Vert {\Vert x_j\Vert }^2\le {\psi }^2\sum _{j=1}^I{\Vert x_j\Vert }^2={\psi }^2{\Vert x\Vert }^2\text{,}$$

and the result follows.

Proof of Theorem 7

Using Lemma 16 we have

$$\Vert {\rm Y}\Vert \le \underset{i}{max}{\psi }_i\text{,}$$

with

$${\psi }_i=\sum _{j=1}^I\Vert {\Pi }_i^{1/2}{\Psi }_{i,j}{\Pi }_j^{1/2}\Vert =\sum _{j=1}^I\Vert \sum _{k:i,j\in {\mathcal{O}}_k}\underset{i}{\overset{1/2}{\Pi }}{\Psi }_{i,j}^{\left(k\right)}{\Pi }_j^{1/2}\Vert \le \sum _{j=1}^I\sum _{k:i,j\in {\mathcal{O}}_k}\Vert \underset{i}{\overset{1/2}{\Pi }}{\Psi }_{i,j}^{\left(k\right)}{\Pi }_j^{1/2}\Vert =\sum _{k\in {\mathcal{I}}_i}\sum _{j\in {\mathcal{O}}_k}\Vert \underset{i}{\overset{1/2}{\Pi }}\underset{i,j}{\overset{\left(k\right)}{\Psi }}\underset{j}{\overset{1/2}{\Pi }}\Vert \text{.}$$

Hence,

$$\Vert {\rm Y}\Vert \le \underset{i}{max}{\varphi }_i\text{,}$$

and the result follows.

A.3. Proof of Theorem 8

We need the following lemma.

Lemma 17

If$M=\left[\begin{array}{cc}\hfill I\hfill & \hfill C\hfill \\ \hfill C^{\ast }\hfill & \hfill I\hfill \end{array}\right]\ge 0$, then$\Vert C\Vert \le 1$.

Proof

Let $u^T=\left[\begin{array}{cc}\hfill x^T\hfill & \hfill y^T\hfill \end{array}\right]$. Then, for any $u$,

$$u^{\ast }Mu={\Vert x\Vert }^2+{\Vert y\Vert }^2+2y^{T}Cx\ge 0\text{.}$$

Choose $x$ and $y$ such that $\Vert x\Vert =\Vert y\Vert =1$ and $y^{\ast }Cx=-\Vert C\Vert$. Then, the inequality above becomes $1-\Vert C\Vert \ge 0$, and the result follows.

Proof of Theorem 8

Recall that $\Pi =\mathrm{diag}\{{\Pi }_1,\dots ,{\Pi }_I\}$ and let $D=\tilde{\Pi }{\Pi }^{-1}$, with $\tilde{\Pi }\in \mathcal{P}$. Then,

$${\tilde{\Pi }}^{1/2}\Psi {\tilde{\Pi }}^{1/2}=D^{1/2}{\rm Y}{D}^{1/2}\text{,}$$

and we have

$$\kappa \left({\tilde{\Pi }}^{1/2}\Psi {\tilde{\Pi }}^{1/2}\right)=\kappa \left(D^{1/2}{\rm Y}{D}^{1/2}\right)={\kappa }^2\left({\left({\rm Y}D\right)}^{1/2}\right)\text{.}$$ (A.3)

Let $\overline{\sigma }\left(A\right)$ and $\underset{¯}{\sigma }\left(A\right)$ denote the largest and smaller singular values of $A$, respectively. Now

$$\overline{\sigma }\left({\left({\rm Y}D\right)}^{1/2}\right)=\underset{x\ne 0}{max}\frac{\Vert {\left({\rm Y}D\right)}^{1/2}x\Vert }{\Vert x\Vert }=\underset{x\ne 0}{max}\frac{\Vert {{\rm Y}}^{1/2}x\Vert }{\Vert D^{-1/2}x\Vert }\ge \frac{\Vert {{\rm Y}}^{1/2}{x}_0\Vert }{\Vert D^{-1/2}{x}_0\Vert }\text{,}$$

for any $x_0\ne 0$. Similarly,

$$\underset{¯}{\sigma }\left({\left({\rm Y}D\right)}^{1/2}\right)\le \frac{\Vert {{\rm Y}}^{1/2}{y}_0\Vert }{\Vert D^{-1/2}{y}_0\Vert }\text{,}$$

for any $y_0\ne 0$. Let $x_0$ and $y_0$ have unit norm and be such that $D^{-1/2}{x}_0=\underset{¯}{\sigma }\left(D^{-1/2}\right)x_0$ and ${{\rm Y}}^{1/2}{y}_0=\underset{¯}{\sigma }\left({{\rm Y}}^{1/2}\right)y_0$. Then,

$$\kappa \left({\left({\rm Y}D\right)}^{1/2}\right)=\frac{\overline{\sigma }\left({\left({\rm Y}D\right)}^{1/2}\right)}{\underset{¯}{\sigma }\left({\left({\rm Y}D\right)}^{1/2}\right)}\ge \frac{\Vert D^{-1/2}{y}_0\Vert }{\underset{¯}{\sigma }\left(D^{-1/2}\right)}\frac{\Vert {{\rm Y}}^{1/2}{x}_0\Vert }{\underset{¯}{\sigma }\left({{\rm Y}}^{1/2}\right)}\ge \frac{\Vert {{\rm Y}}^{1/2}{x}_0\Vert }{\underset{¯}{\sigma }\left({{\rm Y}}^{1/2}\right)}\text{.}$$

Now, since $D^{-1/2}$ is block diagonal, $x_0$ can be chosen so that its nonzero components correspond to only one block of $D^{-1/2}$. Let $x_{0,b}$ denote the entries of $x_0$ in that block, and ${{\rm Y}}_b^{1/2}$ denote the columns of ${{\rm Y}}^{1/2}$ corresponding to the same block. Then,

$${\Vert {{\rm Y}}^{1/2}{x}_0\Vert }^2=\left|{\left({{\rm Y}}^{1/2}{x}_0\right)}^{\ast }{{\rm Y}}^{1/2}{x}_0\right|=\left|x_{0,b}^{\ast }{{\rm Y}}_b^{1/2}{{\rm Y}}_b^{1/2}{x}_{0,b}\right|=\left|x_{0,b}^{\ast }{x}_{0,b}\right|={\Vert x_0\Vert }^2=1\text{.}$$

Then,

$$\kappa \left({\left({\rm Y}D\right)}^{1/2}\right)\ge \frac{1}{\underset{¯}{\sigma }\left({{\rm Y}}^{1/2}\right)}=\frac{\kappa \left({{\rm Y}}^{1/2}\right)}{\overline{\sigma }\left({{\rm Y}}^{1/2}\right)}\text{.}$$

Let ${\rm Y}={\left[{{\rm Y}}_{i,j}\right]}_{i,j=1,\dots ,I}$ be the block partition of ${\rm Y}$. From Lemma 17, $\Vert {{\rm Y}}_{i,j}\Vert \le 1$, for all $i,j=1,\dots ,I$. Then, from Lemma 16, Lemma 17

$$\overline{\sigma }\left({\rm Y}\right)\le \underset{i}{max}\sum _{j=1}^I\Vert {{\rm Y}}_{i,j}\Vert \le \underset{i}{max}\left|{\mathcal{B}}_i\right|=\beta \text{.}$$

Hence, from (A.3),

$$\kappa \left({\tilde{\Pi }}^{1/2}\Psi {\tilde{\Pi }}^{1/2}\right)\ge \frac{\kappa \left({\rm Y}\right)}{\overline{\sigma }\left({\rm Y}\right)}\ge \frac{\kappa \left({\rm Y}\right)}{\beta }\text{.}$$

The result follows since the inequality above holds for any $\tilde{\Pi }$.

Appendix B. Proofs of Section  4

Proof of Lemma 12

From Kailath, Sayed, and Hassibi (2000, A.1(v)), we have ${\Psi }_{a,a}-{\Psi }_{a,b}{\check{\Sigma }}_b{\Psi }_{b,a}$ and ${\Psi }_{b,b}-{\Psi }_{b,a}{\Psi }_{a,a}{\Psi }_{a,b}$ are invertible, and

$${\Psi }^{-1}=\left[\begin{array}{cc}\hfill {\Sigma }_a\hfill & \hfill -{\Sigma }_a{\Psi }_{a,b}{\Psi }_{b,b}^{-1}\hfill \\ \hfill -{\Sigma }_b{\Psi }_{b,a}{\Psi }_{a,a}^{-1}\hfill & \hfill {\Sigma }_b\hfill \end{array}\right]$$

with ${\Sigma }_a={({\Psi }_{a,a}-{\Psi }_{a,b}{\Psi }_{b,b}^{-1}{\Psi }_{b,a})}^{-1}$ and ${\Sigma }_b={({\Psi }_{b,b}-{\Psi }_{b,a}{\Psi }_{a,a}^{-1}{\Psi }_{a,b})}^{-1}$. The result then follows from (3).

Proof of Lemma 13

Follows immediately by applying Lemma 12 with $x_a=x_1$ and $x_b^T=[x_2,\dots ,x_I]$.

Before proving Theorem 15, we introduce some notation.

Notation 18

For each$i\in \mathcal{I}=\{1,\dots ,I\}$, and$j\in {\mathcal{B}}_i$, let${\mathcal{M}}_i\left(0\right)={\mathcal{M}}_{j,i}\left(0\right)=\left\{i\right\}$. Then, for each$t\in \mathbb{N}$, define recursively the following two sequences of sets

$${\mathcal{M}}_i\left(t\right)=\bigcup _{k\in {\mathcal{M}}_i(t-1)}{\mathcal{B}}_k\text{,}$$

$${\mathcal{M}}_{j,i}\left(t\right)=\bigcup _{k\in {\mathcal{M}}_{j,i}(t-1)}{\mathcal{B}}_k\setminus \left\{j\right\}\text{.}$$

I.e.,${\mathcal{M}}_i\left(t\right)$is the set of indexes of nodes which are$t$edges away from Node$i$, and${\mathcal{M}}_{j,i}\left(t\right)$is the set resulting after removing from${\mathcal{M}}_i\left(t\right)$the indexes of those nodes which are linked to Node$i$through Node$j$. For each$t\in {\mathbb{N}}_0=\mathbb{N}\cup \left\{0\right\}$and$i\in \mathcal{I}$, let${\xi }_i^T\left(t\right)=[x_k^T:k\in {\mathcal{M}}_i\left(t\right)]$,${\xi }_{i,j}^T\left(t\right)=[x_k^T:k\in {\mathcal{M}}_{i,j}\left(t\right)]$,${\Omega }_i\left(t\right)={\left[A_{k,l}\right]}_{k\in \mathcal{I},l\in {\mathcal{M}}_i\left(t\right)}$and${\Omega }_{i,j}\left(t\right)={\left[A_{k,l}\right]}_{k\in \mathcal{I},l\in {\mathcal{M}}_{i,j}\left(t\right)}$. Also, let$({\hat{\xi }}_i\left(t\right),{\Xi }_i\left(t\right))$be the WLS solution of the reduced system

$$y={\Omega }_i\left(t\right){\xi }_i\left(t\right)+v\text{,}$$ (B.1)

i.e.,

$${\hat{\xi }}_i\left(t\right)={\Xi }_i\left(t\right){\Omega }_i^{\ast }\left(t\right)R^{-1}y$$

$${\Xi }_i\left(t\right)={\left({\Omega }_i^{\ast }\left(t\right)R^{-1}{\Omega }_i\left(t\right)\right)}^{-1}$$

and$({\hat{\xi }}_{i,j}\left(t\right),{\Xi }_{i,j}\left(t\right))$be WLS the solution of

$$y={\Omega }_{i,j}\left(t\right){\xi }_{i,j}\left(t\right)+v\text{.}$$

Proof of Theorem 15

Suppose that, at time $t\in \mathbb{N}$, and for each $i\in \mathcal{I}$ and $j\in {\mathcal{B}}_i\setminus \left\{i\right\}$, Node $i$ is able to compute the components $({\check{x}}_i\left(t\right),{\check{\Sigma }}_i\left(t\right))$, corresponding to the state $x_i$, of the solution $({\hat{\xi }}_i\left(t\right),{\hat{\Xi }}_j\left(t\right))$, and the components $({\check{x}}_{i,j}\left(t\right),{\check{\Sigma }}_{i,j}\left(t\right))$, corresponding to the same state, of $({\check{\xi }}_{i,j}\left(t\right),{\check{\Xi }}_{i,j}\left(t\right))$. Since the network is acyclic, for each $i\in \mathcal{I}$ and each $t\in \mathbb{N}$, we have

$${\mathcal{M}}_i(t+1)=\left\{i\right\}\cup \bigcup _{j\in {\mathcal{B}}_i\setminus \left\{i\right\}}{\mathcal{M}}_{i,j}\left(t\right)\text{.}$$

Then, given that Node $i$ receives ${\gamma }_{i,j}\left(t\right)$ and ${\Gamma }_{i,j}\left(t\right)$, from each $j\in {\mathcal{B}}_i\setminus \left\{i\right\}$, $\left|{\mathcal{B}}_i\right|-1$ applications of Lemma 13 ($\left|\mathcal{S}\right|$ denotes the number of elements in the set $\mathcal{S}$), gives that ${\Psi }_{i,i}-{\sum }_{j\in {\mathcal{B}}_i\setminus \left\{i\right\}}{\Gamma }_{j,i}(t-1)$ is invertible, and Node $i$ is able to compute $({\check{x}}_i(t+1),{\check{\Sigma }}_i(t+1))$. Also, $\left|{\mathcal{B}}_i\right|-2$ applications of Lemma 13 give that, for each $j\in {\mathcal{B}}_i\setminus \left\{i\right\}$, ${\Psi }_{i,i}-{\sum }_{j\in {\mathcal{B}}_i\setminus \{i,j\}}{\Gamma }_{j,i}(t-1)$ is invertible and Node $i$ can compute $({\check{x}}_{i,j}(t+1),{\check{\Sigma }}_{i,j}(t+1))$. Then, the result follows after initializing the induction above using $({\check{x}}_i\left(0\right),{\check{\Sigma }}_i\left(0\right))$, at each $i\in \mathcal{I}$, for which no information exchange is required.

At each $t\in {\mathbb{N}}_0$ and $i\in \mathcal{I}$, $({\check{x}}_i\left(t\right),{\check{\Sigma }}_i\left(t\right))$ is the WLS solution of the sub-system (B.1). Since (B.1) is obtained by considering only the nodes in ${\mathcal{M}}_i\left(t\right)$, and ${\mathcal{M}}_i\left(t\right)=\mathcal{I}$, for all $t\ge {\rho }_i$, (22) follows.