Robust Asynchronous Stochastic Gradient-Push: Asymptotically Optimal and Network-Independent Performance for Strongly Convex Functions

Abstract

We consider the standard model of distributed optimization of a sum of functions $F(\mathbf{z})={\sum }_{i=1}^n{f}_i(\mathbf{z})$, where node i in a network holds the function f_i(z). We allow for a harsh network model characterized by asynchronous updates, message delays, unpredictable message losses, and directed communication among nodes. In this setting, we analyze a modification of the Gradient-Push method for distributed optimization, assuming that (i) node i is capable of generating gradients of its function f_i(z) corrupted by zero-mean bounded–support additive noise at each step, (ii) F(z) is strongly convex, and (iii) each f_i(z) has Lipschitz gradients. We show that our proposed method asymptotically performs as well as the best bounds on centralized gradient descent that takes steps in the direction of the sum of the noisy gradients of all the functions f₁(z), …, f_n(z) at each step.

1. Introduction

Distributed systems have attracted much attention in recent years due to their many applications such as large scale machine learning (e.g., in the healthcare domain, Brisimi et al., 2018), control (e.g., maneuvering of autonomous vehicles, Peng et al., 2017), sensor networks (e.g., coverage control, He et al., 2015) and advantages over centralized systems, such as scalability and robustness to faults. In a network comprised of multiple agents (e.g., data centers, sensors, vehicles, smart phones, or various IoT devices) engaged in data collection, it is sometimes impractical to collect all the information in one place. Consequently, distributed optimization techniques are currently being explored for potential use in a variety of estimation and learning problems over networks.

This paper considers the separable optimization problem

$$\underset{\mathbf{z}\in {ℝ}^d}{\min }F(\mathbf{z})≔\sum _{i=1}^n{f}_i(\mathbf{z}),$$ (1)

where the function $f_i:{ℝ}^d\to ℝ$ is held only by agent i in the network. We assume the agents communicate through a directed communication network, with each agent able to send messages to its out-neighbors. The agents seek to collaboratively agree on a minimizer to the global function F(z).

This fairly simple problem formulation is capable of capturing a variety of scenarios in estimation and learning. Informally, z is often taken to parameterize a model, and f_i(z) is a loss function measuring how well z matches the data held by agent i. Agreeing on a minimizer of F(z) means agreeing on a model that best explains all the data throughout the network – and the challenge is to do this in a distributed manner, avoiding techniques such as flooding which requires every node to learn and store all the data throughout the network. For more details, we refer the reader to the recent survey by Nedic et al. (2018).

In this work, we will consider a fairly harsh network environment, including message losses, delays, asynchronous updates, and directed communication. The function F(z) will be assumed to be strongly convex with the individual functions f_i(z) having a Lipschitz continuous gradient. We will also assume that, at every time step, node i can obtain a noisy gradient of its function f_i(z). Our goal will be to investigate to what extent distributed methods can remain competitive with their centralized counterparts in spite of these obstacles.

1.1. Literature Review

Research on models of distributed optimization dates back to the 1980s, see Tsitsiklis et al. (1986). The separable model of (1) was first formally analyzed in Nedic and Ozdaglar (2009), where performance guarantees on a fixed-stepsize subgradient method were obtained. The literature on the subject has exploded since, and we review here only the papers closely related to our work. We begin by discussing works that have focused on the effect of harsh network conditions.

A number of recent papers have studied asynchronicity in the context of distributed optimization. It has been noted that asynchronous algorithms are often preferred to synchronous ones, due to the difficulty of perfectly coordinating all the agents in the network, e.g., due to clock drift. Papers by Recht et al. (2011); Li et al. (2014); Agarwal and Duchi (2011); Lian et al. (2015) and Feyzmahdavian et al. (2016) study asynchronous parallel optimization methods in which different processors have access to a shared memory or parameter server. Recht et al. (2011) present a scheme called HOGWILD!, in which processors have access to the same shared memory with the possibility of overwriting each other’s work. Li et al. (2014) proposes a parameter server framework for distributed machine learning. Agarwal and Duchi (2011) analyze the convergence of gradient-based optimization algorithms whose updates depend on delayed stochastic gradient information due to asynchrony. Lian et al. (2015) improve on the earlier work by Agarwal and Duchi (2011), and study two asynchronous parallel implementations of Stochastic Gradient (SG) for nonconvex optimization; establishing an ${\mathcal{O}}_k(1/\sqrt{k})$ convergence rate for both algorithms. Feyzmahdavian et al. (2016) propose an asynchronous mini-batch algorithm that eliminates idle waiting and allows workers to run at their maximal update rates.

The works mentioned above consider a centralized network topology, i.e., there is a central node (parameter server or shared memory) connected to all the other nodes. On the other hand, in a decentralized setting, nodes communicate with each other over a connected network without depending on a central node (see Figure 1). This setting reduces the communication load on the central node, is not vulnerable to failures of that node, and is more easily scalable.

Figure 1:

Different network topologies.

For analysis of how decentralized asynchronous methods perform we refer the reader to Mansoori and Wei (2017); Tsitsiklis et al. (1986); Srivastava and Nedic (2011); Assran and Rabbat (2018); Nedic (2011); Wu et al. (2018) and Tian et al. (2018). We note that of these works only Tian et al. (2018) is able to obtain an algorithm which agrees on a global minimizer of (1) with non-random asynchronicity, under the assumptions of strong convexity, noiseless gradients and possible delays. On the other hand, the papers Nedic (2011) and Wu et al. (2018) obtain convergence in this situation under assumptions of natural randomness in the algorithm: the former assumes randomly failing links while the latter assumes that nodes make updates in random order.

The study of distributed separable optimization over directed graphs was initiated in Tsianos et al. (2012b), where a distributed approach based on dual averaging with convex functions over a fixed graph was proposed and shown to converge at an ${\mathcal{O}}_k(1/\sqrt{k})$ rate. Some numerical results for such methods were reported in Tsianos et al. (2012a). In Nedic and Olshevsky (2015), a method based on plain gradient descent converging at a rate of ${\mathcal{O}}_k((\ln k)/\sqrt{k})$ was proposed over time-varying graphs. This was improved in Nedic and Olshevsky (2016) to ${\mathcal{O}}_k((\ln k)/k)$ for strongly convex functions with noisy gradient samples. More recent works on optimization over directed graphs are Akbari et al. (2017), which considered online convex optimization in this setting, and Assran and Rabbat (2018), which considered combining directed graphs with delays and asynchronicity. The main tool for distributed optimization is the so-called “push sum” method introduced in Kempe et al. (2003), which is widely used to design communication and optimization schemes over directed graphs. More recent references are Bénézit et al. (2010); Hadjicostis et al. (2016), which provide a more modern and general analysis of this method, and the most comprehensive reference on the subject is the recent monograph by Hadjicostis et al. (2018). We also mention Xi and Khan (2017a); Xi et al. (2018); Nedic et al. (2017), where an approach based on push-sum was explored. A parallel line of work in this setting based on the the ADMM model, where updates are allowed to include a local minimization step, was explored in Brisimi et al. (2018); Chang et al. (2016a,b) and Hong (2017).

The reason directed graphs present a problem is because much of distributed optimization relies on the primitive of “multiplication by a doubly stochastic matrix:” given that each node of a network holds a number x_i, the network needs to compute y_i where x = (x₁, …, x_n)^⊺, y = (y₁, …, y_n)^⊺ and y = Wx for some doubly stochastic matrix W with positive spectral gap. This is pretty easy to accomplish over undirected graphs (see Nedic et al., 2018) but not immediate over directed graphs. A parallel line of research focuses on distributed methods for constructing such doubly stochastic matrices over directed graphs – we refer the reader to Dominguez-Garcia and Hadjicostis (2013); Gharesifard and Cortés (2012); Domínguez-García and Hadjicostis (2014). Unfortunately, to the authors’ best knowledge, no explicit and favorable convergence time guarantees are known for this procedure. Another line of work (Xi and Khan, 2017b) takes a similar approach, based on construction of a doubly stochastic matrix with positive spectral gap after the introduction of auxiliary states. Among works with undirected graphs, Scaman et al. (2017) derived the optimal convergence rates for smooth and strongly convex functions and introduced the multi-step dual accelerated (MSDA) algorithm with optimal linear convergence rate in the deterministic case.

Dealing with message losses has always been a challenging problem for multi-agent optimization protocols. Recently, Hadjicostis et al. (2016) resolved this issue rather elegantly for the problem of distributed average computation by having nodes exchange certain running sums. It was shown in Hadjicostis et al. (2016) that the introduction of these running sums is equivalent to a lossless algorithm on a slightly modified graph. We also refer the reader to the follow-up papers by Su and Vaidya (2016b,a, 2017). We will use the same approach in this work to deal with message losses.

In many applications, calculating the exact gradients can be computationally very expensive or impossible Lan et al. (2018). In one possible scenario, nodes are sensors that collect measurements at every step, which naturally corrupts all the data with noise. Alternatively, communication between agents may insert noise into information transmitted between them. Finally, when f_i(z) measures the fit of a model parameterized by the vector z to the data of agent i, it may be efficient for agent i to randomly select a subset of its data and compute an estimate of the gradient based on only those data points (Alpcan and Bauckhage, 2009). Motivated by these considerations, a literature has arisen studying the effects of stochasticity in the gradients. For example, Srivastava and Nedic (2011) showed convergence of an asynchronous algorithm for constrained distributed stochastic optimization, under the presence of local noisy communication in a random communication network. In Pu and Nedic (2018), two distributed stochastic gradient methods were introduced, and their convergence to a neighborhood of the global minimum (under constant step-size) and to the global minimum (under diminishing stepsize) was analyzed. In work by Sirb and Ye (2016), convergence of asynchronous decentralized optimization using delayed stochastic gradients has been shown.

The algorithms we will study here for stochastic gradient descent are based on the standard “consensus+gradient descent” framework: nodes will take steps in the direction of their gradients and then “reconcile” these steps by moving in the directions of an average of their neighbors in the graph. We refer the reader to Nedic et al. (2018); Yuan et al. (2016), for a more recent and simplified analysis of such methods. It is also possible to take a more modern approach, pioneered in Shi et al. (2015), of using the past history to make updates; such schemes have been shown to achieve superior performance in recent years (see Shi et al., 2015; Sun et al., 2016; Oreshkin et al., 2010; Nedic et al., 2017; Xi and Khan, 2017a; Xi et al., 2018; Qu and Li, 2017; Xu et al., 2015; Qu and Li, 2019; Di Lorenzo and Scutari, 2016); we refer the reader to Pu and Nedic (2018) which took this approach.

One of our main concerns in this paper is to develop decentralized optimization methods which perform as well as their centralized counterparts. Specifically, we will compare the performance of a distributed method for (1) on a network of n nodes with the performance of a centralized method which, at every step, can query all n gradients of the functions f₁(z), …, f_n(z). Since the distributed algorithm gets noise-corrupted gradients, so should the centralized method. Thus, the natural approach is to compare the distributed method to centralized gradient descent which moves in the direction of the sum of the gradients of f₁(z), …, f_n(z).This method of comparison keeps the “computational power” of the two nodes identical.

Traditionally, the bounds derived on distributed methods were considerably worse than those derived for centralized methods. For example, the papers by Nedic and Olshevsky (2015, 2016) had bounds for distributed optimization over directed graphs that were worse than the comparable centralized method (in terms of rate of error decay) by a multiplicative factor that, in the worst case, could be as large as $n^{\mathcal{O}(n)}$. This is typical over directed graphs, though better results are possible over undirected graphs. For example, in Olshevsky (2017), in the model of noiseless, undelayed, synchronous communication over an undirected graph, a distributed subgradient method was proposed whose performance, relative to a centralized method with the same computational power, was worse by a multiplicative factor of n.

The breakthrough papers by Chen and Sayed (2015); Pu and Garcia (2017); Morral et al. (2017), were the first to address this gap. These papers studied the model where gradients are corrupted by noise, which we also consider in this paper. Chen and Sayed (2015) examined the mean-squared stability and convergence of distributed strategies with fixed step-size over graphs and showed the same performance level as that of a centralized strategy, in the small step-size regime. In Pu and Garcia (2017) it was shown that, for a certain stochastic differential equation paralleling network gradient descent, the performance of centralized and distributed methods were comparable. In Morral et al. (2017), it was proved, for the first time, that distributed gradient descent with an appropriately chosen step-size, asymptotically performs similarly to a centralized method that takes steps in the direction of the sum of the noisy gradients, assuming iterates will remain bounded almost surely. This was the first analysis of a decentralized method for computing the optimal solution with performance bounds matching its centralized counterpart.

Both Pu and Garcia (2017) and Morral et al. (2017) were over fixed, undirected graphs with no message loss or delays or asynchronicity. As shown in the paper by Morral et al. (2012), this turns out to be a natural consequence of the analysis of those methods. Indeed, on a technical level, the advantage of working over undirected graphs is that they allow for easy distributed multiplication by doubly-stochastic matrices; it was shown in Morral et al. (2012) that if this property holds only in expectation – that is, if the network nodes can multiply by random stochastic matrices that are only doubly stochastic in expectation – distributed gradient descent will not perform comparably to its centralized counterpart.

In parallel to this work, and in order to reduce communication bottlenecks, Koloskova et al. (2019) propose a decentralized SGD with communication compression that can achieve the centralized baseline convergence rate, up to a constant factor. When the objective functions are smooth but not necessarily convex, Lian et al. (2017) show that Decentralized Parallel Stochastic Gradient Descent (D-PSGD) can asymptotically perform comparably to Centralized PSGD in total computational complexity. However, they argue that D-PSGD requires much less communication cost on the busiest node and hence, can outperform C-PSGD in certain communication regimes. Again, both Koloskova et al. (2019) and Lian et al. (2017) are over fixed undirected graphs, without delays, link failures or asynchronicity. The follow-up work by Lian et al. (2018), extends the D-PSGD to the asynchronous case.

1.2. Our Contribution

We propose an algorithm which we call Robust Asynchronous Stochastic Gradient Push (RASGP) for distributed optimization from noisy gradient samples over directed graphs with message losses, delays, and asynchronous updates. We will assume gradients are corrupted with additive noise represented by independent random variables, with bounded support, and with finite variance at node i denoted by ${\sigma }_i^2$. Our main result is that the RASGP performs as well as the best bounds on centralized gradient descent that moves in the direction of the sum of noisy gradients of f₁(z), …, f_n(z). Our results also hold if the underlying graphs are time-varying as long as there are no message losses. We give a brief technical overview of this result next.

We will assume that each function f_i(z) is μ_i-strongly convex with L_i-Lipschitz gradient, where ∑_i μ_i and L_i > 0, i = 1, …, n. The RASGP will have every node maintain an estimate of the optimal solution which will be updated from iteration to iteration; we will use z_i(k) to denote the value of this estimate held by node i at iteration k. We will show that, for each node i = 1, …, n,

$$\mathbb{E}[{\Vert {\mathbf{z}}_i(k)-{\mathbf{z}}^{*}\Vert }_2^2]=\frac{{\mathrm{\Gamma }}_u{\sum }_{i=1}^n{\sigma }_i^2}{k{({\sum }_{i=1}^n{\mu }_i)}^2}+{\mathcal{O}}_k\left(\frac{1}{k^{1.5}}\right),$$ (2)

where z* ≔ arg min F(z) and Γ_u is the degree of asynchronicity, defined as the maximum number of iterations between two consecutive updates of any agent. The leading term matches the best bounds for (centralized) gradient descent that takes steps in the direction of the sum of the noisy gradients of f₁(z), …, f_n(z), every k/Γ_u iterations (see Nemirovski et al., 2009; Rakhlin et al., 2012). Asymptotically, the performance of the RASGP is network independent: indeed, the only effect of the network or the number of nodes is on the constant factor within the ${\mathcal{O}}_k(1/k^{1.5})$ term above. The asymptotic scaling as ${\mathcal{O}}_k(1/k)$ is optimal in this setting (Rakhlin et al., 2012).

Consider the case when all the functions are identical, i.e., f₁(z) = ⋯ = f_n(z), and Γ_u = 1. In this case, letting μ = μ_i and σ = σ_i, we have that for each i = 1, …, n, (2) reduces to

$$\mathbb{E}[{\Vert {\mathbf{z}}_i(k)-{\mathbf{z}}^{*}\Vert }_2^2]=\frac{{\sigma }^2/n}{k{\mu }^2}+{\mathcal{O}}_k\left(\frac{1}{k^{1.5}}\right).$$

In other words, asymptotically we get the variance reduction of a centralized method that simply averages the n noisy gradients at each step.

The implication of this result is that one can get the benefit of having n independent processors computing noisy gradients in spite of all the usual problems associated with communications over a network (i.e., message losses, latency, asynchronous updates, oneway communication). Of course, the caveat is that one must wait sufficiently long for the asymptotic decay to “kick in,” i.e., for the second term on the right-hand side of (2) to become negligible compared to the first. We leave the analysis of the size of this transient period to future work and note here that it will depend on the network and the number of nodes.¹

The RASGP is a variation on the usual distributed gradient descent where nodes mix consensus steps with steps in the direction of their own gradient, combined with a new step-size trick to deal with asynchrony. It is presented as Algorithm 3 in Section 3. For a formal statement of the results presented above, we refer the reader to Theorem 15 in the body of the paper.

We briefly mention two caveats. The first is that implementation of the RASGP requires each node to use the quantity ${\sum }_{i=1}^n{\mu }_i/n$ in setting its local stepsize. This is not a problem in the setting when all functions are the same, but, otherwise, ${\sum }_{i=1}^n{\mu }_i/n$ is a global quantity not immediately available to each node. Assuming that node i knows μ_i one possibility is to use average consensus to compute this quantity in a distributed manner before running the RASGP (for example using the algorithm described in Section 2 of this paper). The second caveat is that, like all algorithms based on the push-sum method, the RASGP requires each node to know its out-degree in the communication graph.

1.3. Organization of This Paper

We conclude this Introduction with Section 1.4, which describes the basic notation we will use throughout the remainder of the paper. Section 2 does not deal directly with the distributed optimization problem we have discussed, but rather introduces the problem of computing the average in the fairly harsh network setting we will consider in this paper. This is an intermediate problem we need to analyze on the way to our main result. Section 3 provides the RASGP algorithm for distributed optimization, and then states and proves our main result, namely the asymptotically network-independent and optimal convergence rate. Results from numerical simulations of our algorithm to illustrate its performance are provided in Section 4, followed by conclusions in Section 5.

1.4. Notations and Definitions

We assume there are n agents $\mathcal{V}=\{1,\mathrm{...},n\}$, communicating through a fixed directed graph $\mathcal{G}=(\mathcal{V},\mathcal{E})$, where $\mathcal{E}$ is the set of directed arcs. We assume $\mathcal{G}$ does not have self-loops and is strongly connected.

For a matrix A, we will use A_ij to denote its (i,j)th entry. Similarly, v_i and [v]_i will denote the ith entry of a vector v. A matrix is called stochastic if it is non-negative and the sum of the elements of each row equals to one. A matrix is column stochastic if its transpose is stochastic. To a non-negative matrix $\mathbf{A}\in {ℝ}^{n\times n}$ we associate a directed graph ${\mathcal{G}}_{\mathbf{A}}$ with vertex set ${\mathcal{V}}_{\mathbf{A}}=\{1,2,\mathrm{...},n\}$ and edge set ${\mathcal{E}}_{\mathbf{A}}=\{(i,j)|A_{ji}>0\}$. In general, such a graph might contain self-loops. Intuitively, this graph corresponds to the information flow in the update x(k + 1) = Ax(k); indeed, $(i,j)\in {\mathcal{E}}_{\mathbf{A}}$ if the jth coordinate of x(k + 1) depends on the ith coordinate of x(k) in this update.

Given a sequence of matrices A(0), A(1), A(2), …, we denote by A^k₂:k₁, k₂ ≥ k₁, the product of elements k₁ to k₂ of the sequence, inclusive, in the following order:

$${\mathbf{A}}^{k_2:k_1}=\mathbf{A}(k_2)\mathbf{A}(k_2-1)\cdots \mathbf{A}(k_1).$$

Moreover, A^k:k = A(k).

Node i is an in-neighbor of node j, if there is a directed link from i to j. Hence, j would be an out-neighbor of node i. We denote the set of in-neighbors and out-neighbors of node i by $N_i^{-}$ and $N_i^{+}$, respectively. Moreover, we denote the number of in-neighbors and out-neighbors of node i with $d_i^{-}$ and $d_i^{+}$, as its in-degree and out-degree, respectively.

By x_min and x_max we denote min_i x_i and max_i x_i respectively, over all possible indices unless mentioned otherwise. We denote a n × 1 column vector of all ones or zeros by 1_n and 0_n, respectively. We will remove the subscript when the size is clear from the context.

Let $\text{v}\in {ℝ}^d$ be a vector. We denote by ${\text{v}}^{-}\in {ℝ}^d$ a vector of the same length such that

$$v_i^{-}=\{\begin{array}{c}1/v_i,\hfill \\ 0,\hfill \end{array}\begin{array}{c}\text{if}{v}_i\ne 0,\hfill \\ \text{if}{v}_i=0.\hfill \end{array}$$

For all the algorithms we describe, we sometimes use the notion of mass to denote the value an agent holds, sends or receives. With that in mind, we can think of a value being sent from one node, as a mass being transferred.

We use ‖.‖_p to denote the l_p-norm of a vector. We sometimes drop the subscript when referring to the Euclidean l₂ norm.

2. Push-Sum with Delays and Link Failures

In this section we introduce the Robust Asynchronous Push-Sum algorithm (RAPS) for distributed average computation and prove its exponential convergence. Convergence results proved for this algorithm will be used later when we turn to distributed optimization. The algorithm relies heavily on ideas from Hadjicostis et al. (2016) to deal with message losses, delays, and asynchrony. The conference version of this paper Olshevsky et al. (2018) developed RAPS for the delay-free case, and this section may be viewed as an extension of that work.

Pseudocode for the algorithm is given in the box for Algorithm 1. We begin by outlining the operation of the algorithm. Our goal in this section is to compute the average of vectors, one held by each node in the network, in a distributed manner. However, since the RAPS algorithm acts separately in each component, we may, without loss of generality, assume that we want to average scalars rather than vectors. The scalar held by node i will be denoted by x_i(0).

Without loss of generality, we define an iteration by descretizing time into time slots indexed by k = 0,1,2,…. We assume that during each time slot every agent makes at most one update and processes messages sent in previous time slots.

In the setting of no message losses, no delays, no asynchrony, and a fixed, regular, undirected communication graph, the RAPS can be shown to be equivalent to the much simpler iteration

$$\mathbf{x}(t+1)=\mathbf{Wx}(t),$$

where W is an irreducible, doubly stochastic matrix with positive diagonal; standard Markov chain theory implies that $x_i(t)\to (1/n){\sum }_{i=1}^n{x}_i(t)$ in this setting. RAPS does essentially the same linear update, but with a considerable amount of modifications. In particular, we use the central idea of the classic push-sum method (Kempe et al., 2003) to deal with directed communication, which suggests to have a separate update equation for the y-variables, which informs us how we should rescale the x-variables; as well as the central idea of Hadjicostis et al. (2018), which is to repeatedly broadcast sums of previous messages to provide robustness against message loss. While the algorithm in Hadjicostis et al. (2018) handles message losses in a synchronous setting, RAPS can handle delays as well as asynchronicity.

Before getting into details, let us provide a simple intuition behind the RAPS algorithm. Each agent i holds a value (mass) x_i and y_i. At the beginning of every iteration, i wants to split its mass between itself and its out-neighbors $j\in N_i^{+}$. However, to handle message losses, it sends the accumulated x and y mass (running sums which we denote by ${\varphi }_i^x$ and ${\varphi }_i^y$), that i wants to transfer to each of its neighbors, from the start of the algorithm. Therefore, when a neighbor j receives a new accumulated mass from i, it stores it at ${\rho }_{ji}^{\ast }$ and by subtracting the previous accumulated mass ρ_ji it had received from i, j obtains all the mass that i has been trying to send since its last successful communication. Then, j updates its x and y mass by adding the new received masses, and finally, updates its estimate of the average to x/y. To handle delays and asynchronicity, timestamps κ_i are attached to messages outgoing from i.

The pseudocode for the algorithm may appear complicated at first glance; this is because of the considerable complexity required to deal with directed communications, message losses, delays, and asynchrony.

We next describe the algorithm in more detail. First, in the course of executing the algorithm, every agent i maintains scalar variables x_i, y_i, z_i, ${\varphi }_i^x$, ${\varphi }_i^y$, κ_i, ${\rho }_{\mathit{ij}}^x$, ${\rho }_{\mathit{ij}}^y$ and κ_ij for $(j,i)\in \mathcal{E}$. The variables x_i and y_i have the same evolution, however y_i is initialized as 1. Therefore, to save space in describing and analyzing the algorithm, we will use the symbol θ, when a statement holds for both x and y. Similarly, when a statement is the same for both variables x and y, we will remove the superscripts x or y. For example, the initialization ρ_ji(0) = 0 in the beginning of the algorithm means both ${\rho }_{ji}^x(0)=0$ and ${\rho }_{ji}^y(0)=0$.

We briefly mention the intuitive meaning of the various variables. The number z_i represents node i’s estimate of the initial average. The counter ${\varphi }_i^{\theta }(k)$ is the total θ-value sent by i to each of its neighbors from time 0 to k − 1. Similarly, ${\rho }_{ij}^{\theta }(k)$ is the total θ value that i has received from j up to time k − 1. The integer κ_i is a timestamp that i attaches to its messages, and the number κ_ij tracks the latest timestamp i has received from j.

To obtain an intuition for how the algorithm uses the counters ${\varphi }_i^{\theta }(k)$ and ${\rho }_{ij}^{\theta }(k)$, note that, in line 15 of the algorithm, node i effectively figures out the last θ value sent to it by each of its in-neighbors j, by looking at the increment to the ${\rho }_{ij}^{\theta }$. This might seem needlessly involved, but, the underlying reason is that this approach introduces robustness to message losses.

Algorithm 1.

Robust Asynchronous Push-Sum (RAPS)

1:	Initialize the algorithm with y(0) = 1, ϕi(0) = 0, ∀i ∈ {1, …, n} and ρij(0) = 0, κij(0) = 0, $\forall (j,i)\in \mathcal{E}$.
2:	At every iteration k = 0, 1, 2, …, for every node i:
3:	if node i wakes up then
4:	κi ← k;
5:	${\varphi }_i^x\leftarrow {\varphi }_i^x+\frac{x_i}{d_i^{+}+1}$, ${\varphi }_i^y\leftarrow {\varphi }_i^y+\frac{y_i}{d_i^{+}+1}$;
6:	$x_i\leftarrow \frac{x_i}{d_i^{+}+1}$, $y_i\leftarrow \frac{y_i}{d_i^{+}+1}$;
7:	Node i broadcasts (${\varphi }_i^x,{\varphi }_i^y$,κi) to its out-neighbors in $N_i^{+}$.
8:	Processing the received messages
9:	for $({\varphi }_j^x,{\varphi }_j^y,{\kappa }_j^{\prime })$ in the inbox do
10:	if ${{\kappa }^{\prime }}_j>{\kappa }_{ij}$ then
11:	${\rho }_{ij}^{\ast x}\leftarrow {\varphi }_j^x$, ${\rho }_{ij}^{\ast y}\leftarrow {\varphi }_j^y$;
12:	${\kappa }_{ij}\leftarrow {\kappa }_j^{\prime }$;
13:	end if
14:	end for
15:	$x_i\leftarrow x_i+{\sum }_{j\in N_i^{-}}\left({\rho }_{\mathit{ij}}^{\ast x}-{\rho }_{\mathit{ij}}^x\right)$ , $y_i\leftarrow y_i+{\sum }_{j\in N_i^{-}}\left({\rho }_{\mathit{ij}}^{\ast y}-{\rho }_{\mathit{ij}}^y\right)$ ;
16:	${\rho }_{ij}^x\leftarrow {\rho }_{ij}^{\ast x}$, ${\rho }_{ij}^y\leftarrow {\rho }_{ij}^{\ast y},$
17:	$z_i\leftarrow \frac{x_i}{y_i}$;
18:	end if
19:	Other variables remain unchanged.

We next describe in words what the pseudocode above does. At every iteration k, if agent i wakes up, it performs the following actions. First, it divides its values x_i, y_i into $d_i^{+}+1$ parts and broadcasts these to its out-neighbors; actually, what it broadcasts are the accumulated running sums ${\varphi }_i^x$ and ${\varphi }_i^y$. Following Kempe et al. (2003), this is sometimes called the “push step.”

Then, node i moves on to process the messages in its inbox in the following way. If agent i has received a message from node j that is newer than the last one it has received before, it will store that message in ${\rho }_{ij}^{\ast }$ and discard the older messages. Next, i updates its x and y variables by adding the difference of ${\rho }_{ij}^{\ast }$ with the older value ρ_ij, for all in-neighbors j. As mentioned above, this difference is equal to the new mass received. Next, ${\rho }_{ij}^{\ast }$ overwrites ρ_ij in the penultimate step. The last step of the algorithm sets z_i to be the rescaled version of x_i: z_i = x_i/y_i.

In the remainder of this section, we provide an analysis of the RAPS algorithm, ultimately showing that it converges geometrically to the average in the presence of message losses, asynchronous updates, delays, and directed communication. Our first step is to formulate the RAPS algorithm in terms of a linear update (i.e., a matrix multiplication), which we do in the next subsection.

2.1. Linear Formulation

Next we show that, after introducing some new auxiliary variables, Algorithm 1 can be written in terms of a classical push-sum algorithm (Kempe et al., 2003) on an augmented graph. Since the y-variables have the same evolution as the x-variables, here we only analyze the x-variables.

In our analysis, we will associate with each message an effective delay. If a message is sent at time k₁ and is ready to be processed at time k₂, then k₂ − k₁ ≥ 1 is the effective delay experienced by that message. Those messages that are discarded will not have an effective delay associated with them and are considered as lost.

Next, we will state our assumptions on connectivity, asynchronicity, and message loss.

Assumption 1 Suppose:

1. Graph $\mathcal{G}$ is strongly connected and does not have self-loops.

2. The delays on each link are bounded above by some Γ_del ≥ 1.

3. Every agent wakes up and performs updates at least once every Γ_u ≥ 1 iterations.

4. Each link fails at most Γ_f ≥ 0 consecutive times.

5. Messages arrive in the order of time they were sent. In other words, if messages are sent from node i to j at times k₁ and k₂ with (effective) delays d₁ and d₂, respectively, and k₁ < k₂, then we have k₁ + d₁ < k₂ + d₂.

One consequence of Assumption 1 is that the effective delays associated with each message that gets through are bounded above by Γ_d ≔ Γ_del + Γ_u − 1. Another consequence is that for each $(i,j)\in \mathcal{E}$, j receives a message from i successfully, at least once every Γ_s iterations where

$${\mathrm{\Gamma }}_s≔{\mathrm{\Gamma }}_u({\mathrm{\Gamma }}_f+1)+{\mathrm{\Gamma }}_d\ge 2.$$ (3)

Part (e) of Assumption 1 can be assumed without loss of generality. Indeed, observe that outdated messages automatically get discarded in Line 10 of our algorithm. For simplicity, it is convenient to think of those messages as lost. Thus, if this assumption fails in practice, the algorithm will perform exactly as if it had actually held in practice due to Line 10. Making this an assumption, rather than a proposition, lets us slightly simplify some of the arguments and avoid some redundancy throughout this paper.

Let us introduce the following indicator variables: τ_i(k) for i ∈ {1, …, n} which equals to 1 if node i wakes up at time k, and equals 0 otherwise. Similarly, ${\tau }_{ij}^l(k)$, for $(i,j)\in \mathcal{E}$, 1 ≤ l ≤ Γ_d which is 1 if τ_i(k) = 1 and the message sent from node i to j at time k will arrive after experiencing an effective delay of l.² Note that if node i wakes up at time k but the message it sends to j is lost, then ${\tau }_{ij}^l(k)$ will be zero for all l.

We can rewrite the RAPS algorithm with the help of these indicator variables. Let us adopt the notation that x_i(k) refers to x_i at the beginning of round k of the algorithm (i.e., before node i has a chance to go through the list of steps outlined in the algorithm box). We will use the same convention with all of the other variables, e.g., y_i(k), z_i(k), etc. If node i does not wake up at round k, then of course x_i(k + 1) = x_i(k).

Now observe that we can write

$${\varphi }_i^x(k+1)-{\varphi }_i^x(k)={\tau }_i(k)\frac{x_i(k)}{d_i^{+}+1}.$$ (4)

Likewise, we have

$$x_i(k+1)=x_i(k)\left(1-{\tau }_i(k)+\frac{{\tau }_i(k)}{d_i^{+}+1}\right)+\sum _{j\in N_i^{-}}({\rho }_{ij}^x(k+1)-{\rho }_{ij}^x(k)),$$ (5)

which can be shown by considering each case (τ_i(k) = 1 or 0); note that we have used the fact that, in the event that node i wakes up at time k, the variable ${\rho }_{ij}^x(k+1)$ equals the variable ${\rho }_{ij}^{\ast x}$ during execution of Line 16 of the algorithm at time k.

Finally, we have that $\forall (i,j)\in \mathcal{E}$, the flows ${\rho }_{ji}^x$ are updated as follows:

$${\rho }_{ji}^x(k+1)={\rho }_{ji}^x(k)+\sum _{l=1}^{{\mathrm{\Gamma }}_d}{\tau }_{ij}^l(k-l)({\varphi }_i^x(k+1-l)-{\rho }_{ji}^x(k)),$$ (6)

where we make use of the fact that the sum contains only a single nonzero term, since the messages arrive monotonically. To parse the indices in this equation, note that node i actually broadcasts ${\varphi }_i^x(k+1-l)$ in our notation at iteration k − l; by our definitions, ${\varphi }_i^x(k-l)$ is the value of ${\varphi }_i^x$ at the beginning of that iteration. To simplify these relations, we introduce the auxiliary variables $u_{ij}^x$ for all $(i,j)\in \mathcal{E}$, defined through the following recurrence relation:

$$u_{ij}^x(k+1)≔\left(1-\sum _{l=1}^{{\mathrm{\Gamma }}_d}{\tau }_{ij}^l(k)\right)\left(u_{ij}^x(k)+{\varphi }_i^x(k+1)-{\varphi }_i^x(k)\right),$$ (7)

and initialized as $u_{ij}^x(0)≔0$. Intuitively, the variables $u_{ij}^x$ represent the “excess mass” of x_i that is yet to reach node j. Indeed, this quantity resets to zero whenever a message is sent that arrives at some point in the future, and otherwise is incremented by adding the broadcasted mass that is lost. Note that node i never knows $u_{ij}^x(k)$, since it has no idea which messages are lost, and which are not; nevertheless, for purposes of analysis, nothing prevents us from considering these variables.

Let us also define the related quantity,

$$v_{ij}^x(k)≔u_{ij}^x(k)+{\varphi }_i^x(k+1)-{\varphi }_i^x(k),\text{for}k\ge \text{0,}$$

and $v_{ij}^x(k)≔0$ for k < 0. Intuitively, this quantity may be thought of as a forward-looking estimate of the mass that will arrive at node j, if the message sent from node i at time k gets through; correspondingly, it includes not only the previous unsent mass, but the extra mass that will be added at the current iteration.

The key variables for the analysis of our method are the variables we will denote by $x_{ij}^l(k)$. Intuitively, every time a message is sent, but gets lost, we imagine that it has instead arrived into a “virtual node” which holds that mass; once the next message gets through, we imagine that the virtual node has forwarded that mass to its intended destination. This idea originates from Hadjicostis et al. (2016). Because of the delays, however, we need to introduce Γ_d virtual nodes for each such event. If a message is sent from i and arrives at j with effective delay l, we will instead imagine it is received by the virtual node $b_{ij}^l$, then sent to $b_{ij}^{l-1}$ at the next time step, and so forth until it reaches $b_{ij}^1$, and is then forwarded to its destination. These virtual nodes are defined formally later.

Putting that intuition aside, we formally define the variables $x_{ij}^l(k)$ via the following set of recurrence relations:

$$x_{ij}^l(k+1)≔{\tau }_{ij}^l(k)v_{ij}^x(k),l={\mathrm{\Gamma }}_d,$$ (8)

$$x_{ij}^l(k+1)≔{\tau }_{ij}^l(k)v_{ij}^x(k)+x_{ij}^{l+1}(k),1\le l<{\mathrm{\Gamma }}_d,$$ (9)

and $x_{ij}^l(k)≔0$ when both k ≤ 0 and l = 1, …, Γ_d. To parse these equations, imagine what happens when a message is sent from i to j with effective delay of Γ_d at time k. The content of this message becomes the value of $x_{ij}^{{\mathrm{\Gamma }}_d}$ according to (8); and, in each subsequent step, influences $x_{ij}^{{\mathrm{\Gamma }}_d-1},x_{ij}^{{\mathrm{\Gamma }}_d-2}$, and so forth according to (9). Putting (8) and (9) together, we obtain

$$x_{ij}^l(k)=\sum _{t=1}^{{\mathrm{\Gamma }}_d-l+1}{\tau }_{ij}^{t+l-1}(k-t)v_{ij}^x(k-t),$$ (10)

and particularly,

$$x_{ij}^1(k)=\sum _{t=1}^{{\mathrm{\Gamma }}_d}{\tau }_{ij}^t(k-t)v_{ij}^x(k-t).$$ (11)

Note that, as is common in many of the equations we will write, only a single term in the sums can be nonzero (this is not obvious at this point and is a result of Lemma 1).

Before proceeding to the main result of this section, we state the following lemma, whose proof is immediate.

Lemma 1 If ${\tau }_{ij}^l(k)=1$, the following statements are satisfied:

1. ${\tau }_{ij}^{l^{\prime }}(k)=0$ for l′ ≠ l.

2. If l > 0, then ${\tau }_{ij}^s(k+t)=0$ for t = 1, …, l and s = 0, …, l − t.

3. If l < Γ_d, then ${\tau }_{ij}^s(k-t)=0$ for t = 1, …, Γ_d − l and s = l + t, …, Γ_d.

Lemma 2 If ${\tau }_{ij}^l(k)=1$ then $x_{ij}^{l^{\prime }}(k)=0$ for l′ > l.

Proof By Lemma 1(c), ${\tau }_{ij}^{t+l^{\prime }-1}(k-t)=0$ for t ∈ {1, …, Γ_d − l′ + 1}. Hence, by (10) we have,

$$x_{ij}^{l^{\prime }}(k)=\sum _{t=1}^{{\mathrm{\Gamma }}_d-l^{\prime }+1}{\tau }_{ij}^{t+l^{\prime }-1}(k-t)v_{ij}^x(k-t)=0.$$

The next lemma is essentially a restatement of the observation that the content of every $x_{ij}^{l^{\prime }}$ eventually “passes through” $x_{ij}^1$.

Lemma 3 If ${\tau }_{ij}^l(k-l)=1$, l ≥ 1, we have,

$$\sum _{l^{\prime }=1}^l{x}_{ij}^{l^{\prime }}(k-l)=\sum _{t=1}^l{x}_{ij}^1(k-t).$$

Proof We will show $x_{ij}^1(k-t)=x_{ij}^{l-t+1}(k-l)$ for t = 1, …, l. For t = l the equality is trivial. Now suppose t < l. By Lemma 1(a) we have ${\tau }_{ij}^{l-t}(k-l)=0$. Moreover, by part (b) of the same lemma we have, ${\tau }_{ij}^{s^{\prime }}(k-l+t^{\prime })=0$ for t′ = l − t − 1 and s′ = l−t−t′. Hence, $x_{ij}^{l-t-t^{\prime }+1}(k-l+t^{\prime })=x_{ij}^{l-t-t^{\prime }}(k-l+t^{\prime }+1)$. Combining these equations for t′ = 0, …, l−t−1, we get $x_{ij}^1(k-t)=x_{ij}^{l-t+1}(k-l)$. ■

The following lemma is the key step of a linear formulation of RAPS.

Lemma 4 For k = 0,1,… and $(i,j)\in \mathcal{E}$ we have:

$${\rho }_{ji}^x(k+1)-{\rho }_{ji}^x(k)=x_{ij}^1(k),$$ (12)

$$u_{ij}^x(k+1)+{\rho }_{ji}^x(k+1)+\sum _{l=1}^{{\mathrm{\Gamma }}_d}{x}_{ij}^l(k+1)={\varphi }_i^x(k+1).$$ (13)

Parsing these equations, (12) simply states that the value of $x_{ij}^1(k)$ can be thought of as impacting ${\rho }_{ji}^x$ at time k; recall that the content of $x_{ij}^1(k)$ is a message that was sent from node i to j at time k − l with an effective delay of l, for some 1 ≤ l ≤ Γ_d (cf. Equation 11). On the other hand, (13) may be thought of a “conservation of mass” equation. All the mass that has been sent out by node i has either: (i) been lost (in which case it is in $u_{ij}^x$), (ii) affected node j (in which case it is in ${\rho }_{ji}^x$), or (iii) is in the process of reaching node j but delayed (in which case it is in some $x_{ij}^l$).

Although this lemma is arguably obvious, a formal proof is surprisingly lengthy. For this reason, we relegate it to the Appendix.

We next write down a matrix form of our updates. As a first step, define the (n + m′) × 1 column vector χ(k) ≔ [x(k)^⊺, x¹(k)^⊺, …, x^Γ_d(k)^⊺,u^x(k)^⊺]^⊺, where m′ ≔ (Γ_d+1)m, $m≔\left|\mathcal{E}\right|$, x(k) collects all x_i(k), x^l(k) collects all $x_{ij}^l(k)$ and, u^x(k) collects all $u_{ij}^x(k)$. Define ψ(k) by collecting y-values similarly.

Now, we have all the tools to show the linear evolution of χ(k). By Equations (4), (5) and (12) we have,

$$x_j(k+1)=x_j(k)\left(1-{\tau }_j(k)+\frac{{\tau }_j(k)}{d_j^{+}+1}\right)+\sum _{i\in N_j^{-}}{x}_{ij}^1(k).$$ (14)

Moreover, by the definitions of x_ij, v_ij and (4) it follows,

$$\begin{gathered} x_{ij}^{{\mathrm{\Gamma }}_d}(k+1)={\tau }_{ij}^{{\mathrm{\Gamma }}_d}(k)\left[u_{ij}^x(k)+\frac{x_i(k)}{d_i^{+}+1}\right], \\ {x}_{ij}^l(k+1)={\tau }_{ij}^l(k)\left[u_{ij}^x(k)+\frac{x_i(k)}{d_i^{+}+1}\right]+x_{ij}^{l+1}(k). \end{gathered}$$ (15)

Finally, by (4) and (7) we obtain,

$$u_{ij}^x(k+1)=(1-\sum _{l=1}^{{\mathrm{\Gamma }}_d}{\tau }_{ij}^l(k))\left(u_{ij}^x(k)+{\tau }_i(k)\frac{x_i(k)}{d_i^{+}+1}\right).$$ (16)

Using (14) to (16) we can write the evolution of χ(k) and ψ(k) in the following linear form:

$$\begin{gathered} \mathit{\chi }(k+1)=\mathbf{M}(k)\mathit{\chi }(k), \\ \mathit{\psi }(k+1)=\mathbf{M}(k)\mathit{\psi }(k), \end{gathered}$$ (17)

where $\mathbf{M}(k)\in {ℝ}^{(n+m^{\prime })\times (n+m^{\prime })}$ is an appropriately defined matrix.

We have thus completed half of our goal: we have shown how to write RAPS as a linear update. Next, we show that the corresponding matrices are column-stochastic.

Lemma 5 M(k) is column stochastic and its positive elements are at least $1/({\max }_i\{d_i^{+}\}+1)$. Moreover, for i = 1, …,n, M_ii(k) are positive.

This lemma can be proved “by inspection.” Indeed, M(k) is column stochastic if and only if, for every χ(k), we have 1^Tχ(k + 1) = 1^Tχ(k). Thus one just needs to demonstrate that no mass is ever “lost,” i.e., that a decrease/increase in the value of one node is always accompanied by an increase/decrease of the value of another node, which can be done just by inspecting the equations. A formal proof is nonetheless given next.

Proof To show that M(k) is column stochastic, we study how each element of χ(k) influences χ(k + 1).

For i = 1, …, n, the ith column of M(k) represents how x_i(k) influences χ(k + 1). We will use (14) to (16) to find these coefficients.

First, x_i(k) influences x_i(k + 1) with the coefficient $1-{\tau }_i(k)+{\tau }_i(k)/(d_i^{+}+1)>0$. For $j\in N_i^{+},x_i(k)$ influences $x_{ij}^l(k+1)$ by ${\tau }_{ij}^l(k)/(d_i^{+}+1)$ and $u_{ij}^x(k+1)$ with coefficient $({\tau }_i(k)-{\sum }_{l=1}^{{\mathrm{\Gamma }}_d}{\tau }_i(k){\tau }_{ij}^l(k))/(d_i^{+}+1)$. Summing these coefficients up results in 1.

For l = 2, …, Γ_d, $(i,j)\in \mathcal{E},x_{ij}^l(k)$ influences $x_{ij}^{l-1}(k+1)$ with coefficient 1 and $x_{ij}^1(k)$ influences $x_j(k+1)$ with coefficient 1.

Finally, $u_{ij}^x(k)$ influences $x_{ij}^l(k+1)$ with coefficient ${\tau }_{ij}^l(k)$ and $u_{ij}^x(k+1)$ with $(1-{\sum }_{l=1}^d{\tau }_{ij}^l(k))$, which sum up to 1.

Note that all the coefficients above are at least $1/({\max }_i\{d_i^{+}\}+1)$. ■

An important result of this lemma is the sum preservation property, i.e.,

$$\begin{gathered} \sum _{i=1}^{n+m^{\prime }}{\chi }_i(k)=\sum _{i=1}^n{x}_i(0), \\ \sum _{i=1}^{n+m^{\prime }}{\psi }_i(k)=n. \end{gathered}$$ (18)

For further analysis, we augment the graph $\mathcal{G}$ to $\mathcal{H}(k)≔{\mathcal{G}}_{\mathbf{M}(k)}=({\mathcal{V}}_A,{\mathcal{E}}_A(k))$ by adding the following virtual nodes: $b_{ij}^l$ for l = 1, …, Γ_d and $(i,j)\in \mathcal{E}$, which hold the values $x_{ij}^l$ and $y_{ij}^l$; We also add the nodes c_ij for $(i,j)\in \mathcal{E}$ which hold the values $u_{ij}^x$ and $u_{ij}^y$.

In $\mathcal{H}(k)$, there is a link from $b_{ij}^l$ to $b_{ij}^{l-1}$ for 1 < l ≤ d and from $b_{ij}^1$ to j as they forward their values to the next node. Moreover, if ${\tau }_{ij}^l(k)=1$ for some 1 ≤ l ≤ Γ_d, then there is a link from both c_ij and i to $b_{ij}^l$.

If ${\tau }_{ij}^l(k)=0$ for 1 ≤ l ≤ Γ_d then c_ij has a self loop, and if also ${\tau }_i(k)=1$, there’s a link from i to c_ij. All non-virtual agents $i\in \mathcal{V}$, have self-loops all the time (see Fig. 2).

Figure 2:

Augmented graph $\mathcal{H}(k)$ for different scenarios.

Recursions (17) and Lemma 5 may thus be interpreted as showing that the RAPS algorithm can be thought of as a push-sum algorithm over the augmented graph sequence $\{\mathcal{H}(k)\}$, where each agent (virtual and non-virtual) holds an x-value and a y-value which evolve similarly and in parallel.

2.2. Exponential Convergence

The main result of this section is exponential convergence of RAPS to initial average, stated next.

Theorem 6 Suppose Assumption 1 holds. Then RAPS converges exponentially to the initial mean of agent values. i.e.,

$$\left|z_i(k)-\frac{1}{n}\sum _{i=1}^n{x}_i(0)\right|\le \delta {\mathrm{\lambda }}^k{\Vert \mathbf{x}(0)\Vert }_1,$$

where $\delta ≔\frac{1}{1-n{\alpha }^6},\mathrm{\lambda }≔{(1-n{\alpha }^6)}^{1/(2n{\mathrm{\Gamma }}_s)}$ and α ≔ (1/n)^nΓ_s.

It is worth mentioning that even though $1/(1-\mathrm{\lambda })=\mathcal{O}(n^{p(n)})$ where $p(n)=\mathcal{O}(n)$, this is a bound for a worst case scenario and on average, as it can be seen in numerical simulations, RAPS performs better. Moreover, when the graph $\mathcal{G}$ satisfies certain properties, such as regularity, and also there is no link delays and failures, we have $1/(1-\mathrm{\lambda })=\mathcal{O}(n^3)$ (see Theorem 1 in Nedic and Olshevsky, 2016). More broadly, that paper establishes that 1/(1 − λ) will scale with the mixing rate of the underlying Markov process.

Unfortunately, this theorem does not follow immediately from standard results on exponential convergence of push-sum. The reason is that the connectivity conditions assumed for such theorems are not satisfied here: there will not always be paths leading to virtual nodes from non-virtual nodes. Nevertheless, with some suitable modifications, the existence of paths from virtual nodes to other virtual nodes is sufficient, as we will show next.

Before proving the theorem, we need the following lemmas and definitions. Given a sequence of graphs ${\mathcal{G}}^0$ , ${\mathcal{G}}^1$ , ${\mathcal{G}}^2$ , …, we will say node b is reachable from node a in time period k₁ to k₂ (k₁ < k₂), if there exists a sequence of directed edges e_k1, e_{k1 + 1}, …, e_k2 such that e_k is in ${\mathcal{G}}^k$ , the destination of e_k is the origin of e_k+1 for k₁ ≤ k < k₂, and the origin of e_k1 is a and the destination of e^k₂ is b.

Our first lemma provides a standard lower bound on the entries of the column-stochastic matrices from (17).

Lemma 7 M^{k+nΓ_s−1:k} has positive first n rows, for any k ≥ 0. The positive elements of this matrix are at least

$$\alpha ={(1/n)}^{n{\mathrm{\Gamma }}_s}.$$

Proof By Lemma 5, each node $j\in \mathcal{V}$ has self-loops at every iteration in the augmented graph $\mathcal{H}$ . Since $\mathcal{G}$ is strongly connected, the set of reachable non-virtual nodes from any node $a_h\in {\mathcal{V}}_A$ strictly increases every Γ_s iterations. Hence, M^{k+nΓ_s−1:k} has positive first n rows. Moreover, since all positive elements of M are at least 1/n, the positive elements of M^{k+nΓ_s−1:k} are at least (1/n)^nΓ_s. ■

Next, we give a reformulation of the push-sum update that will be key to showing the exponential convergence of the algorithm. The proof is a minor variation of Lemma 4 in Nedic and Olshevsky (2016).

Lemma 8 Consider the vectors $\mathbf{u}(k)\in {ℝ}^d,\mathbf{v}(k)\in {ℝ}_{+}^d$, and square matrix $\mathbf{A}(k)\in {ℝ}_{+}^{d\times d}$, for k ≥ 0 such that,

$$\begin{gathered} \mathbf{u}(k+1)=\mathbf{A}(k)\mathbf{u}(k), \\ \mathbf{v}(k+1)=\mathbf{A}(k)\mathbf{v}(k). \end{gathered}$$ (19)

Also suppose u_i(k) = 0 if v_i(k) = 0, ∀k,i. Define ${\mathbf{u}}^{-}(k)\in {ℝ}^d$ as:

$$u_i^{-}(k)≔\{\begin{array}{cc}1/u_i(k),\hfill & if{u}_i(k)\ne 0,\hfill \\ 0\hfill & if{u}_i(k)=0.\hfill \end{array}$$

Define r(k) ≔ u(k) ∘ v⁻(k), where ∘ denotes the element-wise product of two vectors. Then we have,

$$\mathbf{r}(k+1)=\mathbf{B}(k)\mathbf{r}(k),$$

where $\mathbf{B}(k)\in {ℝ}_{+}^{d\times d}$ is defined as,

$$\mathbf{B}(k)≔\mathrm{diag}({\mathbf{v}}^{-}(k+1))\mathbf{A}(k)\mathrm{diag}(\mathbf{v}(k)).$$

Proof Since u_i(k) = 0 if v_i(k) = 0, u_i(k) = r_i(k)v_i(k) holds for all i, k. Substituting in (19) we obtain,

$$r_i(k+1)v_i(k+1)=\sum _{j=1}^d{A}_{ij}(k)r_j(k)v_j(k).$$

Since, by definition r_i(k) = 0 if v_i(k) = 0, ∀k,i, we get

$$r_i(k+1)=v_i^{-}(k+1)\sum _{j=1}^d{A}_{ij}(k)r_j(k)v_j(k).$$

Therefore,

$$\mathbf{r}(k+1)=\mathrm{diag}({\mathbf{v}}^{-}(k+1))\mathbf{A}(k)\mathrm{diag}(\mathbf{v}(k))\mathbf{r}(k).$$

■

Our next corollary, which follows immediately from the previous lemma, characterizes the dichotomy inherent in push-sum with virtual nodes: every row either adds up to one or zero.

Corollary 9 Consider the matrix B(k) defined in Lemma 8. Let us define the index set J^k ≔ {i|v_i(k) ≠ 0}. If i ∉ J^k, the ith column of B(k) and ith row of B(k − 1) only contain zero entries. Moreover,

$$\begin{gathered} \mathbf{B}(k)1_d=\mathrm{diag}({\mathbf{v}}^{-}(k+1))\mathbf{A}(k)\mathbf{v}(k) \\ =\mathrm{diag}({\mathbf{v}}^{-}(k+1))\mathbf{v}(k+1)=\left[\begin{array}{c}1\mathit{or}0\\ ⋮\\ 1\mathit{or}0\end{array}\right]. \end{gathered}$$

Hence, the ith row of B(k) sums to 1 if and only if v_i(k + 1) ≠ 0 or i ∈ J^k+1.

Our next lemma characterizes the relationship between zero entries in the vectors χ(k) and ψ(k).

Lemma 10 χ_h(k) = 0 whenever ψ_h(k) = 0 for h = 1, …, n + m′, k ≥ 0.

Proof First we note that $\mathit{\psi }(0)={[1_n^{⊺},0_{m^{\prime }}^{⊺}]}^{⊺}$ and each node $i\in \mathcal{V}$ has a self-loop in graph $\mathcal{H}(k)$ for all k ≥ 0; hence, ψ_h(k) ≥ 0 for all h and particularly, ψ_i(k) > 0 for i = 1, …, n. Now suppose h > n and corresponds to a virtual agent $a_h\in {\mathcal{V}}_A$. If ψ_h(k) = 0, it means a_h has already sent all its y-value to another node or has not received any y-value yet. In either case, that node also has no remaining x-value as well and χ_h(k) = 0. ■

Let us define ${\mathit{\psi }}^{-}(k)\in {ℝ}^{n+m^{\prime }}$, k ≥ 0 by

$${\psi }_i^{-}(k)≔\{\begin{array}{cc}1/{\psi }_i(k),\hfill & \mathrm{if}{\psi }_i(k)\ne 0,\hfill \\ 0,\hfill & \mathrm{if}{\psi }_i(k)=0.\hfill \end{array}$$ (20)

Moreover, we define the vector z(k) by setting z(k) ≔ χ(k) ∘ ψ⁻(k). By (17) and Lemma 10, we can use Lemma 8 to obtain,

$$\mathbf{z}(k+1)=\mathbf{P}(k)\mathbf{z}(k),$$

where P(k) ≔ diag(ψ⁻(k + 1))M(k)diag(ψ(k)). Let us define

$$I^k≔\{i|{\psi }_i(k)>0\}.$$

Then, by Corollary 9 we have each z_i(k + 1), i ∈ I^k+1, is a convex combination of z_j(k)’s for j ∈ I^k. Therefore,

$$\begin{gathered} \underset{i\in I^{k+1}}{\max }{z}_i(k+1)\le \underset{i\in I^k}{\max }{z}_i(k), \\ \underset{i\in I^{k+1}}{\min }{z}_i(k+1)\ge \underset{i\in I^k}{\min }{z}_i(k). \end{gathered}$$ (21)

These equations will be key to the analysis of the algorithm. We stress that we have not shown that the quantity min_i z_i(k) is non-decreasing; rather, we have shown that the related quantity, where the minimum is taken over I^k, the set of nonzero entries of ψ(k), is non-increasing.

Our next lemma provides lower and upper bounds on the entries of the vector ψ(k).

Lemma 11 For k ≥ 0 and 1 ≤ i ≤ n we have:

$$n\alpha \le {\psi }_i(k)\le n.$$

Moreover, for n + 1 ≤ h ≤ n + m′ and k ≥ 1 we have either ψ_h(k) = 0 or,

$$n{\alpha }^2\le {\psi }_h(k)\le n.$$

Proof We have,

$$\mathit{\psi }(k)={\mathbf{M}}^{k-1:0}\left[\begin{array}{c}{1}_n\\ 0_{m^{\prime }}\end{array}\right],$$

If k < nΓ_s, positive entries of M^k−1:0 are at least (1/n)^k. Hence, positive entries of ψ(k) are at least,

$${\left(\frac{1}{n}\right)}^k\ge {\left(\frac{1}{n}\right)}^{n{\mathrm{\Gamma }}_s-1}=n\alpha .$$

Now suppose k ≥ nΓ_s. M^k−1:0 is the product of M^{k−1:k−nΓ_s} and another column stochastic matrix. By Lemma 7, M^{k−1:k−nΓ_s} has positive first n rows, and positive entries of at least α. Thus, M^k−1:0 has positive first n rows, and positive entries of at least α as well. We obtain for 1 ≤ i ≤ n,

$${\psi }_i(k)\ge n\alpha ,\text{for}k\ge 1.$$

For n + 1 ≤ h ≤ n + m′m, suppose ψ_h corresponds to a virtual node a_h corresponding to some link $(i,j)\in \mathcal{E}$. If ψ_h(k) is positive, it is carrying a value sent from i at k − nΓ_s or later, which has experienced link failure or delays. This is because each value gets to its destination after at most Γ_s iterations. Since i has self-loops all the time, a_h is reachable from i in period k − nΓ_s to k − 1; Hence, $M_{hi}^{k-1:k-n{\mathrm{\Gamma }}_s}\ge \alpha$, and it follows,

$${\psi }_h(k)\ge \alpha {\psi }_i(k-n{\mathrm{\Gamma }}_s)\ge n{\alpha }^2.$$

Also, due to sum preservation property, we have ψ_h(k) ≤ n, for all h and k ≥ 0. ■

Using Lemma 8 again, it follows,

$$\mathbf{z}(k+n{\mathrm{\Gamma }}_s)=\widehat{\mathbf{P}}(k)\mathbf{z}(k),$$

where,

$$\widehat{\mathbf{P}}(k)≔\mathrm{diag}({\mathit{\psi }}^{-}(k+n{\mathrm{\Gamma }}_s)){\mathbf{M}}^{k+nLs-1:k}\mathrm{diag}(\mathit{\psi }(k)).$$ (22)

Next, we are able to find a lower bound on the positive elements of $\widehat{\mathbf{P}}(k)$. The proof of the following corollary is immediate.

Corollary 12 By (22) and Lemma 11 we have:

1. ${\widehat{P}}_{\mathit{ij}}(k)>0$ for 1 ≤ i,j ≤ n.

2. Positive entries of first n columns of $\widehat{P}(k)$ are at least (1/n)α(nα) = α². Similarly, the last m′ columns have positive entries of at least α³.

3. For h > n, if h ∈ I^k+nΓ_s then ${\widehat{P}}_{hi}(k)>0$ for some 1 ≤ i ≤ n.

Our next lemma, which is the final result we need before proving the exponential convergence rate of RAPS, provides a quantitative bound for how multiplication by the matrix P shrinks the range of a vector.

Lemma 13 Let t ≥ 0 and ${\{\mathbf{u}(k)\}}_{k\ge 0}\in {ℝ}^{n+m^{\prime }}$ be a sequence of vectors such that,

$$\mathbf{u}(k+1)=\widehat{\mathbf{P}}(kn{\mathrm{\Gamma }}_s+t)\mathbf{u}(k).$$

Define

$$s_t(k)≔\underset{i\in I^{kn{\mathrm{\Gamma }}_s+t}}{\max }{u}_i(k)-\underset{i\in I^{kn{\mathrm{\Gamma }}_s+t}}{\min }{u}_i(k).$$

Then,

$$s_t(k+2)\le (1-n{\alpha }^6)s_t(k).$$

Proof Let us define

$$r_t(k)≔\underset{1\le i\le n}{\max }{u}_i(k)-\underset{1\le i\le n}{\min }{u}_i(k).$$

By Corollary 12 for j ∈ I^(k+1)nΓ_s+t the jth row of $\widehat{\mathbf{P}}(kn{\mathrm{\Gamma }}_s+t)$ has at least one positive entry in the first n columns. Thus, because u_j(k + 1) is maximized/minimized when all of the weight is put on the largest/smallest possible entry of u_j(k), we have:

$$\begin{gathered} u_j(k+1)\le {\alpha }^3\underset{1\le i\le n}{\max }{u}_i(k)+(1-{\alpha }^3)\underset{i\in I^{kn{\mathrm{\Gamma }}_s+t}}{\max }{u}_i(k), \\ {u}_j(k+1)\ge {\alpha }^3\underset{1\le i\le n}{\min }{u}_i(k)+(1-{\alpha }^3)\underset{i\in I^{kn{\mathrm{\Gamma }}_s+t}}{\min }{u}_i(k), \end{gathered}$$

Therefore,

$$s_t(k+1)\le {\alpha }^3{r}_t(k)+(1-{\alpha }^3)s_t(k).$$ (23)

Moreover, by a similar argument for j ≤ n,

$$\begin{gathered} u_j(k+1)\le {\alpha }^3\sum _{i=1}^n{u}_i(k)+(1-n{\alpha }^3)\underset{i\in I^{kn{\mathrm{\Gamma }}_s+t}}{\max }{u}_i(k), \\ {u}_j(k+1)\ge {\alpha }^3\sum _{i=1}^n{u}_i(k)+(1-n{\alpha }^3)\underset{i\in I^{kn{\mathrm{\Gamma }}_s+t}}{\min }{u}_i(k). \end{gathered}$$

Thus,

$$r_t(k+1)\le (1-n{\alpha }^3)s_t(k).$$

Combining with (23) and noting that r_t(k) ≤ s_t(k) and s_t(k + 1) ≤ s_t(k) we obtain,

$$\begin{gathered} s_t(k+2)\le {\alpha }^3(1-n{\alpha }^3)s_t(k)+(1-{\alpha }^3)s_t(k+1) \\ \le {\alpha }^3(1-n{\alpha }^3)s_t(k)+(1-{\alpha }^3)s_t(k) \\ =(1-n{\alpha }^6)s_t(k).■ \end{gathered}$$

Proof of Theorem 6 Using Lemma 13 with t = 0 and u(k) = z(knΓ_s) we get s₀(k) ≤ (1 − nα⁶)^⌊k/2⌋s₀(0) and lim_k→∞ s₀(k) = 0. Moreover by (21), z_max(k) is a non-increasing sequence and by z_min(k) is non-decreasing. Thus,

$$\underset{k\to \infty ,h\in I^k}{\lim }{\mathbf{z}}_h(k)=L_{\infty }.$$ (24)

We have:

$$\begin{gathered} L_{\infty }=L_{\infty }\underset{k\to \infty }{\lim }\frac{{\sum }_{i=1}^{n+m^{\prime }}{\psi }_i(k)}{{\sum }_{i=1}^{n+m^{\prime }}{\psi }_i(k)} \\ =\underset{k\to \infty }{\lim }\left(\frac{{\sum }_{i=1}^{n+m^{\prime }}{z}_i(k){\psi }_i(k)}{n}+\frac{{\sum }_{i=1}^{n+m^{\prime }}(L_{\infty }-z_i(k)){\psi }_i(k)}{n}\right) \\ =\underset{k\to \infty }{\lim }\left(\frac{{\sum }_{i=1}^{n+m^{\prime }}{\chi }_i(k)}{n}+\frac{{\sum }_{i=1}^{n+m^{\prime }}(L_{\infty }-z_i(k)){\psi }_i(k)}{n}\right) \\ =\frac{{\sum }_{i=1}^n{x}_i(0)}{n}. \end{gathered}$$

In the above, we used (18) and (24), the boundedness of ψ_i(k), and the fact that ψ_i(k) = 0 for i ∉ I^k.

Finally, to show the exponential convergence rate, we go back to s₀(k). We have for k ≥ 1,

$$\begin{gathered} s_0(k)\le {(1-n{\alpha }^6)}^{\lfloor k/2\rfloor }{s}_0(0)\le {(1-n{\alpha }^6)}^{(k-1)/2}{s}_0(0), \\ {s}_0(0)\le \sum _{i=1}^{n+m^{\prime }}|z_i(0)|=\sum _{i=1}^n|x_i(0)|={\Vert \mathbf{x}(0)\Vert }_1, \end{gathered}$$

where the first equality holds because I⁰ = {1, …, n} and y_i(0) = 1. Therefore, we have for i ∈ I^k,

$$\begin{gathered} \mid z_i(k)-\frac{1^{⊺}\mathbf{x}(0)}{n}\mid \le z_{\max }(k)-z_{\min }(k) \\ \le s_0(\lfloor k∕n{\mathrm{\Gamma }}_s\rfloor ) \\ \le {(1-n{\alpha }^6)}^{(\lfloor \frac{k}{n{\mathrm{\Gamma }}_s}\rfloor -1)∕2}{\parallel \mathbf{x}(0)\parallel }_1 \\ \le {(1-n{\alpha }^6)}^{(\frac{k}{n{\mathrm{\Gamma }}_s}-1-1)∕2}{\parallel \mathbf{x}(0)\parallel }_1 \\ =\frac{1}{1-n{\alpha }^6}{\left({(1-n{\alpha }^6)}^{1∕(2n{\mathrm{\Gamma }}_s)}\right)}^k{\parallel \mathbf{x}(0)\parallel }_1 \\ =\delta {\mathrm{\lambda }}^k{\parallel \mathbf{x}(0)\parallel }_1. \end{gathered}$$

where $\delta =\frac{1}{1-n{\alpha }^6}$ and λ = (1 − nα⁶)^1/(2nΓ_s). Note that {1, …, n} ⊆ I^k, ∀k. ■

Remark: Observe that our proof did not really use the initialization ψ(0) = 1, except to observe that the elements ψ(0) are positive, add up to n, and the implication that ψ(k) satisfies the bounds of Lemma 11. In particular, the same result would hold if we viewed time 1 as the initial point of the algorithm (so that ψ(1) is the initialization), or similarly any time k. We will use this observation in the next subsection.

2.3. Perturbed Push-Sum

In this subsection, we begin by introducing the Perturbed Robust Asynchronous Push-Sum algorithm, obtained by adding a perturbation to the x-values of (non-virtual) agents at the beginning of every iteration they wake up.

We show that, if the perturbations are bounded, the resulting z(k) nevertheless tracks the average of χ(k) pretty well. Such a result is a key step towards analyzing distributed optimization protocols. In this general approach to the analyses of distributed optimization methods, we follow Ram et al. (2010) where it was first adopted; see also Nedic and Olshevsky (2016) and Nedic and Olshevsky (2015) where it was used.

Adopting the notations introduced earlier and by the linear formulation (17) we have,

$$\mathit{\chi }(k+1)=\mathbf{M}(k)(\mathit{\chi }(k)+\mathbf{\Delta }(k)),\text{for}k\ge 0,$$

Algorithm 2.

Perturbed Robust Asynchronous Push-Sum

1:	Initialize the algorithm with y(0) = 1 ϕi(0) = 0, ∀i ∈ i ∈ {1, …, n} and ρij(0) = 0, κij(0) = 0, $\forall (j,i)\in \mathcal{E}$ and Δ(0) = 0.
2:	At every iteration k = 0, 1, 2, …, for every node i:
3:	if node i wakes up then
4:	xi ← xi + Δi(k)
5:	Lines 4 to 17 of Algorithm 1
6:	end if
7:	Other variables remain unchanged.

where $\mathbf{\Delta }(k)\in {ℝ}^{n+m^{\prime }}$ collects all perturbations Δ_i(k) in a column vector with Δ_h(k) ≔ 0 for n < h ≤ n + m’. We may write this in a convenient form as follows.

$$\begin{gathered} \mathit{\chi }(k+1)=\mathbf{M}(k)(\mathit{\chi }(k)+\mathbf{\Delta }(k)) \\ =\sum _{t=1}^k{\mathbf{M}}^{k:t}\mathbf{\Delta }(k)+{\mathbf{M}}^{k:0}\mathit{\chi }(0). \end{gathered}$$

Define for k ≥ 1,

$$\begin{array}{ll}{\mathit{\chi }}^t(k)≔{\mathbf{M}}^{k-1:t}\mathbf{\Delta }(t),\hfill & 1\le t\le k,\hfill \\ {\mathit{\chi }}^0(k)≔{\mathbf{M}}^{k-1:0}\mathit{\chi }(0),\hfill & t=0.\hfill \end{array}$$ (25)

We obtain

$$\mathit{\chi }(k)=\sum _{t=0}^{k-1}{\mathit{\chi }}^t(k),k\ge 1.$$ (26)

Define z^t(k) ≔ χ^t(k) ∘ ψ⁻(k) for 0 ≤ t ≤ k (cf. Equation 20). We have

$$\mathbf{z}(k)=\sum _{t=0}^{k-1}{\mathbf{z}}^t(k).$$ (27)

We may view each z^t(k) as the outcome of a push-sum algorithm, initialized at time t, and apply Theorem 6. This immediately yields the following result, with part (b) an immediate consequence of part (a).

Theorem 14 Suppose Assumption 1 holds. Consider the sequence {z_i(k)}, 1 ≤ i ≤ n, generated by Algorithm 2. Then,

1. For k = 1,2,….

   $$\left|z_i(k)-\frac{1^{⊺}\mathit{\chi }(k)}{n}\right|\le \delta {\mathrm{\lambda }}^k{\Vert \mathbf{x}(0)\Vert }_1+\sum _{t=1}^{k-1}\delta {\mathrm{\lambda }}^{k-t}{\Vert \mathbf{\Delta }(t)\Vert }_1.$$
2. If lim_t→∞ ‖Δ(t)‖₁ = 0 then,

   $$\underset{k\to \infty }{lim}\left|z_i(k)-\frac{1^{⊺}\mathit{\chi }(k)}{n}\right|=0.$$

3. Robust Asynchronous Stochastic Gradient-Push (RASGP)

In this section we present the main contribution of this paper, a distributed stochastic gradient method with asymptotically network-independent and optimal performance over directed graphs which is robust to asynchrony, delays, and link failures.

Recall that we are considering a network $\mathcal{G}$ of n agents whose goal is to cooperatively solve the following minimization problem

$$\text{minimize}F(\mathbf{z})≔\sum _{i=1}^n{f}_i(\mathbf{z}),\text{over}\mathbf{z}\in {ℝ}^d,$$

where each $f_i:{ℝ}^d\to ℝ$ is a strongly convex function only known to agent i. We assume agent i has the ability to obtain noisy gradients of the function f_i.

The RASGP algorithm is given as Algorithm 3. Note that we use the notation ${\widehat{\mathbf{g}}}_i(k)$ for a noisy gradient of the function f_i(z) at z_i(k) i.e.,

$${\widehat{\mathbf{g}}}_i(k)={\mathbf{g}}_i(k)+{\mathit{\epsilon }}_i,$$

where g_i(k) ≔ ∇f_i(z_i(k)) and ε_i is a random vector.

The RASGP is based on a standard idea of mixing consensus and gradient steps, first analyzed in Nedic and Ozdaglar (2009). The push-sum scheme of Section 2, inspired by Hadjicostis et al. (2016), is used instead of the consensus scheme, which allows us to handle delays, asynchronicity, and message losses; this is similar to the approach taken in Nedic and Olshevsky (2015). We note that a new step-size strategy is used to handle asynchronicity: when a node wakes up, it takes steps with a step-size proportional to the sum of all the step-sizes during the period it slept. As far as we are aware, this idea is new.

We will be making the following assumption on the noise vectors.

Assumption 2 ε_i is an independent random vector with bounded support, i.e., ‖ε_i‖ ≤ b_i, i = 1, …, n. Moreover, $\mathbb{E}[{\mathit{\epsilon }}_i]=0$ and $\mathbb{E}[{\Vert {\mathit{\epsilon }}_i\Vert }^2]\le {\sigma }_i^2$.

Next, we state and prove the main result of this paper, which states the linear convergence rate of Algorithm 3.

Theorem 15 Suppose that:

1. Assumptions 1 and 2 hold.

2. Each objective function f_i(z) is μ_i-strongly convex over ${ℝ}^d$.

3. The gradients of each f_i(z) are L_i-Lipschitz continuous, i.e., for all ${\mathbf{z}}_1,{\mathbf{z}}_2\in {ℝ}^d$,

   $$\Vert {\mathbf{g}}_i({\mathbf{z}}_1)-{\mathbf{g}}_i({\mathbf{z}}_2)\Vert \le L_i\Vert {\mathbf{z}}_1-{\mathbf{z}}_2\Vert .$$

Then, the RASGP algorithm with the step-size α(k) = n/(μk) for k ≥ 1 and α(0) = 0, will converge to the unique optimum z* with the following asymptotic rate: for all i = 1, …, n, we have

$$\mathbb{E}[{\Vert {\mathbf{z}}_i(k)-{\mathbf{z}}^{*}\Vert }^2]\le \frac{{\mathrm{\Gamma }}_u{\sigma }^2}{k{\mu }^2}+{\mathcal{O}}_k\left(\frac{1}{k^{1.5}}\right),$$

where ${\sigma }^2≔{\sum }_i{\sigma }_i^2$, μ = ∑_i μ_i.

Algorithm 3.

Robust Asynchronous Stochastic Gradient-Push (RASGP)

1:	Initialize the algorithm with y(0) = 1, ${\varphi }_i^x(0)=0$ , ${\varphi }_i^y(0)=0$ , κi(0) = −1, ∀i ∈ {1, …, n} and ${\rho }_{\mathit{ij}}^x(0)=0$ , ${\rho }_{\mathit{ij}}^y(0)=0$ , κij(0) = −1, $\forall (j,i)\in \mathcal{E}$ .
2:	At every iteration k = 0, 1, 2, …, for every node i:
3:	if node i wakes up then
4:	${\beta }_i(k)={\sum }_{t={\kappa }_i+1}^k\alpha (t)$;
5:	${\mathbf{x}}_i\leftarrow {\mathbf{x}}_i-{\beta }_i(k){\widehat{\mathbf{g}}}_i(k)$;
6:	κi ← k;
7:	${\varphi }_i^x\leftarrow {\varphi }_i^x+\frac{x_i}{d_i^{+}+1}$, ${\varphi }_i^y\leftarrow {\varphi }_i^y+\frac{y_i}{d_i^{+}+1}$;
8:	${\mathbf{x}}_i\leftarrow \frac{{\mathbf{x}}_i}{d_i^{+}+1}$, $y_i\leftarrow \frac{y_i}{d_i^{+}+1}$;
9:	Node i broadcasts ( ${\mathit{\varphi }}_i^x$ , ${\varphi }_i^y$ , κi) to its out-neighbors: $N_i^{+}$
10:	Processing the received messages
11:	for $({\mathit{\varphi }}_j^x,{\varphi }_j^y,{\kappa }_j^{\prime })$ in the inbox do
12:	if ${\kappa }_j^{\prime }>{\kappa }_{ij}$ then
13:	${\mathit{\rho }}_{ij}^{\ast x}\leftarrow {\mathit{\varphi }}_j^x$, ${\rho }_{ij}^{\ast y}\leftarrow {\varphi }_j^y$;
14:	${\kappa }_{ij}\leftarrow {\kappa }^{\prime }$
15:	end if
16:	end for
17:	${\mathbf{x}}_i\leftarrow {\mathbf{x}}_i+{\sum }_{j\in N_i^{-}}\left({\rho }_{\mathit{ij}}^{\ast x}-{\rho }_{\mathit{ij}}^x\right)$ , $y_i\leftarrow y_i+{\sum }_{j\in N_i^{-}}\left({\rho }_{\mathit{ij}}^{\ast y}-{\rho }_{\mathit{ij}}^y\right)$ ;
18:	${\mathit{\rho }}_{ij}^x\leftarrow {\mathit{\rho }}_{ij}^{\ast x}$, ${\rho }_{ij}^y\leftarrow {\rho }_{ij}^{\ast y}$;
19:	${\mathbf{z}}_i\leftarrow \frac{{\mathbf{x}}_i}{y_i}$;
20:	end if
21:	Other variables remain unchanged.

Remark 16 We note that each agent stores variables x_i, y_i, κ_i, z_i, ${\mathit{\varphi }}_i^x$ , ${\varphi }_i^y$ and ${\mathit{\rho }}_{\mathit{ij}}^x$ , ${\rho }_{\mathit{ij}}^y$ , κ_ij for all in-neighbors $j\in N_i^{-}$ . Hence, the memory requirement of the RASGP algorithm for each agent is $\mathcal{O}(d_i^{-})$ for each agent i.

We next turn to the proof of Theorem 15. First, we observe that Algorithm 3 is a specific case of multi-dimensional Perturbed Robust Asynchronous Push-Sum. In other words, each coordinate of vectors x_i, z_i, ${\mathit{\varphi }}_i^x$ and ${\rho }_{\mathit{ij}}^x$ will experience an instance of Algorithm 2. Hence, there exists an augmented graph sequence $\{\mathcal{H}(k)\}$ where the Algorithm 3 is equivalent to perturbed push-sum consensus on $\mathcal{H}(k)$ where each agent $a_h\in {\mathcal{V}}_A$ holds vectors x_h and y_h. In other words, we will be able to apply Theorem 14 to analyze Algorithm 3.

Our first step is to show how to decouple the action of Algorithm 3 coordinate by coordinate. For each coordinate 1 ≤ ℓ ≤ d, let ${\mathit{\chi }}^l\in {ℝ}^{n+m^{\prime }}$ stack up the ℓth entries of x-values of all agents (virtual and non-virtual) in ${\mathcal{V}}_A$ . Additionally, define ${\mathbf{\Delta }}^{\ell }(k)\in {ℝ}^{n+m^{\prime }}$ to be the vector stacking up the ℓth entries of perturbations. i.e.,

$${[{\mathbf{\Delta }}^{\ell }(k)]}_i≔\{\begin{array}{cc}-{\beta }_i(k){[{\widehat{\mathbf{g}}}_i(k)]}_{\ell },\hfill & \mathrm{if}i\in \mathcal{V},{\tau }_i(k)=1,\hfill \\ 0,\hfill & \text{otherwise}.\hfill \end{array}$$

Then, by the definition of the algorithm, we have for all ℓ = 1, …, d,

$$\begin{array}{l}{\mathit{\chi }}^{\ell }(k+1)=\mathbf{M}(k)\left({\mathit{\chi }}^{\ell }(k)+{\mathbf{\Delta }}^{\ell }(k)\right),\\ \mathit{\psi }(k+1)=\mathbf{M}(k)\mathit{\psi }(k).\end{array}$$ (28)

These equations write out the action of Algorithm 3 on a coordinate-by-coordinate basis.

In order to prove Theorem 15, we need a few tools and lemmas. As already mentioned, our first step will be to argue that Algorithm 3 converges by application of Theorem 14. This requires showing the boundedness of the perturbations Δ^ℓ(k), which, as we will show, reduces to showing the vectors z_i(k) are bounded. The following lemma will be useful to establish this boundedness.

Lemma 17 (Nedic and Olshevsky, 2016, Lemma 3) Let $q:{ℝ}^d\to ℝ$ be a ν-strongly convex function with ν > 0 which has Lipschitz gradients with constant L. let $\mathbf{v}\in {ℝ}^d$ and $\mathbf{u}\in {ℝ}^d$ defined by,

$$\mathbf{u}=\mathbf{v}-\alpha (\nabla q(\mathbf{v})+p(\mathbf{v})),$$

where α ∈ (0,ν/8L²] and $p:{ℝ}^d\to {ℝ}^d$ is a mapping such that,

$$\Vert p(\mathbf{v})\Vert \le c,\mathit{for\; all}\mathbf{v}\in {\mathbb{R}}^d.$$

Then, there exists a compact set $\mathcal{S}\subset {ℝ}^d$ and a scalar R such that,

$$\Vert \mathbf{u}\Vert \le \{\begin{array}{cc}\Vert \mathbf{v}\Vert ,\hfill & \mathit{for\; all}\mathbf{v}\notin \mathcal{S},\hfill \\ R,\hfill & \mathit{for\; all}\mathbf{v}\in \mathcal{S},\hfill \end{array}$$

where,

$$\begin{gathered} \mathcal{S}≔\{\mathbf{z}|q(\mathbf{z})\le q(0)+2\frac{\nu }{8L^2}({\Vert q(0)\Vert }^2+c^2)\}\cup B\left(0,\frac{4c}{\nu }\right), \\ R≔\underset{\mathbf{z}\in \mathcal{S}}{\max }\{\Vert \mathbf{z}\Vert +\frac{\nu }{8L^2}\Vert \nabla q(\mathbf{z})\Vert \}+\frac{\nu c}{8L^2}. \end{gathered}$$

We now argue that the iterates generated by Algorithm 3 are bounded.

Lemma 18 The iterates z_i(k) generated by Algorithm 3 will remain bounded.

Proof Let us adopt the notation ψ⁻ from previous sections and define ${\mathbf{z}}^l(k)≔{\mathit{\chi }}^l(k)\circ {\mathit{\psi }}^{-}(k)\in {ℝ}^{n+m^{\prime }}$. Moreover, adopt the notation z_h for virtual agent a_h, h = n + 1, …, n+m′, as z_h(k) ≔ x_h(k)/ψ_h(k). Also define ${\mathbf{u}}^l\in {ℝ}^{n+m^{\prime }}$ by

$${\mathbf{u}}^{\ell }(k)≔{\mathit{\chi }}^{\ell }(k)+{\mathbf{\Delta }}^{\ell }(k).$$

Since the perturbations are only added to the non-virtual agents, which have strictly positive y-values, we conclude [u^ℓ(k)]_h = 0 if ψ_h(k) = 0. Hence, the assumptions of Lemma 8 and Corollary 9 are satisfied. Adopting the definition of I^k and P(k) from previous sections, we get for i ∈ I^k+1.

$${[z^{\ell }(k+1)]}_i=\sum _{j\in I^k}{P}_{ij}(k)\frac{{[u^{\ell }(k)]}_j}{{\psi }_j(k)}.$$

Combining the equation above for ℓ = 1,…d we obtain:

$${\mathbf{z}}_i(k+1)=\sum _{j\in I^k}{P}_{ij}(k)\frac{{\mathbf{u}}_j(k)}{{\psi }_j(k)},$$ (29)

where ${\mathbf{u}}_j(k)\in {ℝ}^d$ is created by collecting the jth entries of all u^ℓ(k), i.e.,

$${\mathbf{u}}_i(k)=\{\begin{array}{cc}{\mathbf{x}}_i(k)-{\beta }_i(k){\widehat{\mathbf{g}}}_i(k),\hfill & \mathrm{if}i\in \mathcal{V}\text{and}{\tau }_i(k)=1,\hfill \\ {\mathbf{x}}_i(k),\hfill & \text{otherwise}.\hfill \end{array}$$

Now consider each term on the right hand side of (29) for j ∈ I^k. Suppose j ≤ n and τ_j(k) = 1, then we have:

$$\frac{{\mathbf{u}}_j(k)}{y_j(k)}={\mathbf{z}}_j(k)-\frac{{\beta }_j(k)}{y_j(k)}(\nabla f_j({\mathbf{z}}_j(k))+{\mathit{\epsilon }}_j(k)).$$

Since lim_k→∞ α(k) = 0 and k − κ_i(k) = 0 and k − κ_i(k) ≤ Γ_u, lim_k→∞ β_j(k) = 0. Moreover, by Lemma 11, y_j(k) is bounded below; thus, lim_k→∞ β_j(k)/y_j(k) = 0 and there exists k_j such that for k ≥ k_j, $,{\beta }_j(k)/y_j(k)\in \left(0,{\mu }_j/8L_j^2\right]$. Applying Lemma 17, it follows that for each j there exists a compact set ${\mathcal{S}}_j$ and a scalar R_j such that for k ≥ k_j, if τ_j(k) = 1,

$$\Vert \frac{{\mathbf{u}}_j(k)}{y_j(k)}\Vert \le \{\begin{array}{c}\Vert {\mathbf{z}}_j(k)\Vert \hfill \\ R_j,\hfill \end{array}\begin{array}{c}{\text{if}z}_j(k)\notin S_j,\hfill \\ {\text{if}z}_j(k)\in S_j.\hfill \end{array}$$ (30)

Moreover, if τ_j(k) = 0 or j > n we have,

$$\frac{{\mathbf{u}}_j(k)}{y_j(k)}={\mathbf{z}}_j(k).$$ (31)

Let k_z ≔ max_i k_i. Using mathematical induction, we will show that for all k ≥ k_z:

$$\underset{i\in I^k}{\max }\Vert {\mathbf{z}}_i(k)\Vert \le \overline{R},$$ (32)

where $\overline{R}≔\max \{{\max }_i{R}_i,{\max }_{j\in I^{k_z}}\Vert {\mathbf{z}}_j(k_z)\Vert \}$. Equation (32) holds for k = k_z. Suppose it is true for some k ≥ k_z. Then by (30) and (31) we have,

$$\Vert \frac{{\mathbf{u}}_i(k)}{y_i(k)}\Vert \le \max \{R_i,\Vert {\mathbf{z}}_i(k)\Vert \}\le \overline{R}.$$ (33)

Also by (29), for i ∈ I^k+1, z_i(k + 1) is a convex combination of u_j(k)/y_j(k)’s, where j ∈ I^k. Hence,

$$\Vert {\mathbf{z}}_i(k+1)\Vert \le \sum _{j\in I^k}{P}_{ij}\Vert \frac{{\mathbf{u}}_j(k)}{{\psi }_j(k)}\Vert \le \overline{R}.$$

Define $B_z≔\max \{\overline{R},{\max }_{i\in I^k,k<k_z}\Vert {\mathbf{z}}_i(k)\Vert \}$ and we have ‖z_i(k)‖ ≤ B_z, ∀k ≥ 0. ■

We next explore a convenient way to rewrite Algorithm 3. Let us introduce the quantity w_i(k), which can be interpreted as the x-value of agent i, if it performed a gradient step at every iteration, even when asleep:

$${\mathbf{w}}_i(k)=\{\begin{array}{cc}{\mathbf{x}}_i(k)-\left({\sum }_{t={\kappa }_i(k)+1}^{k-1}\alpha (t)\right){\mathbf{g}}_i(k),\hfill & \mathrm{if}i\in \mathcal{V},\hfill \\ {\mathbf{x}}_i(k),\hfill & \text{otherwise}.\hfill \end{array}$$ (34)

Also, define ${\mathbf{w}}^{\ell }\in {ℝ}^{n+m^{\prime }}$ by collecting the ℓth dimension of all w_i’s and $\overline{\mathbf{w}}(k)≔({\sum }_{i=1}^{n+m^{\prime }}{\mathbf{w}}_i(k))/n$. Moreover, define ${\mathbf{g}}^{\ell }\in {ℝ}^{n+m^{\prime }}$ by collecting the ℓth value of gradients of all agents (0 for virtual agents), i.e.,

$${[{\mathbf{g}}^{\ell }(k)]}_i=\{\begin{array}{cc}{[{\mathbf{g}}_i(k)]}_{\ell }\hfill & \mathrm{if}i\in \mathcal{V},\hfill \\ 0,\hfill & \text{otherwise}.\hfill \end{array}$$

Additionally, define ${\widehat{\mathit{\epsilon }}}_i(k)\in {ℝ}^d$ as the noise injected to the system at time k by agent i, i.e.,

$${\widehat{\mathit{\epsilon }}}_i(k)=\{\begin{array}{cc}{\beta }_i(k){\mathit{\epsilon }}_i(k),\hfill & \mathrm{if}i\in \mathcal{V}\text{and}{\tau }_i(k)=1,\hfill \\ 0,\hfill & \text{otherwise},\hfill \end{array}$$

and ${\widehat{\mathit{\epsilon }}}^l(k)\in {ℝ}^{n+m^{\prime }}$ as the vector collecting the ℓth values of all ${\widehat{\mathit{\epsilon }}}_i(k)$’s.

We then have the following lemma.

Lemma 19

$${\mathbf{w}}^{\ell }(k+1)=\mathbf{M}(k)\left({\mathbf{w}}^{\ell }(k)-\alpha (k){\mathbf{g}}^{\ell }(k)-{\widehat{\mathit{\epsilon }}}^{\ell }\right).$$ (35)

Proof We consider two cases:

• If τ_i(k) = 0, then (35) reduces to w_i(k + 1) = w_i(k) − α(k)g_i(k); noting that, because node i did not update at time k we have that g_i(k) = g_i(k + 1) and this is the correct update.

• For all other nodes (i.e., for both virtual nodes and nodes with τ_i(k) = 1, we have ${[{\mathbf{w}}^{\ell }(k)-\alpha (k){\widehat{\mathbf{g}}}^{\ell }(k)-{\widehat{\mathit{\epsilon }}}^{\ell }(k)]}_i={[{\mathit{\chi }}^{\ell }(k)+{\mathbf{\Delta }}^{\ell }(k)]}_i$ in (28). Since χ^ℓ(k+1) = M(k)(χ^ℓ(k) + Δ^ℓ(k)) and, using the definition of w_i(k), we have that for these nodes,

  $${\mathbf{w}}_i(k+1)={\mathbf{x}}_i(k+1);$$

(28) implies the conclusion. ■

This lemma allows us to straightforwardly analyze how the average of w(k) evolves. Indeed, summing all the elements of (35) and dividing by n for each ℓ = 1, …, d we obtain,

$$\begin{gathered} \overline{\mathbf{w}}(k+1)=\overline{\mathbf{w}}(k)-\frac{\alpha (k)}{n}\sum _{i=1}^n{\mathbf{g}}_i(k)-\frac{1}{n}\sum _{i=1}^n{\widehat{\mathit{\epsilon }}}_i(k) \\ =\overline{\mathbf{w}}(k)-\frac{\alpha (k)}{n}\sum _{i=1}^n\nabla f_i{(\overline{\mathbf{w}}(k))}_i-\frac{1}{n}\sum _{i=1}^n{\widehat{\mathit{\epsilon }}}_i(k)-\frac{\alpha (k)}{n}\sum _{i=1}^n({\mathbf{g}}_i(k)-\nabla f(\overline{\mathbf{w}}(k))). \end{gathered}$$ (36)

We next give a sequence of lemmas to the effect that all the quantities generated by the algorithm are close to each other over time. Define,

$$\overline{\mathbf{x}}(k)=\frac{1}{n}\sum _{a_h\in {\mathcal{V}}_A}{\mathbf{x}}_h(k).$$

where, recall, ${\mathcal{V}}_A$ is our notation for all the nodes in the augmented graph (i.e., including virtual nodes). Moreover, we will extend the definition of from Line 4 of Algorithm 3 to all k via the same formula ${\beta }_i(k)≔{\sum }_{t={\kappa }_i(k)+1}^k\alpha (t)$. Our first lemma will show that each z_i(k) closely tracks $\overline{\mathbf{x}}(k)$.

Lemma 20 Using Algorithm 3 with α(k) = n/(kμ), under the assumptions of Theorem 15, we have for each i, $\Vert {\mathbf{z}}_i(k+1)-\overline{\mathbf{x}}(k+1)\Vert ={\mathcal{O}}_k(1/k)$.

Proof By Theorem 14(a) we have for each ℓ,

$$\left|{[{\mathbf{z}}^{\ell }(k+1)]}_i-\frac{1^{⊺}{\mathit{\chi }}^{\ell }(k+1)}{n}\right|\le \delta {\mathrm{\lambda }}^k{\Vert {\mathit{\chi }}^{\ell }(0)\Vert }_1+\sum _{t=1}^k\delta {\mathrm{\lambda }}^{k-t}{\Vert {\mathbf{\Delta }}^{\ell }(t)\Vert }_1.$$

Summing the above inequality for ℓ = 1, …, d we obtain,

$${\Vert {\mathbf{z}}_i(k+1)-\overline{\mathbf{x}}(k+1)\Vert }_1\le \sum _{j=1}^n\left(\delta {\mathrm{\lambda }}^k{\Vert {\mathbf{x}}_j(0)\Vert }_1+\sum _{t=1}^k\delta {\mathrm{\lambda }}^{k-t}{\beta }_i(t){\tau }_i(t){\Vert {\widehat{\mathbf{g}}}_j(t)\Vert }_1\right).$$

Moreover,

$${\beta }_i(k)=\sum _{t={\kappa }_i(k)+1}^k\frac{n}{\mu t}\le \frac{n}{\mu }\left(\frac{k-{\kappa }_i(k)}{{\kappa }_i(k)+1}\right).$$ (37)

But,

$${\kappa }_i(k)<k\le {\kappa }_i(k)+{\mathrm{\Gamma }}_u.$$

Since Γ_u ≥ 1, we obtain

$$k\le ({\kappa }_i(k)+1){\mathrm{\Gamma }}_u,$$

or,

$$\frac{1}{{\kappa }_i(k)+1}\le \frac{{\mathrm{\Gamma }}_u}{k}.$$

Thus, from (37) we have,

$${\beta }_i(k)\le \frac{n{\mathrm{\Gamma }}_u^2}{\mu k}.$$ (38)

Define,

$$M_j≔\underset{\Vert \mathbf{z}\Vert \le B_z}{\max }{\Vert {\mathbf{g}}_j(\mathbf{z})\Vert }_1,$$ (39)

and observe that M_j is finite by Lemma 18. Also τ_j(k) ≤ 1. We obtain,

$${\Vert {\mathbf{z}}_i(k+1)-\overline{\mathbf{x}}(k+1)\Vert }_1\le \sum _{j=1}^n\left(\delta {\mathrm{\lambda }}^k{\Vert {\mathbf{x}}_j(0)\Vert }_1+\sum _{t=1}^k\delta {\mathrm{\lambda }}^{k-t}\frac{n{\mathrm{\Gamma }}_u^2}{\mu t}(M_j+b_j)\right).$$

Let RHS denote the right hand side of the relation above. We have,

$$\begin{gathered} RHS=\sum _{j=1}^n\left(\delta {\mathrm{\lambda }}^k{\Vert {\mathbf{x}}_j(0)\Vert }_1+\frac{\delta n{\mathrm{\Gamma }}_u^2}{\mu }(M_j+b_j)\left(\sum _{t=1}^{\lfloor \frac{k}{2}\rfloor }\frac{{\mathrm{\lambda }}^{k-t}}{t}+\sum _{t=\lfloor \frac{k}{2}\rfloor +1}^k\frac{{\mathrm{\lambda }}^{k-t}}{t}\right)\right) \\ \le \sum _{j=1}^n\left(\delta {\mathrm{\lambda }}^k{\Vert {\mathbf{x}}_j(0)\Vert }_1+\frac{\delta n{\mathrm{\Gamma }}_u^2}{\mu }(M_j+b_j)\left(\frac{k}{2}{\mathrm{\lambda }}^{\frac{k}{2}}+\frac{2}{(1-\mathrm{\lambda })k}\right)\right)={\mathcal{O}}_k\left(\frac{1}{k}\right), \end{gathered}$$

where we used the following relations,

$$\begin{gathered} \sum _{t=1}^{\lfloor \frac{k}{2}\rfloor }\frac{{\mathrm{\lambda }}^{k-t}}{t}\le \lfloor \frac{k}{2}\rfloor {\mathrm{\lambda }}^{k-\lfloor \frac{k}{2}\rfloor }\le \frac{k}{2}{\mathrm{\lambda }}^{\frac{k}{2}}, \\ \sum _{t=\lfloor \frac{k}{2}\rfloor +1}^k\frac{{\mathrm{\lambda }}^{k-t}}{t}\le \sum _{t=0}^{\lceil \frac{k}{2}\rceil -1}\frac{{\mathrm{\lambda }}^t}{\lfloor \frac{k}{2}\rfloor +1}\le \frac{2}{(1-\mathrm{\lambda })k}. \end{gathered}$$

Finally, ‖v‖₂ ≤ ‖v‖₁ for all vectors v, completes the proof. ■

An immediate consequence of this lemma is that the quantities $\overline{\mathbf{x}}(k)$ and $\overline{\mathbf{w}}(k)$ are close to each other.

Lemma 21 Using Algorithm 3 with α(k) = n/(kμ), under the assumptions of Theorem 15, we have, $\Vert \overline{\mathbf{x}}(k)-\overline{\mathbf{w}}(k)\Vert ={\mathcal{O}}_k(1/k)$.

Proof By definition of $\stackrel{‒}{\mathbf{w}}$ we have,

$$\overline{\mathbf{x}}(k)-\overline{\mathbf{w}}(k)=\frac{1}{n}\sum _{i=1}^n\left(\sum _{t={\kappa }_i(k)+1}^{k-1}\alpha (t)\right){\mathbf{g}}_i(k).$$

Using (38) we have,

$$\Vert \overline{\mathbf{x}}(k)-\overline{\mathbf{w}}(k)\Vert \le \frac{1}{n}\sum _{i=1}^n{\beta }_i(k)M_i\le \sum _{i=1}^n\frac{{\mathrm{\Gamma }}_u^2{M}_i}{n\mu k}={\mathcal{O}}_k\left(\frac{1}{k}\right),$$

where M_i was defined through (39). ■

We next remark on a couple of implications of the past series of lemmas.

Corollary 22 We have $\Vert {\mathbf{z}}_i(k)-\overline{\mathbf{w}}(k)\Vert ={\mathcal{O}}_k(\frac{1}{k})$.

Lemma 23 $\Vert {\mathbf{g}}_i(k)-\nabla f_i(\overline{\mathbf{w}}(k))\Vert ={\mathcal{O}}_k(\frac{1}{k})$.

Proof Since ∇f_i is L_i-Lipschitz, we have,

$$\Vert {\mathbf{g}}_i(k)-\nabla f_i(\overline{\mathbf{w}}(k))\Vert \le L_i\Vert {\mathbf{z}}_i(k)-\overline{\mathbf{w}}(k)\Vert .$$

Using Corollary 22, the lemma is proved. ■

We are now in a position to rewrite Algorithm 3 as a sort of perturbed gradient descent. Let us define,

$$\mathit{\eta }(k)≔\frac{1}{\mu k}\sum _{i=1}^n({\mathbf{g}}_i(k)-\nabla f_i(\overline{\mathbf{w}}(k))).$$

By Lemma 23, $\mathit{\eta }(k)={\mathcal{O}}_k(1/k^2)$. Therefore, there exists B_η such that η(k) ≤ B_η/k² for all k ≥ 1.

By (36) we have,

$$\overline{\mathbf{w}}(k+1)=\overline{\mathbf{w}}(k)-\frac{1}{\mu k}\nabla F(\overline{\mathbf{w}}(k))-\overline{\mathit{\epsilon }}(k)-\overline{\mathit{\eta }}(k),$$ (40)

where

• The function $F≔{\sum }_{i=1}^n{f}_i\in {ℝ}^d\to ℝ$ is μ-strongly-convex with L-Lipschitz gradient, where $L≔{\sum }_{i=1}^n{L}_i$.

• The noise $\overline{\mathit{\epsilon }}(k)≔({\sum }_{i=1}^n{\widehat{\mathit{\epsilon }}}_i(k))/n$ is bounded (i.e., $\overline{\mathit{\epsilon }}(k)\in B(0,r_e)$), with probability one, where r_e ≔ (Γ_u/μ)∑_j b_j), and $\mathbb{E}[\overline{\mathit{\epsilon }}(k)]=0$.

In other words, with the exception of the η(k) term, what we have is exactly a stochastic gradient descent method on the function F(⋅).

The following lemmas bound $\overline{\mathit{\epsilon }}(k)$. Let us define ν_i(k) = k − κ_i(k) as the number of iterations agent i has skipped since it’s last update. By Assumption 1, ν_i(k) ≤ Γ_u.

Lemma 24 We have ${\beta }_i(k)={\mathcal{O}}_k(1/k)$, ∀i. Moreover,

$${\beta }_i(k)\le \frac{n{\nu }_i(k)}{\mu k}+{\mathcal{O}}_k(k^{-2}).$$

Proof Since ν_i(k) ≤ Γ_u, ∀i, we have for κ_i(k) ≥ 1,

$$\begin{gathered} {\beta }_i(k)=\sum _{t={\kappa }_i(k)+1}^k\frac{n}{\mu t}\le \frac{n}{\mu }\ln \left(\frac{k}{{\kappa }_i(k)}\right)\le \frac{n}{\mu }\ln \left(\frac{k}{k-{\nu }_i(k)}\right) \\ =\frac{n}{\mu }\ln \left(\frac{{\nu }_i(k)}{k-{\nu }_i(k)}\right)\le \frac{n{\nu }_i(k)}{\mu (k-{\nu }_i(k))}=\frac{n{\nu }_i(k)}{\mu k}+{\mathcal{O}}_k(k^{-2}).■ \end{gathered}$$

Corrllary 25 $\mu k\Vert \overline{\mathit{\epsilon }}(k)\Vert$ is bounded.

Lemma 26 There exists B_ϵ > 0 such that We have,

$$\mathbb{E}[{\Vert \overline{\mathit{\epsilon }}(k)\Vert }^2]\le \frac{{\mathrm{\Gamma }}_u^2}{{\mu }^2{k}^2}{\sigma }^2+\frac{B_{ϵ}}{k^4}.$$

Proof Using Lemma 24, we have for k > Γ_u,

$$\begin{gathered} \mathbb{E}[{\Vert \overline{\mathit{\epsilon }}(k)\Vert }^2]=\mathbb{E}[{\Vert \frac{1}{n}\sum _{i=1}^n{\beta }_i(k){\mathit{\epsilon }}_i(k){\tau }_i(k)\Vert }^2]=\frac{1}{n^2}\sum _{i=1}^n{\beta }_i^2(k)\mathbb{E}[{\Vert {\mathit{\epsilon }}_i(k)\Vert }^2] \\ \le \frac{1}{n^2}\sum _{i=1}^n{\beta }_i^2(k){\sigma }_i^2\le \frac{{\mathrm{\Gamma }}_u^2}{{\mu }^2{k}^2}{\sigma }^2+{\mathcal{O}}_k(k^{-4}), \end{gathered}$$

where the second equality is the result of the noise terms being independent and zero-mean. ■

Our next observation is a technical lemma which is essentially a rephrasing of Lemma 17 above.

Lemma 27 There exists a constant B_w and time k_w such that $\Vert \overline{\mathbf{w}}(k)\Vert \le B_w$ with probability one, for k ≥ k_w.

Proof We have

$$\overline{\mathbf{w}}(k+1)=\overline{\mathbf{w}}(k)-\frac{1}{\mu k}[\nabla F(\overline{\mathbf{w}}(k))+\mu k(\overline{\mathit{\epsilon }}(k)+\mathit{\eta }(k))],$$

where $\mu k\Vert \overline{\mathit{\epsilon }}(k)+\mathit{\eta }(k)\Vert$ is bounded. Moreover, there exists k_w such that for k ≥ k_w, $\frac{1}{\mu k}\in \left(0,\mu /8L^2\right]$. Therefore, by Lemma 17 there exists a compact set ${\mathcal{S}}_w$ and a scalar R_w > 0 such that for k ≥ k_w,

$$\Vert \overline{\mathbf{w}}(k+1)\Vert \le \{\begin{array}{cc}\Vert \overline{\mathbf{w}}(k)\Vert ,\hfill & \mathrm{for}\overline{\mathbf{w}}\in {\mathcal{S}}_w,\hfill \\ R_w,\hfill & \mathrm{for}\overline{\mathbf{w}}\in {\mathcal{S}}_w.\hfill \end{array}$$

Therefore, setting $B_w≔\max \{R_w,\Vert \overline{\mathbf{w}}(k_w)\Vert \}$ will complete the proof. ■

As a consequence of this lemma, because ${\Vert \mathit{\eta }(k)\Vert }_2\le B_{\eta }$, this lemma implies there is a constant B₁ such that for k ≥ k_w,

$$\Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }-\frac{1}{\mu k}\nabla F(\overline{\mathbf{w}}(k))-\overline{\mathit{\epsilon }}(k)\Vert \le B_1,$$ (41)

with probability one. This now puts us in a position to show that $\overline{\mathbf{w}}(k)$ converges in mean square to the optimal solution.

Lemma 28 $\mathbb{E}[{\Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }\Vert }^2]\to 0$.

Proof Using the definition of k_w from Lemma 27, we have that for k ≥ k_w,

$$\mathbb{E}[{\Vert \overline{\mathbf{w}}(k+1)-{\mathbf{z}}^{\ast }\Vert }^2]\le \mathbb{E}[{\Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }-\frac{1}{\mu k}\nabla F(\overline{\mathbf{w}}(k))-\overline{\mathit{\epsilon }}(k)\Vert }^2+2\Vert \mathit{\eta }(k)\Vert \Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }-\frac{1}{\mu k}\nabla F(\overline{\mathbf{w}}(k))-\overline{\mathit{\epsilon }}(k)\Vert +{\Vert \mathit{\eta }(k)\Vert }^2].$$

We will bound each of the terms on the right. We begin with the easiest one, which is the last one:

$${\Vert \mathit{\eta }(k)\Vert }^2\le \frac{B_{\eta }^2}{k^4}.$$ (42)

The middle term is bounded as

$$2\Vert \mathit{\eta }(k)\Vert \Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }-\frac{1}{\mu k}\nabla F(\overline{\mathbf{w}}(k))-\overline{\mathit{\epsilon }}(k)\Vert \le \frac{2B_{\eta }{B}_1}{k^2},$$ (43)

where we used (41).

Finally, we turn to the first term which we denote by T₁:

$$T_1\le \mathbb{E}{\Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }\Vert }^2-\frac{2}{\mu k}\mathbb{E}[\nabla F{(\overline{\mathbf{w}}(k))}^{⊺}(\overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast })]+\frac{L^2}{{\mu }^2{k}^2}\mathbb{E}[{\Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }\Vert }^2]+\mathbb{E}[{\Vert \overline{\mathit{\epsilon }}(k)\Vert }^2],$$

where we used the usual inequality ${\Vert \nabla F(\overline{\mathbf{w}}(k))\Vert }^2\le L^2{\Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }\Vert }^2$ which follows from ∇F(⋅) being L-Lipschitz. Now, using the standard inequality

$$\begin{gathered} \nabla F{(\overline{\mathbf{w}}(k))}^{\text{T}}(\overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast })\ge F(\overline{\mathbf{w}}(k))-F({\mathbf{z}}^{\ast })+\frac{\mu }{2}{\Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }\Vert }^2 \\ \ge \mu {\Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }\Vert }^2, \end{gathered}$$

and Lemma 26 we obtain,

$$T_1\le \left(1-\frac{2}{k}+\frac{L^2}{{\mu }^2{k}^2}\right)\mathbb{E}[{\Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }\Vert }^2]+\frac{{\mathrm{\Gamma }}_u^2}{{\mu }^2{k}^2}{\sigma }^2+\frac{B_{ϵ}}{k^4}.$$ (44)

Now putting together (42), (43), and (44), we get,

$$\mathbb{E}[{\Vert \overline{\mathbf{w}}(k+1)-{\mathbf{z}}^{\ast }\Vert }^2]\le \left(1-\frac{2}{k}+\frac{L^2}{{\mu }^2{k}^2}\right)\mathbb{E}[{\Vert \overline{\mathbf{w}}(k)-x^{\ast }\Vert }^2]+\frac{{\mathrm{\Gamma }}_u^2{\sigma }^2}{{\mu }^2{k}^2}+\frac{2B_{\eta }{B}_1}{k^2}+\frac{B_{\eta }^2+B_{ϵ}}{k^4}.$$

For large enough k, we can bound the inequality above as,

$$\mathbb{E}[{\Vert \overline{\mathbf{w}}(k+1)-{\mathbf{z}}^{\ast }\Vert }^2]\le \left(1-\frac{1.5}{k}\right)\mathbb{E}[{\Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }\Vert }^2]+\frac{B_2}{k^2},$$ (45)

where $B_2={\mathrm{\Gamma }}_u^2{\sigma }^2/{\mu }^2+2B_{\eta }{B}_1+B_{\eta }^2+B_{ϵ}$. Using Lemma 29, stated next, we conclude $\mathbb{E}[{\Vert \overline{\mathbf{w}}(k+1)-{\mathbf{z}}^{\ast }\Vert }^2]\to 0$. ■

Lemma 29 Let a > 1, b ≥ 0 and {x_t} be a non-negative sequence which satisfies,

$$x_{t+1}\le \left(1-\frac{a}{t}\right)x_t+\frac{b}{t^2},\mathit{for}t\ge t^{\prime }>0.$$

Then for all t ≥ t’ we have,

$$x_t\le \frac{m}{t},$$

where m ≔ max{t′x_t′,b/(a − 1)}.

This lemma is stated and proved for t′ = 1 in (Rakhlin et al., 2012, Lemma 3), and the case of general t′ follows immediately.

We are almost ready to complete the proof of Theorem 15; all that is needed is to refine the convergence rate of $\overline{\mathbf{w}}(k)$ to x*. Now as a consequence of (45) and Lemma 29, we may use the inequality $\mathbb{E}[|X|]\le \sqrt{\mathbb{E}[X^2]}$ to obtain that

$$\mathbb{E}[\Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }\Vert ]={\mathcal{O}}_k\left(\frac{1}{\sqrt{k}}\right).$$ (46)

Furthermore, by the finite support of $\mu k\overline{\mathit{\epsilon }}(k)$, by Corollary 25, we also have that

$$\mathbb{E}[\Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }-\frac{1}{\mu k}\nabla F(\overline{\mathbf{w}}(k))-\overline{\mathit{\epsilon }}(k)\Vert ]={\mathcal{O}}_k\left(\frac{1}{\sqrt{k}}\right).$$ (47)

We now use these observations to provide a proof of our main result.

Proof of Theorem 15 Essentially, we rewrite the proof of Lemma 28, but now using the fact that $\mathbb{E}[\Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }\Vert ]={\mathcal{O}}_k(1/\sqrt{k})$ from (46). This allows us to make two modification to the arguments of that lemma. First, we can now replace (43) by

$$\mathbb{E}[2\Vert \mathit{\eta }(k)\Vert \Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }-\frac{1}{\mu k}\nabla F(\overline{\mathbf{w}}(k))-\overline{\mathit{\epsilon }}(k)\Vert ]\le \frac{2B_{\eta }}{k^2}{\mathcal{O}}_k\left(\frac{1}{\sqrt{k}}\right),$$ (48)

where we used (47). Second, putting together (42), (48), and (44), we obtain:

$$\begin{gathered} \mathbb{E}[{\Vert \overline{\mathbf{w}}(k+1)-{\mathbf{z}}^{\ast }\Vert }^2]\le \left(1-\frac{2}{k}+\frac{L^2}{{\mu }^2{k}^2}\right)\mathbb{E}[{\Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }\Vert }^2] \\ +\mathbb{E}[{\Vert \overline{\mathit{\epsilon }}(k)\Vert }^2]+\frac{B_{\eta }^2}{k^4}+\frac{2B_{\eta }}{k^2}{\mathcal{O}}_k\left(\frac{1}{\sqrt{k}}\right). \end{gathered}$$

which, again using the fact that $\mathbb{E}[{\Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }\Vert }^2]={\mathcal{O}}_k(1/\sqrt{k})$, we simply rewrite as,

$$\mathbb{E}[{\Vert \overline{\mathbf{w}}(k+1)-{\mathbf{z}}^{\ast }\Vert }^2]\le \left(1-\frac{2}{k}\right)\mathbb{E}[{\Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }\Vert }^2]+\mathbb{E}[{\Vert \overline{\mathit{\epsilon }}(k)\Vert }^2]+{\mathcal{O}}_k\left(\frac{1}{k^{2.5}}\right).$$

To save space, let us define $a_k≔\mathbb{E}[{\Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }\Vert }^2]$. Multiplying both sides of relation above by k² we obtain,

$$a_{k+1}{k}^2\le a_k\left(1-\frac{2}{k}\right)k^2+\mathbb{E}[{\Vert \overline{\mathit{\epsilon }}(k)\Vert }^2]k^2+{\mathcal{O}}_k(k^{-0.5}).$$

Note that,

$$\left(1-\frac{2}{k}\right)k^2=k^2-2k<{(k-1)}^2.$$

Thus,

$$a_{k+1}{k}^2\le a_k{(k-1)}^2+\mathbb{E}[{\Vert \overline{\mathit{\epsilon }}(k)\Vert }^2]k^2+{\mathcal{O}}_k(k^{-0.5}).$$

Summing the relation above for k = 0, …, T implies,

$$a_{T+1}{T}^2\le \sum _{k=0}^T\mathbb{E}[{\Vert \overline{\mathit{\epsilon }}(k)\Vert }^2]k^2+{\mathcal{O}}_T(T^{0.5}).$$

Now, let us estimate the first term on the right hand side of relation above,

$$\sum _{k=0}^T\mathbb{E}[{\Vert \overline{\mathit{\epsilon }}(k)\Vert }^2]k^2\le \sum _{k=0}^T\sum _{i=1}^n\frac{{\beta }_i^2(k)}{n^2}{\sigma }_i^2{\tau }_i(k)k^2=\sum _{i=1}^n\frac{{\sigma }_i^2}{{\mu }^2}\sum _{k=0}^T{\nu }_i{(k)}^2{\tau }_i(k)+{\mathcal{O}}_T(T^{-1}),$$

where we used Lemma 24 in the last equality. Define t_i(j) as the j’th time agent i has woken up, and set t_i(0) = −1. Then we can rewrite the relation above as,

$$\sum _{k=0}^T{\nu }_i{(k)}^2{\tau }_i(k)=\sum _{j=1}^{t_i(j)\le T}{(t_i(j)-t_i(j-1))}^2\le \sum _{j=1}^{t_i(j)\le T}{\mathrm{\Gamma }}_u(t_i(j)-t_i(j-1))\le {\mathrm{\Gamma }}_u(T+1).$$

Combining relations above and then dividing both sides by T² we obtain,

$$a_{T+1}\le \frac{{\mathrm{\Gamma }}_u{\sigma }^2}{{\mu }^{2}T}+{\mathcal{O}}_T(T^{-1.5}).$$ (49)

We next argue that the same guarantee holds for every z_i(k). Indeed, for each i = 1, …, m,

$$\begin{gathered} {\Vert {\mathbf{z}}_i(k)-{\mathbf{z}}^{\ast }\Vert }^2={\Vert {\mathbf{z}}_i(k)-\overline{\mathbf{w}}(k)+\overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }\Vert }^2 \\ ={\Vert {\mathbf{z}}_i(k)-\overline{\mathbf{w}}(k)\Vert }^2+2\Vert {\mathbf{z}}_i(k)-\overline{\mathbf{w}}(k)\Vert \Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }\Vert +{\Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }\Vert }^2. \end{gathered}$$

Now from Corollary 22, we know that with probability one, ${\Vert {\mathbf{z}}_i(k)-\mathbf{w}(k)\Vert }_2={\mathcal{O}}_k(1/k)$. Taking expectation of both sides and using (49) along with the usual bound $\mathbb{E}[|X|]\le \sqrt{\mathbb{E}[X^2]}$, we have

$$\mathbb{E}[{\Vert {\mathbf{z}}_i(k)-{\mathbf{z}}^{\ast }\Vert }^2]={\mathcal{O}}_k\left(\frac{1}{k^2}\right)+{\mathcal{O}}_k\left(\frac{1}{k^{1.5}}\right)+\mathbb{E}[{\Vert \overline{\mathbf{w}}(k)-{\mathbf{z}}^{\ast }\Vert }^2].$$

Putting this together with (49) completes the proof. ■

3.1. Time-Varying Graphs

We remark that Theorems 6, 14 and 15 all extend verbatim to the case of time-varying graphs with no message losses. Indeed, only one problem appears in extending the proofs in this paper to time-varying graphs: a node i may send a message to node j; that message will be lost; and afterwards node i never sends anything to node j again. In this case, Lemmas 7 and 11 do not hold. Indeed, examining Lemma 11, we observe what can very well happen is that all of χ_i(k) and ψ_i(k) are “lost” over time into messages that never arrive. However, as long as no messages are lost, the proofs in this paper extend to the time-varying case verbatim. On a technical level, the results still hold if ${\mathbf{u}}_{\mathit{ij}}^x(k)=0$ , $u_{\mathit{ij}}^y(k)=0$ (virtual node $c_{ij}\in V_A$ holds no lost message), when link (i,j) is removed from the network at time k, and the graph $\mathcal{G}$ stays strongly connected (or B-connected, i.e., there exists a positive integer B such that the union of every B consecutive graphs is strongly connected).

3.2. On the Bounds for Delays, Asynchrony, and Message Losses

It is natural to what extent the assumption of finite upper bounds on delays, asynchrony, and message losses are really necessary. A natural example which falls outside our framework is a fixed graph G, where, at each time step, every link in G appears with probability 1/2. A more general model might involve a different probability p_e of failure for each edge e.

We observe that our result can already handle this case in the following manner. For simplicity, let us stick with the scenario where every link appears with probability 1/2. Then the probability that, after time t, some link has not appeared is at most m(1/2)^t, where m is the number of edges in G. This implies that if we choose B = O(log(mnT)), then with hight probability, the sequence of graphs G₁,…G_T is B-connected.

Thus our theorem applies to this case, albeit at the expense of some logarithmic factors due to the choice of B. We remark that it is possible to get rid of these factors by directly analyzing the decrease in $\mathbb{E}[{\Vert z(t)-z^{\ast }\Vert }_2^2]$ coming from the random choice of graph G. Since our arguments are already quite lengthy, we do not pursue this generalization here, and refer the reader to Lobel and Ozdaglar (2010); Srivastava and Nedic (2011) where similar arguments have been made.

4. Numerical Simulations

4.1. Setup

In this section, we simulate the RASGP algorithm on two classes of graphs, namely, random directed graphs and bidirectional cycle graphs. The main objective function is chosen to be a strongly convex and smooth Support Vector Machine (SVM), i.e. $F(\mathit{\omega },\gamma )=\frac{1}{2}({\Vert \mathit{\omega }\Vert }^2+{\gamma }^2)+C_N{\sum }_{j=1}^{N}h(b_j({\mathbf{A}}_j^{⊺}\mathit{\omega }+\gamma ))$ where $\mathit{\omega }\in {ℝ}^{d-1}$ and $\gamma \in ℝ$ are the optimization variables, and ${\mathbf{A}}_j\in {ℝ}^{d-1},b_j\in \{-1,+1\}$, j = 1, …, N are the data points and their labels, respectively. The coefficient $C_N\in ℝ$ penalizes the points outside of the soft margin. We set C_N = c/N, c = 500 in our simulations, which depends on the total number of data points. Here, $h:ℝ\to ℝ$ is the smoothed hinge loss, initially introduced in Rennie and Srebro (2005), defined as follows:

$$h(\xi )=\{\begin{array}{cc}-0.5-\xi ,\hfill & \text{if}\xi <0,\hfill \\ 0.5{(1-\xi )}^2,\hfill & \text{if}0\le \xi <1,\hfill \\ 0,\hfill & \text{if}1\le \xi .\hfill \end{array}$$

To solve this problem in a distributed way, we suppose all data points are spread among agents. Hence, the local objective functions are $f_i({\mathit{\omega }}_i,{\gamma }_i)=\frac{1}{2n}({\Vert \mathit{\omega }\Vert }^2+{\gamma }^2)+C_N{\sum }_{j\in D_i}h(b_j({\mathbf{A}}_j^{⊺}\mathit{\omega }+\gamma ))$, where D_i ⊂ {1,2, …, N} is an index set for data points of agent i and N is the total number of data points. We choose the size of the data set for each local function to be a constant (|D_i| = 50), thus N = 50n. It is easy to check that each f_i has Lipschitz gradients and is strongly convex with μ_i = 1/n.

We will compare our results with a centralized gradient descent algorithm, which updates every Γ_u iterations using the step-size sequence α_c(k) = Γ_u/(μk), in the direction of the sum of the gradients of all agents.

To make gradient estimates stochastic, we add a uniformly distributed noise ${\mathit{\epsilon }}_i\sim \mathcal{U}{[-b/2,b/2]}^d$ to the gradient estimates of each agent and ${\mathit{\epsilon }}_c\sim \mathcal{U}{[-\sqrt{nb/2},\sqrt{nb/2}]}^d$ to the gradient of the centralized gradient descent, where $\mathbf{U}{[b_1,b_2]}^d$ denotes the uniform distribution of size d over the interval [b₁,b₂), b₁ < b₂. Note that ε_i and ε_c are bounded and have zero mean and $\mathbb{E}[{\Vert {\epsilon }_i\Vert }^2]=d{b}^2/12$ and $\mathbb{E}[{\Vert {\epsilon }_c\Vert }^2]=d{b}^2/12$. We set b = 4 for all simulations.

Agents wake up with probability P_w and links fail with probability P_f, unless they reach their maximum allowed value where the algorithm forces the agent to wake up or the link to work successfully. The link delays are chosen uniformly between 1 to Γ_del.

Each data set D_i is synthetically generated by picking 25 data points around each of the centers (1,1) and (3,3) with multivariate normal distributions, labeled −1 and +1, respectively. In generating strongly connected random graphs, we pick each edge with a probability of 0.5 and then check if the resulting graph is strongly connected; if it isn’t, we repeat the process. Since the initial step-sizes for the distributed algorithm can be very large (e.g., α(1) = 50 for n = 50), to stabilize the algorithms, both algorithms are started with k₀ = 100. This wouldn’t affect the asymptotic convergence performance. Moreover, the initial point of the centralized algorithm and all agents in RASGP are chosen as 1_d.

Let us denote by $\mathbf{z}(k)≔(1/n){\sum }_{i=1}^n{\text{z=}}_i(k)$ the average of z-values of non-virtual agents. Then, we define optimization errors $E_{dist}≔{\Vert \mathbf{z}(k)-{\mathbf{z}}^{\ast }\Vert }^2$ and $E_c(k)≔{\Vert {\mathbf{x}}_c(k)-{\mathbf{z}}^{\ast }\Vert }^2$ for RASGP and Centralized stochastic gradient descent, respectively.

Since our performance guarantees are for the expectation of (squared) errors, for each network setting, we perform up to 1000 Monte-Carlo simulations and use their corresponding performance to estimate the average behavior of the algorithms. Since accurately estimating the true expected value requires an extremely large number of simulations, in order to alleviate the effect of spikes and high variance, we take the following steps. First a batch of simulations are performed and their average is calculated. Next, to obtain a smoother plot, an average over every 100 iterations is taken. And finally, the median of these outputs over all the batches is our estimate of the expected value.

We report two figures for each setting: one including the errors E_dist and E_c, and another one including k × E_dist and k × E_c to demonstrate the convergence rates.

Finally, to study the non-asymptotic behavior of RASGP and its dependence on network size n, we have compared the performance of the centralized stochastic gradient descent and RASGP over a bidirectional cycle graph, with error variances of $n^2{\widehat{\sigma }}^2$ and ${\sigma }_i^2={\widehat{\sigma }}^2$, respectively. Then, we plot the ratio E_c(k)/E_dist(k) over n, for different iterations k.

4.2. Results

Our simulation results are consistent with our theoretical claims (due to the performance of centralized and decentralized methods growing closer over time) and show the achievement of an asymptotic network-independent convergence rate.

Fig. 3 shows that when there is no link failure or delay and all agents wake up at every iteration (Γ_s = 2), RASGP and centralized gradient descent have very similar performance. When we allow links to have delays and failures (see Fig. 4), as well as asynchronous updates (see Fig. 5), it takes longer for RASGP to reach its asymptotic convergence rate.

Figure 3:

Results on a directed cycle graph of size n = 50, synchronous with no delays and link failures (P_w = 1, P_f = 0, Γ_del = Γ_f = 0, Γ_u = 1, Γ_s = 2).

Figure 4:

Results on a directed cycle graph of size n = 50, synchronous with delays and link failures (P_w = 1, P_f = 0.3, Γ_del = Γ_f = 3, Γ_u = 1, Γ_s = 7).

Figure 5:

Results on a directed cycle graph of size n = 50, asynchronous with delays and link failures (P_w = 0.5, P_f = 0.3, Γ_del = Γ_f = 3, Γ_u = 3, Γ_s = 17).

We observe that, with all the other parameters fixed, the RASGP performs better on a random graph than on a cycle graph (see Figs. 5 and 6). A possible reason is that the cycle graph has a higher diameter or mixing time compared to the random graph, resulting in a slower decay of the consensus error.

Figure 6:

Results on a directed random graph of size n = 50, asynchronous with delays and link failures (P_w = 0.5, P_f = 0.3, Γ_del = Γ_f = 3, Γ_u = 3, Γ_s = 17).

We notice that by fixing the network size, increasing the number of iterations brings us closer to linear speed-up (see Fig. 7). On the other hand, when fixing the number of iterations, increasing the number of nodes, after a certain point, does not help speeding up the optimization. Moreover, by allowing link delays and failures (see Fig. 7b) we require more iterations to achieve network independence.

Figure 7:

Error ratio over network size. Shaded areas correspond to 1-standard-deviation of the performance.

5. Conclusions

The main result of this paper is to stablish asymptotically, network independent performance for a distributed stochastic optimization method over directed graphs with message losses, delays, and asynchronous updates. Our work raises several open questions.

The most natural question raised by this paper concerns the size of the transients. How long must the nodes wait until the network-independent performance bound is achieved? The answer, of course, will depend on the network, but also on the number of nodes, the degree of asynchrony, and the delays. Understanding how this quantity scales is required before the algorithms presented in this work can be recommended to practitioners.

More generally, it is interesting to ask which problems in distributed optimization can achieve network-independent performance, even asymptotically. For example, the usual bounds for distributed subgradient descent (see, e.g., Nedic et al., 2018) depend on the spectral gap of the underlying network; various worst-case scalings with the number of nodes can be derived, and the final asymptotics are not network-independent. It is not immediately clear whether this is due to the analysis, or a fundamental limitation that will not be overcome.

Appendix A. Proof of Lemma 4

Proof We use mathematical induction. For k = 0 we have $x_{ij}^l(0)=0$, ∀l and $u_{ij}^x(0)={\varphi }_i^x(0)={\rho }_{ji}^x(0)=0$. By (6) and the definition of $u_{ij}^x$ and $x_{ij}^l$ we obtain,

$$\begin{gathered} {\rho }_{ji}^x(1)=0, \\ {u}_{ij}^x(1)=(1-\sum _{l=1}^{{\mathrm{\Gamma }}_d}{\tau }_{ij}^l(0)){\varphi }_i^x(1), \\ \sum _{l=1}^{{\mathrm{\Gamma }}_d}{x}_{ij}^l(1)=(\sum _{l=1}^{{\mathrm{\Gamma }}_d}{\tau }_{ij}^l(0)){\varphi }_i^x(1) \end{gathered}$$

Equation (12) is concluded from first equation above and (13) results by summing up all three equations above.

Now assume this lemma is true for k = 0, …, K − 1. We want to show it will be true for k = K as well. In the following, LHS and RHS denote the left-hand-side and right-hand-side of (12) for k = K. By (6) we have,

$$LHS=\sum _{l=1}^{{\mathrm{\Gamma }}_d}{\tau }_{ij}^l(K-l)[{\varphi }_i^x(K+1-l)-{\rho }_{ji}^x(K)].$$

Using (11) we obtain,

$$RHS=\sum _{l=1}^{{\mathrm{\Gamma }}_d}{\tau }_{ij}^l(K-l)v_{ij}^x(K-l).$$

Hence, it suffices to show that:

$$\sum _{l=1}^{{\mathrm{\Gamma }}_d}{\tau }_{ij}^l(K-l)[{\varphi }_i^x(K+1-l)-{\rho }_{ji}^x(K)-v_{ij}^x(K-l)]=0.$$ (50)

By part (e) of Assumption 1, at most one of the ${\tau }_{ij}^l(K-l)$, l = 1, …, Γ_d is non-zero. If all are zeros, the result follows. Now suppose ${\tau }_{ij}^l(K-l)=1$ for some l. Equation (50) becomes,

$${\varphi }_i^x(K+1-l)-{\rho }_{ji}^x(K)-v_{ij}^x(K-l)=0.$$

Plugging in the definition of $v_{ij}^x$, after rearrangement we obtain,

$${\varphi }_i^x(K-l)-u_{ij}^x(K-l)={\rho }_{ji}^x(K).$$ (51)

By the induction hypothesis, (12) holds for k = K − t, t = 1, …, l. Therefore,

$${\rho }_{ji}^x(K+1-t)-{\rho }_{ji}^x(K-t)=x_{ij}^1(K-t).$$

Hence,

$$\begin{gathered} {\rho }_{ji}^x(K)={\rho }_{ji}^x(K-l)+\sum _{t=1}^l({\rho }_{ji}^x(K+1-t)-{\rho }_{ji}^x(K-t)) \\ ={\rho }_{ji}^x(K-l)+\sum _{t=1}^l{x}_{ij}^1(K-t) \\ ={\rho }_{ji}^x(K-l)\sum _{l^{\prime }=1}^d{x}_{ij}^{l^{\prime }}(K-l)(\text{Lemma}3) \\ ={\rho }_{ji}^x(K-l)+\sum _{l^{\prime }=1}^d{x}_{ij}^{l^{\prime }}(K-l).(\text{Lemma}2) \end{gathered}$$

Moreover, by the induction hypothesis, (13) holds for k = K − l, thus,

$${\varphi }_i^x(K-l)-u_{ij}^x(K-l)={\rho }_{ji}^x(K-l)+\sum _{l^{\prime }=1}^{{\mathrm{\Gamma }}_d}{x}_{ij}^{l^{\prime }}(K-l).$$

Combining the two relations above we conclude (51).

To show (13), consider the following equations which are direct results of the definitions and (12) that we just showed for k = K:

$$\begin{gathered} u_{ij}^x(K+1)=(1-\sum _{l=1}^{{\mathrm{\Gamma }}_d}{\tau }_{ij}^l(K)v_{ij}^x(K)), \\ {\rho }_{ji}^x(K+1)={\rho }_{ji}^x(K)+x_{ij}^1(K), \\ \sum _{l=1}^{{\mathrm{\Gamma }}_d}{x}_{ij}^l(K+1)=\sum _{l=2}^{{\mathrm{\Gamma }}_d}{x}_{ij}^l(K)+\sum _{l=1}^{{\mathrm{\Gamma }}_d}{\tau }_{ij}^l(K)v_{ij}^x(K). \end{gathered}$$

Summing up both sides of the equations above we have,

$$\begin{gathered} LHS=u_{ij}^x(K+1)+{\rho }_{ji}^x(K+1)+\sum _{l=1}^{{\mathrm{\Gamma }}_d}{x}_{ij}^l(K+1), \\ RHS=\sum _{l=1}^{{\mathrm{\Gamma }}_d}{x}_{ij}^l(K)+{\rho }_{ji}^x(K)+v_{ij}^x(K) \\ =\sum _{l=1}^{{\mathrm{\Gamma }}_d}{x}_{ij}^l(K)+{\rho }_{ji}^x(K)+u_{ij}^x(K)-{\varphi }_i^x(k)+{\varphi }_i^x(K+1)={\varphi }_i^x(K+1). \end{gathered}$$

The last equality holds because of the induction hypothesis (13) for k = K − 1, hence completing the proof. ■