CONSISTENT SPECTRAL CLUSTERING OF NETWORK BLOCK MODELS UNDER LOCAL DIFFERENTIAL PRIVACY

Abstract

The stochastic block model (SBM) and degree-corrected block model (DCBM) are network models often selected as the fundamental setting in which to analyze the theoretical properties of community detection methods. We consider the problem of spectral clustering of SBM and DCBM networks under a local form of edge differential privacy. Using a randomized response privacy mechanism called the edge-flip mechanism, we develop theoretical guarantees for differentially private community detection, demonstrating conditions under which this strong privacy guarantee can be upheld while achieving spectral clustering convergence rates that match the known rates without privacy. We prove the strongest theoretical results are achievable for dense networks (those with node degree linear in the number of nodes), while weak consistency is achievable under mild sparsity (node degree greater than $\sqrt{n}$). We empirically demonstrate our results on a number of network examples.

1. Introduction

In the field of network data analysis, a common problem is community detection, the algorithmic discovery of clusters or communities of dense connection. There exist numerous parametric network models that exhibit community structure. Two fundamental models used in the analysis of community detection algorithms are the stochastic block model (SBM), first introduced in Holland et al. (1983), and the degree-corrected block model (DCBM) of Karrer and Newman (2011), a popular SBM extension that allows for networks with more realistic heterogeneity in node degree. In addition to theoretical interest, this class of models has proven useful in modeling patterns of interaction in various settings ranging from social networks and political science to the natural sciences (e.g., Wang and Wong, 1987; Daudin et al., 2008; Decelle et al., 2011; Karrer and Newman, 2011; Kim et al., 2018; Lee and Wilkinson, 2019, and references therein).

We consider the problem of estimating latent community structure in networks using SBM and DCBM via the method of spectral clustering. In the absence of privacy, this problem has gained popularity, as spectral clustering has been shown to be a computationally tractable method for group estimation with satisfying theoretical guarantees (McSherry, 2001; Ng et al., 2002; Rohe et al., 2011; Qin and Rohe, 2013; Lei and Rinaldo, 2015; Joseph and Yu, 2016; Binkiewicz et al., 2017; Abbe, 2018; Abbe et al., 2021). In this setting, the clusters of a given network are informed by the relationships contained within that network. These relationships are represented by the edges of the network, which may reflect friendships, partnerships, collaborations, communications, financial transactions, similarities, or other interactions between some set of entities, say, people or businesses.

In many cases, these relationships may constitute sensitive or confidential information, and so we may desire a clustering that reveals high-level structures in the network while preserving the privacy of low-level relationships. To provide such a privacy guarantee, we turn to a form of local differential privacy (DP) for networks (see, e.g., Imola et al. (2021) for networks, and Duchi et al. (2013) for more general ideas on local DP). We assume that the nodes of the network are known but that their relationships are sensitive. We protect these relationships through a randomized-response mechanism applied to the edges of the network. Previous works, such as Mülle et al. (2015) and Karwa et al. (2017), have used similar techniques to obtain a synthetic network that satisfies $\epsilon$-DP (of Dwork et al., 2006) with respect to the edges of the network. Recent works (Qin et al., 2017; Imola et al., 2021) have emphasized that these synthetic networks can be constructed in a distributed fashion, requiring that no single party has knowledge of the full set of true network relationships. We construct our synthetic networks in such a manner, then apply a modified spectral clustering algorithm to obtain consistent estimates of group membership for SBM and DCBM networks.

The problem of performing DP community detection has previously been given empirical consideration (Mülle et al., 2015; Nguyen et al., 2016; Qin et al., 2017), but to our knowledge, our analysis of the cost of privacy in this setting is the first theoretical treatment of the subject.¹ Our work generalizes the non-private results of Lei and Rinaldo (2015) to the DP setting, and we demonstrate certain conditions under which our DP estimator matches the known non-private convergence rates. We attribute these results to certain desirable properties of the edge-flip mechanism that we explore in some detail. While this method performs well for dense networks, the magnitude of error introduced by the local DP mechanism results in a slowing of the convergence rate for sparse networks and a requirement that node degree grow faster than $\sqrt{n}$. We derive these results in the context of more general finite-sample and asymptotic bounds on misclassification rates with privacy.

The paper is structured as follows: We first introduce the network models and notation in Section 2. In Section 3, we review differential privacy and define the edge-flip mechanism, and we demonstrate that the edge-flip mechanism can be viewed as a mixture distribution (Lemma 3.7), leading to two corollaries: a closure property for SBM and a more general post-processing step that allows us to preserve the expected spectral embedddings of edge-flipped networks. In Section 4, we propose a modified spectral clustering algorithm based on this method—with concentration bounds for the estimation of spectral embeddings of SBM and DCBM networks proven in Supplementary Material, Lemma A.1. Our main theoretical results on finite-sample bounds on misclassification and asymptotic convergence rates for private spectral clustering are presented in Section 5. Proofs of these results are deferred to Supplementary Material, Section A. In Section 6, we evaluate the performance of private spectral clustering on both simulated and observed networks. Finally, we conclude with a discussion of limitations and open questions in Section 7.

2. Network Models And Notation

SBM and DCBM are models for binary, undirected networks without covariates. Edges in both networks occur as independent Bernoulli random variables. Each of the $n$ nodes is assigned to one of $k$ communities, or blocks, and these memberships are denoted in the parameter $\theta \in {[k]}^n$ (where $\left[k\right]=\left\{1,\dots ,k\right\}$). In SBM, the probability of a given edge occurring between nodes $i$ and $j$ depends only on the blocks to which $i$ and $j$ belong, with those block-to-block edge probabilities recorded in the symmetric matrix $B\in {(0,1]}^{k\times k}$. In DCBM, edge probabilities further depend on a parameter $\psi \in {(0,1]}^n$ that determines the relative expected node degree for each node. If we let $Y$ be the symmetric $n\times n$ adjacency matrix from a DCBM, the elements of $Y$ are then distributed as

$$Y_{ij}\stackrel{\text{ ind }}{~}\{\begin{array}{ll}\text{ Bernoulli }\left({\psi }_i{\psi }_j{B}_{{\theta }_i{\theta }_j}\right),\hfill & i<j\hfill \\ 0,\hfill & i=j\hfill \\ Y_{ji},\hfill & i>j.\hfill \end{array}$$

For such a network, we will write $Y~\text{DCBM}\left(\theta ,\psi ,B\right)$. The stochastic block model can be regarded as a special case of the DCBM, where $\psi =1_n$, a vector of $n$ ones, i.e.,

$$\text{SBM}\left(\mathit{\theta },B\right)\stackrel{D}{=}\text{DCBM}\left(\mathit{\theta },1_n,B\right)$$ (2.1)

where $\stackrel{D}{=}$ indicates equality in joint distribution of network edges.

We let $C_j=\left\{i:{\theta }_i=j\right\}$ denote the set of nodes in the $j$-th community. We denote the size of the $j$-th community in an SBM or DCBM (i.e., the number of nodes in a block) as $n_j=\left|C_j\right|$, the smallest of which we denote by $n_{\text{min}}={\text{min}}_{j\in \left[k\right]}{n}_j$, the largest by $n_{\text{max}}={\text{max}}_{j\in \left[k\right]}{n}_j$, and the second-largest as $n_{\text{max}}^{\prime }$. As in Lei and Rinaldo (2015), we denote the effective size of the $j$-th block in a DCBM as ${\tilde{n}}_j={\sum }_{i\in C_j}{\psi }_i^2$ and the largest effective size of a block as ${\tilde{n}}_{\text{max}}={\text{max}}_{j\in \left[k\right]}{\tilde{n}}_j$. As a measure of heterogeneity within the $j$-th block, we use ${\nu }_j=n_j^{-2}\left({\sum }_{i\in C_j}{\psi }_i^{-2}\right)\left({\sum }_{i\in C_j}{\psi }_i^2\right)\in \left[1,\infty \right)$. When ${\nu }_j=1$, all nodes in the $j$-th community have the same expected degree, and larger values of ${\nu }_j$ denote greater heterogeneity.

We use ${\lambda }_B$ to denote the smallest absolute nonzero eigenvalue of the matrix $B$. The largest entry in $B$ is denoted $\text{max}B={\text{max}}_{ij}{B}_{ij}$. We represent networks via adjacency matrices, e.g., $Y\in {\{0,1\}}^{n\times n}$. The $i$-th row of the matrix $Y$ is denoted $Y_{i\text{*}}$. We use $\parallel x{\parallel }_p$ to denote the ${\ell }_p$ norm of a vector $x,\parallel A{\parallel }_F$ to denote the Frobenius norm of a matrix, and $\parallel A\parallel$ to denote the operator norm of the matrix $A$, i.e., $\parallel A\parallel ={\text{sup}}_{\parallel x{\parallel }_2=1}\parallel Ax{\parallel }_2$.

In asymptotic notation, we write $a_n=o\left(b_n\right)$ when $\left|a_n/b_n\right|\to 0$ as $n\to \infty ;a_n=\omega \left(b_n\right)$ when $\left|a_n/b_n\right|\to \infty$ as $n\to \infty ;a_n=O\left(b_n\right)$ when $\left|a_n/b_n\right|\le C$ for some $C>0$ and all $n$; $a_n=\Omega \left(b_n\right)$ when $\left|a_n/b_n\right|\ge C$ for some $C>0$ and all $n$; and $a_n=\Theta \left(b_n\right)$ when $a_n=O\left(b_n\right)$ and $a_n=\Omega \left(b_n\right)$. Finally, we write $X_n=O_P\left(b_n\right)$ if for any $\alpha >0$ there exists a constant $C$ such that $P\left(\left|X_n/b_n\right|>C\right)<\alpha$ for all large $n$.

3. Privacy Mechanism as a Mixture Distribution

3.1. Defining Privacy.

The framework of differential privacy offers more than just a formal privacy definition: it offers many privacy definitions that build on each other. To fully appreciate the privacy implications of a given definition—and thereby to justify our specific choice of definition—we briefly review the core concepts of DP and discuss the nuanced distinctions between privacy definitions that could be applied to the problem at hand. Consider first the typical definition of DP:

Definition 3.1 (Differential Privacy (Dwork et al., 2006)). Suppose that $\epsilon >0,\delta \in \left[0,1\right)$. Let $𝒴,𝒵$ be sets, and let $d:𝒴\times 𝒴\to {\mathbb{Z}}_{\ge 0}$ be an integer-valued metric. Let $ℳ:𝒴\to 𝒵$ be a randomized algorithm. We say $ℳ$ satisfies $\left(\epsilon ,\delta \right)$-differential privacy with respect to $d$ if for any $y,y^{\prime }\in 𝒴$ satisfying $d\left(y,y^{\prime }\right)=1$ and any $E\subseteq 𝒵$,

$$P\left(ℳ\left(y\right)\in E\right)\le e^{\epsilon }\cdot P\left(ℳ\left(y^{\prime }\right)\in E\right)+\delta .$$ (3.1)

If $\delta =0$, we say $ℳ$ satisfies $\epsilon$-differential privacy.

Such an algorithm $ℳ$ is called a privacy mechanism, and the elements $y$ and $y^{\prime }$ are often referred to as databases. In non-network settings, the metric $d$ is usually chosen to be the Hamming distance, such that (3.1) can be interpreted thus: changing any one record in an arbitrary database $y\in 𝒴$ does not significantly affect the distribution of $ℳ\left(y\right)$.² Consequently, one can gain virtually no information about a single record based on the private output $ℳ\left(y\right)$. The parameters $\epsilon$ and $\delta$ quantify the strength of the privacy guarantee, with smaller values conferring a stronger guarantee. We focus on the case where $\delta =0$, a setting often referred to as pure differential privacy. The parameter $\epsilon$ is commonly called the privacy-loss budget.

In the network setting, there are two primary choices for the metric $d$ that are frequently employed, and these correspond to the notions of edge DP and node DP (Hay et al., 2009; Kasiviswanathan et al., 2013):

Definition 3.2 (Edge DP). Let $Y,Y^{\prime }$ be two networks with $n$ nodes. Then $d_{\text{edge }}\left(Y,Y^{\prime }\right)$ is given by the total number of edges that differ between $Y$ and $Y^{\prime }$. If $ℳ$ satisfies $\left(\epsilon ,\delta \right)-$ DP with respect to $d_{\text{edge}}$, then we say $ℳ$ satisfies $\left(\epsilon ,\delta \right)$-edge differential privacy.

Definition 3.3 (Node DP). Let $Y,Y^{\prime }$ be two networks with $n$ nodes. Then $d_{\text{node}}\left(Y,Y^{\prime }\right)$ is given by the minimum number of nodes in $Y$ whose incident edges can be modified in order to obtain $Y^{\prime }$. If $ℳ$ satisfies $\left(\epsilon ,\delta \right)-\text{DP}$ with respect to $d_{\text{node}}$, then we say $ℳ$ satisfies $\left(\epsilon ,\delta \right)$-node differential privacy.

Since $d_{\text{edge}}\left(Y,Y^{\prime }\right)\ge d_{\text{node}}\left(Y,Y^{\prime }\right)$, the definition of node DP is more strict than the definition of edge DP. Indeed, node DP implies edge DP, but the reverse is not true. In many cases, the strictness of node DP precludes meaningful analysis (Kasiviswanathan et al., 2013). This is the case, for example, in the problem we wish to solve. Suppose $\hat{\theta }\left(Y\right)$ is an $\epsilon$-DP estimator of the group membership $\theta$ for the network $Y$ with respect to a distance measure $d$. This implies that when we look at any given node’s estimated label, ${[\hat{\theta }(\cdot )]}_i$, across two networks $Y,Y^{\prime }$ satisfying $d\left(Y,Y^{\prime }\right)=1$, it must be that

$$\frac{P\left({[\hat{\mathit{\theta }}\left(Y\right)]}_i=\ell \right)}{P\left({\left[\hat{\mathit{\theta }}\left(Y^{\prime }\right)\right]}_i=\ell \right)}\in \left[e^{-\epsilon },e^{\epsilon }\right]\forall \ell .$$

For edge DP $\left(d=d_{\text{edge}}\right)$, this means that changing any single edge in the network cannot significantly affect the distribution of the estimated label for a given node. For node DP $\left(d=d_{\text{node}}\right)$, however, we can take any given node and change any subset (or even all) of its incident edges without significantly affecting the distribution of its estimated label. Since our goal is to infer labels from precisely these edges, this notion of privacy is too strict for our purposes. For this reason, we will focus on edge-based definitions of DP, which aim to protect the privacy of individual relationships within the network.³

So far, the DP definitions we have considered all fall under the umbrella of central DP. In central DP, we assume that one party, often called the trusted curator, has complete knowledge of the true database $y$. This arrangement is not always desirable, which motivates the concept of local DP (Kasiviswanathan et al., 2011; Duchi et al., 2013). In local DP, records in the database are held by a number of distributed parties (e.g., users of a social network), and only these parties require true knowledge of their records. To facilitate some central analysis of the database, a randomized algorithm is independently applied to each record to produce a local differentially private view of that record. These local DP views may then be shared with a central processor for further analysis. Thus, in contrast with central DP, where a single mechanism is applied at the database level by a single database owner, local DP applies privacy mechanisms at the record level, eliminating the need for a trusted curator, while still allowing for centralized analysis of the data.

Applying local DP to the edge setting, we have further choices yet. Possibly the most widely known definition of local DP in the edge setting is edge local DP of Qin et al. (2017), but we will instead focus on relationship DP as defined in Imola et al. (2021). This definition is more tailored to the setting of undirected networks, where it provides a stronger privacy guarantee.⁴ In relationship DP (as in edge local DP), each node reports a private view (or summary) of its neighbor list, the set of nodes with which it shares an edge. The formal definition is given below.

Definition 3.4 (Relationship DP (Imola et al., 2021)). Let $\epsilon >0$, and let ${ℳ}_1,\dots ,{ℳ}_n$ be randomized algorithms with domain ${\{0,1\}}^n$. The algorithm $ℳ\left(Y\right)=\left({ℳ}_1\left(Y_{1\text{*}}\right),\dots ,{ℳ}_n\left(Y_{n\text{*}}\right)\right)$ is said to satisfy $\epsilon -$relationship DP if for each $E\subseteq \text{Range}\left(ℳ\right)$ and networks $Y,Y^{\prime }$ satisfying $d_{\text{edge}}\left(Y,Y^{\prime }\right)=1$

$$P\left(ℳ\left(Y\right)\in E\right)\le e^{\epsilon }\cdot P\left(ℳ\left(Y^{\prime }\right)\in E\right).$$

3.2. The Edge-Flip Mechanism.

The specific privacy mechanism we employ is a simple randomized-response mechanism that produces an $\epsilon$-relationship-DP synthetic copy ${ℳ}_{\epsilon }\left(Y\right)$ of the original network $Y$ by randomly flipping edges in $Y$. This can be performed locally by assigning each node to flip and self-report a subset of its edges. In this way, no single party ever needs full knowledge of the true adjacency matrix $Y$. Since differential privacy is preserved under post-processing (Dwork et al., 2006), any analysis performed on the synthetic network ${ℳ}_{\epsilon }\left(Y\right)$ preserves $\epsilon$–relationship DP. Edge-flipping mechanisms have been utilized in several earlier papers and interpreted under various edge DP definitions, including (central) edge DP (Mülle et al., 2015; Karwa and Slavković, 2016; Karwa et al., 2017), edge local DP (Qin et al., 2017), and relationship DP (Imola et al., 2021). We include here an explicit formulation of the edge-flipping procedure from Imola et al. (2021), which we term the symmetric edge-flip mechanism.

Definition 3.5 (Symmetric Edge-Flip Mechanism). Suppose that $\epsilon >0$. For $i\in \left[n\right]$, let ${ℳ}_i:{\{0,1\}}^n\to {\{0,1\}}^n$ be such that

$${\left[{ℳ}_1\left(x\right)\right]}_j\stackrel{\text{ind}}{=}\{\begin{array}{lll}0\hfill & i\ge j\hfill & \hfill \\ 1-x_j\hfill & i<j\hfill & w.p.\frac{1}{1+e^{\epsilon }}\hfill \\ x_j\hfill & i<j\hfill & w.p.\frac{1}{1+e^{\epsilon }},\hfill \end{array}$$

and let

$$T\left(Y\right)=\left[\begin{array}{c}{ℳ}_1\left(Y_{1\text{*}}\right)\\ ⋮\\ {ℳ}_n\left(Y_{n\text{*}}\right)\end{array}\right].$$

Then the $n\times n$ symmetric edge-flip mechanism is the mechanism ${ℳ}_{\epsilon }\left(Y\right)=T\left(Y\right)+$${[T\left(Y\right)]}^T$.

Theorem 3.6. The symmetric edge-flip mechanism ${ℳ}_{\epsilon }$ satisfies $\epsilon$-relationship DP.

Proof. The proof of this follows from Theorem 3 of Imola et al. (2021) and the corresponding discussion. For completeness, we give a formal proof in Supplementary Material, Section A.5. □

The simplicity of the edge-flip mechanism affords it several key advantages: In practice, it is easy and flexible to implement. In theory, the distribution of the resulting synthetic network is transparent and tractable. In fact, in Lemma 3.7 we show that the network generated by the edge-flip mechanism is a mixture of the original non-private network with an Erdős-Rényi network. This in turn leads to closure and statistical inference properties that are important and useful, both theoretically and practically. These properties are demonstrated in the corollaries that follow.

Lemma 3.7. Let $Y\in {\{0,1\}}^{n\times n}$ be a random, undirected, binary network. Let $Z~G\left(n,1/2\right)$ be an Erdős-Rényi random graph with $n$ nodes and edge probability $1/2$, and let $U_{ij}\stackrel{ind}{~}\text{Bernoulli}\left(2/\left(2e^{\epsilon }+1\right)\right)$ for $1\le i<j\le n$. Define ${ℳ}^{\prime }\left(Y\right)$ to be the symmetric mixture network such that for $i<j$,

$${ℳ}^{\prime }{(Y)}_{ij}={\left(Z_{ij}\right)}^{U_{ij}}{\left(Y_{ij}\right)}^{\left(1-U_{ij}\right)}$$

Then ${ℳ}^{\prime }\left(Y\right)$ is equal in distribution to ${ℳ}_{\epsilon }\left(Y\right)$.

Proof. It is sufficient to show that the conditional distributions of ${ℳ}^{\prime }\left(Y\right)\mid Y$ and ${ℳ}_{\epsilon }\left(Y\right)\mid Y$ are equivalent. Conditioned on $Y$, the entries of ${ℳ}^{\prime }\left(Y\right)$ and ${ℳ}_{\epsilon }\left(Y\right)$ are independent, binary random variables, and

$$\begin{gathered} P\left({ℳ}^{\prime }{(Y)}_{ij}=1\mid Y\right)=E\left[{ℳ}^{\prime }{(Y)}_{ij}\mid Y\right] \\ =E\left[E\left[{ℳ}^{\prime }{(Y)}_{ij}\mid Y,Z,U\right]\mid Y\right] \\ =E\left[U_{ij}{Z}_{ij}+\left(1-U_{ij}\right)Y_{ij}\mid Y\right] \\ =\frac{1}{e^{\epsilon }+1}+\frac{e^{\epsilon }-1}{e^{\epsilon }+1}{Y}_{ij} \\ =\frac{1}{e^{\epsilon }+1}\left(1-Y_{ij}\right)+\frac{e^{\epsilon }}{e^{\epsilon }+1}{Y}_{ij} \\ =E\left[{ℳ}_{\epsilon }{(Y)}_{ij}\mid Y\right] \\ =P\left({ℳ}_{\epsilon }{(Y)}_{ij}=1\mid Y\right). \end{gathered}$$

□

Lemma 3.7 provides useful intuition about the edge-flip mechanism. As $\epsilon \to \infty$, our synthetic network ${ℳ}_{\epsilon }\left(Y\right)$ approaches the true network $Y$, and as $\epsilon \to 0$, we approach an Erdős-Rényi network. For block models, this is a welcome property, as it means the community structure of the network is exactly preserved. For the SBM in particular, we have closure of the model family under ${ℳ}_{\epsilon }$. The following corollary follows from a simple extension of the proof of Lemma 3.7.

Corollary 3.8. If $Y~\text{SBM}\left(\theta ,B\right)$, then ${ℳ}_{\epsilon }\left(Y\right)~\text{SBM}\left(\theta ,{\tau }_{\epsilon }\left(B\right)\right)$, where

$${\tau }_{\epsilon }\left(B\right)=\frac{1}{e^{\epsilon }+1}{\mathbf{1}}_k{\mathbf{1}}_k^T+\frac{e^{\epsilon }-1}{e^{\epsilon }+1}B.$$ (3.2)

The closure property of Corollary 3.8 may be used to immediately obtain consistency results for SBM under differential privacy via application of existing non-private results. This approach, however, comes with two major limitations: First, applying the results of Lei and Rinaldo (2015) to the private network ${ℳ}_{\epsilon }\left(Y\right)$ yields error bounds in terms of the spectrum of ${\tau }_{\epsilon }\left(B\right)$, while for the non-private $Y$, we obtain bounds in terms of the spectrum of $B$. This makes it difficult to make meaningful, general comparisons between the resulting error bounds. Secondly, while closure holds for SBM, it does not hold for the broader DCBM family. As a result, this approach offers no theoretical insight on how to perform consistent community detection outside of the specific case of the SBM.

To facilitate comparisons between private and non-private error bounds and in consideration of the broader DCBM family, we will explore a more generally applicable framework here, relying instead on a weaker property of the edge-flip mechanism that holds for any random binary network. Via a small “downshift” transformation to an edge-flipped network, we can recover a network whose expectation matches that of the original network, up to a scaling factor.

Corollary 3.9. Let $Y\in {\{0,1\}}^{n\times n}$ be a random, undirected, binary network, and let $\left(d_{\epsilon }\circ {ℳ}_{\epsilon }\right)\left(Y\right)={ℳ}_{\epsilon }\left(Y\right)-{\left(e^{\epsilon }+1\right)}^{-1}\left(1_n{1}_n^T-I_n\right)$ be the downshifted edge-flipped network. Then

$$E\left(d_{\epsilon }\circ {ℳ}_{\epsilon }\right)\left(Y\right)=\frac{e^{\epsilon }-1}{e^{\epsilon }+1}EY.$$

Proof. This follows from a slight rewriting of the derivation in the proof of Lemma 3.7, namely

$$\begin{gathered} E\left[{ℳ}_{\epsilon }\left(Y\right)\right] =E\left[E\left[{ℳ}_{\epsilon }\left(Y\right)\mid Y\right]\right] \\ =\frac{1}{e^{\epsilon }+1}\left({\mathbf{1}}_n{\mathbf{1}}_n^T-I_n\right)+\frac{e^{\epsilon }-1}{e^{\epsilon }+1}EY. \end{gathered}$$

□

Corollary 3.9 transforms a {0, 1}-valued network into a weighted network in exchange for the following property: for any random, undirected, binary network $Y$, the matrices $E\left[Y\right]$ and $E\left[\left(d_{\epsilon }\circ {ℳ}_{\epsilon }\right)\left(Y\right)\right]$ share the same eigenvectors. This fact leads to a more general post-processing step under DP that allows us to preserve the expected spectral embeddings of edge-flipped networks.

4. Modified Clustering Methods with Concentration Bounds

Our goal is to use spectral clustering to estimate the unobserved block membership parameter $\theta$ while satisfying $\epsilon$-relationship DP. A variety of related clustering algorithms exist under the umbrella term of spectral clustering (Ng et al., 2002; Von Luxburg, 2007; Rohe et al., 2011; Lei and Rinaldo, 2015; Binkiewicz et al., 2017). In our setting, we focus on the method of adjacency spectral clustering with $k$ known, following the framework of Lei and Rinaldo (2015). In adjacency spectral clustering, we place the leading $k$ eigenvectors (by absolute value of corresponding eigenvalues) of the observed adjacency matrix $Y$ into an $n\times k$ matrix $\hat{U}$. We consider each row of $\hat{U}$ to be the spectral embedding of the respective node in the network, and we perform a clustering process over these embeddings: for SBM, we perform simple $k$-means, and for DCBM, we normalize the embeddings to have unit norm before performing $k$-medians clustering. The resulting estimated cluster memberships serve as our estimator, $\hat{\theta }$

This approach is theoretically justified by demonstrating that the observed embeddings $\hat{U}$ concentrate around a set of “expected” embeddings $U$ that satisfy appropriate geometric properties. In particular, let $Y~\text{DCBM}\left(\theta ,\psi ,B\right)$, let $P$ be the $n\times n$ matrix with entries $P_{ij}={\psi }_i{\psi }_j{B}_{{\theta }_i{\theta }_j}$, and let $U$ be the $n\times k$ matrix that holds the leading $k$ eigenvectors of $P$. (Note that $E\left[Y\right]=P$ except on the diagonal.) In the case of SBM, it can be shown that the rows of $U$ correspond to $k$ distinct points, and two nodes belong to the same block if and only if their expected embeddings are equal. For general DCBM, the rows of $U$ fall along $k$ rays emanating from the origin, with each ray corresponding to a unique block (Jin, 2015; Lei and Rinaldo, 2015).

Unfortunately, these geometric properties are not, in general, preserved by the edge-flipping routine: the embeddings of ${ℳ}_{\epsilon }\left(Y\right)$ do not have the same geometric properties as the embeddings of $Y$. This limitation prevents direct application of standard DCBM results to ${ℳ}_{\epsilon }\left(Y\right)$ without modification. However, from Corollary 3.9, we know that the “downshifted” network $\left(d_{\epsilon }\circ {ℳ}_{\epsilon }\right)\left(Y\right)$ shares the same eigenvectors as the original $Y$ in expectation, suggesting that the embeddings of $\left(d_{\epsilon }\circ {ℳ}_{\epsilon }\right)\left(Y\right)$ and $Y$ will concentrate in the same locations. Since $\left(d_{\epsilon }\circ {ℳ}_{\epsilon }\right)\left(Y\right)$ is a weighted network, these concentration results require further extensions of the standard results for binary networks (e.g., Lei and Rinaldo, 2015). Indeed, we state and prove explicit concentration bounds in Supplementary Material, Lemma A.1. This is the key fact that powers the main results of Section 5.

To perform spectral clustering on the edge-flipped network, then, we need only to modify the common algorithms to use the downshifted adjacency matrix $\left(d_{\epsilon }\circ {ℳ}_{\epsilon }\right)\left(Y\right)$ in place of ${ℳ}_{\epsilon }\left(Y\right)$. Our proposed modified algorithms for SBM and DCBM are given in Algorithms 1 and 2, respectively. The $k$-means and $k$-medians problems, as well as their approximations, are defined in Supplementary Material, Section A.1.

To evaluate the performance of the clustering algorithms, we assume knowledge of ground truth group labels $\theta$ and measure two forms of misclassification. The first is an overall measure of misclassification that captures the proportion of misidentified nodes, up to a permutation of labelings. Let $S_{\left[k\right]}$ denote the set of all permutations $\sigma :\left[k\right]\to \left[k\right]$, and let $\mathbb{I}(\cdot )$ denote an indicator function. Then the overall misclassification for the estimated groups $\hat{\theta }$ is given by

$$L\left(\theta ,\hat{\theta }\right)=\underset{\sigma \in S_{\left[k\right]}}{\text{min}}\frac{1}{n}\sum _{i=1}^n\mathbb{I}\left(\sigma \left({\hat{\theta }}_i\right)\ne {\theta }_i\right)).$$ (4.1)

As in Lei and Rinaldo (2015), we also consider the worst-case misclassification within a single community in order to account for trivialities ⁵. This error is given by

$$\tilde{L}\left(\theta ,\hat{\theta }\right)=\underset{j\in \left[k\right]}{\text{max }}\underset{\sigma \in S_{\left[k\right]}}{\text{min}}\frac{1}{\left|C_j\right|}\sum _{i\in C_j}\mathbb{I}\left(\sigma \left({\hat{\theta }}_i\right)\ne j\right).$$ (4.2)

5. Misclassification Bounds for Private Clustering

We generalize the results of Lei and Rinaldo (2015) developed for the non-private network setting to derive finite-sample misclassification bounds for spectral clustering of SBM and DCBM under local differential privacy using the modified spectral clustering algorithms proposed in Section 4 (Algorithms 1 and 2). We argue in Section 4 and prove in Supplementary Material, Section A that the spectral embeddings of $Y$ and the weighted network $\left(d_{\epsilon }\circ {ℳ}_{\epsilon }\right)\left(Y\right)$ will concentrate in the same locations. In Lemma A.1, we also demonstrated that the concentration weakens with a stronger privacy guarantee (i.e., smaller $\epsilon$). In this section, we use these concentration results to derive misclassification bounds for private spectral clustering of SBM and DCBM; all proofs are provided in Section A of the Supplementary Material. We begin with the results for SBM.

5.1. SBM.

Theorem 5.1. Let $Y~\text{SBM}\left(\theta ,B\right)$ with $\text{max }B\ge \text{log\hspace{0.17em}}n/n$, and minimum absolute eigenvalue ${\lambda }_B>0$. Let $\epsilon \in \left(0,\infty \right]$, and let ${\hat{\theta }}_{\epsilon }$ be the result of Algorithm 1 on ${ℳ}_{\epsilon }\left(Y\right)$ (where ${ℳ}_{\infty }\left(Y\right)=Y)$, using an approximation error of $\gamma$. Let

$$g_{\epsilon }\left(B\right)=\{\begin{array}{ll}\frac{e^{\epsilon }+1}{e^{\epsilon }-1}\left(\text{max\hspace{0.17em}}B+\frac{1}{e^{\epsilon }-1}\right),\hfill & \epsilon <\infty \hfill \\ \text{max\hspace{0.17em}}B,\hfill & \epsilon =\infty \hfill \end{array}$$ (5.1)

There exists a universal constant $c_1$ such that with probability at least $1-n^{-1}$,

$$\tilde{L}\left(\mathit{\theta },{\hat{\mathit{\theta }}}_{\epsilon }\right)\le c_1\frac{\left(2+\gamma \right)kn}{n_{min}^2{\lambda }_B^2}{g}_{\epsilon }\left(B\right)$$

and

$$L\left(\mathit{\theta },{\hat{\mathit{\theta }}}_{\epsilon }\right)\le c_1\frac{\left(2+\gamma \right)k{n}_{max}^{\prime }}{n_{min}^2{\lambda }_B^2}{g}_{\epsilon }\left(B\right)$$

provided that

$$n_{min}>{\lambda }_B^{-1}\sqrt{c_1\left(2+\gamma \right)kn{g}_{\epsilon }\left(B\right)}.$$ (5.2)

Taking $\epsilon =\infty$ (i.e., the non-private setting) in Theorem 5.1 recovers a result that is exactly equivalent to the results for SBM in Lei and Rinaldo (2015). Consequently, the cost of privacy on misclassification is captured by $g_{\epsilon }\left(B\right)$. Noting that $g_{\epsilon }\left(B\right)$ is a decreasing function in $\epsilon$ and that ${\text{lim}}_{\epsilon \to \infty }{g}_{\epsilon }\left(B\right)=g_{\infty }\left(B\right)$, Theorem 5.1 provides a smooth interpolation between the known finite-sample misclassification bounds without privacy and the corresponding bounds under local DP.

We can use the finite-sample bounds from Theorem 5.1 to construct particular asymptotic regimes under which ${\hat{\theta }}_{\epsilon }$ is consistent and derive convergence rates. For a given sequence of SBM networks, it suffices to show that

$$\frac{kn}{n_{\text{min}}^2}{g}_{\epsilon }\left(B\right)=o\left(1\right)$$

If this holds, condition (5.2) is met for large $n$, and both measures of misclassification, $\tilde{L}\left(\theta ,{\hat{\theta }}_{\epsilon }\right)$ and $L\left(\theta ,{\hat{\theta }}_{\epsilon }\right)$, tend to zero.

Since $g_{\epsilon }\left(B\right)$ explains the difference in classification performance between private and non-private clustering, a comparison of convergence rates can be boiled down to the differences in asymptotic behavior of $g_{\epsilon }\left(B\right)$. We note that if $\epsilon =\infty$, then $g_{\epsilon }\left(B\right)=\text{max }B$. This will serve as our baseline. On the other hand, if $\epsilon <\infty$, then $g_{\epsilon }\left(B\right)=O\left(\text{max }B+{\zeta }_{\epsilon }^{-1}+{\zeta }_{\epsilon }^{-2}\right)$, where ${\zeta }_{\epsilon }=e^{\epsilon }-1$, an increasing function of $\epsilon$ satisfying ${\zeta }_0=0$. (See Supplementary Material, Fact A.2.) Thus,

$$\begin{array}{rr}\hfill \tilde{L}\left(\theta ,{\hat{\theta }}_{\epsilon }\right)& \hfill =\{\begin{array}{ll}{O}_P\left(\frac{kn}{n_{\text{min}}^2{\lambda }_B^2}\text{max }B\right)\hfill & \epsilon =\infty \hfill \\ O_P\left(\frac{kn}{n_{\text{min}}^2{\lambda }_B^2}\left(\text{max }B+{\zeta }_{\epsilon }^{-1}+{\zeta }_{\epsilon }^{-2}\right)\right)\hfill & \epsilon <\infty \hfill \end{array}\\ \hfill \text{ and }L\left(\theta ,{\hat{\theta }}_{\epsilon }\right)& \hfill =\{\begin{array}{ll}{O}_P\left(\frac{k{n}_{\text{max\hspace{0.17em}}}^{\prime }}{n_{\text{min}}^2{\lambda }_B^2}\text{max }B\right)\hfill & \epsilon =\infty \hfill \\ O_P\left(\frac{k{n}_{\text{max\hspace{0.17em}}}^{\prime }}{n_{\text{min}}^2{\lambda }_B^2}\left(\text{max }B+{\zeta }_{\epsilon }^{-1}+{\zeta }_{\epsilon }^{-2}\right)\right)\hfill & \epsilon <\infty .\hfill \end{array}\end{array}$$

From this, it is clear that if $\epsilon$ and $\text{max\hspace{0.17em}}B$ are bounded away from zero, we can obtain the same convergence rate under local DP as in the absence of privacy. For example, for a fixed privacy budget and a sequence of dense SBMs (i.e., one for which the edge probabilities do not tend to zero), we can effectively employ local DP at the same cost, asymptotically, as in the non-private setting (up to a constant). For a sparse SBM (one in which $\text{max\hspace{0.17em}}B\to 0$) with fixed privacy budget, however, this does not hold, as $g_{\epsilon }\left(B\right)=\Theta \left(1\right)$ for $\epsilon <\infty$, but $g_{\infty }\left(B\right)=\text{max\hspace{0.17em}}B=o\left(1\right)$. Thus the convergence rate for sparse SBM slows by a factor of ${(\text{max\hspace{0.17em}}B)}^{-1}$.

This slowing of convergence for sparse SBM limits the extent of sparsity that can be handled under this local DP mechanism. Consider a regime in which $k$ is fixed, communities grow proportionally $\left(n_{\text{min}}=\Theta \left(n\right)\right)$, and $B$ changes with $n$ only via some scaling parameter ${\alpha }_n\to 0$, i.e., $B={\alpha }_n{B}_0$ for some fixed matrix $B_0$. In this case, we have $k$ constant, $\text{max }B=\Theta \left({\alpha }_n\right),{\lambda }_B=\Theta \left({\alpha }_n\right)$, and so the worst-case misclassification is

$$\tilde{L}\left(\mathit{\theta },{\hat{\mathit{\theta }}}_{\epsilon }\right)=\{\begin{array}{ll}{O}_P\left(\frac{1}{n{\alpha }_n}\right)\hfill & \epsilon =\infty \hfill \\ O_P\left(\frac{{\alpha }_n+{\zeta }_{\epsilon }^{-1}+{\zeta }_{\epsilon }^{-2}}{n{\alpha }_n^2}\right)\hfill & \epsilon <\infty .\hfill \end{array}$$ (5.3)

Theorem 5.1 allows us to choose ${\alpha }_n$ as small as $\text{log\hspace{0.17em}}n/n$, so we have consistency for non-private clustering down to $\text{log\hspace{0.17em}}n/n$. For private clustering with a fixed privacy budget, however, we need ${\alpha }_n=\omega \left(n^{-1/2}\right)$. The only way to achieve greater sparsity is to allow the privacy budget to grow arbitrarily large. For example, choosing $\epsilon =\text{log\hspace{0.17em}}\left(1+{\alpha }_n^{-1}\right)$, we have ${\zeta }_{\epsilon }^{-1}={\alpha }_n$, and thus we can recover the same convergence rate and accommodate the same level of sparsity as without privacy—but at the cost of allowing unbounded privacy loss for large $n$.

5.2. DCBM.

Generalizing the theoretical results for SBM to DCBM requires a bit of care. As described in Section 3, in the more general setting of DCBM, the expected embeddings for a given block fall along distinct rays emanating from the origin—in contrast with distinct points in the case of SBM. It is for this reason that Algorithm 2 is used for DCBM. The key theorethical result from this more general treatment is given below.

Theorem 5.2. Let $Y~\text{DCBM}\left(\theta ,\psi ,B\right)$ with ${\text{max\hspace{0.17em}}}_{i\in C_j}{\psi }_i=1$ for $j\in \left[k\right],\text{max\hspace{0.17em}}B\ge \text{log\hspace{0.17em}}n/n$, and minimum absolute eigenvalue ${\lambda }_B>0$. Let $\epsilon \in \left(0,\infty \right]$, and let ${\hat{\theta }}_{\epsilon }$ be the result of Algorithm 2 on ${ℳ}_{\epsilon }\left(Y\right)$ (where ${ℳ}_{\infty }\left(Y\right)=Y$), using an approximation error of $\gamma$. Let $g_{\epsilon }\left(B\right)$ as in Theorem 5.1. There exists a universal constant $c_2$ such that with probability at least $1-n^{-1}$,

$$L\left(\mathit{\theta },{\hat{\mathit{\theta }}}_{\epsilon }\right)\le c_2\frac{\left(2.5+\gamma \right)}{{\tilde{n}}_{min}{\lambda }_B}\sqrt{\frac{k}{n}\left(\sum _{j=1}^k{n}_j^2{\nu }_j\right)g_{\epsilon }\left(B\right)}$$

provided that

$$n_{min}>\frac{c_2\left(2.5+\gamma \right)\sqrt{kn{g}_{\epsilon }\left(B\right){\sum }_{j=1}^k{n}_j^2{\nu }_j}}{{\tilde{n}}_{min}{\lambda }_B}.$$ (5.4)

As was the case with SBM, the results of Theorem 5.2 for $\epsilon =\infty$ (non-private setting) match the original results of Lei and Rinaldo (2015). The cost of privacy is once again determined by the function $g_{\epsilon }\left(B\right)$. In this case, however, the bound changes with the square root of $g_{\epsilon }\left(B\right)$.

Comparing the results of Theorems 5.1 and 5.2, the more general result involves additional parameters, of course, but also generally provides a weaker result. Ignoring constants, condition (5.4) is more stringent than (5.2), as $\sqrt{{\sum }_{j=1}^k{n}_j^2{\nu }_j}/{\tilde{n}}_{\text{min}}\ge \sqrt{k}$. The resulting upper bound on $L\left(\theta ,{\hat{\theta }}_{\epsilon }\right)$ also results in weaker convergence rates. In keeping with Lei and Rinaldo (2015), Theorem 5.2 offers no bound on $\tilde{L}\left(\theta ,{\hat{\theta }}_{\epsilon }\right)$, as its proof bounds only the total number of misclassified nodes in the network. A trivial bound for $\tilde{L}$ can be obtained by observing that:

$$\tilde{L}\left(\theta ,{\hat{\theta }}_{\epsilon }\right)\le \left(n/n_{\text{min}}\right)L\left(\theta ,{\hat{\theta }}_{\epsilon }\right)$$ (5.5)

To simplify matters, we illustrate an asymptotic regime in which communities grow proportionally $(n_j=\Theta \left(n/k\right)$ for all $j\in \left[k\right])$ and assume that there exists a global lower bound $0<a\le {\psi }_i$ for $i\in \left[n\right]$. Under these conditions, we can state a simple asymptotic result.

Lemma 5.3. Suppose $Y$ satisfies the conditions of Theorem 5.2, and suppose further that $0<a\le {\psi }_i\le 1$ for all $i\in \left[n\right]$ and $n_j=\Theta \left(n/k\right)$ for all $j\in \left[k\right]$. Then

$$\frac{k^2\sqrt{g_{\epsilon }\left(B\right)}}{a^3{\lambda }_B\sqrt{n}}=o\left(1\right)\Rightarrow L\left(\theta ,{\hat{\theta }}_{\epsilon }\right)=O_P\left(\frac{k\sqrt{g_{\epsilon }\left(B\right)}}{a^3{\lambda }_B\sqrt{n}}\right)\text{. }$$

To compare this to the results seen in Eq. (5.3), consider again the case when $B={\alpha }_n{B}_0$ for some fixed matrix $B_0$ and sequence ${\alpha }_n\to 0$. Then since $k$ is fixed, $n/n_{\text{min}}=\Theta \left(1\right)$, and so combining Lemma 5.3 with Eq. (5.5) yields a convergence rate of:

$$\tilde{L}\left(\theta ,{\hat{\theta }}_{\epsilon }\right)=\{\begin{array}{ll}{O}_P\left(\frac{1}{a^3\sqrt{n{\alpha }_n}}\right)\hfill & \epsilon =\infty \hfill \\ O_P\left(\frac{\sqrt{{\alpha }_n+{\zeta }_{\epsilon }^{-1}+{\zeta }_{\epsilon }^{-2}}}{a^3{\alpha }_n\sqrt{n}}\right)\hfill & \epsilon <\infty .\hfill \end{array}$$

For SBM, where $a=1$—or more generally for any regime in which $a$ is bounded away from zero—this bound is precisely the square root of the bound obtained in Eq. (5.3). Once again, we see that the convergence rate for sparse networks slows under private methods, and while some sparsity is attainable with privacy through this method, to accommodate the same level of sparsity with privacy as without, we would again need to allow the privacy-loss budget $\epsilon$ to grow arbitrarily large.

6. Empirical Evaluations

6.1. Simulation Studies.

The theoretical results above suggest that the edge-flip privacy mechanism can be used to achieve convergence for spectral clustering of SBM and DCBM networks that is similar in rate for dense networks, while slower for sparse networks. However, existing work suggests that the established non-private bounds from which we developed our theory may not be tight (Lei and Rinaldo, 2015; Athreya et al., 2016). Here we evaluate these results through simulation to assess how empirical performance compares with theoretical. To facilitate this analysis, we consider two special cases of the SBM and DCBM.

Definition 6.1 (Symmetric SBM). The symmetric SBM is an SBM network consisting of $n$ nodes, $k$ equal-sized blocks, and $B=p{I}_k+r{1}_k{1}_k^T$. For such a network, we write $Y~\text{SSBM}\left(n,k,p,r\right)$.

Definition 6.2 (Symmetric DCBM). The symmetric DCBM is a DCBM network consisting of $n$ nodes, $k$ equal-sized blocks, $B=p{I}_k+r{1}_k{1}_k^T$, and $\psi \in {[a,1]}^n$ taking value 1 for the first node of a given block and values randomly drawn from Uniform $\left(a,1\right)$ elsewhere. For such a network, we write $Y~\text{SDCBM}\left(n,k,p,r,a\right)$.

Starting with SBM, we simulate two regimes following the symmetric SBM. In the first setting, we use a dense symmetric SBM. We consider 16 values of $n$ ranging from $n=30$ to $n=12000$ and $\epsilon \in \left\{0.5,0.75,1,1.5,2,3,4,\infty \right\}$, where $\epsilon =\infty$ represents the original network (i.e., no privacy). For each $n,\epsilon$ pair, we draw 100 networks $Y^{\left(i\right)}~\text{SSBM}\left(n,k=3,p=0.2,r=0.05\right)$. We then apply Algorithm 1 to ${ℳ}_{\epsilon }\left(Y^{\left(i\right)}\right)$, and report the mean overall misclassification rate $L\left({\theta }^{\left(i\right)},{\hat{\theta }}^{\left(i\right)}\right)$ of Eq. (4.1). The results of this experiment are plotted on log–log axes in the left panel of Figure 1. The bounds from Section 5 suggest polynomial convergence rates in this setting that should be approximately the same without or without privacy. Indeed, on a log–log scale, there are no clear differences in these curves, although all seem to exhibit downward curvature suggesting faster than polynomial convergence.

Figure 1:

Proportion of misclassified nodes in simulated SBM networks for various values of $\epsilon ,n$. Left (dense): $Y~\text{SSBM}\left(n,k=3,p=0.2,r=0.05\right)$, Right (sparse): $Y~\text{SSBM}\left(n,k=2,p=1.5n^{-.3},r=.15n^{-.3}\right)$

For the second setting, we use a sparse symmetric SBM. Here we consider 13 values of $n$ ranging from $n=10$ to $n=12800$ and the same eight values of $\epsilon$ as in the first setting. For each $n,\epsilon$ pair, we draw 100 networks $Y^{\left(i\right)}~\text{SSBM}\left(n,k=2,p=1.5n^{-.3},r=.15n^{-.3}\right)$, then apply Algorithm 1 to ${ℳ}_{\epsilon }\left(Y^{\left(i\right)}\right)$, as in the first setting. The results are plotted on log–log axes in the right panel of Figure 1. Here, the curves corresponding to each value of $\epsilon$ are no longer parallel: for smaller values of $\epsilon$, the convergence rate slows.

For DCBM, we simulate two additional regimes, one dense and one sparse, in a manner similar to the SBM simulations. In the dense setting, we use a dense symmetric DCBM with the same values of $n,\epsilon$ as used in the dense SBM simulations. For each $n,\epsilon$ pair, we draw 100 networks $Y^{\left(i\right)}~\text{SDCBM}\left(n,k=3,p=0.4,r=0.05,a=0.3\right)$. In the sparse setting, we use the same $\epsilon$ values as in the other settings and 12 values of $n$ ranging from $n=20$ to $n=12800$. Then we draw 100 networks $Y^{\left(i\right)}~\text{SDCBM}\left(n,k=2,p=2n^{-.25},r=0.1n^{-.25},a=0.3\right)$. In both DCBM settings, we report the mean overall misclassification rate after applying Algorithm 2. The results of these simulations are plotted on log–log axes in Figure 2. In contrast with the SBM simulations, the DCBM simulations generally exhibit slower convergence. Similar to the SBM simulations, we see a clear slowing of convergence rate for smaller values of $\epsilon$ in the sparse setting.

Figure 2:

Proportion of misclassified nodes in simulated DCBM networks for various values of $\epsilon ,n$. Left (dense): $Y~\text{SDCBM}\left(n,k=3,p=0.4,r=0.05,a=0.3\right)$, Right (sparse): $Y~\text{SSBM}\left(n,k=2,p=1.5n^{-.3},r=.15n^{-3}\right)$.

While the results of these simulations are generally consistent with the theory of Section 5, our simulations suggest the bounds given here and in Lei and Rinaldo (2015) are not tight, in the sense that the observed convergence rates appear to beat the theoretical guarantees, sometimes by considerable order. This appears consistent with the empirical findings of Athreya et al. (2017, Section 5), where a related method is evaluated in the non-private setting. In the private, sparse setting, the precise nature in which convergence slows with $\epsilon$ is not fully captured by our theoretical results. Specifically, for larger values of $\epsilon$, the convergence properties appear more similar to the non-private setting than suggested by the theoretical bounds.

6.2. Performance on Observed Networks.

To assess the practicality of these methods, we applied the edge-flip mechanism and Algorithm 2 to several real-world datasets, then compared the performance of spectral clustering on the private networks over various privacy budgets. We show the trade-offs of privacy loss and accuracy.

The first dataset we considered is a small, classical network used in the SBM literature: Hansell’s friendship data (Hansell, 1984), which were used in Wang and Wong (1987); Snijders and Nowicki (1997). This is a directed network of $n=27$ (13 male, 14 female) students, and the presence of an edge $\left(i,j\right)$ indicates that student $i$ considers student $j$ to be a friend. We symmetrized the network by setting $Y_{ij}=\text{max\hspace{0.17em}}\left\{Y_{ij},Y_{ji}\right\}$, and we used the students’ sexes as ground-truth labels.

For this small network, we considered $\epsilon \in \left\{0.5,0.75,1,1.5,2,3,4,\infty \right\}$, and for each value of $\epsilon$, we applied the privacy mechanism and spectral clustering algorithm 500 times. Box plots showing the overall classification accuracy, $1-L\left(\theta ,\hat{\theta }\right)$ from Eq. (4.1), for each value of $\epsilon$ are given in Figure 3. Accuracy remains high for values of $\epsilon >2$; for such a small network, this is about as good as we can expect.

Figure 3:

Trade-off of privacy-loss and accuracy of private spectral clustering on a small dataset, Hansell’s friendship data (Hansell, 1984), with varying privacy-loss $\epsilon$, and “NP” denoting non-private data.

Next, we considered two datasets from the DCBM literature. The first network consists of $n=1137$ Facebook users from Simmons College (Traud et al., 2012; Chen et al., 2018), and edges represent friendships. Each student belongs to one of $k=4$ class years (2006 to 2009). These class years are used as ground-truth group memberships, which are inferred through their friendships. The second dataset is the well-known network of political blogs from Adamic and Glance (2005), which has been widely used in the DCBM literature (Karrer and Newman, 2011; Jin, 2015; Chen et al., 2018; Abbe, 2018). This network consists of $n=1222$ blogs on U.S. politics, which have been categorized as either left-leaning or right-leaning ($k=2$ groups). An edge in this network represents the existence of a hyperlink between the two blogs, and these hyperlinks are used to infer the left- or right-leaning label for each node. The performance of the private spectral clustering methods on these networks is shown in Figure 4. In both of these datasets, we see a steady decline in performance as $\epsilon$ decreases, without a clear inflection point.

Figure 4:

Trade-off of privacy-loss and accuracy of private spectral clustering on well-known DCBM datasets, with varying privacy-loss $\epsilon$, and “NP” denoting non-private data. Left: Facebook friendships of Simmons College students (Traud et al., 2012), Right: political blogs network (Adamic and Glance, 2005)

The theoretical and simulated results from earlier suggest that these methods are best paired with suitably large or dense networks. A relevant collection of networks can be found, for example, in Andris et al. (2015), where networks of Democrat and Republican members of the U.S. House of Representatives are constructed based on voting similarity. Using their methods and data from Lewis et al. (2021), we reconstructed these networks for a number of sessions of the U.S. House of Representatives and U.S. Senate (Hehir, 2022b). ⁶

Changes over time in the patterns of voting among members of Congress offer us a range of networks that span nearly complete separation of the parties (particularly in recent years) to networks that exhibit greater levels of connectivity across parties. Figure 5 depicts two such networks, with the 70th U.S. Senate on the left and the 110th U.S. Senate on the right. In the 70th Senate, 13% of cross-party pairs are connected, while 76% and 59% of Democrat and Republican pairs are connected (respectively). In the 110 th Senate, 9% of cross-party pairs are connected, while $98\text{\%}$. and $88\text{\%}$ of Democrat and Republican pairs are connected (respectively).

Figure 5:

Visualization of congressional voting networks for 70th U.S. Senate (left) and 110th U.S. Senate (right). Democrats are depicted as blue nodes and Republican as red.

For the 70th Senate $\left(n=87\right)$, 110th Senate $\left(n=98\right)$, 70th House $\left(n=425\right)$, and 110th House $\left(n=423\right)$, we applied the private spectral clustering methods 50 times for each value of $\epsilon \in \left\{0.25,0.33,0.5,0.75,1,1.5,2,3,4,\infty \right\}$. The results are depicted in Figure 6. The high density (and relatively low variability of degree) in these networks enables impressive clustering performance: for the House networks, clustering performance is solid for all $\epsilon$ considered; in the smaller Senate networks, we see a drop in performance for values of $\epsilon <0.5$.

Figure 6:

Trade-off of privacy-loss and accuracy of private spectral clustering on U.S. Congressional voting networks. Clockwise from top left: 110th House of Representatives, 70th House of Representatives, 70th Senate, 110th Senate

7. Discussion

Just as SBM and DCBM are fundamental network models featuring community structure, the edge-flip mechanism is a fundamental network privacy mechanism providing local differential privacy on networks at the edge level. A great advantage of the edge-flip mechanism is that it produces a synthetic network with clear distributional properties. In particular, the explicit mixture distribution of ${ℳ}_{\epsilon }\left(Y\right)$ described in Lemma 3.7—in contrast with the unclear distributional properties of some alternative mechanisms—greatly facilitates the process of accounting for privacy in various statistical procedures. Moreover, by nature of producing a synthetic network, any number of statistical procedures can be applied to the single output ${ℳ}_{\epsilon }\left(Y\right)$ with a fixed privacy budget $\epsilon$.

Our work demonstrates how to use the clear distribution of ${ℳ}_{\epsilon }\left(Y\right)$ to extend the theory of consistent community detection to the local DP setting. This is, to our knowledge, the first attempt to address this important problem. Algorithms 1 and 2 represent modest extensions to the algorithms originally proposed in Lei and Rinaldo (2015), requiring only the addition of a small “downshift” transformation to unlock a powerful privacy guarantee with a clear accounting of the cost to clustering performance. In fact, Corollary 3.8 suggests that for SBM networks in particular, this downshift transformation is optional, as ${ℳ}_{\epsilon }\left(Y\right)$ is itself an SBM whose community structure is identical to $Y$.

Focusing on the specific case of SBM, we may ask whether it is preferable to perform clustering for $\text{SBM}$ on $\left(d_{\epsilon }\circ {ℳ}_{\epsilon }\right)\left(Y\right)$, as prescribed by Algorithm 1, or to proceed with ordinary spectral clustering on ${ℳ}_{\epsilon }\left(Y\right)$. per the closure property of Corollary 3.8. By combining the closure property with the $\epsilon =\infty$ case of Theorem 5.1, one can obtain performance bounds in terms of the matrix ${\tau }_{\epsilon }\left(B\right)$ of Eq. (3.2). In some cases, these bounds may be superior to the bounds resulting from Theorem 5.1 alone.⁷ Nonetheless, we have opted to present the results for the downshifted procedure for the sake of generality. This allows us to keep the SBM misclassification bounds in terms of the smallest absolute eigenvalue of $B$ instead of that of ${\tau }_{\epsilon }\left(B\right)$, where a general relationship between these two quantities is elusive.

The benefits of the edge-flip mechanism’s tractable distribution are, of course, not limited to the network models considered in this paper, nor to the field of network clustering. For example, Karwa et al. (2017) have previously shown how to adjust inferential procedures for exponential random graph models using the edge-flip mechanism. These examples are likely only scratching the surface of the theory that can be extended to accommodate privacy under the edge-flip mechanism.

The primary drawback to the edge-flip mechanism is its relationship to sparse networks. We posit that this limitation is inherent to working with a local-DP synthetic network, in which the magnitude of noise required for privacy begins to dwarf the signal present in a sparse network. Looking at the edge-flip mechanism in particular, for a sparse network $Y$, the mixture ${ℳ}_{\epsilon }\left(Y\right)$ will be considerably more dense, and a disproportionate number of its edges will result from the privacy mechanism, not the original network of interest. Although these changes preserve key properties of SBM and DCBM networks, reductions in sparsity dilute the signal-to-noise ratio considerably. While the theory supports the claim that these methods are consistent for SBM and DCBM networks exhibiting mild sparsity, empirical performance for sparse networks indicates a considerable utility loss when a very strong privacy guarantee is applied to a sparse network. Other privacy mechanisms preserving the sparsity of a network, especially non-local mechanisms, could be considered for greater performance in these cases.

The trade-off of privacy-loss and accuracy that we observe for label recovery on dense networks such as the Congressional voting networks, where considerable accuracy is maintained at $\epsilon =0.5$, suggests that the methods employed here are of more than just theoretical interest. By comparison, in their work with exponential random graph models and the edge-flip mechanism, Karwa et al. (2017) use values of $\epsilon$ ranging from 3 to 6 on a network of similar size. Nonetheless, it would be useful to develop theoretical results for alternative privacy mechanisms that can accommodate greater sparsity.

Lastly, while the results given here focus on spectral clustering of SBM and DCBM, some of the insights from these techniques are suggestive of further theoretical developments in related fields. For example, concentration bounds for the spectral embeddings of the downshifted private network could potentially be extended to the more general family of random dot product graphs (Athreya et al., 2017). Doing this would open a wide range of avenues for future work.