MODEL-BASED CLUSTERING OF LARGE NETWORKS

Abstract

We describe a network clustering framework, based on finite mixture models, that can be applied to discrete-valued networks with hundreds of thousands of nodes and billions of edge variables. Relative to other recent model-based clustering work for networks, we introduce a more flexible modeling framework, improve the variational-approximation estimation algorithm, discuss and implement standard error estimation via a parametric bootstrap approach, and apply these methods to much larger data sets than those seen elsewhere in the literature. The more flexible framework is achieved through introducing novel parameterizations of the model, giving varying degrees of parsimony, using exponential family models whose structure may be exploited in various theoretical and algorithmic ways. The algorithms are based on variational generalized EM algorithms, where the E-steps are augmented by a minorization-maximization (MM) idea. The bootstrapped standard error estimates are based on an efficient Monte Carlo network simulation idea. Last, we demonstrate the usefulness of the model-based clustering framework by applying it to a discrete-valued network with more than 131,000 nodes and 17 billion edge variables.

1. Introduction

According to Fisher [(1922), page 311], “the object of statistical methods is the reduction of data.” The reduction of data is imperative in the case of discrete-valued networks that may have hundreds of thousands of nodes and billions of edge variables. The collection of such large networks is becoming more and more common, thanks to electronic devices such as cameras and computers. Of special interest is the identification of influential subsets of nodes and high-density regions of the network with an eye to break down the large network into smaller, more manageable components. These smaller, more manageable components may be studied by more advanced statistical models, such as advanced exponential family models [e.g., Frank and Strauss (1986), Strauss and Ikeda (1990), Wasserman and Pattison (1996), Snijders et al. (2006), Hunter and Handcock (2006)].

An example is given by signed networks, such as trust networks, which arise in World Wide Web applications. Users of internet-based exchange networks are invited to classify other users as either −1 (untrustworthy) or +1 (trustworthy). Trust networks can be used to protect users and enhance collaboration among users [Kunegis, Lommatzsch and Bauckhage (2009), Massa and Avesani (2007)]. A second example is the spread of infectious disease through populations by way of contacts among individuals [Britton and O'Neill (2002), Groendyke, Welch and Hunter (2011)]. In such applications, it may be of interest to identify potential super-spreaders—that is, individuals who are in contact with many other individuals and who could therefore spread the disease to many others—and dense regions of the network through which disease could spread rapidly.

The current article advances the model-based clustering of large networks in at least four ways. First, we introduce a simple and flexible statistical framework for parameterizing models based on statistical exponential families [e.g., Barndorff-Nielsen (1978)] that advances existing model-based clustering techniques. Model-based clustering of networks was pioneered by Snijders and Nowicki (1997). The simple, unconstrained parameterizations employed by Snijders and Nowicki (1997) and others [e.g., Nowicki and Snijders (2001), Airoldi et al. (2008), Daudin, Picard and Robin (2008), Zanghi et al. (2010), Mariadassou, Robin and Vacher (2010)] make sense when networks are small, undirected and binary, and when there are no covariates. In general, though, such parameterizations may be unappealing from both a scientific point of view and a statistical point of view, as they may result in nonparsimonious models with hundreds or thousands of parameters. An important advantage of the statistical framework we introduce here is that it gives researchers a choice: they can choose interesting features of the data, specify a model capturing those features, and cluster nodes based on the specified model. The resulting models are therefore both parsimonious and scientifically interesting.

Second, we introduce approximate maximum likelihood estimates of parameters based on novel variational generalized EM (GEM) algorithms, which take advantage of minorization-maximization (MM) algorithms [Hunter and Lange (2004)] and have computational advantages. For unconstrained models, tests suggest that the variational GEM algorithms we propose can converge quicker and better avoid local maxima than alternative algorithms; see Sections 6 and 7. In the presence of parameter constraints, we facilitate computations by exploiting the properties of exponential families [e.g., Barndorff-Nielsen (1978)]. In addition, we sketch how the variational GEM algorithm can be extended to obtain approximate Bayesian estimates.

Third, we introduce bootstrap standard errors to quantify the uncertainty about the approximate maximum likelihood estimates of the parameters, whereas other work has ignored the uncertainty about the approximate maximum likelihood estimates. To facilitate these bootstrap procedures, we introduce Monte Carlo simulation algorithms that generate sparse networks in much less time than conventional Monte Carlo simulation algorithms. In fact, without the more efficient Monte Carlo simulation algorithms, obtaining bootstrap standard errors would be infeasible.

Finally, while model-based clustering has been limited to networks with fewer than 13,000 nodes and 85 million edge variables [see the largest data set handled to date, Zanghi et al. (2010)], we demonstrate that we can handle much larger, nonbinary networks by considering an internet-based data set with more than 131,000 nodes and 17 billion edge variables, where “edge variables” comprise all observations, including node pairs between which no edge exists. Many internet-based companies and websites, such as http://amazon.com, http://netflix.com and http://epinions.com, allow users to review products and services. Because most users of the World Wide Web do not know each other and thus cannot be sure whether to trust each other, readers of reviews may be interested in an indication of the trustworthiness of the reviewers themselves. A convenient and inexpensive approach is based on evaluations of reviewers by readers. The data set we analyze in Section 7 comes from the website http://epinions.com, which collects such data by allowing any user i to evaluate any other user j as either untrustworthy, coded as y_ij = −1, or trustworthy, coded as y_ij = +1, where y_ij = 0 means that user i did not evaluate user j [Massa and Avesani (2007)]. The resulting network consists of n = 131,827 users and N = n(n − 1) = 17,378,226,102 observations. Since each user can only review a relatively small number of other users, the network is sparse: the vast majority of the observations y_ij are zero, with only 840,798 negative and positive evaluations. Our modeling goal, broadly speaking, is both to cluster the users based on the patterns of trusts and distrusts in this network and to understand the features of the various clusters by examining model parameters.

The rest of the article is structured as follows: A scalable model-based clustering framework based on finite mixture models is introduced in Section 2. Approximate maximum likelihood and Bayesian estimation are discussed in Sections 3 and 4, respectively, and an algorithm for Monte Carlo simulation of large networks is described in Section 5. Section 6 compares the variational GEM algorithm to the variational EM algorithm of Daudin, Picard and Robin (2008). Section 7 applies our methods to the trust network discussed above.

2. Models for large, discrete-valued networks

We consider n nodes, indexed by integers 1, …, n, and edges y_ij between pairs of nodes i and j, where y_ij can take values in a finite set of M elements. By convention, y_ii = 0 for all i, where 0 signifies “no relationship.” We call the set of all edges y_ij a discrete-valued network, which we denote by y, and we let $\mathcal{Y}$ denote the set of possible values of y. Special cases of interest are (a) undirected binary networks y, where y_ij ∈ {0, 1} is subject to the linear constraint y_ij = y_ji for all i < j; (b) directed binary networks y, where y_ij ∈ {0, 1} for all i, j; and (c) directed signed networks y, where y_ij ∈ {−1, 0, 1} for all i, j.

A general approach to modeling discrete-valued networks is based on exponential families of distributions [Besag (1974), Frank and Strauss (1986)]:

$$P_{\theta }(\mathbf{Y}=\mathbf{y}\mid \mathbf{x})=\exp \left[{\theta }^{⊺}\mathbf{g}\right(\mathbf{x},\mathbf{y})-\psi (\theta \left)\right],\mathbf{y}\in y,$$ (2.1)

where θ is the vector of canonical parameters and g(x, y) is the vector of canonical statistics depending on a matrix x of covariates, measured on the nodes or the pairs of nodes, and the network y, and ψ(θ) is given by

$$\psi \left(\theta \right)=\log \sum _{{\mathbf{y}}^{\prime }\in y}\exp \left[{\theta }^{⊺}\mathbf{g}\right(\mathbf{x},{\mathbf{y}}^{\prime }\left)\right],\theta \in {\mathbb{R}}^p,$$ (2.2)

and ensures that P_θ(Y = y | x) sums to 1.

A number of exponential family models have been proposed [e.g., Holland and Leinhardt (1981), Frank and Strauss (1986), Wasserman and Pattison (1996), Snijders et al. (2006), Hunter and Handcock (2006)]. In general, though, exponential family models are not scalable: the computing time to evaluate the likelihood function is exp(N log M), where N = n(n − 1)/2 in the case of undirected edges and N = n(n − 1) in the case of directed edges, which necessitates time-consuming estimation algorithms [e.g., Snijders (2002), Hunter and Handcock (2006), Møller et al. (2006), Koskinen, Robins and Pattison (2010), Caimo and Friel (2011)].

We therefore restrict attention to scalable exponential family models, which are characterized by dyadic independence:

$$P_{\theta }(\mathbf{Y}=\mathbf{y}\mid \mathbf{x})=\prod _{i<j}^n{P}_{\theta }(D_{\mathit{ij}}=d_{\mathit{ij}}\mid \mathbf{x}),$$ (2.3)

where D_ij ≡ D_ij(Y) corresponds to Y_ij in the case of undirected edges and (Y_ij, Y_ji) in the case of directed edges. The subscripted i < j and superscripted n mean that the product in (2.3) should be taken over all pairs (i, j) with 1 ≤ i < j ≤ n; the same is true for sums as in (3.5).

Dyadic independence has at least three advantages: (a) it facilitates estimation, because the computing time to evaluate the likelihood function scales linearly with N; (b) it facilitates simulation, because dyads are independent; and (c) by design it bypasses the so-called model degeneracy problem: if N is large, some exponential family models without dyadic independence tend to be ill-defined and impractical for modeling networks [Strauss (1986), Handcock (2003), Schweinberger (2011)].

A disadvantage is that most exponential families with dyadic independence are either simplistic [e.g., models with identically distributed edges, Erdős and Rényi (1959), Gilbert (1959)] or nonparsimonious [e.g., the p₁ model with O(n) parameters, Holland and Leinhardt (1981)].

We therefore assume that the probability mass function has a K-component mixture form as follows:

$$\begin{array}{cc}\hfill P_{\gamma ,\theta }(\mathbf{Y}=\mathbf{y}\mid \mathbf{x})& =\sum _{\mathbf{z}\in \mathcal{Z}}{P}_{\theta }(\mathbf{Y}=\mathbf{y}\mid \mathbf{x},\mathbf{Z}=\mathbf{z})P_{\gamma }(\mathbf{Z}=\mathbf{z})\hfill \\ \hfill & =\sum _{\mathbf{z}\in \mathcal{Z}}\prod _{i<j}^n{P}_{\theta }(D_{\mathit{ij}}=d_{\mathit{ij}}\mid \mathbf{x},\mathbf{Z}=\mathbf{z})P_{\gamma }(\mathbf{Z}=\mathbf{z}),\hfill \end{array}$$ (2.4)

where Z denotes the membership indicators Z₁, …, Z_n with distributions

$${\mathbf{Z}}_i\mid {\gamma }_1,\dots ,{\gamma }_K\stackrel{i.i.d.}{\sim }\text{Multinomial}(1;{\gamma }_1,\dots ,{\gamma }_K)$$ (2.5)

and $\mathcal{Z}$ denotes the support of Z. In some applications, it may be desired to model the membership indicators Z_i as functions of x by using multinomial logit or probit models with Z_i as the outcome variables and x as predictors [e.g., Tallberg (2005)]. We do not elaborate on such models here, but the variational GEM algorithms discussed in Sections 3 and 4 could be adapted to such models.

Mixture models represent a reasonable compromise between model parsimony and complexity. In particular, the assumption of conditional dyadic independence does not imply marginal dyadic independence, which means that the mixture model of (2.4) captures some degree of dependence among the dyads. We give two specific examples of mixture models below.

Example 1. The p₁ model of Holland and Leinhardt (1981) for directed, binary-valued networks may be modified using a mixture model. The original p₁ models the sequence of in-degrees (number of incoming edges of nodes) and out-degrees (number of outgoing edges of nodes) as well as reciprociated edges, postulating that the dyads are independent and that the dyadic probabilities are of the form

$$P_{\theta }(D_{\mathit{ij}}=d_{\mathit{ij}})=\exp \left[\right({\alpha }_i+{\beta }_j)y_{\mathit{ij}}+({\alpha }_j+{\beta }_i)y_{\mathit{ij}}+\rho y_{\mathit{ij}}{y}_{\mathit{ji}}-{\psi }_{\mathit{ij}}(\theta \left)\right],$$ (2.6)

where θ = (α₁, …, α_n, β₁, …, β_n, ρ) and exp{−ψ_ij(θ)} is a normalizing constant. Following Holland and Leinhardt (1981), the parameters α_i may be interpreted as activity or productivity parameters, representing the tendencies of nodes i to “send” edges to other nodes; the parameters β_j may be interpreted as attractiveness or popularity parameters, representing the tendencies of nodes j to “receive” edges from other nodes; and the parameter ρ may be interpreted as a mutuality or reciprocity parameter, representing the tendency of nodes i and j to reciprocate edges.

A drawback of this model is that it requires 2n + 1 parameters. Here, we show how to extend it to a mixture model that is applicable to both directed and undirected networks as well as discrete-valued networks, that is much more parsimonious, and that allows identification of influential nodes.

Observe that the dyadic probabilities of (2.6) are of the form

$$P_{\theta }(D_{\mathit{ij}}=d_{\mathit{ij}})\propto \exp \left[{\theta }_1^{⊺}{\mathbf{g}}_1\right(d_{\mathit{ij}})+{\theta }_{2i}^{⊺}{\mathbf{g}}_{2i}(d_{\mathit{ij}})+{\theta }_{2j}^{⊺}{\mathbf{g}}_{2j}(d_{\mathit{ij}}\left)\right],$$ (2.7)

where θ₁ = ρ is the reciprocity parameter and θ_2i = (α_i, β_i)^⊺ and θ_2j = (α_j, β_j)^⊺ are the sending and receiving propensities of nodes i and j, respectively. The corresponding statistics are the reciprocity indicator g₁(d_ij) = y_ijy_ji and the sending and receiving indicators g_2i(d_ij) = (y_ij, y_ji)^⊺ and g_2j(d_ij) = (y_ji, y_ji)^⊺ of nodes i and j, respectively. A mixture model modification of the p₁ model postulates that, conditional on Z, the dyadic probabilities are independent and of the form

$$P_{\theta }(D_{\mathit{ij}}=d_{\mathit{ij}}\mid Z_{\mathit{ik}}=Z_{\mathit{jl}}=1)\propto \exp \left[{\theta }_1^{⊺}{\mathbf{g}}_1\right(d_{\mathit{ij}})+{\theta }_{2k}^{⊺}{\mathbf{g}}_{2k}(d_{\mathit{ij}})+{\theta }_{2l}^{⊺}{\mathbf{g}}_{2l}(d_{\mathit{ij}}\left)\right],$$ (2.8)

where the parameter vectors θ_2k and θ_2l depend on the components k and l to which the nodes i and j belong, respectively. The mixture model version of the p₁ model is therefore much more parsimonious provided K ⪡ n and was proposed by Schweinberger, Petrescu-Prahova and Vu (2012) in the case of undirected, binary-valued networks. Here, the probabilities of (2.7) and (2.8) are applicable to both undirected and directed networks as well as discrete-valued networks, because the functions g_1k and g_2l may be customized to fit the situation and may even depend on covariates x, though we have suppressed this possibility in the notation. Finally, the mixture model version of the p₁ model admits model-based clustering of nodes based on indegrees or outdegrees or both. A small number of nodes with high indegree or outdegree or both is considered to be influential: if the corresponding nodes were to be removed, the network structure would be impacted.

Example 2. The mixture model of Nowicki and Snijders (2001) assumes that, conditional on Z, the dyads are independent and the conditional dyadic probabilities are of the form

$$P_{\pi }(D_{\mathit{ij}}=d\mid Z_{\mathit{ik}}=Z_{\mathit{jl}}=1)={\pi }_{d;kl}.$$ (2.9)

In other words, conditional on Z, the dyad probabilities are constant across dyads and do not depend on covariates. It is straightforward to add covariates by writing the conditional dyad probabilities in canonical form:

$$P_{\theta }(D_{\mathit{ij}}=d_{\mathit{ij}}\mid \mathbf{x},Z_{\mathit{ik}}=Z_{\mathit{jl}}=1)\propto \exp \left[{\theta }_1^{⊺}{\mathbf{g}}_1\right(\mathbf{x},d_{\mathit{ij}})+{\theta }_{kl}^{⊺}{\mathbf{g}}_2(\mathbf{x},d_{\mathit{ij}}\left)\right],$$ (2.10)

where the canonical statistic vectors g₁(x, d_ij) and g₂(x, d_ij) may depend on the covariates x. If the canonical parameter vectors θ_kl are constrained by the linear constraints θ_kl = θ_k + θ_l, where θ_k and θ_l are parameter vectors of the same dimension as θ_kl, then the mixture model version of the p₁ model arises. In other words, the mixture model version of the p₁ model can be viewed as a constrained version of the Nowicki and Snijders (2001) model. While the constrained version can be used to cluster nodes based on degree, the unconstrained version can be used to identify, for instance, high-density regions of the network, corresponding to subsets of nodes with large numbers of within-subset edges. These regions may then be studied individually in more detail by using more advanced statistical models such as exponential family models without dyadic independence as proposed by, for example, Holland and Leinhardt (1981), Frank and Strauss (1986), Strauss and Ikeda (1990), Wasserman and Pattison (1996), Snijders et al. (2006) or Hunter and Handcock (2006).

Other examples. Other mixture models for networks have been proposed by Tallberg (2005), Handcock, Raftery and Tantrum (2007) and Airoldi et al. (2008). However, these models scale less well to large networks, so we confine attention here to examples 1 and 2.

3. Approximate maximum likelihood estimation

A standard approach to maximum likelihood estimation of finite mixture models is based on the classical EM algorithm, taking the complete data to be (Y, Z), where Z is unobserved [Dempster, Laird and Rubin (1977)]. However, the E-step of an EM algorithm requires the computation of the conditional expectation of the complete data log-likelihood function under the distribution of Z | Y, which is intractable here even in the simplest cases [Daudin, Picard and Robin (2008)].

As an alternative, we consider so-called variational EM algorithms, which can be considered as generalizations of EM algorithms. The basic idea of variational EM algorithms is to construct a tractable lower bound on the intractable log-likelihood function and maximize the lower bound, yielding approximate maximum likelihood estimates. Celisse, Daudin and Pierre (2011) have shown that approximate maximum likelihood estimators along these lines are—at least in the absence of parameter constraints—consistent estimators.

We assume that all modeling of Y can be conditional on covariates x and define

$${\pi }_{d;ij,kl,\mathbf{x}}\left(\theta \right)=P_{\theta }(D_{\mathit{ij}}=d\mid Z_{\mathit{ik}}=Z_{\mathit{jl}}=1,\mathbf{x}).$$

However, for ease of presentation, we drop the notational dependence of π_d;ij,kl,x on i, j, x and make the homogeneity assumption

$${\pi }_{d;ij,kl,\mathbf{x}}\left(\theta \right)={\pi }_{d;kl}\left(\theta \right)\text{for all}i,j,\mathbf{x},$$ (3.1)

which is satisfied by the models in examples 1 and 2. Exponential parameterizations of π_d;kl(θ), as in (2.6) and (2.10), may or may not be convenient. An attractive property of the variational EM algorithm proposed here is that it can handle all possible parameterizations of π_d;kl(θ). In some cases (e.g., example 1), exponential parameterizations are more advantageous than others, while in other cases (e.g., example 2), the reverse holds.

3.1. Variational EM algorithm

Let A(z) ≡ P(Z = z) be an auxiliary distribution with support $\mathcal{Z}$. Using Jensen's inequality, the log-likelihood function can be bounded below as follows:

$$\begin{array}{cc}\hfill \log P_{\gamma ,\theta }(\mathbf{Y}=\mathbf{y})& =\log \sum _{\mathbf{z}\in \mathcal{Z}}\frac{P_{\gamma },\theta (\mathbf{Y}=\mathbf{y},\mathbf{Z}=\mathbf{z})}{A\left(\mathbf{z}\right)}A\left(\mathbf{z}\right)\hfill \\ \hfill & \ge \sum _{\mathbf{z}\in \mathcal{Z}}\left[\log \frac{P_{\gamma ,\theta }(\mathbf{Y}=\mathbf{y},\mathbf{Z}=\mathbf{z})}{A\left(\mathbf{z}\right)}\right]A\left(\mathbf{z}\right)\hfill \\ \hfill & =E_A\left[\log P_{\gamma ,\theta }\right(\mathbf{Y}=\mathbf{y},\mathbf{Z}=\mathbf{z}\left)\right]-E_A\left[\log A\right(\mathbf{Z}\left)\right].\hfill \end{array}$$ (3.2)

Some choices of A(z) give rise to better lower bounds than others. To see which choice gives rise to the best lower bound, observe that the difference between the log-likelihood function and the lower bound is equal to the Kullback–Leibler divergence from A(z) to P_γ,θ(Z = z | Y = y):

$$\begin{array}{cc}\hfill \log P& {}_{\gamma ,\theta }(\mathbf{Y}=\mathbf{y})-\sum _{\mathbf{z}\in \mathcal{Z}}\left[\log \frac{P_{\gamma ,\theta }(\mathbf{Y}=\mathbf{y},\mathbf{Z}=\mathbf{z})}{A\left(\mathbf{z}\right)}\right]A\left(\mathbf{z}\right)\hfill \\ \hfill & =\sum _{\mathbf{z}\in \mathcal{Z}}\left[\log P_{\gamma ,\theta }\right(\mathbf{Y}=\mathbf{y}\left)\right]A\left(\mathbf{z}\right)-\sum _{\mathbf{z}\in \mathcal{Z}}\left[\log \frac{P_{\gamma ,\theta }(\mathbf{Y}=\mathbf{y},\mathbf{Z}=\mathbf{z})}{A\left(\mathbf{z}\right)}\right]A\left(\mathbf{z}\right)\hfill \\ \hfill & =\sum _{\mathbf{z}\in \mathcal{Z}}\left[\log \frac{A\left(\mathbf{z}\right)}{P_{\gamma },\theta (\mathbf{Z}=\mathbf{z}\mid \mathbf{Y}=\mathbf{y})}\right]A\left(\mathbf{z}\right).\hfill \end{array}$$ (3.3)

If the choice of A(z) were unconstrained in the sense that we could choose from the set of all distributions with support $\mathcal{Z}$, then the best lower bound is obtained by the choice A(z) = P_γ,θ(Z = z | Y = y), which reduces the Kullback–Leibler divergence to 0 and makes the lower bound tight. If the optimal choice is intractable, as is the case here, then it is convenient to constrain the choice to a subset of tractable choices and substitute a choice which, within the subset of tractable choices, is as close as possible to the optimal choice in terms of Kullback–Leibler divergence. A natural subset of tractable choices is given by introducing the auxiliary parameters α = (α₁,…,α_n) and setting

$$A\left(\mathbf{z}\right)=P_{\alpha }(\mathbf{Z}=\mathbf{z})=\prod _{i=1}^n{P}_{{\alpha }_i}({\mathbf{Z}}_i={\mathbf{z}}_i),$$ (3.4)

where the marginal auxiliary distributions P_αi(Z_i = z_i) are Multinomial(1;α_i1,…,α_iK). In this case, the lower bound may be written

$$\begin{array}{cc}\hfill {\text{LB}}_{\text{ML}}(\gamma ,\theta ;\alpha )& =E_{\alpha }\left[\log P_{\gamma ,\theta }\right(\mathbf{Y}=\mathbf{y},\mathbf{Z}=\mathbf{z}\left)\right]-E_{\alpha }\left[\log P_{\alpha }\right(\mathbf{Z}\left)\right]\hfill \\ \hfill & =\sum _{i<j}^n\sum _{k=1}^K\sum _{l=1}^K{\alpha }_{ik}{\alpha }_{jl}\log {\pi }_{d_{ij};kl}\left(\theta \right)+\sum _{i=1}^n\sum _{k=1}^K{\alpha }_{ik}(\log {\gamma }_k-\log {\alpha }_{ik})\hfill \end{array}$$ (3.5)

Because equation (3.4) assumes independence, the Kullback–Leibler divergence between P_α(Z = z) and P_γ,θ(Z = z | Y = y), and thus the tightness of the lower bound, is determined by the dependence of the random variables Z₁,…,Z_n conditional on Y. If the random variables Z₁,…,Z_n are independent conditional on Y, then, for each i, there exists α_i such that P_αi(Z_i = z_i) = P_γ,θ(Z_i = z_i | Y = y), which reduces the Kullback–Leibler divergence to 0 and makes the lower bound tight. In general, the random variables Z₁,…,Z_n are not independent conditional on Y and the Kullback–Leibler divergence (3.3) is thus positive.

Approximate maximum likelihood estimates of γ and θ can be obtained by maximizing the lower bound in (3.5) using variational EM algorithms of the following form, where t is the iteration number:

• E-step: Letting γ^(t) and θ^(t) denote the current values of γ and θ, maximize LB_ML(γ^(t), θ^(t);α) with respect to α. Let α^(t+1) denote the optimal value of α and compute E_α^(t+1) [log P_γ,θ(Y = y, Z = z)].

• M-step: Maximize E_α^(t+1) [log P_γ,θ(Y = y, Z = z)] with respect to γ and θ, which is equivalent to maximizing LB_ML(γ,θ;α^(t+1)) with respect to γ and θ.

The method ensures that the lower bound is nondecreasing in the iteration number:

$${\mathrm{LB}}_{\mathrm{ML}}({\gamma }^{\left(t\right)},{\theta }^{\left(t\right)};{\alpha }^{\left(t\right)})\le {\mathrm{LB}}_{\mathrm{ML}}({\gamma }^{\left(t\right)},{\theta }^{\left(t\right)};{\alpha }^{(t+1)})$$ (3.6)

$$\le {\mathrm{LB}}_{\mathrm{ML}}({\gamma }^{(t+1)},{\theta }^{(t+1)};{\alpha }^{(t+1)}),$$ (3.7)

where inequalities (3.6) and (3.7) follow from the E-step and M-step, respectively.

It is instructive to compare the variational EM algorithm to the classical EM algorithm as applied to finite mixture models. The E-step of the variational EM algorithm minimizes the Kullback–Leibler divergence between A(z) and P_{γ^(t),θ^(t)}(Z = z | Y = y). If the choice of A(z) were unconstrained, then the optimal choice would be A(z) = P_{γ^(t),θ^(t)}(Z = z | Y = y). Therefore, in the unconstrained case, the E-step of the variational EM algorithm reduces to the E-step of the classical EM algorithm, so the classical EM algorithm can be considered to be the optimal variational EM algorithm.

3.1.1. Generalized E-step: An MM algorithm

To implement the E-step, we exploit the fact that the lower bound is nondecreasing as long as the E-step and M-step increase the lower bound. In other words, we do not need to maximize the lower bound in the E-step and M-step. Indeed, increasing rather than maximizing the lower bound in the E-step and M-step may have computational advantages when n is large. In the literature on EM algorithms, the advantages of incremental E-steps and incremental M-steps are discussed by Neal and Hinton (1993) and Dempster, Laird and Rubin (1977), respectively. We refer to the variational EM algorithm with either an incremental E-step or an incremental M-step or both as a variational generalized EM, or variational GEM, algorithm.

Direct maximization of LB_ML(γ^(t), θ^(t);α) is unattractive: equation (3.5) shows that the lower bound depends on the products α_ikα_jl and, therefore, fixed-point updates of α_ik along the lines of [Daudin, Picard and Robin (2008)] depend on all other α_jl. We demonstrate in Section 6 that the variational EM algorithm with the fixed-point implementation of the E-step can be inferior to the variational GEM algorithm when K is large.

To separate the parameters of the maximization problem, we increase LB_ML(γ^(t), θ^(t);α) via an MM algorithm [Hunter and Lange (2004)]. MM algorithms can be viewed as generalizations of EM algorithms [Hunter and Lange (2004)] and are based on iteratively constructing and then optimizing surrogate (minorizing) functions to facilitate the maximization problem in certain situations. We consider here the surrogate function

$$Q_{\mathrm{ML}}({\gamma }^{\left(t\right)},{\theta }^{\left(t\right)};{\alpha }^{\left(t\right)},\alpha )=\sum _{i<j}^n\sum _{k=1}^K\sum _{l=1}^K\left({\alpha }_{ik}^2\frac{{\alpha }_{jl}^{\left(t\right)}}{2{\alpha }_{ik}^{\left(t\right)}}+{\alpha }_{jl}^2\frac{{\alpha }_{ik}^{\left(t\right)}}{2{\alpha }_{jl}^{\left(t\right)}}\right)\log {\pi }_{d_{ij};kl}\left({\theta }^{\left(t\right)}\right)+\sum _{i=1}^n\sum _{k=1}^K{\alpha }_{ik}\left(\log {\gamma }_k^{\left(t\right)}-\log {\alpha }_{ik}^{\left(t\right)}-\frac{{\alpha }_{ik}}{{\alpha }_{ik}^{\left(t\right)}}+1\right),$$ (3.8)

which we show in Appendix A to have the following two properties:

$$Q_{\mathrm{ML}}({\gamma }^{\left(t\right)},{\theta }^{\left(t\right)},{\alpha }^{\left(t\right)};\alpha )\le {\mathrm{LB}}_{\mathrm{ML}}({\gamma }^{\left(t\right)},{\theta }^{\left(t\right)};\alpha )\text{for all}\alpha ,$$ (3.9)

$$Q_{\mathrm{ML}}({\gamma }^{\left(t\right)},{\theta }^{\left(t\right)},{\alpha }^{\left(t\right)};{\alpha }^{\left(t\right)})={\mathrm{LB}}_{\mathrm{ML}}({\gamma }^{\left(t\right)},{\theta }^{\left(t\right)};{\alpha }^{\left(t\right)}).$$ (3.10)

In the language of MM algorithms, conditions (3.9) and (3.10) establish that Q_ML(γ^(t), θ^(t), α^(t);α) is a minorizer of LB_ML(γ^(t), θ^(t);α) at α^(t). The theory of MM algorithms implies that maximizing the minorizer with respect to forces LB_ML(γ^(t), θ^(t);α) uphill [Hunter and Lange (2004)]. This maximization, involving n separate quadratic programming problems of K variables α_i under the constraints α_ik ≥ 0 for all k and ${\sum }_{k=1}^K{\alpha }_{ik}=1$, may be accomplished quickly using the method described by Stefanov (2004). When n is large, it is much easier to update α by maximizing the Q_ML function, which is the sum of functions of the individual α_i, than by maximizing the LB_ML function, in which the α parameters are not separated in this way. We therefore arrive at the following replacement for the E-step:

• Generalized E-step: For i = 1, …, n, increase Q_ML(γ(t), θ(t), α(t);α) as a function of α_i subject to α_ik ≥ 0 for all k and ${\sum }_{k=1}^K{\alpha }_{ik}=1$. Let α^(t+1) denote the new value of α.

3.1.2. More on the M-step

To maximize LB_ML(γ, θ;α^(t+1)) in the M-step, examination of (3.5) shows that maximization with respect to γ and θ may be accomplished separately. In fact, for γ, there is a simple, closed-form solution:

$${\gamma }_k^{(t+1)}=\frac{1}{n}\sum _{i=1}^n{\alpha }_{ik}^{(t+1)},k=1,\dots ,K.$$ (3.11)

Concerning θ, if there are no constraints on π(θ) other than ${\sum }_{d\in \mathcal{D}}{\pi }_{d;kl}\left(\theta \right)=1$, it is preferable to maximize with respect to π = π(θ) rather than θ, because there are closed-form expressions for π^(t+1) but not for θ^(t+1). Maximization with respect to π is accomplished by setting

$${\pi }_{d;kl}^{(t+1)}=\frac{\sum _{i<j}^n{\alpha }_{ik}^{(t+1)}{\alpha }_{jl}^{(t+1)}I(D_{ij}=d)}{\sum _{i<j}^n{\alpha }_{ik}^{(t+1)}{\alpha }_{jl}^{(t+1)}},d\in \mathcal{D},k,l=1,\dots ,K.$$ (3.12)

If the homogeneity assumption (3.1) does not hold, then closed-form expressions for π may not be available. In some cases, as in the presence of categorical covariates, closed form expressions for π are available, but the dimension of π, and thus computing time, increases with the number of categories.

If equations (2.1) and (2.3) hold, then the exponential parametrization π(θ) may be inverted to obtain an approximate maximum likelihood estimate of after the approximate MLE of π is found using the variational GEM algorithm. One method for accomplishing this inversion exploits the convex duality of exponential families [Barndor-Nielsen (1978), Wainwright and Jordan (2008)] and is explained in Appendix B.

If, in addition to the constraint ${\sum }_{d\in \mathcal{D}}{\pi }_{d;kl}\left(\theta \right)=1$, additional constraints on π are present, the maximization with respect to π may either decrease or increase computing time. Linear constraints on π can be enforced by Lagrange multipliers and reduce the dimension of π and thus computing time. Nonlinear constraints on π, as in example 1, may not admit closed form updates of π and thus may require iterative methods. If so, and if the nonlinear constraints stem from exponential family parameterizations of π(θ) with natural parameter vector θ as in example 1, then it is convenient to translate the constrained maximization problem into an unconstrained problem by maximizing LB_ML(γ, θ;α^(t+1)) with respect to θ and exploiting the fact that LB_ML(γ, θ;α^(t+1)) is a concave function of θ owing to the exponential family membership of π_d;kl(θ) [Barndorff-Nielsen (1978), page 150]. We show in Appendix C how the exponential family parameterization can be used to derive the gradient and Hessian of the lower bound of LB_ML(γ, θ;α^(t+1)) with respect to θ, which we exploit in Section 7 using a Newton–Raphson algorithm.

3.2. Standard errors

Although we maximize the lower bound LB_ML(γ, θ;α) of the log-likelihood function to obtain approximate maximum likelihood estimates, standard errors of the approximate maximum likelihood estimates $\widehat{\gamma }$ and $\widehat{\theta }$ based on the curvature of the lower bound LB_ML(γ, θ;α) may be too small. The reason is that even when the lower bound is close to the log-likelihood function, the lower bound may be more curved than the log-likelihood function [Wang and Titterington (2005)]; indeed, the higher curvature helps ensure that LB_ML(γ, θ;α) is a lower bound of the log-

likelihood function log P_γ,θ (Y = y) in the first place. As an alternative, we approximate the standard errors of the approximate maximum likelihood estimates of γ and θ by a parametric bootstrap method [Efron (1979)] that can be described as follows:

• (1)Given the approximate maximum likelihood estimates of γ and θ, sample B data sets.
• (2)For each data set, compute the approximate maximum likelihood estimates of γ and θ.

In addition to fast maximum likelihood algorithms, the parametric bootstrap method requires fast simulation algorithms. We propose such an algorithm in Section 5.

3.3. Starting and stopping

As usual with EM-like algorithms, it is a good idea to use multiple different starting values with the variational EM due to the existence of distinct local maxima. We find it easiest to use random starts in which we assign the values of α⁽⁰⁾ and then commence with an M-step. This results in values γ⁽⁰⁾ and θ⁽⁰⁾, then the algorithm continues with the first E-step, and so on. The initial ${\alpha }_{ik}^{\left(0\right)}$ are chosen independently uniformly randomly on (0, 1), then each ${\alpha }_i^{\left(0\right)}$ is multiplied by a normalizing constant chosen so that the elements of ${\alpha }_i^{\left(0\right)}$ sum to one for every i.

The numerical experiments of Section 7 use 100 random restarts each. Ideally, more restarts would be used, yet the size of the data sets with which we work makes every run somewhat expensive. We chose the number 100 because we were able to parallelize on a fairly large scale, essentially running 100 separate copies of the algorithm. Larger numbers of runs, such as 1000, would have forced longer run times since we would have had to run some of the trials in series rather than in parallel.

As a convergence criterion, we stop the algorithm as soon as

$$\frac{\mid {\mathrm{LB}}_{\mathrm{ML}}({\gamma }^{(t+1)},{\theta }^{(t+1)};{\alpha }^{(t+1)})-{\mathrm{LB}}_{\mathrm{ML}}({\gamma }^{\left(t\right)},{\theta }^{\left(t\right)};{\alpha }^{\left(t\right)})\mid }{\mid {\mathrm{LB}}_{\mathrm{ML}}({\gamma }^{(t+1)},{\theta }^{(t+1)};{\alpha }^{(t+1)})\mid }<{10}^{-10}.$$

We consider the relative change in the objective function rather than the absolute change or the changes in the parameters themselves because (1) even small changes in the parameter values can result in large changes of the objective function, and (2) the objective function is a lower bound of the log-likelihood, so small absolute changes of the objective function may not be worth the computational effort.

4. Approximate Bayesian estimation

The key to Bayesian model estimation and model selection is the marginal likelihood, defined as

$$P(\mathbf{Y}=\mathbf{y})={\int }_{\Gamma }{\int }_{ϴ}\sum _{\mathbf{z}\in \mathcal{Z}}{P}_{\gamma ,\theta }(\mathbf{Y}=\mathbf{y},\mathbf{Z}=\mathbf{z})p(\gamma ,\theta )d\gamma d\theta ,$$ (4.1)

where p(γ, θ) is the prior distribution of γ and θ. To ensure that the marginal likelihood is well-defined, we assume that the prior distribution is proper, which is common practice in mixture modeling [McLachlan and Peel (2000), Chapter 4]. A lower bound on the log marginal likelihood can be derived by introducing an auxiliary distribution with support $\mathcal{Z}$ × Γ × Θ, where Γ is the parameter space of γ and Θ is the parameter space of θ. A natural choice of auxiliary distributions is given by

$$A_{\alpha }(\mathbf{z},\gamma ,\theta )\equiv \left[\prod _{i=1}^n{P}_{{\alpha }_{\mathbf{Z},i}}({\mathbf{Z}}_i={\mathbf{z}}_i)\right]p_{{\alpha }_{\gamma }}\left(\gamma \right)\left[\prod _{i=1}^L{P}_{{\alpha }_{\theta }}\left({\theta }_i\right)\right],$$ (4.2)

where α denotes the set of auxiliary parameters α_Z = (α_Z,1, …, α_Z,n), α_γ and α_θ.

A lower bound on the log marginal likelihood can be derived by Jensen's inequality:

$$\begin{array}{cc}\hfill \log P(\mathbf{Y}=\mathbf{y})& =\log {\int }_{\Gamma }{\int }_{ϴ}\sum _{\mathbf{z}\in \mathcal{Z}}\frac{P_{\gamma ,\theta }(\mathbf{Y}=\mathbf{y},\mathbf{Z}=\mathbf{z})p(\gamma ,\theta )}{A_{\alpha }(\mathbf{z},\gamma ,\theta )}{A}_{\alpha }(\mathbf{z},\gamma ,\theta )d\gamma d\theta \hfill \\ \hfill & \ge E_{\alpha }\left[\log P_{\gamma ,\theta }\right(\mathbf{Y}=\mathbf{y},\mathbf{Z}=\mathbf{z}\left)p\right(\gamma ,\theta \left)\right]-E_{\alpha }\left[\log A_{\alpha }\right(\mathbf{Z},\gamma ,\theta \left)\right],\hfill \end{array}$$ (4.3)

where the expectations are taken with respect to the auxiliary distribution A_α (z, γ, θ).

We denote the right-hand side of (4.3) by LB_B(α_γ, α_θ; α_Z). By an argument along the lines of (3.3), one can show that the difference between the log marginal likelihood and LB_B(α_γ, α_θ; α_Z) is equal to the Kullback–Leibler divergence from the auxiliary distribution A_α(z, γ, θ) to the posterior distribution P(Z = z, γ, θ |Y = y):

$$\begin{array}{cc}\hfill \log P& (\mathbf{Y}=\mathbf{y})-{\int }_{\Gamma }{\int }_{ϴ}\sum _{\mathbf{z}\in \mathcal{Z}}\left[\log \frac{P_{\gamma ,\theta }(\mathbf{Y}=\mathbf{y},\mathbf{Z}=\mathbf{z})p(\gamma ,\theta )}{A_{\alpha }(\mathbf{z},\gamma ,\theta )}\right]A_{\alpha }(\mathbf{z},\gamma ,\theta )d\gamma d\theta \hfill \\ \hfill & ={\int }_{\Gamma }{\int }_{ϴ}\sum _{\mathbf{z}\in \mathcal{Z}}\left[\log \frac{A_{\alpha }(\mathbf{z},\gamma ,\theta )}{P(\mathbf{Z}=\mathbf{z},\gamma ,\theta \mid \mathbf{Y}=\mathbf{y})}\right]A_{\alpha }(\mathbf{z},\gamma ,\theta )d\gamma d\theta .\hfill \end{array}$$ (4.4)

The Kullback–Leibler divergence between the auxiliary distribution and the posterior distribution can be minimized by a variational GEM algorithm as follows, where t is the iteration number:

• Generalized E-step: Letting ${\alpha }_{\gamma }^{\left(t\right)}$ and ${\alpha }_{\theta }^{\left(t\right)}$ denote the current values of α_γ and α_θ, increase LB_B(${\alpha }_{\gamma }^{\left(t\right)}$, ${\alpha }_{\theta }^{\left(t\right)}$; α_Z) with respect to α_Z. Let ${\alpha }_{\mathbf{Z}}^{(t+1)}$ denote the new value of α_Z.

• Generalized M-step: Choose new values ${\alpha }_{\gamma }^{(t+1)}$ and ${\alpha }_{\theta }^{(t+1)}$ that increase LB_B(α_γ, α_θ; ${\alpha }_{\mathbf{Z}}^{(t+1)}$) with respect to α_γ and α_θ.

By construction, iteration t of a variational GEM algorithm increases the lower bound LB_B(α_γ, α_θ; α_Z):

$${\mathrm{LB}}_B({\alpha }_{\gamma }^{\left(t\right)},{\alpha }_{\theta }^{\left(t\right)};{\alpha }_{\mathbf{Z}}^{\left(t\right)})\le {\mathrm{LB}}_B({\alpha }_{\gamma }^{\left(t\right)},{\alpha }_{\theta }^{\left(t\right)};{\alpha }_{\mathbf{Z}}^{(t+1)})$$ (4.5)

$$\le {\mathrm{LB}}_B({\alpha }_{\gamma }^{(t+1)},{\alpha }_{\theta }^{(t+1)};{\alpha }_{\mathbf{Z}}^{(t+1)}).$$ (4.6)

A variational GEM algorithm approximates the marginal likelihood as well as the posterior distribution. Therefore, it tackles Bayesian model estimation and model selection at the same time.

Variational GEM algorithms for approximate Bayesian inference are only slightly more complicated to implement than the variational GEM algorithms for approximate maximum likelihood estimation presented in Section 3. To understand the difference, we examine the analogue of (3.5):

$${\mathrm{LB}}_B({\alpha }_{\gamma },{\alpha }_{\theta };{\alpha }_{\mathbf{Z}})=\sum _{i<j}^n\sum _{k=1}^K\sum _{l=1}^K{\alpha }_{\mathbf{Z},ik}{\alpha }_{\mathbf{Z},jl}{E}_{\alpha }\left[\log {\pi }_{d_{ij};kl}\right(\theta \left)\right]+E_{\alpha }\left[\log P_{\gamma }\right(\mathbf{Z}=\mathbf{z}\left)\right]+E_{\alpha }\left[\log p\right(\gamma ,\theta \left)\right]-E_{\alpha }\left[\log A\right(\mathbf{Z}=\mathbf{z},\gamma ,\theta \left)\right].$$ (4.7)

If the prior distributions of γ and θ are given by independent Dirichlet and Gaussian distributions and the auxiliary distributions of Z₁, …, Z_n, γ and θ are given by independent Multinomial, Dirichlet and Gaussian distributions, respectively, then the expectations on the right-hand side of (4.7) are tractable, with the possible exception of the expectations E_α [log π_d;kl(θ)]. Under the exponential parameterization

$${\pi }_{d;kl}\left(\theta \right)=\text{exp}\left\{{\theta }^{⊺}\mathbf{g}\left(d\right)-\log \sum _{d^{\prime }\in \mathcal{D}}\text{exp}\left[{\theta }^{⊺}\mathbf{g}\right(d^{\prime }\left)\right]\right\},$$ (4.8)

the expectations can be written as

$$E_{\alpha }\left[\log {\pi }_{d;kl}\right(\theta \left)\right]=E_{\alpha }{\left[\theta \right]}^{⊺}\mathbf{g}\left(d\right)-E_{\alpha }\left\{\log \sum _{d^{\prime }\in \mathcal{D}}\text{exp}\left[{\theta }^{⊺}\mathbf{g}\right(d^{\prime }\left)\right]\right\}$$ (4.9)

and are intractable. We are not aware of parameterizations under which the expectations are tractable. We therefore use exponential parameterizations and deal with the intractable nature of the resulting expectations by invoking Jensen's inequality:

$$E_{\alpha }\left[\log {\pi }_{d;kl}\right(\theta \left)\right]\ge E_{\alpha }{\left[\theta \right]}^{⊺}\mathbf{g}\left(d\right)-\log \sum _{d^{\prime }\in \mathcal{D}}{E}_{\alpha }\left\{\text{exp}\right[{\theta }^{⊺}\mathbf{g}\left(d^{\prime }\right)\left]\right\}.$$ (4.10)

The right-hand side of (4.10) involves expectations of independent log-normal random variables, which are tractable. We thus obtain a looser, yet tractable, lower bound by replacing E_α[log π_d;kl(θ)] in (4.7) by the right-hand side of inequality (4.10).

To save space, we do not address the specific numerical techniques that may be used to implement the variational GEM algorithm here. In short, the generalized E-step is based on an MM algorithm along the lines of Section 3.1.1. In the generalized M-step, numerical gradient-based methods may be used. A detailed treatment of this Bayesian estimation method and its implementation, using a more complicated prior distribution, may be found in Schweinberger, Petrescu-Prahova and Vu (2012); code related to this article is available at http://sites.stat.psu.edu/~dhunter/code/.

5. Monte Carlo simulation

Monte Carlo simulation of large, discrete-valued networks serves at least three purposes:

• (a)to generate simulated data to be used in simulation studies;
• (b)to approximate standard errors of the approximate maximum likelihood estimates by parametric bootstrap;
• (c)to assess model goodness of fit by simulation.

A crude Monte Carlo approach is based on sampling Z by cycling through all n nodes and sampling D_ij | Z by cycling through all n(n − 1)/2 dyads. However, the running time of such an algorithm is O(n²), which is too slow to be useful in practice, because each of the goals listed above tends to require numerous simulated data sets.

We propose Monte Carlo simulation algorithms that exploit the fact that discrete-valued networks tend to be sparse in the sense that one element of $\mathcal{D}$ is much more common than all other elements of $\mathcal{D}$. An example is given by directed, binary-valued networks, where $\mathcal{D}=\left\{\right(0,0),(0,1),(1,0),(1,1\left)\right\}$ is the sample space of dyads and $(0,0)\in \mathcal{D}$ tends to dominate all other elements of $\mathcal{D}$.

Assume there exists an element b of $\mathcal{D}$, called the baseline, that dominates the other elements of $\mathcal{D}$ in the sense that π_b;kl ≫ 1 − π_b;kl for all k and l. The Monte Carlo simulation algorithm exploiting the sparsity of large, discrete-valued networks can be described as follows:

• (1)Sample Z by sampling M ~ Multinomial(n; γ₁, …, γ_K) and assigning nodes 1, …, M₁ to component 1, nodes M₁ + 1, …, M₁ + M₂ to component 2, etc.
• (2)
  Sample Y | Z as follows: for each 1 ≤ k ≤ l ≤ K,

  • (a)
    sample the number of dyads S_kl with nonbaseline values, S_kl ~ Binomial(N_kl, 1 − π_b;kl), where N_kl is the number of pairs of nodes belonging to components k and l;
  • (b)
    sample S_kl out of N_kl pairs of nodes i < j without replacement;
  • (c)
    for each of the S_kl sampled pairs of nodes i < j, sample the nonbase-line value D_ij according to the probabilities π_d;kl/(1 − π_b;kl), $d\in \mathcal{D}$, d ≠ b.

In general, if the degree of any node (i.e., the number of nonbaseline values for all dyad variables incident on that node) has a bounded expectation, then the expected number of nonbaseline values S = Σ_k≤l S_kl in the network scales with n and the expected running time of the Monte Carlo simulation algorithm scales with $n{K}^2\mid \mathcal{D}\mid$. If K is small and n is large, then the Monte Carlo approach that exploits the sparsity of large, discrete-valued networks is superior to the crude Monte Carlo approach.

6. Comparison of algorithms

We compare the variational EM algorithm based on the fixed-point (FP) implementation of the E-step along the lines of Daudin, Picard and Robin (2008) to the variational GEM algorithm based on the MM implementation of the E-step by applying them to two data sets. The first data set comes from the study on political blogs by Adamic and Glance (2005). We convert the binary network of political blogs with two labels, liberal (+1) and conservative (−1), into a signed network by assigning labels of receivers to the corresponding directed edges. The resulting network has 1490 nodes and 2,218,610 edge variables. The second data set is the Epinions data set described in Section 1 with more than 131,000 nodes and more than 17 billion edge variables.

We compare the two algorithms using the unconstrained network mixture model of (2.9) with K = 5 and K = 20 components. For the first data set, we allow up to 1 hour for K = 5 components and up to 6 hours for K = 20 components. For the second data set, we allow up to 12 hours for K = 5 components and up to 24 hours for K = 20 components. For each data set, for each number of components and for each algorithm, we carried out 100 runs using random starting values as described in Section 3.3.

Figure 1 shows trace plots of the lower bound LB_ML(γ^(t), θ^(t); α^(t)) of the log-likelihood function, where red lines refer to the lower bound of the variational EM algorithm with FP implementation and blue lines refer to the lower bound of the variational GEM algorithm with MM implementation. The variational EM algorithm seems to outperform the variational GEM algorithm in terms of computing time when K and n are small. However, when K or n are large, the variational GEM algorithm appears far superior to the variational EM algorithm in terms of the lower bounds. The contrast is most striking when K is large, though the variational GEM seems to outperform the variational EM algorithm even when K is small and n is large. We believe that the superior performance of the variational GEM algorithm stems from the fact that it separates the parameters of the maximization problem and reduces the dependence of the updates of the variational parameters α_ik, as discussed in Section 3.1.1, while the variational EM algorithm tends to be trapped in local maxima.

Fig. 1.

Trace plots of the lower bound LB_ML(γ^(t), θ^(t); α^(t)) of the log-likelihood function for 100 runs each of the variational EM algorithm with FP implementation (red) and variational GEM algorithm with MM implementation (blue), applied to the unconstrained network mixture model of (2.9) for two different data sets.

Thus, if K and n are small and a computing cluster is available, it seems preferable to carry out a large number of runs using the variational EM algorithm in parallel, using random starting values as described in Section 3.3. However, if either K or n is large, it is preferable to use the variational GEM algorithm. Since the variational GEM algorithm is not prone to be trapped in local maxima, a small number of long runs may be all that is needed.

7. Application

Here, we address the problem of clustering the n = 131,000 users of the data set introduced in Section 1 according to their levels of trustworthiness, as indicated by the network of +1 and −1 ratings given by fellow users. To this end, we first introduce the individual “excess trust” statistics

$$e_i\left(\mathbf{y}\right)=\sum _{1\le j\le n,j\ne i}{y}_{ji}.$$

Since e_i(y) is the number of positive ratings received by user i in excess of the number of negative ratings, it is a natural measure of a user's individual trustworthiness. Our contention is that consideration of the overall pattern of network connections results in a more revealing clustering pattern than a mere consideration of the e_i(y) statistics, and we support this claim by considering three different clustering methods: A parsimonious network model using the e_i(y) statistics, the fully unconstrained network model of (2.9), and a mixture model that considers only the e_i(y) statistics while ignoring the other network structure.

For each method, we assume that the number of categories, K, is five. Partly, this choice is motivated by the fact that formal model selection methods such as the ICL criterion suggested by Daudin, Picard and Robin (2008), which we discuss in Section 9, suggest dozens if not hundreds of categories, which complicate summary and interpretation. Since the reduction of data is the primary task of statistics [Fisher (1922)], we want to keep the number of categories small and follow the standard practice of internet-based companies and websites, such as http://amazon.com and http://netflix.com, which use five categories to classify the trustworthiness of reviewers, sellers and service providers.

Our parsimonious model, which enjoys benefits over the other two alternatives as we shall see, is based on

$$P_{\theta }(D_{ij}=d_{ij}\mid Z_{ik}=Z_{jl}=1)\propto \text{exp}\left[{\theta }^{-}\right(y_{ij}^{-}+y_{ji}^{-})+{\theta }^{+}(y_{ij}^{+}+y_{ji}^{+})+{\theta }_k^{\Delta }{y}_{ji}+{\theta }_l^{\Delta }{y}_{ij}+{\theta }^{--}{y}_{ij}^{-}{y}_{ji}^{-}+{\theta }^{++}{y}_{ij}^{+}{y}_{ji}^{+}],$$ (7.1)

where $y_{ij}^{-}=I(y_{ij}=-1)$ and $y_{ij}^{+}=I(y_{ij}=1)$ are indicators of negative and positive edges, respectively. The parameters in model (7.1) are not identifiable, because $y_{ij}=y_{ij}^{+}-y_{ij}^{-}$ and $y_{ij}=y_{ji}^{+}-y_{ji}^{-}$. We therefore constrain the positive edge parameter θ⁺ to be 0. Model (7.1) assumes in the interest of model parsimony that the propensities to form negative and positive edges and to reciprocate negative and positive edges do not vary across clusters; however, the flexibility afforded by this modeling framework enables us to define cluster-specific parameters for any of these propensities if we wish. The conditional probability mass function of the whole network is given by

$$P_{\theta }(\mathbf{Y}=\mathbf{y}\mid \mathbf{Z}=\mathbf{z})\propto \text{exp}\left[{\theta }^{-}\sum _{i<j}^n(y_{ij}^{-}+y_{ji}^{-})+\sum _{k=1}^K{\theta }_k^{\Delta }{t}_k(\mathbf{y},\mathbf{z})+{\theta }^{--}\sum _{i<j}^n{y}_{ij}^{-}{y}_{ji}^{-}+{\theta }^{++}\sum _{i<j}^n{y}_{ij}^{+}{y}_{ji}^{+}\right],$$ (7.2)

where $t_k(\mathbf{y},\mathbf{z})={\sum }_{i=1}^n{z}_{ik}{e}_i\left(\mathbf{y}\right)$ is the total excess trust for all nodes in the kth category. The ${\theta }_k^{\Delta }$ parameters are therefore measures of the trustworthiness of each of the categories. Furthermore, these parameters are estimated in the presence of—that is, after correcting for—the reciprocity effects as measured by the parameters θ⁻⁻ and θ⁺⁺, which summarize the overall tendencies of users to reciprocate negative and positive ratings, respectively. Thus, θ⁻⁻ and θ⁺⁺ may be considered to measure overall tendencies toward lex talionis and quid pro quo behaviors.

One alternative model we consider is the unconstrained network model obtained from (2.9). With five components, this model comprises four mixing parameters λ₁, …, λ₄ in addition to the π_d;kl parameters, of which there are 105: there are nine types of dyads d whenever k ≠ l, contributing $8\left(\begin{array}{c}\hfill 5\hfill \\ \hfill 2\hfill \end{array}\right)=80$ parameters, and six types of dyads d whenever k = l, contributing an additional 5(5) = 25 parameters. Despite the increased flexibility afforded by model (2.9), we view the loss of interpretability due to the large number of parameters as a detriment. Furthermore, more parameters opens up the possibility of overfitting and, as we discuss below, appears to make the lower bound of the log-likelihood function highly multi-modal.

Our other alternative model is a univariate mixture model applied to the e_i(y) statistics directly, which assumes that the individual excesses e_i(y) are independent random variables sampled from a distribution with density

$$f\left(x\right)=\sum _{j=1}^5{\lambda }_j\frac{1}{{\sigma }_j}\varphi \left(\frac{x-{\mu }_j}{{\sigma }_j}\right),$$ (7.3)

where λ_j, μ_j and σ_j are component-specific mixing proportions, means and standard deviations, respectively, and ϕ(·) is the standard normal density. Traditional univariate approaches like this are less suitable than network-based clustering approaches not only because by design they neither consider nor inform us about the topology of the network, which may be relevant, but also because the individual excesses are not independent: these e_i(y) are functions of edges, and edges may be dependent owing to reciprocity (and other forms of dependence not modeled here), which decades of research [e.g., Davis (1968), Holland and Leinhardt (1981)] have shown to be important in shaping social networks. Unlike the univariate mixture model of (7.3), the mixture model we employ for networks allows for such dependence.

We use a variational GEM algorithm to estimate the network model (7.2), where the M-step is executed by a Newton–Raphson algorithm using the gradient and Hessian derived in Appendix C with a maximum of 100 iterations. It stops earlier if the largest absolute value in the gradient vector is less than 10⁻¹⁰. By contrast, the unconstrained network model following from (2.9) employs a variational GEM algorithm using the exact M-step update (3.12). The variational GEM algorithm stops when either the relative change in the objective function is less than 10⁻¹⁰ or 6000 iterations are performed. Most runs require the full 6000 iterations. To estimate the normal mixture model (7.3), we use the R package mixtools [Benaglia et al. (2009)].

To diagnose convergence of the algorithm for fitting the model (7.2), we present the trace plot of the lower bound of the log-likelihood function LB_ML(γ^(t), θ^(t); α^(t)) in Figure 2(a) and the trace plot of the cluster-specific excess parameters ${\theta }_k^{\Delta }$ in Figure 2(b). Both figures are based on 100 runs, where the starting values are obtained by the procedure described in Section 3.3. The results suggest that all 100 runs seem to converge to roughly the same solution. This fact is somewhat remarkable, since many variational algorithms appear very sensitive to their starting values, converging to multiple distinct local optima [e.g., Daudin, Pierre and Vacher (2010), Salter-Townshend and Murphy (2013)]. For instance, the 100 runs for the unconstrained network model (2.9) produced essentially a unique set of estimates for each set of random starting values. Similarly, the normal mixture model algorithm produces many different local maxima, even after we try to correct for label-switching by choosing random starting values fairly tightly clustered by their mean values.

Fig. 2.

(a) Trace plot of the lower bound LB_ML(γ^(t), θ^(t); α^(t)) of the log-likelihood function and (b) cluster-specific excess parameters ${\theta }_k^{\Delta }$, using 100 runs with random starting values.

Figure 3 shows the observed excesses e₁(y),…,e_n(y) grouped by clusters for the best solutions, as measured by likelihood or approximate likelihood, found for each of the three clustering methods. It appears that the clustering based on the parsimonious network model does a better job of separating the e_i(y) statistics into distinct subgroups—though this is not the sole criterion used—than the clusterings for the other two models, which are similar to each other. In addition, if we use a normal mixture model in which the variances are restricted to be constant across components, the results are even worse, with one large cluster and multiple clusters with few nodes.

Fig. 3.

Observed values of excess trust e_i(y), grouped by highest-probability component of i, for (a) parsimonious network mixture model (7.2) with 12 parameters, (b) normal mixture model (7.3) with 14 parameters, and (c) unconstrained network mixture model (2.9) with 109 parameters.

In Figure 4, we “ground truth” the clustering solutions using external information: the average ratings of 659,290 articles, grouped according to the highest-probability category of the article's author. While in Figure 3 the size of each cluster is the number of users in that cluster, in Figure 4 the size of each cluster is the number of articles written by users in that cluster. The widths of the boxes in Figures 3 and 4 are proportional to the square roots of the cluster sizes.

Fig. 4.

Average ratings of 659,290 articles, grouped according to the highest-probability category of the article's author, for (a) parsimonious network mixture model (7.2) with 12 parameters, (b) normal mixture model (7.3) with 14 parameters, and (c) unconstrained network mixture model (2.9) with 109 parameters. The ordering of the five categories, which is the same as in Figure 3, indicates that the unconstrained network mixture model does not even preserve the correct ordering of the median average ratings.

As an objective criterion to compare the three models, we fit one-way ANOVA models where responses are article ratings and fixed effects are the group indicators of the articles' authors. The adjusted R² values are 0.262, 0.165 and 0.172 for the network mixture model, the normal mixture model and the unconstrained network mixture model, respectively. In other words, the latent structure detected by the 12-component network mixture model of (7.2) explains the variation in article ratings better than the 14-parameter univariate mixture model or the 109-parameter unconstrained network model.

Table 1 reports estimates of the θ parameters from model (7.2) along with 95% confidence intervals reported in that table obtained by simulating 500 networks using the method of Section 5 and the parameter estimates obtained via our algorithm. For each network, we run our algorithm for 1000 iterations starting at the M-step, where the α parameters are initialized to reflect the “true” component to which each node is assigned by the simulation algorithm by setting α_ik = 10⁻¹⁰ for k not equal to the true component and α_ik = 1 – 4 × 10⁻¹⁰ otherwise. This is done to eliminate the so-called label-switching problem, which is rooted in the invariance of the likelihood function to switching the labels of the 5 components and which can affect bootstrap samples in the same way it can affect Markov chain Monte Carlo samples from the posterior of finite mixture models [Stephens (2000)]. The sample 2.5% and 97.5% quantiles form the confidence intervals shown. In addition, we give density estimates of the five trustworthiness bootstrap samples in Figure 5. Table 1 shows that some clusters of users are much more trustworthy than others. In addition, there is statistically significant evidence that users rate others in accordance with both lex talionis and quid pro quo, since both θ⁻⁻ and θ⁺⁺ are positive. These findings suggest that the ratings of pairs of users i and j are, perhaps unsurprisingly, dependent and not free of self-interest.

Table 1.

95% Confidence intervals based on parametric bootstrap using 500 simulated networks, with 1000 iterations for each network. The statistic Σ_i e_i(y)Z_ik equals Σ_i Σ_j≠i y_jiZ_ik, where Z_ik = 1 if user i is a member of cluster k and Z_ik = 0 otherwise

Parameter	Statistic	Parameter estimate	Confidence interval
Negative edges (θ−)	${\sum }_{ij}{y}_{ij}^{-}$	−24.020	(−24.029, −24.012)
Positive edges (θ+)	${\sum }_{ij}{y}_{ij}^{+}$	0	—
Negative reciprocity (θ−−)	${\sum }_{ij}{y}_{ij}^{-}{y}_{ji}^{-}$	8.660	(8.614, 8.699)
Positive reciprocity (θ++)	${\sum }_{ij}{y}_{ij}^{+}{y}_{ji}^{+}$	9.899	(9.891, 9.907)
Cluster 1 trustworthiness (${\theta }_1^{\Delta }$)	Σi ei(y)Zi1	−6.256	(−6.260, −6.251)
Cluster 2 trustworthiness (${\theta }_2^{\Delta }$)	Σi ei(y)Zi2	−7.658	(−7.662, −7.653)
Cluster 3 trustworthiness (${\theta }_3^{\Delta }$)	Σi ei(y)Zi3	−9.343	(−9.348, −9.337)
Cluster 4 trustworthiness (${\theta }_4^{\Delta }$)	Σi ei(y)Zi4	−11.914	(−11.919, −11.908)
Cluster 5 trustworthiness (${\theta }_5^{\Delta }$)	Σi ei(y)Zi5	−15.212	(−15.225, −15.200)

Fig. 5.

Kernel density estimates of the five bootstrap samples of the trustworthiness parameters, shifted so that each component's estimated parameter value (shown in the legend) equals zero.

Finally, a few remarks concerning the parametric bootstrap are appropriate. While we are encouraged by the fact that bootstrapping is even feasible for problems of this size, there are aspects of our investigation that will need to be addressed with further research. First, the bootstrapping is so time-consuming that we were forced to rely on computing clusters with multiple computing nodes to generate a bootstrap sample in reasonable time. Future work could focus on more efficient bootstrapping. Some work on efficient bootstrapping was done by Kleiner et al. (2011), but it is restricted to simple models and not applicable here.

Second, when the variational GEM algorithm is initialized at random locations, it may converge to local maxima whose LB_ML(γ, θ; α) values are inferior to the solutions attained when the algorithm is initialized at the “true” values used to simulate the networks. While it is not surprising that variational GEM algorithms converge to local maxima, it is surprising that the issue shows up in some of the simulated data sets but not in the observed data set. One possible explanation is that the structure of the observed data set is clear cut, but that the components of the estimated model are not sufficiently separated. Therefore, the estimated model may place nonnegligible probability mass on networks where two or more subsets of nodes are hard to distinguish and the variational GEM algorithm may be attracted to local maxima.

Third, some groups of confidence intervals, such as the first four trustworthiness parameter intervals, have more or less the same width. We do not have a fully satisfying explanation for this result; it may be a coincidence or it may have some deeper cause related to the difficulty of the computational problem.

In summary, we find that the clustering framework we introduce here provides useful results for a very large network. Most importantly, the sensible application of statistical modeling ideas, which reduces the unconstrained 109-parameter model to a constrained 12-parameter model, produces vastly superior results in terms of interpretability, numerical stability and predictive performance.

8. Discussion

The model-based clustering framework outlined here represents several advances. An attention to standard statistical modeling ideas relevant in the network context improves model parsimony and interpretability relative to fully unconstrained clustering models, while also suggesting a viable method for assessing precision of estimates obtained. Algorithmically, our advances allow us to apply a variational EM idea, recently applied to network clustering models in numerous publications [e.g., Nowicki and Snijders (2001), Airoldi et al. (2008), Daudin, Picard and Robin (2008), Zanghi et al. (2010), Mariadassou, Robin and Vacher (2010)], to networks far larger than any that have been considered to date. We have applied our methods to networks with over a hundred thousand nodes and signed edges, indicating how they extend to categorical-valued edges generally or models that incorporate other covariate information. In practice, these methods could have myriad uses, from identifying high-density regions of large networks to selecting among competing models for a single network to testing specific network effects of scientific interest when clustering is present.

To achieve these advances, we have focused exclusively on models exhibiting dyadic independence conditional on the cluster memberships of nodes. It is important to remember that these models are not dyadic independence models overall, since the clustering itself introduces dependence. However, to more fully capture network effects such as transitivity, more complicated models may be needed, such as the latent space models of Hoff, Raftery and Handcock (2002), Schweinberger and Snijders (2003) or Handcock, Raftery and Tantrum (2007). A major drawback of latent space models is that they tend to be less scalable than the models considered here. An example is given by the variational Bayesian algorithm developed by Salter-Townshend and Murphy (2013) to estimate the latent space model of Handcock, Raftery and Tantrum (2007). The running time of the algorithm is O(n²) and it has therefore not been applied to networks with more than n = 300 nodes and N = 89,700 edge variables. An alternative to the variational Bayesian algorithm of Salter-Townshend and Murphy (2013) based on case-control sampling was proposed by Raftery et al. (2012). However, while the computing time of this alternative algorithm is O(n), the suggested preprocessing step, which requires determining the shortest path length between pairs of nodes, is O(n²). As a result, the largest network Raftery et al. (2012) analyze is an undirected network with n = 2716 nodes and N = 3,686,970 edge variables.

In contrast, the running time of the variational GEM algorithm proposed here is O(n) in the constrained and O(f(n)) in the unconstrained version of the Nowicki and Snijders (2001) model, where f(n) is the number of edge variables whose value is not equal to the baseline value. It is worth noting that f(n) is O(n) in the case of sparse graphs and, therefore, the running time of the variational GEM algorithm is O(n) in the case of sparse graphs. Indeed, even in the presence of the covariates, the running time of the variational GEM algorithm is $O\left(n{\prod }_{i=1}^I{C}_i\right)$ with categorical covariates, where I is the number of covariates and C_i is the number of categories of the ith covariate. We have demonstrated that the variational GEM algorithm can be applied to networks with more than n = 131,000 nodes and N = 17 billion edge variables.

While the running time of O(n) shows that the variational GEM algorithm scales well with n, in practice, the “G” in “GEM” is an important contributor to the speed of the variational GEM algorithm: merely increasing the lower bound using an MM algorithm rather than actually maximizing it using a fixed-point algorithm along the lines of Daudin, Picard and Robin (2008) appears to save much computing time for large networks, though an exhaustive comparison of these two methods is a topic for further investigation.

An additional increase in speed might be gained by exploiting acceleration methods such as quasi-Newton methods [Press et al. (2002), Section 10.7], which have shown promise in the case of MM algorithms [Hunter and Lange (2004)] and which might accelerate the MM algorithm in the E-step of the variational GEM algorithm. However, application of these methods is complicated in the current modeling framework because of the exceptionally large number of auxiliary parameters introduced by the variational augmentation.

We have neglected here the problem of selecting the number of clusters. Daudin, Picard and Robin (2008) propose making this selection based on the so-called ICL criterion, but it is not known how the ICL criterion behaves when the intractable incomplete-data log-likelihood function in the ICL criterion is replaced by a variational-method lower bound. In our experience, the magnitude of the changes in the maximum lower bound value achieved with multiple random starting parameters is at least as large as the magnitude of the penalization imposed on the log-likelihood by the ICL criterion. Thus, we have been unsuccessful in obtaining reliable ICL-based results for very large networks. More investigation of this question, and of the selection of the number of clusters in general, seems warranted.

By demonstrating that scientifically interesting clustering models can be applied to very large networks by extending the variational-method ideas developed for network data sets recently in the statistical literature, we hope to encourage further investigation of the possibilities of these and related clustering methods.

The source code, written in C++, and data files used in Sections 6 and 7 are publicly available at http://sites.stat.psu.edu/~dhunter/code.

APPENDIX A: OBTAINING A MINORIZER OF THE LOWER BOUND

The lower bound LB_ML(γ, θ; α) of the log-likelihood function can be written as

$${\mathrm{LB}}_{\mathrm{ML}}(\gamma ,\theta ;\alpha )=\sum _{i<j}^n\sum _{k=1}^K\sum _{l=1}^K{\alpha }_{ik}{\alpha }_{jl}\log {\pi }_{d_{ij};kl}\left(\theta \right)+\sum _{i=1}^n\sum _{k=1}^K{\alpha }_{ik}(\log {\gamma }_k-\log {\alpha }_{ik}).$$ (A.1)

Since log π_dij;kl(θ) < 0 for all θ, the arithmetic-geometric mean inequality implies that

$${\alpha }_{ik}{\alpha }_{jl}\log {\pi }_{d_{ij};kl}\left(\theta \right)\ge \left({\alpha }_{ik}^2\frac{{\widehat{\alpha }}_{jl}}{2{\widehat{\alpha }}_{ik}}+{\alpha }_{jl}^2\frac{{\widehat{\alpha }}_{ik}}{2{\widehat{\alpha }}_{jl}}\right)\log {\pi }_{d_{ij};kl}\left(\theta \right)$$ (A.2)

[Hunter and Lange (2004)], with equality if ${\alpha }_{ik}={\widehat{\alpha }}_{ik}$ and ${\alpha }_{jl}={\widehat{\alpha }}_{jl}$. In addition, the concavity of the logarithm function gives

$$-\log {\alpha }_{ik}\ge -\log {\widehat{\alpha }}_{ik}-\frac{{\alpha }_{ik}}{{\widehat{\alpha }}_{ik}}+1$$ (A.3)

with equality if ${\alpha }_{ik}={\widehat{\alpha }}_{ik}$. Therefore, function Q_ML(γ, θ, α; $\widehat{\alpha }$) as defined in (3.8) possesses properties (3.9) and (3.10).

APPENDIX B: CONVEX DUALITY OF EXPONENTIAL FAMILIES

We show how closed-form expressions of θ in terms of π can be obtained by exploiting the convex duality of exponential families. Let

$${\psi }^{\ast }\left(\mu \right)=\underset{\theta }{\text{sup}}\{{\theta }^{⊺}\mu -\psi (\theta \left)\right\}$$ (B.1)

be the Legendre–Fenchel transform of ψ(θ), where μ ≡ μ(θ) = E_θ [g(Y)] is the mean-value parameter vector and the subscripts k and l have been dropped. By Barndorff-Nielsen [(1978), page 140] and Wainwright and Jordan [(2008), pages 67 and 68], the Legendre–Fenchel transform of ψ(θ) is self-inverse and, thus, ψ(θ) can be written as

$$\psi \left(\theta \right)=\underset{\mu }{\text{sup}}\{{\theta }^{⊺}\mu -{\psi }^{\ast }(\mu \left)\right\}=\underset{\pi }{\text{sup}}\left\{{\theta }^{⊺}\mu \right(\pi )-{\psi }^{\ast }(\mu \left(\pi \right)\left)\right\},$$ (B.2)

where $\mu \left(\pi \right)={\sum }_{d\in \mathcal{D}}\mathbf{g}\left(d\right){\pi }_d$ and ${\psi }^{\ast }\left(\mu \right(\pi \left)\right)={\sum }_{d\in \mathcal{D}}{\pi }_d\log {\pi }_d$. Therefore, closed-form expressions of θ in terms of π may be found by maximizing θ^⊺μ(π) − ψ*(μ(π)) with respect to π.

APPENDIX C: GRADIENT AND HESSIAN OF LOWER BOUND

We are interested in the gradient and Hessian with respect to the parameter vector θ of the lower bound in (A.1). The two examples of models considered in Section 2 assume that the conditional dyad probabilities π_dij;kl(θ) take the form

$${\pi }_{d_{ij};kl}\left(\theta \right)=\exp \left[{\eta }_{kl}{\left(\theta \right)}^{⊺}\mathbf{g}\right(d_{ij})-{\psi }_{kl}(\theta \left)\right],$$ (C.1)

where η_kl(θ) = A_klθ is a linear function of parameter vector θ and A_kl is a matrix of suitable order depending on components k and l. It is convenient to absorb the matrix A_kl into the statistic vector g(d_ij) and write

$${\pi }_{d_{ij};kl}\left(\theta \right)=\exp \left[{\theta }^{⊺}{\mathbf{g}}_{kl}^{\ast }\right(d_{ij})-{\psi }_{kl}(\theta \left)\right],$$ (C.2)

where ${\mathbf{g}}_{kl}^{\ast }\left(d_{ij}\right)={\mathbf{A}}_{kl}^{⊺}\mathbf{g}\left(d_{ij}\right)$. Thus, we may write

$${\text{LB}}_{\text{ML}}(\gamma ,\theta ;\alpha )=\sum _{i<j}^n\sum _{k=1}^K\sum _{l=1}^K{\alpha }_{ik}{\alpha }_{jl}\left[{\theta }^{⊺}{\mathbf{g}}_{kl}^{\ast }\right(d_{ij})-{\psi }_{kl}(\theta \left)\right]+\text{const},$$ (C.3)

where “const” denotes terms which do not depend on θ and

$${\psi }_{kl}\left(\theta \right)=\log \sum _{d\in \mathcal{D}}\exp \left[{\theta }^{⊺}{\mathbf{g}}_{kl}^{\ast }\right(d\left)\right].$$ (C.4)

Since the lower bound LB_ML(γ, θ; α) is a weighted sum of exponential family log-probabilities, it is straightforward to obtain the gradient and Hessian of LB_ML(γ, θ; α) with respect to θ, which are given by

$${\nabla }_{\theta }{\text{LB}}_{\text{ML}}(\gamma ,\theta ;\alpha )=\sum _{i<j}^n\sum _{k=1}^K\sum _{l=1}^K{\alpha }_{ik}{\alpha }_{jl}\left\{{\mathbf{g}}_{kl}^{\ast }\right(d_{ij})-E_{\theta }[{\mathbf{g}}_{kl}^{\ast }\left(D_{ij}\right)\left]\right\}$$ (C.5)

and

$${\nabla }_{\theta }^2{\text{LB}}_{\text{ML}}(\gamma ,\theta ;\alpha )=-\sum _{i<j}^n\sum _{k=1}^K\sum _{l=1}^K{\alpha }_{ik}{\alpha }_{jl}{E}_{\theta }\left[{\mathbf{g}}_{kl}^{\ast }\right(D_{ij}\left){\mathbf{g}}_{kl}^{\ast }{\left(D_{ij}\right)}^{⊺}\right],$$ (C.6)

respectively.

In other words, the gradient and Hessian of LB_ML(γ, θ; α) with respect to α are weighted sums of expectations—the means, variances and covariances of statistics. Since the sample space of dyads $\mathcal{D}$ is finite and, more often than not, small, these expectations may be computed by complete enumeration of all possible values of $d\in \mathcal{D}$ and their probabilities.