Testing and Modeling Dependencies Between a Network and Nodal Attributes

Abstract

Network analysis is often focused on characterizing the dependencies between network relations and node-level attributes. Potential relationships are typically explored by modeling the network as a function of the nodal attributes or by modeling the attributes as a function of the network. These methods require specification of the exact nature of the association between the network and attributes, reduce the network data to a small number of summary statistics, and are unable provide predictions simultaneously for missing attribute and network information. Existing methods that model the attributes and network jointly also assume the data are fully observed. In this article we introduce a unified approach to analysis that addresses these shortcomings. We use a previously developed latent variable model to obtain a low dimensional representation of the network in terms of node-specific network factors. We introduce a novel testing procedure to determine if dependencies exist between the network factors and attributes as a surrogate for a test of dependence between the network and attributes. We also present a joint model for the network relations and attributes, for use if the hypothesis of independence is rejected, that can capture a variety of dependence patterns and be used to make inference and predictions for missing observations.

1 Introduction

A common goal in the analysis of network data is to characterize the dependence between network relations and a set of node-specific attributes. For example in recent years many studies in the social sciences have examined the relationship between individuals’ friendship networks and their health measures, such as happiness, smoking and drinking behavior, and obesity (e.g. Veenstra et al. (2013) and references therein). Similarly in the biological sciences, scientists are interested in the relationship between how proteins interact and their biological importance (see Butland et al. (2005) for example). In each of these applications, the data consists of two parts: the network relations {y_i,j : i, j ∈ {1, …, n}} representing a measure of the directed relationship between each pair of nodes i and j, and p-variate nodal attributes {x_i : i ∈ {1, …, n}}. In the case of a social network, the nodes, network relations Y = {y_ij}, and attributes X = {x_i}, often represent people, their friendships, and their demographic and behavioral characteristics, respectively.

Traditional approaches to describing the dependence between a network and attributes rely on statistical methods that model either the network conditional on the attributes, {Y|X}, or the attributes conditional on the network, {X|Y}. In the social sciences, this first perspective parallels the theory of “social selection”, whereby individuals’ attributes influence the formation of their social relations, and the second perspective is motivated by “social influence” theory whereby individuals’ relations affect their attributes.

Methods that model the network as a function of the attributes, {Y|X}, commonly specify a regression framework for the dependence: the probability of the relation y_i,j is a function of β^Tx_i,j where x_i,j = f(x_i, x_j), x_i and x_j are the attributes for nodes i and j, and β is an unknown parameter vector. The covariate vector x_i,j typically includes terms for each attribute of the sender node i and receiver node j, as well as interaction terms measuring the similarity between the sender and receiver attributes. These interaction terms are frequently specified as the absolute difference between the attributes, an indicator of whether an attribute is the same for both the sender and receiver node (in the case of discrete attributes), or the product of the nodes’ attributes. Examples of network models that can accommodate such a regression term are exponentially parameterized random graph models (ERGM) (Frank and Strauss (1986), Wasserman and Pattison (1996), Snijders et al. (2006), Hunter and Handcock (2006)) and latent variable models (Hoff et al. (2002), Hoff (2005)). This latter class of models regresses a function of the network on both attribute terms and node-specific latent variables; Austin et al. (2013) proposed a slight modification to this model class where the network is expressed as a function of nodal latent variables and the latent variables themselves are regressed on the attributes.

Methods for assessing the impact of the network on nodal attributes, {X|Y}, often regress each node’s attributes on the attributes of other nodes in the network according to the network relations. For example, Christakis and Fowler (2007) use a logistic regression model to estimate the degree to which an individual’s obesity status can be explained by the obesity status of individuals in their social network (children, neighbors, spouse, etc.) (see Cohen-Cole and Fletcher (2008) and Lyons (2011) for additional discussion of these methods). Other similar models include the auto-regressive network effects models of Erbring and Young (1979) and Marsden and Friedkin (1993) and the p^* social influence models of Robins et al. (2001). All of these models are univariate, focusing a single attribute of interest that is possibly subject to social influence.

While modeling the network and attributes as functions of one another is able to provide some insight into their dependence structure, there are two primary drawbacks to utilizing these methods for analysis. First, neither modeling framework allows for simultaneous inference about the dependencies between and among the network relations and attributes. For example, when analyzing data on an adolescent friendship network and individuals’ health behaviors there may be interest in whether smoking habits and obesity status are conditionally independent given the network. Addressing this question of dependence between attributes conditional on the network is impossible using either of the conditional modeling frameworks. A second limitation of these methods is that they are unable to accommodate, and provide predictions for, datasets that have both missing network and attribute information. In the conditional modeling frameworks either the network or attributes are assumed to be fully observed.

Fellows and Handcock (2012) proposed a new class of models for jointly modeling a network and attributes called exponential-family random network models which is a combination of an ERGM and a Gibbs random field. This joint model addresses the first limitation of the conditional models and could potentially (with modification) address the second limitation of imputing missing data. However these models, like ERGMs, require careful specification to avoid model degeneracy issues (Handcock (2003), Schweinberger (2011)) and require explicit elicitation of the possible relationships between the network and attributes. Kim and Leskovec (2011) and Kim and Leskovec (2012) proposed a simple joint attribute and network model for which mathematical analysis on network connectivity and degree distributions is tractable. A limitation of this model class is that it only accommodates categorical attributes and assumes no missing attribute or network information. Both of these existing joint modeling frameworks lack traditional procedures for testing whether the joint model is appropriate, i.e., if dependencies even exist between the network and attributes.

In this article, we propose a unified approach to the analysis of network and attribute data. Our approach builds off existing network models and inference procedures and makes two key contributions to statistical network methodology. The first contribution is a testing strategy for determining whether dependencies exist between the network and attributes. The second contribution, for use when the test rejects independence, is a joint modeling framework for the network and attributes that allow for predictions for missing information and inference on the dependencies between and within the network and attributes. Neither of these objectives can be addressed with existing network analysis methods. Our proposed methodology can be summarized as follows. Investigating the dependence between network data Y and attribute data X is difficult since network data are often high dimensional, containing relational information on each pair of nodes. For this reason, in Section 2 we propose using a previously developed network model to represent the (n × n) matrix of network relations Y with a low-dimensional structure defined by an (n × r) matrix N of node-specific network factors (r ≪ n). These network factors N are not observed directly and hence are estimated from the observed network Y using a network model.

In Section 3 we introduce a novel testing strategy for evaluating whether dependencies exist between the network Y and attributes X by formally testing for correlation between the estimated network factors N and the attributes X. A conceptual representation of this testing framework is shown in Figure 1. If the network is independent of the attributes, then any functions of the network, specifically the network factors, are also independent of the attributes. A primary advantage of this approach is that the overall relationship between the network and an arbitrary number of attributes can be assessed without needing to construct a complex regression model of the network relations on the attributes or perform variable selection. In Section 4 we show via simulation that there is little loss in power for the test due to not observing the network factors directly.

Figure 1.

The primary patterns in the network Y are represented by r node-specific factors N. To determine if dependencies exist between the network Y and the p nodal attributes X, we propose a testing for a relationship between the network factors N and attributes X.

In Section 5, we propose a novel joint model for the network and attributes, for use when the hypothesis of independence between the network factors and attributes is rejected, that is able to capture dependence such as that detected by the test. This joint model allows for simultaneous estimation and inference on the dependence between and within the network and attributes, as well as provides methodology for handling and predicting missing network and attribute data. We show that the joint model conditional on the attributes can be viewed as a reduced rank regression of the network relations on attribute interactions. This further motivates the joint model as a mechanism for parsimoniously characterizing attribute and network dependence. In Section 6 our proposed methodology in used to analyze data from the National Longitudinal Study of Adolescent Health. In a cross validation experiment, we demonstrate that predictions of missing attribute data can be improved by basing imputations on both observed network and attribute information rather than attribute data alone. We conclude with a discussion in Section 7.

2 Calculation of node-specific network factors

The latent space network models in Hoff et al. (2002) and latent variable models in Hoff (2005) and Hoff (2009) provide parsimonious representations of the patterns in a network using node-specific latent factors. These models have been shown to capture a variety of network dependence patterns such as homophily, transitivity, reciprocity, and heterogeneity in node sociability and popularity. We consider an extension of the model presented in Hoff (2009) that contains additive and multiplicative latent effects, as well as structure for within dyad correlation. This model is a mosaic composed of common network effects well established in the literature. Ultimately, we use this model to obtain a low-dimensional representation of the network in terms of interpretable node-specific factors. We describe the model for continuous network data, and at the end of this section we briefly discuss how these methods can be extended to model ordinal or binary relations.

Let y_i,j represent a continuous measure of the directed relation from node i to node j. For example, in an adolescent friendship network, relation y_i,j may represent the aggregate call time of phone calls initiated by person i to person j in the last month. We consider the following model:

$$y_{i,j}=\mu +a_i+b_j+{\mathit{u}}_i^T{\mathit{\upsilon }}_j+e_{i,j},a_i,b_j\in ℝ,{\mathit{u}}_i,{\mathit{\upsilon }}_j\in {ℝ}^k.$$ (1)

The overall mean relation is represented by μ and the random error by e_i,j. The additive sender effect a_i and receiver effect b_j are often interpreted as a measure of node i’s sociability (i.e. outgoingness) and node j’s popularity respectively. The multiplicative interaction effect ${\mathit{u}}_i^T{\mathit{\upsilon }}_j$ can capture higher order dependence, such as network transitivity, balance, and clustering (Hoff (2005)). One interpretation of the these effects stems from the concept of an underlying social space (McFarland and Brown (1973), Faust (1988)), whereby nodes that are close to one another in the underlying space exhibit similar network patterns. In this context, the node-specific sender factors u_i and receiver factors υ_i can be interpreted as k-dimensional representations of the underlying outgoing (sending) and incoming (receiving) behaviors of node i. A similar interpretation was used to motivate the latent position models in Hoff et al. (2002).

The random errors are modeled as Gaussian, independent across dyads, and correlated within a dyad:

$${(e_{i,j},e_{j,i})}^T\stackrel{\mathrm{i}\mathrm{i}\mathrm{d}}{~}{\text{normal}}_2(\mathbf{0},{\sigma }_e^2(\begin{array}{cc}\hfill 1\hfill & \hfill \rho \hfill \\ \hfill \rho \hfill & \hfill 1\hfill \end{array}\left)\right).$$ (2)

The additive and multiplicative node-specific factors are also modeled as Gaussian and independent across nodes:

$${(a_i,b_i,{\mathit{u}}_i^T,{\mathit{\upsilon }}_i^T)}^T\stackrel{\mathrm{i}\mathrm{i}\mathrm{d}}{~}{\text{normal}}_{2+2k}(\mathbf{0},{\mathrm{\Sigma }}_{abu\upsilon }){\mathrm{\Sigma }}_{abu\upsilon }=\left(\begin{array}{cc}\hfill {\mathrm{\Sigma }}_{ab}\hfill & \hfill {\mathrm{\Sigma }}_{ab,u\upsilon }\hfill \\ \hfill {\mathrm{\Sigma }}_{u\upsilon ,ab}\hfill & \hfill {\mathrm{\Sigma }}_{u\upsilon }\hfill \end{array}\right).$$ (3)

The within dyad correlation ρ is interpreted as a measure of network relation reciprocity and together with the additive effects a_i and b_j induces the covariance structure from the social relations model (Warner et al. (1979), Wong (1982)).

A Bayesian estimation procedure for this network model has been implemented in the ‘amen’ package in the open source computing software program R; however the implementation restricts Σ_ab,uυ = 0. Under this restriction the model can capture third-order dependence patterns between relation “cycles”, such as {y_i,j, y_j,k, y_k,i} or {y_i,j, y_j,k, y_i,k} where the edges create a closed loop (ignoring direction), but not between noncyclic relation triples such as {y_i,j, y_j,i, y_k,i}. By allowing the additive and multiplicative effects to be dependent (i.e. Σ_ab,uυ ≠ 0) as in (3), the model is able to capture a larger class of dependencies. Specifically, it can capture correlation among sets of relation triples where each relation in the set shares at least one node with another relation in the set (i.e. dependence between {y_i,j, y_j,k, y_k,l}, but not between {y_i,j, y_j,k, y_m,l}). One justification for allowing such dependence is that latent factors that act additively, affecting node popularity and sociability, plausibly also impact the network in a multiplicative manner. A modified version of the ‘amen’ R package that supports Bayesian parameter estimation for the network model presented here is available at the corresponding author’s website.

Motivation via the singular value decomposition

A key strength of the network model in (1) is its ability to capture a variety of common network phenomena, however an alternative motivation for the model stems from its relationship to the singular value decomposition (SVD) (see Hoff (2009) and Azari and Airoldi (2012) for discussion of similar motivations for related models). The singular value decomposition is a matrix factorization that is commonly used to obtain an approximation of a matrix M by another matrix M̂ which is of reduced rank and contains the main patterns of the original matrix M. The SVD-based approximation M̂ is the optimal matrix approximation of its rank with respect to squared error loss. Here we show that the model in (1) is similar to an SVD-based approximation of the network Y, and hence can be viewed as a low dimensional representation of the network that captures the dominant patterns in the relations.

The network model in (1) can be written in matrix form as Y = M + E, where

$$M=\mu {\mathbf{1}}_{\mathit{n}}{{\mathbf{1}}_{\mathit{n}}}^T+\mathit{a}{{\mathbf{1}}_{\mathit{n}}}^T+{\mathbf{1}}_{\mathit{n}}{\mathit{b}}^T+U{V}^T,$$ (4)

1_n is an (n × 1) vector whose elements all equal one, a and b are (n × 1) vectors of the additive sender and receiver factors, U and V are (n × k) matrices of multiplicative factors, and E is an (n × n) matrix of errors.

The singular value decomposition of an arbitrary (n × n) matrix Y is written Y = ACB^T, where A and B are orthogonal (n × n) matrices and C is an (n × n) diagonal matrix with non-negative decreasing diagonal elements. The rank-k matrix that best approximates Y based on squared-error loss is given by M̂ = ÂĈB̂^T where  = A[, 1: k], Ĉ = C[1 : k, 1: k] and B̂ = B[, 1: k]. Absorbing Ĉ into  and/or B̂, the best rank-k approximation is written M̂ = ĂB̆^T. Letting μ_A, μ_B ∈ ℝ^k contain the columns means of Ă and B̆ respectively, M̂ can be expressed:

$$\hat{M}={\mu }_{AB}{\mathbf{1}}_{\mathit{n}}{{\mathbf{1}}_{\mathit{n}}}^T+\tilde{\mathit{a}}{{\mathbf{1}}_{\mathit{n}}}^T+{\mathbf{1}}_{\mathit{n}}{\tilde{\mathit{b}}}^T+\tilde{A}{\tilde{B}}^T,$$ (5)

$$\begin{array}{ccc}\tilde{A}=(\stackrel{⌣}{A}-{\mathbf{1}}_{\mathit{n}}{{\mathit{\mu }}_{\mathit{A}}}^T),\hfill & \tilde{\mathit{a}}=\tilde{A}{\mathit{\mu }}_{\mathit{B}},\hfill & {\mu }_{AB}={{\mathit{\mu }}_{\mathit{A}}}^T{\mathit{\mu }}_{\mathit{B}},\hfill \\ \tilde{B}=(\stackrel{⌣}{B}-{\mathbf{1}}_{\mathit{n}}{{\mathit{\mu }}_{\mathit{B}}}^T),\hfill & \tilde{\mathit{b}}=\tilde{B}{\mathit{\mu }}_{\mathit{A}},\hfill & \hfill \end{array}$$

where à and B̃ represent multiplicative factors with mean-zero columns, ã and b̃ represent mean-zero row and column factors, and μ_AB is an overall mean.

Observe that the representation in (5) resembles that in (4) for the network model. This illustrates the additive and multiplicative effects structure in the network model is similar to a rank-k matrix approximation of the network. Note that in the decomposition in (5), there is functional dependence between the overall mean, additive, and multiplicative effects. Since there are no such restrictions in the network model, the latent network effects represent a slightly larger class of approximations than that given by the set of matrices with rank-k.

Non-identifiability of multiplicative factors

From the representation in (4), it is evident that the multiplicative network effects (U, V) individually are non-identifiable: the probability model for Y in (1) is the same with multiplicative latent factors (U, V) as it is with factors (UA^T, VA⁻¹) for any nonsingular (k × k) matrix A. In other words, all multiplicative factors which satisfy {(UA^T, VA⁻¹) : A (k × k) nonsingular} are equivalent with respect to the probability model for Y. In the absence of prior information which prefers one parameter over another within the same equivalence class, inference for the relative likeliness of parameter values should be conducted over the equivalence classes. Performing inference over the equivalence classes of (U, V) is equivalent to performing inference on the posterior distribution of UV^T. This issue is discussed further in Sections 3 and 5.

Non-continuous network measures

Observed network information is often not continuous. For example it is common for network data to be binary where y_i,j is an indicator of whether the relation between nodes i and j exceeds some threshold, or ordinal where y_i,j represents, for instance, the relative rank of node j from the perspective of node i. To model non-continuous relations, the network model in (1) can be generalized by modeling y_i,j = ℓ(z_i,j) where z_i,j is a continuous measure of the pairwise relation and ℓ is a function defining the relationship between z_i,j and y_i,j. The latent continuous network measure z_i,j is then modeled using the network model in (1) in place of y_i,j. In the case of binary data, a thresholding function may be appropriate (corresponding to the probit model), and in the ordinal case the ordered probit can be considered. Hoff et al. (2013) discusses additional ℓ functions which account for censoring of binary and ordinal relations when nodes are restricted on the number of relations they can send (i.e. the number of non-zero relations in a row of Y). We employ one of these link functions in Section 6 to analyze fixed rank nomination data from the National Longitudinal Study of Adolescent Health using the proposed testing and joint modeling methodology introduced here.

3 Testing for dependencies

The goals in an analysis of network and attribute data are often threefold: 1) to determine whether dependencies exist between the network and attributes, 2) to model and estimate these dependencies, and finally 3) to make inference and possibly make predictions for missing data. The first step in any such analysis is to formally test for dependencies between the network and attributes. In this section, we describe a novel, simple approach to performing such a test.

A classical approach to determining whether there is an association between the nodal attributes X_n×p and network relations Y_n×n would be to test whether dependencies exist between X and the rows of Y or between X and the columns of Y. This would involve hypothesizing that each attribute is uncorrelated with each node’s outgoing relations (H₀: Cov(X[, i], Y [j,]) = 0 for all i,j) or incoming relations (H₀: Cov(X[, i], Y [,j]) = 0 for all i,j) and investigating the evidence against these claims. However, conventional multivariate analysis tests are not applicable to these problems since these tests address relationships between p + n variables based on n observations.

We propose an alternative testing approach using the the estimated latent network factors N_n×(2k+2) = [a, b, U, V] from the network model in (1). The nodal attributes X = [x₁, …, x_n]^T are independent of the network Y if and only if the attributes are independent of any function of the network. As described in Section 2, the network factors N provide a simplified representation of the network. Thus, we propose testing for dependence between the latent network factors N and attributes X on the basis that rejecting such a test would imply dependence between the network Y and attributes X (see Figure 1). However, the latent network factors N are not observed so in practice they must be estimated from the observed network Y. In this section we present a test for dependence between the estimated network factors and attributes that borrows from classic multivariate testing theory, discuss invariances in the test, and describe an exact likelihood ratio testing procedure. We also discuss alternative interpretations of the test that do not involve distributional assumptions on both the latent factors and attributes.

Suppose the nodal attributes {x_i : i ∈ {1, …, n}} are continuous and mean-zero, and let ${\mathit{n}}_i={(a_i,b_i,{\mathit{u}}_i^T,{\mathit{\upsilon }}_i^T)}^T$ denote the (estimated) latent network factors for node i. We propose testing for linear dependence between the network factors and attributes using a classical multivariate test based on the assumption that the network factors and attributes are samples from a multivariate normal distribution:

$${({\mathit{x}}_i^T,a_i,b_i,{\mathit{u}}_i^T,{\mathbf{\upsilon }}_i^T)}^T={({\mathit{x}}_i^T,{\mathit{n}}_i^T)}^T\stackrel{\mathrm{i}\mathrm{i}\mathrm{d}}{~}{\text{normal}}_{p+2+2k}\left(\right(\begin{array}{c}\hfill \mathbf{0}\hfill \\ \hfill \mathbf{0}\hfill \end{array}),{\mathrm{\Sigma }}_{XN}=(\begin{array}{cc}\hfill {\mathrm{\Sigma }}_X\hfill & \hfill {\mathrm{\Sigma }}_{X,N}\hfill \\ \hfill {\mathrm{\Sigma }}_{N,X}\hfill & \hfill {\mathrm{\Sigma }}_N\hfill \end{array}\left)\right).$$ (6)

The null and alternative hypotheses for this test are

$${\mathrm{H}}_0:{\mathrm{\Sigma }}_{X,N}=0\mathrm{v}\mathrm{s}.{\mathrm{H}}_1:{\mathrm{\Sigma }}_{X,N}\ne 0\text{based on}(6).$$ (7)

Network model and test invariances

As mentioned in Section 2, the network model in (1) is invariant under transformations of the multiplicative latent factors. Formally, this non-identifiability can be expressed as an invariance of the probability model under transformations of network factors {n_i : i ∈ {1, …, n}} by elements of group

$${𝒢}_N=\{{\mathit{G}}_N=(\begin{array}{ccc}\hfill {\mathrm{I}}_2\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill A^T\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill A^{-1}\hfill \end{array}):A_{k\times k}\text{nonsingular}\},$$

which act via multiplication on the left: ${\mathit{n}}_i={(a_i,b_i,{\mathit{u}}_i^T,{\mathit{\upsilon }}_i^T)}^T\to {\mathit{G}}_N{\mathit{n}}_i={(a_i,b_i,A^T{\mathit{u}}_i^T,A^{-1}{\mathit{\upsilon }}_i^T)}^T$. It would be undesirable for the test in (7) to depend on which latent factor estimates in the set

$$\{\{{\mathit{G}}_N{\mathit{n}}_i:i\in \{1,\dots ,n\}\}:{\mathit{G}}_N\in {𝒢}_N\}$$ (8)

are selected to represent the network. As discussed in Section 2, the probability of the network Y is the same for all latent factor estimates in this set, and hence all elements within a set are equivalent based on the likelihood. Define 𝒢 to be the extension of group 𝒢_N to transformations of ${({\mathit{x}}_i^T,{\mathit{n}}_i^T)}^T$:

$$𝒢=\{\mathit{G}=(\begin{array}{cc}\hfill I_p\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill {\mathit{G}}_N\hfill \end{array}):{\mathit{G}}_N\in {𝒢}_N\},$$

which acts via left multiplication and leaves x_i unchanged. We define 𝒢 in order to relate the invariance in the network model parameterization to the test in (7).

The testing problem in (7) is itself invariant under left multiplication of ${({\mathit{x}}_i^T,{\mathit{n}}_i^T)}^T$ by elements in the group ℱ, where ℱ is defined

$$ℱ=\{\mathit{F}=(\begin{array}{cc}\hfill B^X\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill B^N\hfill \end{array}):B_{p\times p}^X,B_{(2k+2)\times (2k+2)}^N\text{nonsingular}\}.$$

An ℱ-invariant test is a test for (7) that produces the same results for all attributes and network factors that are equivalent under group ℱ. Observe that 𝒢 is a subgroup of ℱ. This implies that an ℱ-invariant test will also respect the 𝒢-invariances in the network relations probability model. In other words, all latent network factor estimates in the same equivalence set given by (8) will generate the same test results for (7) under an ℱ-invariant test.

Likelihood ratio test

There is no uniformly most powerful invariant test for (7), however the likelihood ratio test is ℱ-invariant, unbiased (Perlman and Olkin (1980)), and generally performs well. Let N = [a, b, U, V] be the (n × (2k + 2)) matrix of network factors. The likelihood ratio test statistic can be written

$$\mathrm{\Lambda }=\frac{{\text{max}}_{{\mathrm{\Sigma }}_{XN}}L({\mathrm{\Sigma }}_{XN}|N,X)}{{\text{max}}_{{\mathrm{\Sigma }}_X,{\mathrm{\Sigma }}_N}{L}_0({\mathrm{\Sigma }}_X,{\mathrm{\Sigma }}_N|N,X)}=\prod _{i=1}^{p^(2k+2)}{(1-r_i^2)}^{-n/2}$$ (9)

where L₀ and L refer to the likelihood corresponding to the multivariate normal model in (6) with and without restricting Σ_N,X = 0. The term $r_i^2$ is the ith eigenvalue of

$${(X^{T}X)}^{-1/2}(X^{T}N){(N^{T}N)}^{-1}(N^{T}X){(X^{T}X)}^{-1/2},$$

and its positive square root is commonly referred to as the ith canonical correlation between N and X. This correlation represents the largest correlation obtainable between a linear combination of attributes and a linear combination of the network factors such that the linear combinations are uncorrelated with the respective combinations used to obtain the first i − 1 correlations. The test based on (9) rejects the null hypothesis for large values of Λ and was shown to have monotonically increasing power as a function of each population canonical correlation (Anderson and Gupta (1964)).

Under the null hypothesis, W = Λ^−2/n has a Wilks’ Lambda U(p, 2k + 2, n − (2k + 2)) distribution, which is equivalent to the product of independent, Beta distributed random variables (Muirhead (1982)):

$$W~U(p,2k+2,n-(2k+2))=\prod _{i=1}^p\text{Beta}(\frac{n-(2k+2)-p+i}{2},\frac{2k+2}{2}).$$

The α-quantiles for this distribution can be obtained via Monte Carlo estimation and used to perform exact level-α tests for (7).

Alternative interpretation of the test

The test in (7) was derived as the likelihood ratio test for a model where both the network factors and attributes are samples from a normal distribution. However in some cases these assumptions may not be appropriate. Fortunately, alternative interpretations of the test exist that do not rely on such assumptions. The likelihood ratio test in (9) for the test in (7) is the same as the likelihood ratio test to determine whether the coefficients in a linear regression are nonzero, where either the network factors are regressed on the attributes or the attributes are regressed on the network factors. These conditional tests can be expressed

$${\mathrm{H}}_0:{\beta }_{X|N}=0\mathrm{v}\mathrm{s}.{\mathrm{H}}_1:{\beta }_{X|N}\ne 0\text{based on}{\mathit{x}}_i|{\mathit{n}}_i\stackrel{\mathrm{i}\mathrm{i}\mathrm{d}}{~}\text{normal}({\beta }_{X|N}{\mathit{n}}_i,{\mathrm{\Sigma }}_{X|N}),\text{and}$$ (10)

$${\mathrm{H}}_0:{\beta }_{N|X}=0\mathrm{v}\mathrm{s}.{\mathrm{H}}_1:{\beta }_{N|X}\ne 0\text{based on}{\mathit{n}}_i|{\mathit{x}}_i\stackrel{\mathrm{i}\mathrm{i}\mathrm{d}}{~}\text{normal}({\beta }_{N|X}{\mathit{x}}_i,{\mathrm{\Sigma }}_{N|X}),$$ (11)

where β_X|N and β_N|X are (p × (2 + 2k)) and ((2 + 2k) × p) matrices, respectively. If the nodal attributes were specified as part of the study design or are binary or ordinal, it may be inappropriate to model them as Gaussian as is done in (6). Instead it may be preferable to test for dependence between the attributes and network factors via the conditional formulation in (11), where no distributional assumptions are placed on X. The likelihood ratio test for the tests in (7), (10) and (11) are identical, so the testing framework presented is appropriate if the assumption of normality is reasonable for one or both of the network factors and attributes.

4 Simulation study

To analyze data with the test outlined in Section 3, the network latent factors N must be estimated from the observed network Y. We expect this to result in a decrease in the power of the test in (7) compared to if the network factors were able to be observed directly. Furthermore, we expect a greater decrease in power when the observed network relations are less informative (i.e. binary rather than continuous). In this section we present a simulation study that quantifies the degree to which power is lost when the network factors are not observed and must be estimated from observed network relations.

Consider the network model in (1) with one multiplicative effect (k = 1), zero mean (μ = 0), and independent standard normal errors (ρ = 0, ${\sigma }_e^2=1$):

$$y_{i,j}=a_i+b_j+u_i{\upsilon }_j+e_{i,j},a_i,b_j,u_i,{\upsilon }_j\in ℝ,e_{i,j}~\text{normal}(0,1).$$ (12)

We consider the case where one nodal attribute is of interest (p = 1) and the attribute and latent network factors have one of the following covariance structures:

1. Cov[(x_i, a_i, b_i, u_i, υ_i)] = Σ_XN = I + γE_x,a,

2. Cov[(x_i, a_i, b_i, u_i, υ_i)] = Σ_XN = I + γE_x,u.

E_x,a is the (5 × 5) matrix of zeros with a one in the entires corresponding to Cov[x, a] and Cov[a, x], and E_x,u is defined analogously. In scenario A the attribute and each network factor are uncorrelated, except the additive sender factor a_i and the attribute x_i which have correlation γ. Similarly, in scenario B correlation γ exists between the sender multiplicative factor u_i and the attribute x_i.

Monte Carlo estimates of the power based on the level-0.05 likelihood ratio test in (9) for the test in (7) were computed for squared correlation values γ² ∈ {−0.05, 0, 0.05, 0.1, 0.15, 0.2}, network sizes n ∈ {25, 50, 100} and three decreasingly informative observations of the network:

1. N = [a, b, U, V] is observed;

2. N is estimated from a continuous network Y according to (1);

3. N is estimated from a binary network B_d, where B_d is defined as B_d = {b_i,j : b_i,j = 1 if y_i,j > y_d, 0 otherwise} and y_d is chosen such that the proportion of network relations greater than y_d (i.e. the network density) is d.

Notice that the binary network B_d is a deterministic function of the continuous network Y. We consider the binary networks with density 0.5 and 0.15. The former case represents a relatively dense binary network with many observed relations, whereas the latter case reflects more common network seen in survey data where information about only a small number of nodes’ relations are available. For the continuous network Y and binary networks B_d, the Bayesian estimation procedure was used to obtain estimates of the latent network factors. A probit function was specified for the binary networks. The additive factors a and b were estimated by their posterior means, and the multiplicative factors U and V were estimated by the first left and right singular vector of the posterior mean of the multiplicative effect UV^T in (4).

Figure 2 shows the power estimates for the two covariance structures A and B and the four network observations (N, Y, B_0.5, B_0.15) based on 2,000 simulations. A single power curve is shown for the latent network factors N since the correlation structures A and B are equivalent with respect to the invariances of the test in (7). Most notably, Figure 2 illustrates there is relatively little power lost when the network factors are estimated from an observed continuous or binary network, even when the network size is small. The power of the test is slightly larger when dependence exists between the attribute and an additive factor compared to when it exists between the attribute and a multiplicative factor for continuous and binary network observations. This is likely a consequence of the relative ease with which additive effects are estimated compared to interaction effects. As expected, the power of the test decreases as the observed network information becomes less informative (N → Y → B_0.5 → B_0.15), although for even moderate network sizes the power loss is negligible.

Figure 2.

Power when testing for independence between a single attribute x_i and network factors {a_i, b_i, u_i, ρ_i} based on four types of network observations (latent network factors N, continuous network Y, binary network B_0.50, binary network B_0.15).

5 Joint model for the network and nodal attributes

If the test in Section 3 rejects the null hypothesis of independence between the attributes and network factors, there is often interest in estimating and making inference on the dependencies, as well as predicting missing network and attribute information. Addressing such inference objectives requires joint modeling of the network Y and attributes X. We propose jointly modeling the network Y and attributes X via a model composed of the network relations model in (1) and (2), and the latent factor and attribute model in (6). For completeness, we include all components of the joint model below.

$$y_{i,j}=\mu +a_i+b_j+{\mathit{u}}_i^T{\mathit{\upsilon }}_j+e_{i,j}$$ (13)

$${(e_{i,j},e_{j,i})}^T\stackrel{\mathrm{i}\mathrm{i}\mathrm{d}}{~}{\text{normal}}_2(\mathbf{0},{\sigma }_e^2(\begin{array}{cc}\hfill 1\hfill & \hfill \rho \hfill \\ \hfill \rho \hfill & \hfill 1\hfill \end{array}\left)\right)$$ (14)

$${({\mathit{x}}_i^T,a_i,b_i,{\mathit{u}}_i^T,{\mathit{\upsilon }}_i^T)}^T={({\mathit{x}}_i^T,{\mathit{n}}_i^T)}^T\stackrel{\mathrm{i}\mathrm{i}\mathrm{d}}{~}{\text{normal}}_{p+2+2k}\left(\right(\begin{array}{c}\hfill \mathbf{0}\hfill \\ \hfill \mathbf{0}\hfill \end{array}),{\mathrm{\Sigma }}_{XN}=(\begin{array}{cc}\hfill {\mathrm{\Sigma }}_X\hfill & \hfill {\mathrm{\Sigma }}_{X,N}\hfill \\ \hfill {\mathrm{\Sigma }}_{N,X}\hfill & \hfill {\mathrm{\Sigma }}_N\hfill \end{array}\left)\right)$$ (15)

Inference for the dependence and conditional dependencies between the attributes and network is based on the covariance matrix Σ_XN.

Simplified parameterization

The non-identifiability of the latent factors discussed in Sections 2 and 3 translates to non-identifiability of portions of the covariance matrix Σ_XN. However, by restricting the covariance matrix to have specific structure, the 𝒢-invariance of the network model due to the multiplicative latent factors can be removed.

We propose reparameterizing the model for the latent factors and attributes in (15) by

$$({\mathit{x}}_i^T,a_i,b_i,{\mathit{u}}_i^T,{\mathit{\upsilon }}_i^T)\stackrel{\mathrm{i}\mathrm{i}\mathrm{d}}{~}{\text{normal}}_{p+2+2k}\left(\right(\begin{array}{c}\hfill \mathbf{0}\hfill \\ \hfill \mathbf{0}\hfill \\ \hfill \mathbf{0}\hfill \end{array}),{\mathrm{\Sigma }}_{XN}=(\begin{array}{ccc}\hfill {\mathrm{\Sigma }}_{Xab}\hfill & \hfill {\mathrm{\Sigma }}_{Xab,U}\hfill & \hfill {\mathrm{\Sigma }}_{Xab,V}\hfill \\ \hfill {\mathrm{\Sigma }}_{U,Xab}\hfill & \hfill D\hfill & \hfill {\mathrm{\Sigma }}_{U,V}\hfill \\ \hfill {\mathrm{\Sigma }}_{V,Xab}\hfill & \hfill {\mathrm{\Sigma }}_{V,U}\hfill & \hfill D\hfill \end{array}\left)\right),$$ (16)

where D is a diagonal matrix with decreasing elements along the diagonal. The joint model defined by (13), (14), and (16) is not invariant to transformations of the network factors and attributes by elements in the group 𝒢, however it continues to possess non-identifiability with respect to signs of the entries in U and V. Specifically, the probability of the observed network Y and attributes X is the same with parameters {U, V, Σ_XN} as it is with parameters

$$\{US,VS,(\begin{array}{ccc}\hfill I_{p+2}\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill S\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill S\hfill \end{array}\left){\mathrm{\Sigma }}_{XN}\right(\begin{array}{ccc}\hfill I_{p+2}\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill S\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill S\hfill \end{array}\left)\right\},$$ (17)

where S_k×k is any diagonal matrix with ±1’s along the diagonal. Similar to the multiplicative network factors estimation in Section 2, inference for the parameters’ relative likeliness in this model should be over equivalence classes defined by the relations in (17).

Relation to reduced rank regression

The expectation of the network relations conditional on the attributes based on (13) resembles that of a reduced rank regression on (multiplicative) attribute interaction effects. This is noteworthy as the motivations underlying reduced rank regression parallel many of the motivations for this network modeling framework.

The expectation of the network factors conditional on the attributes can be written

$$\mathrm{E}[{(a_i,b_i,{\mathit{u}}_i^T,{\mathit{\upsilon }}_i^T)}^T|{\mathit{x}}_i]={({\beta }_{a|X}{\mathit{x}}_i,{\beta }_{b|X}{\mathit{x}}_i,{({\beta }_{U|X}{\mathit{x}}_i)}^T,{({\beta }_{V|X}{\mathit{x}}_i)}^T)}^T,$$

where β_a|X, β_b|X are (p × 1) vectors and β_U|X and β_V|X are ((2 + 2k) × p) matrices of coefficients based on Σ_XN. Since the latent factors for different nodes are modeled as independent, the expectation of the network relations in (13) conditional on the attributes is

$$\mathrm{E}[y_{i,j}|{\mathit{x}}_i,{\mathit{x}}_j]=\mu +{\beta }_{a|X}{\mathit{x}}_i+{\beta }_{b|X}{\mathit{x}}_j+{\mathit{x}}_i^T{\beta }_{U|X}^T{\beta }_{V|X}{\mathit{x}}_j.$$

The interaction term ${\mathit{x}}_i^T{\beta }_{U|X}^T{\beta }_{V|X}{\mathit{x}}_j$ represents a linear combination of all possible pairwise products between the p sender and p receiver attributes, resulting in p² linear effects. The coefficients on these linear effects are given by the (k × k) matrix ${\beta }_{U|X}^T{\beta }_{V|X}$, whose rank is at most equal to the minimum of p and k. Therefore, if the number of attributes is greater than the number of multiplicative network factors (p > k), linear constraints will exist among the p² effect coefficients. In reduced rank regression the coefficient matrix corresponding to the regression of a multivariate outcome on a multivariate predictor is restricted to be reduced rank (Anderson (1951), see Reinsel and Velu (1998) for a comprehensive review). This approach to parameter dimension reduction is motivated by improvement in parameter estimation and interpretation. A similar goal exists in network modeling and is achieved here using the latent network factors. Modeling dependencies between the latent network factors N and attributes X instead of between the network relations Y and attributes X directly allows us to parsimoniously estimate and characterize complex (multiplicative) dependencies without defining a complicated regression model for the network relations. This approach is especially advantageous when the number of attributes is large and/or it is likely at most a small number of attribute pairs are related to the network.

Estimation

Estimation of the parameters in the joint network and attribute model is straightforward in a Bayesian context, where inference is based on the joint posterior distribution of the network factors {a, b, U, V} and parameters {${\sigma }_e^2$, ρ, Σ_XN} given the data {X, Y}. Since an analytic expression of the posterior distribution is not available, it is approximated by samples generated from a Markov chain Monte Carlo (MCMC) algorithm. The MCMC procedure implemented in the R package ‘amen’ for the model described in Section 2, where the additive and multiplicative factors are uncorrelated, was adapted for the joint model presented here. Details regarding the families of prior distributions considered and the corresponding MCMC algorithm are included in the appendix. Code is provided at the corresponding author’s website.

6 Analysis of AddHealth data

We consider data from a survey of 389 high-school students from the National Longitudinal Study of Adolescent Health (AddHealth) (Harris et al. (2009)) and investigate whether evidence exists that student friendships are related to student health behaviors and grade point average (GPA). The data we use includes same-sex friendship nomination data, whereby students identified the top five friends of their sex, in addition to demographic and behavioral information. The data considered here can be described as follows:

• network information - R = {r_i,j}: r_i,j is the rank of student j in student i’s listing of friends (5 = highest, 1 = lowest) or 0 if student i did not list student j;

• nodal attributes - X = [x^exercise, x^drink, x^gpa]: standardized measures of exercise frequency, drinking frequency, and grade point average;

• nodal covariate - W = [w^grade]: student grade level (9, 10, 11, or 12).

Students in the same grade and adjacent grades are more likely to be friends than students many grades apart. For this reason, we refine our question of interest to be whether students’ attributes (exercise, drinking, and GPA) are associated with their network relations’ while controlling for their grade.

We use the fixed rank nomination likelihood introduced in Hoff et al. (2013) to model the observed network ranks and restriction that at most five friends could be listed on the survey. This likelihood assumes each observed network relation r_i,j is the function of an underlying (latent) continuous measure z_i,j such that the following relation consistencies are satisfied:

$$r_{i,j}>0\Rightarrow z_{i,j}>0,r_{i,j}>r_{i,k}\Rightarrow z_{i,j}>z_{i,k},r_{i,j}=0\text{and student}i\text{listed}<5\text{friends}\Rightarrow z_{i,j}\le 0.$$ (18)

The first association is the function used in probit regression which assumes that if a friendship is reported, the latent friendship value must exceed a given threshold (in this case 0). The second relation assures consistency of the ranks with the latent friendship measures. Finally, the last association posits that friendships between a given student and all students he/she did not list as a friend must be below the friendship threshold if the nominating student listed fewer than five friends.

The network model for the latent relations z_i,j is that given in (1) with additional regression terms for whether students are in the same grade $w_{i,j}^s$ and whether they are in adjacent grades $w_{i,j}^a$

$$z_{i,j}=\mu +{\beta }_s{w}_{i,j}^s+{\beta }_a{w}_{i,j}^a+a_i+b_j+{\mathit{u}}_i^T{\mathit{\upsilon }}_j+e_{i,j},a_i,b_i\in ℝ,{\mathit{u}}_i,{\mathit{\upsilon }}_i\in {ℝ}^k.$$ (19)

Selection of factor dimension k

The multiplicative factor dimensions k for the male and female networks were determined using a method analogous to the scree plot method which is commonly used in factor analysis and principal components analysis. The network model in (18) and (19) was fit to each gender network with k = 8. Let M denote the posterior mean estimate of the multiplicative network effect UV^T, and M̂ represent the rank eight matrix approximation of M based on the singular value decomposition. The total variation in M̂ is equal to the sum of the squared singular values: ${\Vert \hat{M}\Vert }_F^2={\sum }_{\ell =1}^8{\lambda }_{\ell }^2$, where ‖·‖_F denotes the Frobenius norm and λ_i is the ith singular value. Figure 3 shows the proportion of the total variation in M̂ attributed to each multiplicative effect (i.e. ${\lambda }_i^2/{\sum }_{\ell =1}^8{\lambda }_{\ell }^2$). For both the male and female network the large majority of the variation in the network relations explained by the eight multiplicative effects is associated with the first three effects. Thus, the multiplicative effect dimension was selected to be three for both networks.

Figure 3.

Proportion of variation in the posterior mean eight factor multiplicative effect M̂ that is explained by each multiplicative effect.

Testing for dependence

As discussed in the Introduction, a traditional approach to modeling dependence between the network relations and nodal attributes would be to include regression terms in the form of sender, receiver, and interaction effects for the three attributes. Including all such effects using this approach would require 15 regression terms. However, by performing the test of independence proposed in Section 3 based on the latent network factors, we are able to assess the evidence for any relationship between the attributes and network without creating a potentially unnecessarily complex network model or performing any model selection.

The latent network factors for the model in (18) and (19) with k = 3 were estimated for the male and female networks. The additive factors a and b were estimated by their posterior means, and the multiplicative factors U and V were estimated by the first three left and right singular vectors of the posterior mean of the multiplicative effect UV^T. The test of independence between the network factors and the three nodal attributes for the female and male network resulted in p-values < 0.001. Therefore, based on a 0.05 level test, we reject the null hypothesis of independence between the student attributes and their network relations after accounting for grade structure.

Jointly modeling the network and attributes

The rejections of the tests of independence between the attributes and network suggests the network factors are informative for nodal attribute data. To investigate this claim we performed a 20-fold cross validation on each sex dataset in which 5% of data for each attribute was treated as missing in each experiment. We compared predictions for the missing attributes based on the observed attributes alone to predictions based on both the network and observed attributes. The predictions based solely on the attributes were the fitted values from a regression of each attribute on all other attributes. The predictions based on the network and attributes were the posterior mean estimates from the Bayesian estimation procedure for the joint network and attribute model introduced in Section 5. For each sex dataset, a Markov chain was run for 500 iterations of burn-in followed by an additional 500,000 iterations and samples were thinned to every 25th iteration, resulting in 20,000 simulated values for each missing element. The average effective sample size was 2,607 for the male network and 734 for the female network.

Table 1 shows the mean squared error over the 20 cross validations for each attribute and each sex dataset. The predictions based on the network and attributes improved upon the predictions based on the attributes alone for both sexes and all attributes. The improvement was greatest for male drinking frequency and female GPA where prediction mean squared error was reduced by about 15%. This illustrates that when dependence exists between the network and attributes, improvements in the predictions of missing values can be obtained by using both the network and attribute information.

Table 1.

Mean squared error for predictions from 20-fold cross validation.

	Males			Females
Method	Exercise	Drinking	GPA	Exercise	Drinking	GPA
Regression (attributes only)	1.89	3.24	2.38	1.67	2.38	2.29
Joint model (attributes & network)	1.75	2.69	2.18	1.61	2.17	1.93
% improvement	7.4	17.0	8.4	3.6	8.8	15.7

7 Discussion

In this article we introduced an original approach to testing whether dependencies exist between a network and attribute data, and a joint modeling framework for the network and attributes, which relies on a simplified representation of the network in terms of latent node-specific factors. As discussed in the Introduction, many others have proposed methodology for investigating the relationship between network and attribute data. The most common methods involve regressing either the network on functions of the nodal attributes or each node’s attributes on functions of the attributes of the node’s neighbors in the network. Frequently final models are settled upon after some, often undocumented, model selection procedure which not only alters the interpretation of the results, but also adds an additional element of subjectively to the analysis. The key distinction between the previous methods and those presented here is that our method does not involve any model selection procedures and simultaneously tests and estimates first and second order dependencies between the network and attributes.

Although the methods presented here are able to address questions about association between attributes and network relations based on a single cross-sectional dataset, the ultimate goal of much research in the social and biological sciences is to determine causal relationships using temporal network and attribute data. For example, sociologists are interested in determining whether students’ friendships impact their health behaviors or whether health behaviors cause changes in their friendship networks (Bauman and Ennett (1996)). Snijders et al. (2007) and Steglich et al. (2010) propose methods for modeling the co-evolution of networks and behaviors using actor-oriented processes whereby actors dictate changes in their behaviors and outgoing ties. These methods focus on binary data, rely on method of moments estimation procedures, and are not applicable to cross-section datasets since they model network and attribute changes conditional on the first observation. A key advantage of the latent variable model approach here is that it warrants a formal testing procedure void of any model selection procedures. Extending these methods to allow for testing of causal relationships between a network and attributes is a future research area of the authors.

A historically difficult problem not addressed here is how to select the number of multiplicative factors in the network model. In Section 6 we illustrated a procedure similar to the scree plot method used frequently to choose the number of factors in factor analysis and the number of eigenvectors in principal component analysis. An alternative approach would be to incorporate the dimension selection into the model by placing a prior on the number of factors, similar to that proposed in Hoff (2007) for the singular value decomposition, and simultaneously estimate the multiplicative latent factor dimension along with the other parameters. However this would greatly increase the complexity of the model and computation time of estimation.

Appendix

A Bayesian estimation procedure

In this section we outline the Bayesian estimation procedure used to obtain parameter estimates for the joint attribute and network model in (15). This procedure is extremely similar to that implemented in the ‘amen’ package in the statistical computing program R. We present the simple case here where the observed network Y is continuous, there are no regression terms in the network model, and there is no missing data. For details on accommodating non-continuous network data see Hoff et al. (2013) and for including regression terms see Hoff (2005).

Model -

$$\begin{array}{cc}\hfill y_{i,j}& =\mu +a_i+b_j+{\mathit{u}}_i^T{\mathit{\upsilon }}_j+e_{i,j},\hfill \\ \hfill {(e_{i,j},e_{j,i})}^T& \stackrel{\mathrm{i}\mathrm{i}\mathrm{d}}{~}{\text{normal}}_2(\mathbf{0},{\sigma }_e^2(\begin{array}{cc}\hfill 1\hfill & \hfill \rho \hfill \\ \hfill \rho \hfill & \hfill 1\hfill \end{array}\left)\right)\hfill \\ \hfill {({\mathit{x}}_i^T,a_i,b_i,{\mathit{u}}_i^T,{\mathit{\upsilon }}_i^T)}^T& \stackrel{\mathrm{i}\mathrm{i}\mathrm{d}}{~}{\text{normal}}_{p+2+2k}(\mathbf{0},{\mathrm{\Sigma }}_{XN})\hfill \end{array}$$

Prior distributions -

$$\begin{array}{cc}\hfill {\sigma }_e^{-2}& ~\text{gamma}(1/2,1/2)\hfill \\ \hfill \rho & ~\text{uniform}(-1,1)\hfill \\ \hfill {\mathrm{\Sigma }}_{XN}^{-1}& ~\text{Wishart}(p+2+2k+1,(\begin{array}{cc}\hfill {\mathrm{\Sigma }}_{X0}^{-1}\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill {\mathrm{I}}_{2+2k}\hfill \end{array}\left)\right)\hfill \end{array}$$

Markov chain Monte Carlo algorithm -

Given initial values of all latent variables {a, b, U, V} and parameters {Σ_XN, ρ, ${\sigma }_e^2$}, the algorithm proceeds as follows:

1. Sample a, b|Y, X, U, V, Σ_XN, ρ, ${\sigma }_e^2$ (normal).

2. Sample Σ_XN|Y, X, a, b, U, V, ρ, ${\sigma }_e^2$ (inverse-Wishart).

3. Update ρ using a Metropolis-Hastings step with proposal ρ^*|ρ ~ truncated normal_[−1,1] (ρ, ${\sigma }_e^2$);

4. Sample ${\sigma }_e^2|Y$, X, a, b, U, V, ρ, Σ_XN (inverse-gamma).

5. For each latent factor i:

   • Sample U[, i]|Y, X, a, b, U[, −i], V, ρ, ${\sigma }_e^2$, Σ_XN (normal);
   • Sample V [, i]|Y, X, a, b, U, V [, −i], ρ, ${\sigma }_e^2$, Σ_XN (normal).

Although the estimation algorithm is not constructed based on the unique parameterization of the model, each sample of network factors from the posterior distribution can be transformed using the covariance matrix Σ_XN sample to represent a sample from (16). Inference for the relative likeliness of parameter values is based on the posterior distribution over the parameter equivalence classes associated with representations congruent with (17).