Skip to main content
PMC home page
Springer logoLink to Springer
. 2022 Apr 9;87(1):1–11. doi: 10.1007/s11336-022-09861-x

Guest Editors’ Introduction to The Special Issue “Network Psychometrics in Action”: Methodological Innovations Inspired by Empirical Problems

Maarten Marsman 1,3,✉, Mijke Rhemtulla 2
PMCID: PMC9021145  PMID: 35397084

In 1904, Spearman reported the positive manifold—a pattern of exclusively positive correlations among cognitive test scores—for which he proposed the g-factor theory of general intelligence in 1927. According to g-factor theory, general intelligence is a stable trait that cannot be directly observed but gives rise to observed test scores. The g-factor model quickly became the dominant theory of intelligence and, while the theory was revised and expanded, the fundamental idea went unchallenged for almost a century. Modifications to the g-factor theory (e.g., multi-factor models, hierarchical factor models, and their merger) were proposed and debated at length (e.g., Carroll, 1993; Gardner, 1983; Horn & Cattell, 1966; McGrew & Flanagan, 1998; Sternberg, 1985; Thurstone, 1935). Still, all these theories held intelligence and its facets to be unobservable underlying abilities. The theory’s dominance spread from intelligence to personality research, where similar debates about the underlying factor structure of personality continue to hold great interest, and to other areas of psychology where common factors came to characterize other traits, behaviors, and attitudes. Spearman’s idea now permeates many areas of psychological research.

Although alternative characterizations of the nature of psychological phenomena such as the positive manifold were proposed (e.g., the sampling model of Thomson (1916) and the gene–environment interaction model of Dickens and Flynn (2001)), the g-factor model received a challenge almost a century later, when van der Maas and colleagues published “A dynamical model of general intelligence” (van der Maas et al., 2006). This landmark paper introduced the mutualism model, which accounts for the positive manifold in cognitive test scores by invoking a network of mutually interacting components. The mutualism model challenged the general factor theory by suggesting that intelligent behavior—and the observed pattern of positive correlations among test scores—could evolve during development from mutualistic interactions between cognitive, behavioral, and biological factors rather than unknown common causes. That is, the very same positive manifold that was assumed to arise as a result of an underlying latent factor could, it turned out, stem from a dynamical system of positively interacting components (van der Maas, Kan, Marsman, & Stevenson, 2017; van der Maas, Savi, Hofman, Kan, & Marsman, 2019). The mutualism model of intelligence offered a radically new framework for thinking about how a psychological attribute might arise and how it might be related to its constituent parts. And, as the past 15 years have shown, the proposed network approach proved to be a promising addition to the pantheon of psychometric theories (e.g., Marsman et al., 2018).

In the 15 years since van der Maas et al.’s (2006) paper was published, its central idea has sparked an entirely new psychometric subfield. This subfield, which has come to be known as network psychometrics, defines psychological constructs (e.g., intelligence, mental disorders, personality traits, and attitudes) as complex systems of behavioral, cognitive, environmental, and biological factors (Borsboom, 2017; Cramer et al., 2012; Dalege et al., 2016; Savi, Marsman, van der Maas, & Maris, 2019). These biopsychosocial systems need not contain hidden units, in principle, and are defined by the local interactions between the system’s elements, which form a network. Psychometric network theory asks how psychological phenomena emerge from these local interactions, while psychometric network analysis aims to infer these local interactions from empirical data.

While the field has made great strides, it is still in its youth. The co-development of network theories and models with technical methodological advances and software releases shapes its short history. In the following sections, we briefly review this history of methodological innovations in light of the maturation of psychological network theory. We then discuss the status of the methods regarding what questions they can and cannot yet answer and how the papers in this special issue contribute to three of the most pressing open questions.

Historical Trajectory of Network Modeling in Psychology

Borsboom (2008) took van der Maas et al.’s (2006) mutualism idea and considered it as an alternative framework for conceptualizing psychopathological disorders, suggesting that these, too, be seen as “causal networks consisting of symptoms and direct causal relations between them” (p. 1089). In the conclusion of that paper, Borsboom noted that, as of 2008, “there [was] currently no worked-out psychometric theory to go with [the network] perspective” (p. 1106), and he called for further empirical and psychometric work to elaborate on the network perspective on psychopathology. That paper marks the beginning of a concerted effort to apply network modeling theories and methods to psychological data, and to develop new theories and methods in response to those data and to the research questions of psychologists.

A few years later, seminal developments in theory and software helped the field take off. Cramer, Waldorp, van der Maas, and Borsboom (2010) took the first steps toward developing a network theory of psychopathology and showing how such a network could be modeled and visualized. Borsboom and Cramer (2013) provided examples of how to generate network visualizations from data and compute network properties like path lengths, clustering coefficients, and centrality measures. Their applied examples laid the groundwork for researchers to create network visualizations and compute network metrics from their data. At the same moment, two key software developments arrived. Epskamp, Cramer, Waldorp, Schmittmann, and Borsboom (2012) published qgraph, an R package for visualizing (and later fitting) networks in data. Shortly after that, van Borkulo et al. (2014) introduced IsingFit, an R package for fitting regularized networks to binary (e.g., symptom) data. Armed with software, applied researchers worldwide were newly able to estimate and visualize their data as networks. Many used this opportunity to spark a new way of thinking about attributes within their research areas. Early publications of this era focused on psychopathology, describing network theories of distinct disorders and using networks to explain the links between those disorders (e.g., Cramer, Borsboom, Aggen, & Kendler, (2012; Robinaugh, LeBlanc, Vuletich, & McNally, 2014; Ruzzano, Borsboom, & Geurts, 2015). Another early thread of research posited that personality, too, could be fruitfully conceived as a network of interacting components (e.g., Costantini & Perugini, 2012; Cramer et al., 2012).

While exciting, these early forays into network modeling were limited in what they were able to do: network models were, at this point, typically visualizations of correlations or partial correlations between variables observed at a single time point (although the development of longitudinal methods also began early on, e.g., Bringmann et al., 2013, Bringmann, Lemmens, Huibers, Borsboom, & Tuerlinckx, 2015). Many used l1-regularization (“lasso”) to remove some edges to obtain a sparse network diagram. The interpretable output of these early network applications consisted of (1) a network structure, wherein edges set to zero by the regularization algorithm were interpreted as missing causal links, (2) a set of edge weights, where edges large in absolute value were interpreted as strong and potentially causal direct relations, and (3) a set of node centrality indices, which were interpreted as representing the relative importance of each variable to the system.

The development of network methodology beyond its starting point was propelled by the desire to answer specific empirical questions arising from network theory. While the initial forays into network modeling attempted to answer the most general question, “what is the structure of direct relations among variables in a multivariate dataset?,” methodologists shortly began to work on methods to answer other questions, such as “how do these structures differ across groups?” (van Borkulo et al., 2015; in press), “how does the structure of a network predict future behavior”? (Dalege et al., 2016), and “how can individual networks be used to guide clinical interventions”? (Rubel, Fisher, Husen, & Lutz, 2018; Wichers, Groot, psychosystems, ESM group, & EWS group, 2016). Several of the papers in this special issue further develop answers to these empirically motivated questions.

The early explosion of empirical network analysis applications sparked some strong criticism of the methods and the common interpretations of network models, For example, several papers questioned the stability across repeated samples of network properties such as edge weights and centrality indices (Forbes, Wright, Markon, & Krueger, 2017; Fried et al., 2018; Neal & Neal, in press). Other authors critiqued the prevalent causal interpretation of network structure and centrality indices (e.g., Bringmann et al., 2019; Hallquist, Wright, & Molenaar, 2021; Rodebaugh et al., 2018; Ryan, Bringmann, & Schuurman, 2019; Spiller et al., 2020). Yet other authors have questioned the dichotomy between networks and common factors, when in fact, both may apply to most situations (Bringmann & Eronen, 2018), and each statistical network and factor model has an equivalent in the other framework (Epskamp, Maris, Waldorp, & Borsboom, 2018; Marsman et al., 2018; Waldorp & Marsman, in press). On the estimation end, the practice of fitting network models using l1-regularization has been shown to be suboptimal for most of the types of psychological data that network models are fit to (Williams, & Rast, 2020; Williams, Wysocki, Rhemtulla, & Rast, 2019; Wysocki & Rhemtulla, 2021), and the practice of inferring that edges set to zero by the regularization function are truly zero in the population (or more generally, inferring population networks are sparse when they’ve been estimated using sparse estimation procedures) has likewise been shown to be unjustified (Epskamp, Kruis, & Marsman, 2017; Williams, Briganti, Linkowski, & Mulder, 2021).

This criticism has not happened in a vacuum; methodologists have continued to develop and study new network methodologies in response to and alongside it. In response to questions about the replicability and robustness of network methodologies, researchers have begun to develop methods to quantify the uncertainty around estimated network parameters and to develop confirmatory tests for them (Rodriguez, Williams, Rast, & Mulder, 2020), as well as to investigate empirical evidence for replicability and generalizability of networks estimated on real data (Herrera-Bennett & Rhemtulla, 2021; Funkhouser, Correa, Gorka, Nelson, Phan, & Shankman, 2020). Most critics additionally offer innovative responses to their own critiques. For example, longitudinal and idiographic network methods responded to criticisms of cross-sectional network models (Bringmann et al., 2013; Epskamp, Waldorp, Mõttus, & Borsboom, 2018). New estimation methods were developed to deal with the peculiarities of psychological data (e.g., small samples, small numbers of variables, ordinal and nonnormal distributions, and dense population networks; Haslbeck & Waldorp, 2020; Williams, 2021a; Wysocki & Rhemtulla, 2021). In the face of criticism of centrality indices, researchers introduced new centrality indices and developed predictability indices (e.g., Haslbeck & Waldorp, 2018; Robinaugh, Millner, & McNally, 2016), although more development is clearly needed in this area.

When we published the call for papers for the “Network psychometrics in action” special issue in the final quarter of 2019, the field had matured a bit. We had methods that answered real empirical questions, and we had begun to grasp what our methods could or could not do. Yet, at the same time, some methodological challenges persisted. Therefore, to curate the special issue, we called for papers that “showcase how methodological innovations in the network approach that are inspired by real data can be used to answer important substantive questions.” We hoped to receive manuscripts that addressed challenges within a couple of general themes that pervade the psychological network literature. As the next section and the special issue show, we were not let down.

Three Methodological Challenges that Impede Substantive Research | Contributions to the Special Issue

The special issue’s contributions can be organized into three research themes, focusing on unique substantive questions. The first research theme concerns the discovery of network structure: what does the population network look like, and how can we calibrate our certainty in the estimates of that structure? The second theme concerns confirmatory network methodology: how can we test hypotheses about particular edges and evaluate group differences? The third theme involves the interpretation of an estimated network: how can we identify elements of a network that are important in a conceptual, predictive, or causal sense? We consider each of these themes in turn.

What is the Network’s Structure? and How Robust are Our Estimates of It?

The field started with a methodology that could estimate a network’s structure and parameters but could not quantify their uncertainty. As mentioned above, early research in this field revolved around network visualizations. At the time, network psychometrics was not more sophisticated than a pretty picture (Bringmann, 2016). But there is a real danger of becoming overconfident in the estimated network if one is unaware of the underlying uncertainty (Hinne, Gronau, van den Bergh, & Wagenmakers, 2020; Hoeting, Madigan, Raftery, & Volinsky, 1999). And there often is more uncertainty than researchers wish to acknowledge. For example, Mansueto, Wiers, van Weert, Schouten, and Epskamp (in press) recently showed that it is hard to recover the network structure from longitudinal data in typical sample sizes. Similarly, Fried and colleagues (Fried & Cramer, 2017; Fried et al., 2018) and Forbes and colleagues (2017; 2019a, b) initiated a discussion on the robustness of networks in cross-sectional data. With the limited data that we usually have, it is unlikely that we can be sure that the estimated network is correct and know its parameters with absolute certainty. However evident it may be, it took the field several years to develop the first methods that quantify the uncertainty of our network results. It is thus not surprising that concerns about the reproducibility of published network results have become prevalent.

The robustness of network results now firmly ranks as one of the field’s top priorities. Much of the development that addresses this research priority has focused on quantifying the uncertainty in the estimated parameters (e.g., Epskamp, Borsboom, & Fried, 2018; Jones, Williams, & McNally, 2021; Jongerling, Epskamp, & Williams, 2021). However, we believe that given the complex nature of network structure selection, more work should also focus on quantifying the uncertainty in the selected structure and address questions like, “which structures are plausible for the data at hand?” and “what impact does the uncertainty in the network’s structure have on our parameter estimates and their uncertainty?” Although some elegant Bayesian solutions have been developed that address these questions (e.g., Mohammadi, Massam, & Letac, in press; Mohammadi & Wit, 2015; Pensar, Nyman, Niiranen, & Corander, 2017; Williams 2021b; Williams & Mulder, 2020a), that have been paired with software implementations (Mohammadi & Wit, 2019; Williams & Mulder, 2020b), these solutions have received too little attention in the psychological literature. At the same time, this methodology could still use further development to be applied to the full spectrum of psychological network models and psychometric variables, and to address a broad range of empirical questions (e.g., to quantify uncertainty in centrality measures; Huth, Luigjes, Marsman, Goudriaan, & van Holst, 2021; Jongerling et al., 2021). The development of Bayesian models (e.g., prior specifications) that fit the psychological context is another area that deserves attention.

There are also concerns about the theoretical properties of uncertainty quantification (e.g., producing confidence intervals) in combination with the l1 constraint (i.e., lasso estimation) in frequentist approaches (i.e., Bühlmann, Kalisch, & Meier, 2014, Section 3.1; Pötscher & Leeb, 2009; Williams, 2021c). Therefore, more work should focus on alternative routes to quantify parameter uncertainty, such as Bayesian or empirical Bayesian approaches or alternative forms of regularization. Some elegant Bayesian solutions have already been developed for Gaussian Graphical models (e.g., Mohammadi & Wit, 2015; Williams, 2021b; Williams & Mulder, 2020a), but non-Gaussian models have received little attention (Pensar et al., 2017, offers a recent exception).

Two papers in the special issue aim to model the uncertainty associated with estimating the network and consequently offer more robust inference. For example, Epskamp, Isvoranu, and Chueng (2022: this issue) provide a classical hierarchical approach to aggregate independent network sources into a single estimate of its topology useful for meta-analyses of Gaussian networks. The approach does not use regularization, and its standard maximum likelihood framework is blessed with familiar solutions for standard errors and confidence intervals of the estimated parameters. Marsman, Huth, Waldorp, and Ntzoufras (2022: this issue), on the other hand, offer empirical Bayes and full Bayesian solutions for selecting the structure of an Ising model, a network for binary variables, quantifying the uncertainty in the network’s estimated structure, and the associated parameters. Both contributions offer unique solutions to gauge the uncertainty in estimated networks and deliver robust network results.

How Can We Conduct Confirmatory Tests of the Relationship Between Two Variables and Discover Differences Between Groups?

The current methodological toolbox for psychological networks is mainly exploratory. We use it to estimate a network based on the available data and then interpret the estimated network. But researchers often struggle with interpreting these networks as exploratory. For example, the absence of an edge between two variables in a lasso-estimated network is often viewed as evidence for its exclusion (Williams et al., 2021). But if the lasso edge estimate is equal to zero exactly, what evidence do we have that the edge should, in fact, be excluded from the network? The problem is that we cannot really tell. Current (frequentist) implementations of the lasso estimation procedure cannot separate the evidence for absence from the absence of evidence (i.e., that there is too little information to decide about its inclusion). Moreover, the current implementation of lasso-estimation is also not intended to be a statistical test for edge inclusion or exclusion; it is meant for selecting a single network structure but does not pit structures with a particular edge against those without that edge.

The lack of confirmatory methodology for psychological networks was a serious concern, echoed by reviews taking stock of the field (Fried & Cramer, 2017; Robinaugh, Hoekstra, Toner, & Borsboom, 2020). It is hard to formulate a cumulative science without the ability to build on what we have learned. What is the evidence for including a particular edge in the network? How do network structures compare for the data at hand? What can we say about the sign of network relations? Does the cross-sectional network hold for the population, or are there groups with a systematically different topology? In the past few years, the field has exerted considerable effort to address these questions. For example, van Bork et al. (2019; see also Kan, van der Maas, & Levine, 2019) proposed a test to identify whether data were generated from a sparse network model or a unidimensional factor model. Epskamp (2020) borrowed ideas from structural equation modeling and developed relative fit measures and likelihood-based tests for nested Gaussian Graphical models, and Williams (2021b) and Williams and Mulder (2020a) developed Bayes factor tests to assess the evidence for edge exclusion and order constraints (e.g., their sign) on the relations of these models. Van Borkulo and colleagues (in press) developed a permutation test to assess if two estimated network structures differ, and Jones, Mair, Simon, and Zeileis (2020) developed structural change tests for evaluating the impact of background variables to assess subgroup differences in the structure of Gaussian networks. For the latter, Huth et al. (in press) developed a permutation test variant suited for small sample sizes. To contrast with these classical approaches, Williams, Rast, Pericchi, and Mulder (2020) developed Bayesian solutions for assessing subgroup differences in Gaussian Graphical models (also see Williams (2021b)).

In sum, we have made great strides in developing confirmatory methods in the last three years, but the available methodology is still very limited. Where the Bayesian assessment of GGMs has received much attention, a confirmatory network methodology of non-Gaussian variables is still largely absent. Several papers in the special issue aim to fill that gap. For example, Marsman et al. (2022; this issue) offer Bayesian solutions for assessing edge inclusion for the Ising model, a network for binary variables, addressing similar questions as Williams and Mulder (2020a) did for GGMs. While Epskamp et al. (2022, this issue) offer a classical approach to gauge the heterogeneity of a GGM applied to independent datasets, Lee, Chen, DeSarbo, and Xue (2022; this issue) gauge the heterogeneity of networks of ordinal variables estimated from cross-sectional data. Lee and colleagues introduce an empirical Bayes method for estimating a finite mixture of latent GGMs to model the ordinal variables, using a new penalized Expectation–Maximization procedure to estimate the mixing weights and network parameters. Where the aforementioned approaches of, for example, van Borkulo et al. (in press) and Williams et al., (2020) assess differences between identified subgroups (i.e., observed heterogeneity), the mixture approach of Lee and colleagues allows us to assess if there are unidentified subgroups (i.e., Brusco, Steinley, Hoffman, Davis-Stober, & Wasserman, 2019). Finally, Bodner, Tuerlinckx, Bosmans, & Ceulemans (2021) recently showed how to assess the marginal dependence of two binary variables (e.g., symptom indicators) in a nonparametric way using a permutation test. In this issue, Bodner, Bringmann, Tuerlinckx, de Jonge, and Ceulemans (2022; this issue) use their method to investigate the co-occurrence of symptoms over time and construct symptom networks from the set of significant greater-than-zero values. The obtained individual network structures can then be used to reveal symptom clusters in between-subjects analyses.

What Defining Features of A Network Foster Interpretation, Prediction, and Intervention?

Where initial network analyses focused on network plots, researchers started to wonder how to interpret their network estimates. Which relations or which nodes in the network are important? Centrality measures, borrowed from network science (e.g., Newman, 2004; Newman, Barabási, & Watts, 2006), are often used to identify the important nodes in the estimated structure. But as alluded to before, centrality measures have also received several critiques in recent years. Bringmann et al. (2019) argued that the assumptions underlying centrality measures might not apply to psychological networks. They stressed the importance of considering “for what?” when interpreting a node as important. Dablander and Hinne (2019), for example, showed that the nodes that are flagged by centrality measures might not be the nodes that are important in a causal sense. But despite these critiques and concerns, centrality measures continue to be used due to a lack of a better alternative. At the same time, centrality measures focus exclusively on the network’s nodes. But for assessing causality, it seems reasonable that not only nodes, but also particular network relations are important. In this context, Haslbeck and Waldorp (2018) proposed to use nodewise predictability—the ability of a node or group of nodes to predict others.

Four papers in this issue propose methods focused on the interpretation and use of network models. Brusco, Steinley, and Watts (2022; this issue) propose methods that work on estimated networks, cleverly reordering the rows and columns of an estimated association matrix to produce a maximally interpretable structure. These methods offer a fruitful alternative to existing centrality measures and can say something about which nodes are best able to predict other nodes, that is, which nodes are most central. In a similar vein, Golino, Christensen, Moulder, Kim, and Boker (2022, this issue) introduce a novel clustering method that we can use to identify latent topics in the time series of text data, such as Twitter data. They extend their exploratory graph analysis (Golino, & Epskamp, 2017; Golino et al., 2020) approach to discover latent topics in text data taken from interviews at a single time (Kjellström & Golino, 2019) ) to multiple time points (i.e., a time series). They apply it to estimate latent topics in Twitter data taken during the 2016 US presidential election to identify word clusters and analyze individual dynamics of separate words. Both contributions of Henry, Robinaugh, and Fried (2022, this issue) and Ryan and Hamaker (2022, this issue) offer methods that aim to characterize a network by examining its implications for intervention. Henry and colleagues bring control theory to bear on dynamic psychometric networks, showing how this method, which was developed to optimize production processes, might be used to tailor clinical interventions based on individual networks. Ryan and Hamaker, on the other hand, develop a continuous-time vector autoregressive modeling approach to constructing dynamic individual networks. This model allows the researcher to extract model-implied effects on any node in the network at any time lag. The authors show how these models can be used to form precise predictions about the impact of an intervention. These predictions form the basis for two new centrality measures, total effect, and indirect effect centrality, that indicate the importance of nodes as intervention targets.

Software Contributions

Not only do each of these contributions push the field forward by proposing novel methods to discover and confirm network structure, summarize network properties, and use networks for prediction and intervention. All but one provides us with the software for doing so. In eight papers, four new R packages are introduced (packages: ConNEcT, Bodner et al., 2022; Bodner & Ceulemans, in press; netcontrol, Henry et al., 2022; rbinnet, Marsman et al., 2022; ctnet, Ryan, and Hamaker, 2022), two previously introduced R packages are expanded to include new methods ((psychonetrics, Epskamp, 2020; Epskamp et al., 2022; EGAnet, Golino & Epskamp, 2017; Golino et al., 2022), and one paper provides code in MATLAB and R for implementing the new method (Brusco et al. 2022).

Closing Statement

Having offered a brief overview of the historic trends, the questions facing network psychometrics as we see them, and the contributions provided by the papers in this special issue, we leave our reader to read. We offer our heartfelt thanks to every author who contributed their fine work, and we hope you enjoy this truly excellent set of papers.

Footnotes

The editorial process for this paper was not handled by the editors of the special issue, Maarten Marsman and Mijke Rhemtulla, but by Edward Ip instead.

MM was supported by a Veni grant (451-17-017) from the Netherlands Organization for Scientific Research (NWO).

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  1. Bodner, N., Bringmann, L. F., Tuerlinckx, F., de Jonge, P., & Ceulemans, E. (2022). ConNEcT: A novel network approach for investigating the co-occurrence of binary psychopathological symptoms over time. Psychometrika, this issue. [DOI] [PubMed]
  2. Bodner, N., & Ceulemans, E. (in press). ConNEcT: An R package to build contingency measure-based networks on binary time series. Behavior Research Methods. [DOI] [PubMed]
  3. Bodner N, Tuerlinckx F, Bosmans G, Ceulemans E. Accounting for auto-dependency in binary dyadic time series data: A comparison of model- and permutation-based approaches for testing pairwise associations. British Journal of Mathematical and Statistical Psychology. 2021;74:86–109. doi: 10.1111/bmsp.12222. [DOI] [PubMed] [Google Scholar]
  4. Borsboom D. Psychometric perspectives on diagnostic systems. Journal of Clinical Psychology. 2008;64:1089–1108. doi: 10.1002/jclp.20503. [DOI] [PubMed] [Google Scholar]
  5. Borsboom D. A network theory of mental disorders. World Psychiatry. 2017;16:5–13. doi: 10.1002/wps.20375. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Borsboom D, Cramer AOJ. Network analysis: An integrative approach to the structure of psychopathology. Annual Review of Clinical Psychology. 2013;9:91–121. doi: 10.1146/annurev-clinpsy-050212-185608. [DOI] [PubMed] [Google Scholar]
  7. Bringmann, L. F. (2016). Dynamical networks in psychology: More than a pretty picture? (Unpublished doctoral dissertation). Katholieke Universiteit Leuven.
  8. Bringmann LF, Elmer T, Epskamp S, Krause RW, Schoch D, Wichers M, Wigman JTW, Snippe E. What do centrality measures measure in psychological networks? Journal of Abnormal Psychology. 2019;128:892–903. doi: 10.1037/abn0000446. [DOI] [PubMed] [Google Scholar]
  9. Bringmann LF, Eronen MI. Don’t blame the model: Reconsidering the network approach to psychopathology. Psychological Review. 2018;125:606–615. doi: 10.1037/rev0000108. [DOI] [PubMed] [Google Scholar]
  10. Bringmann LF, Lemmens LHJM, Huibers MJH, Borsboom D, Tuerlinckx F. Revealing the dynamic network structure of the Beck Depression Inventory-II. Psychological Medicine. 2015;45:747–757. doi: 10.1017/S0033291714001809. [DOI] [PubMed] [Google Scholar]
  11. Bringmann LF, Pe ML, Vissers N, Ceulemans E, Borsboom D, VanPaemel F, Kuppens P. Assessing temporal emotion dynamics using networks. Assessment. 2016;23:425–435. doi: 10.1177/1073191116645909. [DOI] [PubMed] [Google Scholar]
  12. Bringmann LF, Vissers N, Wichers M, Geschwind N, Kuppens P, Peeters F, Borsboom D, Tuerlinckx F. A network approach to psychopathology: New insights into clinical longitudinal data. PLoS One. 2013;8:e60188. doi: 10.1371/journal.pone.0060188. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Brusco MJ, Steinley D, Hoffman M, Davis-Stober C, Wasserman S. On Ising models and algorithms for the construction of symptom networks in psychopathological research. Psychological Methods. 2019;24:735–753. doi: 10.1037/met0000207. [DOI] [PubMed] [Google Scholar]
  14. Brusco, M. J., Steinley, D., & Watts, A. L. (2022). Disentangling relationships in symptom networks using matrix permutation methods. Psychometrika, this issue. [DOI] [PubMed]
  15. Bühlmann P, Kalisch M, Meier L. High-dimensional statistics with a view toward applications in biology. Annual Reviews of Statistics and Its Applications. 2014;1:255–278. [Google Scholar]
  16. Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic studies. Cambridge University Press.
  17. Costantini G, Perugini M. The definition of components and the use of formal indexes are key steps for a successful application of network analysis in personality psychology. European Journal of Personality. 2012;26:434–435. [Google Scholar]
  18. Cramer AOJ, Borsboom D, Aggen SH, Kendler KS. The pathoplasticity of dysphoric episodes: Differential impact of stressful life events on the pattern of depressive symptom inter-correlations. Psychological Medicine. 2012;42:957–965. doi: 10.1017/S003329171100211X. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Cramer AOJ, van der Sluis S, Noordhof A, Wichers M, Geschwind N, Aggen SH, Kendler KS, Borsboom D. Dimensions of normal personality as networks in search for equilibrium: You can’t like parties if you don’t like people. European Journal of Personality. 2012;26:414–431. [Google Scholar]
  20. Cramer AOJ, Waldorp LJ, van der Maas HLJ, Borsboom D. Comorbidity: A network perspective. Behavioral and Brain Sciences. 2010;33:137–150. doi: 10.1017/S0140525X09991567. [DOI] [PubMed] [Google Scholar]
  21. Dablander F, Hinne M. Node centrality measures are a poor substitute for causal inference. Scientific Reports. 2019;9:6846. doi: 10.1038/s41598-019-43033-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Dalege J, Borsboom D, van Harreveld F, van den Berg H, Conner M, van der Maas HLJ. Toward a formalized account of attitudes: The causal attitude network (CAN) model. Psychological Review. 2016;123:2–22. doi: 10.1037/a0039802. [DOI] [PubMed] [Google Scholar]
  23. Dickens WT, Flynn JR. Heritability estimates versus large environmental effects: The IQ paradox resolved. Psychological Review. 2001;108:346–369. doi: 10.1037/0033-295x.108.2.346. [DOI] [PubMed] [Google Scholar]
  24. Epskamp E. Psychometric network models from time-series and panel data. Psychometrika. 2022;85:206–231. doi: 10.1007/s11336-020-09697-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Epskamp E, Borsboom D, Fried EI. Estimating psychological networks and their accuracy: A tutorial paper. Behavior Research Methods. 2018;50:195–212. doi: 10.3758/s13428-017-0862-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Epskamp, S., Cramer, A. O. J., Waldorp, L. J., Schmittmann, V. D., & Borsboom, D. (2012). qgraph: Network visualizations of relationships in psychometric data. Journal of Statistical Software, 48(4).
  27. Epskamp, S., Isvoranu, A.-M., & Chueng, M. W.-L. (2022). Meta-analytic Gaussian network aggregation. Psychometrika, this issue. [DOI] [PMC free article] [PubMed]
  28. Epskamp S, Kruis J, Marsman M. Estimating psychopathological networks: Be careful what you wish for. PloS One. 2017;12:e017891. doi: 10.1371/journal.pone.0179891. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Epskamp, S., Maris, G. K. J., Waldorp, L. J., & Borsboom, D. (2018). Network psychometrics. In P. Irwing, D. Hughes, & T. Booth (Eds.), The Wiley handbook of psychometric testing, 2 volume set: A multidisciplinary reference on survey, scale and test development. Wiley.
  30. Epskamp S, Waldorp LJ, Mõttus R, Borsboom D. The Gaussian Graphical Model in cross-sectional and time-series data. Multivariate Behavioral Research. 2018;53:453–480. doi: 10.1080/00273171.2018.1454823. [DOI] [PubMed] [Google Scholar]
  31. Forbes MK, Wright AGC, Markon KE, Krueger RF. Evidence that psychopathology symptom networks have limited replicability. Journal of Abnormal Psychology. 2017;126:969–988. doi: 10.1037/abn0000276. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Forbes, M. K., Wright, A. G. C., Markon, K. E., & Krueger, R. F. (2019a). The network approach to psychopathology; promise versus reality. World Psychiatry,18, 272–273. [DOI] [PMC free article] [PubMed]
  33. Forbes, M. K., Wright, A. G. C., Markon, K. E., & Krueger, R. F. (2019b). Quantifying the reliability and replicability of psychopathology network characteristics. Multivariate Behavioral Research,56, 224–242. [DOI] [PMC free article] [PubMed]
  34. Fried EI, Cramer AOJ. Moving forward: Challenges and directions for psychopathological network theory and methodology. Perspectives on Psychological Science. 2017;12:999–1020. doi: 10.1177/1745691617705892. [DOI] [PubMed] [Google Scholar]
  35. Fried EI, Eidhof MB, Palic S, Costantini G, Huisman-van Dijk HM, Bockting CLH, Engelhard I, Armour C, Nielsen ABS, Karstoft K-I. Replicability and generalizability of posttraumatic stress disorder (PTSD) networks: A cross-cultural multisite study of PTSD symptoms in four trauma patient samples. Clinical Psychological Science. 2018;6:335–351. doi: 10.1177/2167702617745092. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Funkhouser CJ, Correa KA, Gorka SM, Nelson BD, Phan KL, Shankman SA. The replicability and generalizability of internalizing symptom networks across five samples. Journal of Abnormal Psychology. 2020;129:191–203. doi: 10.1037/abn0000496. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Gardner, H. (1983). Frames of mind: The theory of multiple intelligences. Basic Books.
  38. Golino, H., Christensen, A. P., Moulder, R., Kim, S., & Boker, S. M. (2022). Modeling latent topics in social media using dynamic exploratory graph analysis: The case of the right-wing and left-wing trolls in the 2016 US elections. Psychometrika, this issue. [DOI] [PMC free article] [PubMed]
  39. Golino HF, Epskamp S. Exploratory graph analysis: A new approach for estimating the number of dimensions in psychological research. PloS One. 2017;12:e0174035. doi: 10.1371/journal.pone.0174035. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Golino H, Shi D, Garrido LE, Christensen AP, Nieto MD, Sadana R, Martinez-Molina A. Investigating the performance of exploratory graph analysis and traditional techniques to identify the number of latent factors: a simulation and tutorial. Psychological Methods. 2020;25:292. doi: 10.1037/met0000255. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Hallquist MN, Wright AGC, Molenaar PCM. Problems with centrality measures in psychopathology symptom networks: Why network psychometrics cannot escape psychometric theory. Multivariate Behavioral Research. 2021;56:199–223. doi: 10.1080/00273171.2019.1640103. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Haslbeck JMB, Waldorp LJ. How well do network models predict observations? On the importance of predictability in network models. Behavior Research Methods. 2018;50:853–861. doi: 10.3758/s13428-017-0910-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Haslbeck, J. M. B. & Waldorp, L. J. (2020). mgm: Estimating time-varying mixed graphical models in high-dimensional data. Journal of Statistical Software, 93(8).
  44. Henry, T. R., Robinaugh, D. J., & Fried, E. I. (2022). On the control of psychological networks. Psychometrika, this issue. [DOI] [PMC free article] [PubMed]
  45. Herrera-Bennett, A. C., & Rhemtulla, M. (2021). Network replicability & generalizability: Exploring the effects of sampling variability, scale variability, and node reliability. PsyArXiv https://psyarxiv.com/7vkm8/
  46. Hinne M, Gronau QF, van den Bergh D, Wagenmakers E-J. A conceptual introduction to Bayesian model averaging. Advances in Methods and Practices in Psychological Science. 2020;3:200–215. [Google Scholar]
  47. Hoeting JA, Madigan D, Raftery AE, Volinsky CT. Bayesian model averaging: A tutorial. Statistical Science. 1999;14:382–417. [Google Scholar]
  48. Horn JL, Cattell RB. Refinement and test of the theory of fluid and crystallized general intelligences. Journal of Educational Psychology. 1966;57:253–270. doi: 10.1037/h0023816. [DOI] [PubMed] [Google Scholar]
  49. Huth K, Luigjes K, Marsman M, Goudriaan AE, van Holst RJ. Modeling alcohol use disorder as a set of interconnected symptoms - Assessing differences between clinical and population samples and across external factors. Addictive Behaviors. 2021;125:107128. doi: 10.1016/j.addbeh.2021.107128. [DOI] [PubMed] [Google Scholar]
  50. Huth, K., Waldorp, L. J., Luigjes, J., Goudriaan, A. E., van Holst, R. J., & Marsman, M. (in press). A note on the Structural Change Test in finite samples: Using a permutation approach to estimate the sampling distribution. Psychometrika. [DOI] [PMC free article] [PubMed]
  51. Jones PJ, Mair P, Simon T, Zeileis A. Network trees: A method for recursively partitioning covariance structures. Psychometrika. 2020;85:926–945. doi: 10.1007/s11336-020-09731-4. [DOI] [PubMed] [Google Scholar]
  52. Jones PJ, Williams DR, McNally RJ. Sampling variability is not nonreplication: A Bayesian reanalysis of Forbes, Wright, Markon, and Krueger. Multivariate Behavioral Research. 2021;56:249–255. doi: 10.1080/00273171.2020.1797460. [DOI] [PubMed] [Google Scholar]
  53. Jongerling, J., Epskamp, S., & Williams, D. R. (2021). Bayesian uncertainty estimation for Gaussian Graphical Models and centrality indices. PsyArXiv https://psyarxiv.com/7kude/ [DOI] [PubMed]
  54. Kan KJ, van der Maas HLJ, Levine SZ. Extending psychometric network analysis: Empirical evidence against g in favor of mutualism? Intelligence. 2019;73:52–62. [Google Scholar]
  55. Kjellström S, Golino H. Mining concepts of health responsibility using text mining and exploratory graph analysis. Scandinavian Journal of Occupational Therapy. 2019;26:395–410. doi: 10.1080/11038128.2018.1455896. [DOI] [PubMed] [Google Scholar]
  56. Lee, K. H., Chen, Q., DeSarbo, W. S., & Xue, L. (2022). Estimating finite mixtures of ordinal graphical models. Psychometrika, this issue. [DOI] [PubMed]
  57. Marsman M, Borsboom D, Kruis J, Epskamp S, van Bork R, Waldorp LJ, van der Maas HLJ, Maris GKJ. An introduction to Network Psychometrics: Relating Ising network models to item response theory models. Multivariate Behavioral Research. 2018;53:15–35. doi: 10.1080/00273171.2017.1379379. [DOI] [PubMed] [Google Scholar]
  58. Marsman, M., Huth, K., Waldorp, L. J., & Ntzoufras, I. (2022). Objective Bayesian edge screening and structure selection for networks of binary variables. Psychometrika, this issue. [DOI] [PMC free article] [PubMed]
  59. Mansueto, A. C., Wiers, R. W., van Weert, J. C. M., Schouten, B. C., & Epskamp, S. (in press). Investigating the feasibility of idiographic network models. Psychological Methods. [DOI] [PubMed]
  60. McGrew, K., & Flanagan, D. (1998). Intelligence test desk reference (ITDR): The Gf-Gc cross-battery assessment. Pearson Education. Retrieved from https://psycnet.apa.org/record/1998-07192-000
  61. Mohammadi, R., Massam, H., & Letac, G. (in press). Accelerating bayesian structure learning in sparse gaussian graphical models. Journal of the American Statistical Association.
  62. Mohammadi A, Wit EC. Bayesian structure learning in sparse Gaussian graphical models. Bayesian Analysis. 2015;10:109–138. [Google Scholar]
  63. Mohammadi, R., & Wit, E.C. (2019). BDgraph: An R package for Bayesian structure learning in graphical models. Journal of Statistical Software, 89(3).
  64. Neal, Z. P., & Neal, J. W. (in press). Out of bounds? The boundary specification problem for centrality in psychological networks. Psychological Methods. [DOI] [PubMed]
  65. Newman M. Analysis of weighted networks. Physical Review E. 2004;70:056131. doi: 10.1103/PhysRevE.70.056131. [DOI] [PubMed] [Google Scholar]
  66. Newman, M., Barabási, A.-L., & Watts, D. J. (Eds.). (2006). The Structure and dynamics of networks. Princeton University Press.
  67. Pensar J, Nyman H, Niiranen J, Corander J. Marginal pseudo-likelihood learning of discrete Markov network structures. Bayesian Analysis. 2017;12:1195–1215. [Google Scholar]
  68. Pötscher BM, Leeb H. On the distribution of penalized maximum likelihood estimators: The LASSO, SCAD, and thresholding. Journal of Multivariate Analysis. 2009;100:2065–2082. [Google Scholar]
  69. Robinaugh DJ, Hoekstra RHA, Toner ER, Borsboom D. The network approach to psychopathology: A review of the literature 2008–2018 and an agenda for future research. Psychological Medicine. 2020;50:353–366. doi: 10.1017/S0033291719003404. [DOI] [PMC free article] [PubMed] [Google Scholar]
  70. Robinaugh DJ, LeBlanc NJ, Vuletich HA, McNally RJ. Network analysis of persistent complex bereavement disorder in conjugally bereaved adults. Journal of Abnormal Psychology. 2014;123:510–522. doi: 10.1037/abn0000002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  71. Robinaugh, D. J., Millner, A. J., & McNally, R. J. (2016). Identifying highly influential nodes in the complicated grief network. Journal of Abnormal Psychology,125, 747–757. [DOI] [PMC free article] [PubMed]
  72. Rodebaugh TL, Tonge NA, Piccirillo ML, Fried E, Horenstein A, Morrison AS, Goldin P, Gross JJ, Lim MH, Fernandez KC, Blanco C, Schneier FR, Bogdan R, Thompson RJ, Heimberg RG. Does centrality in a cross-sectional network suggest intervention targets for social anxiety disorder? Journal of Consulting and Clinical Psychology. 2018;86:831–844. doi: 10.1037/ccp0000336. [DOI] [PMC free article] [PubMed] [Google Scholar]
  73. Rodriguez, J. E., Williams, D. R., Rast, P., & Mulder, J. (2020). On formalizing theoretical expectations: Bayesian testing of central structures in psychological networks. PsyArXiv https://psyarxiv.com/zw7pf/
  74. Rubel JA, Fisher AJ, Husen K, Lutz W. Translating person-specific network models into personalized treatments: Development and demonstration of the dynamic assessment treatment algorithm for individual networks (DATA-IN) Psychotherapy and Psychosomatics. 2018;87:249–251. doi: 10.1159/000487769. [DOI] [PubMed] [Google Scholar]
  75. Ruzzano L, Borsboom D, Geurts HM. Repetitive behaviors in autism and obsessive-compulsive disorder: New perspectives from a network analysis. Journal of Autism and Developmental Disorders. 2015;45:192–202. doi: 10.1007/s10803-014-2204-9. [DOI] [PubMed] [Google Scholar]
  76. Ryan, O., Bringmann, L. F., & Schuurman, N. K. (2019). The challenge of generating causal hypotheses using network models. PsyArXiv https://psyarxiv.com/ryg69
  77. Ryan, O., & Hamaker, E. L. (2022). Time to intervene: A continuous-time approach to network analysis and centrality. Psychometrika, this issue. [DOI] [PMC free article] [PubMed]
  78. Savi AO, Marsman M, van der Maas HLJ, Maris GKJ. The wiring of intelligence. Perspectives on Psychological Science. 2019;14:1034–1061. doi: 10.1177/1745691619866447. [DOI] [PMC free article] [PubMed] [Google Scholar]
  79. Spearman C. General intelligence, objectively determined and measured. The American Journal of Psychology. 1904;15:201–292. [Google Scholar]
  80. Spearman, C. (1927). The abilities of man: Their nature and assessment. Macmillan and Company.
  81. Spiller, T. R., Levi, O., Neria, Y., Suarez-Jimenez, B., Bar-Haim, Y., & Lazarov, A. (2020). On the validity of the centrality hypothesis in cross-sectional between-subject networks of psychopathology. BMC Medicine, 18(297). [DOI] [PMC free article] [PubMed]
  82. Sternberg, R. J. (1985). Beyond IQ: A triarchic theory of intelligence. Cambridge University Press.
  83. Thomson GH. A hierarchy without a general factor. British Journal of Psychology. 1916;8:271–281. [Google Scholar]
  84. Thurstone, L. L. (1935). The vectors of the mind. Chicago University Press.
  85. van Bork R, Rhemtulla M, Waldorp LJ, Kruis J, Rezvanifar S, Borsboom D. Latent variable models and networks: Statistical equivalence and testability. Multivariate Behavioral Research. 2019;56(2):175–198. doi: 10.1080/00273171.2019.1672515. [DOI] [PubMed] [Google Scholar]
  86. van Borkulo CD, Borsboom D, Epskamp S, Blanken TF, Boschloo L, Schoevers RA, Waldorp LJ. A new method for constructing networks from binary data. Scientific Reports. 2014;4:5918. doi: 10.1038/srep05918. [DOI] [PMC free article] [PubMed] [Google Scholar]
  87. van Borkulo CD, Boschloo L, Borsboom D, Penninx BWJH, Waldorp LJ, Schoevers RA. Association of symptom network structure with the course of depression. JAMA Psychiatry. 2015;72:1219–1226. doi: 10.1001/jamapsychiatry.2015.2079. [DOI] [PubMed] [Google Scholar]
  88. van Borkulo, C. D., van Bork, R., Boschloo, L., Kossakowski, J. J., Tio, P., Schoevers, R. A., Borsboom, D., & Waldorp, L. J. (in press). Comparing network structures on three aspects: A permutation test. Psychological Methods. [DOI] [PubMed]
  89. van der Maas HLJ, Dolan CV, Grasman RPPP, Wicherts JM, Huizenga HM, Raijmakers MEJ. A dynamical model of general intelligence: The positive manifold of intelligence by mutualism. Psychological Review. 2006;113:842–861. doi: 10.1037/0033-295X.113.4.842. [DOI] [PubMed] [Google Scholar]
  90. van der Maas, H. L. J., Kan, K.-J., Marsman, M., & Stevenson, C. E. (2017). Network models for cognitive development and intelligence. Journal of Intelligence, 5(2). [DOI] [PMC free article] [PubMed]
  91. van der Maas, H. L. J., Savi, A. O., Hofman, A., Kan, K.-J., & Marsman, M. (2019). The network approach to general intelligence. In D. J. McFarland (Ed.), General and specific mental abilities (pp. 108–131). Cambridge Scholars Publishing.
  92. Waldorp, L. J., & Marsman, M. (in press). Relations between networks, regression, partial correlation, and latent variable models. Multivariate Behavioral Research. [DOI] [PubMed]
  93. Wichers, M., Groot, P. C., Psychosystems, ESM group, & EWS group (2016). Critical slowing down as a personalized early warning signal for depression. Psychotherapy and Psychosomatics, 85, 114–116. [DOI] [PubMed]
  94. Williams, D. R. (2021a). GGMnonreg: Non-regularized Gaussian Graphical Models in R. PsyArXiv https://psyarxiv.com/p5jk9/
  95. Williams, D. R. (2021b). Bayesian estimation for Gaussian Graphical Models: Structure learning, predictability, and network comparisons. Multivariate Behavioral Research,56, 336–352. [DOI] [PubMed]
  96. Williams, D. R. (2021c). The confidence interval that wasn’t: Bootstrapped “confidence intervals” in l1-regularized partial correlation networks. PsyArXiv https://psyarxiv.com/kjh2f
  97. Williams, D. R., Briganti, G., Linkowski, P., & Mulder, J. (2021). On accepting the null hypothesis of conditional independence in partial correlation networks: A bayesian analysis. PsyArXiv https://psyarxiv.com/7uhx8/
  98. Williams, D. R., & Mulder, J. (2020a). Bayesian hypothesis testing for Gaussian graphical models: Conditional independence and order constraints. Journal of Mathematical Psychology,99, 102441.
  99. Williams, D. R., & Mulder, J. (2020b). BGGM: Bayesian Gaussian graphical models in R. Journal of Open Source Software,5(21), 2111.
  100. Williams DR, Rast P. Back to the basics: Rethinking partial correlation network methodology. British Journal of Mathematical and Statistical Psychology. 2020;73:187–212. doi: 10.1111/bmsp.12173. [DOI] [PMC free article] [PubMed] [Google Scholar]
  101. Williams DR, Rast P, Pericchi LR, Mulder J. Comparing Gaussian graphical models with the posterior predictive distribution and Bayesian model selection. Psychological Methods. 2020;25:653–672. doi: 10.1037/met0000254. [DOI] [PMC free article] [PubMed] [Google Scholar]
  102. Williams DR, Rhemtulla M, Wysocki AC, Rast P. On nonregularized estimation of psychological networks. Multivariate Behavioral Research. 2019;54:719–750. doi: 10.1080/00273171.2019.1575716. [DOI] [PMC free article] [PubMed] [Google Scholar]
  103. Wysocki AC, Rhemtulla M. On penalty parameter selection for estimating network models. Multivariate Behavioral Research. 2021;56:288–302. doi: 10.1080/00273171.2019.1672516. [DOI] [PubMed] [Google Scholar]

Articles from Psychometrika are provided here courtesy of Springer

RESOURCES