The growth and impact of Alzheimer's Disease Centers as measured by social network analysis

Abstract

Importance

Alzheimer's disease (AD) is a devastating neurodegenerative disorder with no effective therapies. In 1984, the National Institute on Aging created the first five AD Centers (ADCs) in an effort to coordinate research efforts into the pathology and treatment of AD. Since that time, the ADC program has expanded to include 27 centers in major medical schools throughout the United States. A major aim of ADCs is to develop shared resources – such as tissue samples and patient populations – and thereby promote large-scale, high-impact studies that go beyond the capabilities of any single investigator or institution working in isolation.

Objective and Design

To quantitatively evaluate the performance of this program over the past quarter century, we systematically harvested every paper published by ADC investigators and used social network analysis to analyze co-publication networks.

Results

We found that the frequency of collaborations has increased dramatically during this time, even after the expansion of ADCs and the recruitment of new investigators plateaued. Moreover, the collaborations established within the context of the ADC program are increasingly inter-institutional, consistent with the overall goal of the program to catalyze multi-center research teams. Most importantly, we determined that collaborative, multi-center ADC research articles are consistently of higher-impact than AD papers as a whole.

Conclusions

We conclude that the ADC program has successfully fostered high-impact, multi-university collaborations, and suggest that its structural and administrative features could be replicated in other fields of patient oriented research.

Introduction

Alzheimer's Disease (AD) is a neurodegenerative disorder that is the leading cause of dementia in people age 65 years and older. At present, there is no cure and there are no survivors; one hundred percent of AD cases end in death. Over 5 million Americans are afflicted with AD, and it recently passed diabetes to become the sixth leading cause of death in the United States¹. Worldwide, over 35 million people live with dementia, ultimately resulting in costs exceeding 600 billion dollars². To put this number in perspective, roughly one percent of the world's gross domestic product (GDP) is used to care for patients with AD and related forms of dementia².

As troubling as these numbers are, future trends are even more alarming. The leading edge of baby boomers has begun to reach retirement age, reflecting a dramatic aging of the US population as a whole. By 2050, nearly 90 million Americans are expected to be 65 years or older, and since the risk of developing AD doubles every five years beyond age 65¹, these demographic trends will result in a dramatic increase in the prevalence of AD. Indeed, the cost of AD now rivals that of cancer and heart disease, but the growing epidemic of AD means that these costs will race ahead of these other diseases in the coming years³. Thus, by 2050, 16 million Americans will have AD, resulting in 1 trillion dollars per year in expenses to Medicare and Medicaid alone¹. Needless to say, this outcome will be catastrophic to patients, their families, and to public health spending and policy. Moreover, US trends are mirrored in both the industrialized and developing world, meaning that the current epidemic of AD is global and will extend far into the future.

Consequently, there is enormous interest in research aiming to improve the diagnosis, treatment, and management of AD. In the United States, the majority of funding for Alzheimer's research comes from the federal government through the National Institute on Aging (NIA) of the National Institutes of Health (NIH)⁴. One major facet of the NIA's efforts has been the creation of AD Centers (ADCs) in major medical schools throughout the country⁵. The first five ADCs were created in 1984, and the program has grown to include 27 different active centers across the United States. The initial ADCs were P50 AD Research Centers (ADRCs), and they include the following mandated Cores: Administrative, Clinical, and Neuropathology. Subsequently, Education and Information Transfer Cores were added to all ADCs. In addition, these ADRCs also included support for multi-year RO1-like research projects as well as a minimum of two non-renewable Pilot grants each year to stimulate new AD research initiatives. In 1990, the NIA introduced a second type of ADC with a different concept and these ADCs are designated AD Core Centers or ADCCs. Like ADRCs, these centers provide core facilities and support for three RO1-like research projects. In addition, they provide essential research infrastructure for independently funded research projects as NIH RO1s, P50s and PO1s or by foundation grants. Currently, ADCCs and ADRCs are referred to collectively as ADCs. Moreover, renewal of each ADC for another 5 year funding cycle requires submission of a competing renewal application that undergoes peer review, and each competitive renewal application cycle for ADC slots is announced by the NIA. Since all qualified institutions can respond to these periodic announcements with an application for an ADC slot, regardless of whether or not they have an existing ADC, there is turnover among the ADCs with the best competing successfully for renewal each cycle^4,5. Additionally, the AD Cooperative Study (ADCS) program was launched in 1991 to design, test, and implement AD clinical trial evaluation methods and to conduct clinical studies of new treatments for AD including the effects of new therapies on the diverse cognitive and behavioral impairments associated with AD. The ADCS program continues to be supported by the NIA and most ADCs participate^4,5.

A principal aim of these centers is to establish the expertise and infrastructure to support independent research initiatives and to catalyze collaborative efforts that transcend university boundaries. Much of this work has been devoted to generating common resources, including collections of tissue samples, patient populations, and electronic databases^4,5. A major accomplishment was the establishment of the National Alzheimer's Coordinating Center (NACC) Database, a standardized repository for data collected during clinical studies of AD⁶. Subsequently, the need for uniform protocols and data collection motivated the creation of a uniform data set (UDS) to standardize clinical observations of AD patients^7,8.

Moreover, collective efforts through ADCs have been made to identify genes and other risk factors associated with AD⁴. The AD Genomics Consortium or ADGC (http://alois.med.upenn.edu/adgc/) was formalized in 2009 to conduct genome-wide association studies (GWAS) to identify genes associated with an increased risk of developing late-onset AD (LOAD). Other goals were to develop, standardize, and validate imaging tools and chemical biomarkers to be used in clinical trials of AD treatments, as well as to elucidate mechanism of AD onset and progression through the establishment of the AD Neuroimaging Initiative (ADNI) in 2004^9,10.

From an administrative standpoint, the goal of the ADC program is to foster collaboration between different universities, and ultimately facilitate large-scale, high-impact research that is beyond the reach of any single investigator or institution. Indeed, the creation of the ADCS, NACC, ADGC and ADNI would not have been possible without the existing ADC network which created the sociological context and culture of a highly collaborative group of investigators open to data sharing that have been central to the success of these ADC-linked programs. Anecdotal evidence suggests that ADCs have contributed to the successful completion of such projects, and many of the most influential Alzheimer's researchers are members of ADCs¹¹. Nevertheless, the extent to which ADCs directly foster high-impact, multi-university collaborations has never been systematically evaluated.

The most common methods for quantitatively measuring scientific productivity are largely based on counting publications and citations. For example, a journal's impact factor or an investigator's H-index¹² are metrics built on the assumption that a given paper's influence (as measured by citation count) correlates with its importance and quality. Although these metrics are admittedly imperfect, they do possess several valuable attributes, including being quantitative, impractical to manipulate, and focused on quality rather than quantity of output¹³. These and similar efforts to objectively measure scientific productivity have been aided immeasurably by the growth and development of analytical tools to harvest and analyze publication records from online databases. Termed ‘metaknowledge’, this growing discipline aims to study the scientific enterprise, with a particular emphasis on the structural, sociological, and technological innovations that underlie scientific discovery¹⁴.

Such methodologies have shown that over the past 50 years, the basic unit of scientific activity has shifted from single investigators towards larger and more complex teams^15,16. This trend is particularly relevant to the study of AD, which encompasses many different basic science disciplines, including neuroscience, biochemistry, molecular genetics, and physiology as well as a wide range of clinical disciplines extending from neurology, geriatrics and psychiatry to radiology, neuropathology, biostatistics, epidemiology and clinical trials. Most importantly, discoveries in the basic science of AD would be entirely impotent without close collaborations with investigators in translational, clinical, and public health disciplines. Therefore, it is crucially important that we understand what best fosters effective multi-disciplinary team science^17,18, particularly given the continued barriers to effective collaboration between different universities¹⁹.

Social network analysis (SNA) is one approach to understand scientific collaborations^20,21. Co-publication of a research article can be visually represented as a link between two or more investigators. Expanding this abstraction to an entire scientific field thereby facilitates its analysis using the power of network statistics. This is an effective way to measure the scope and connectivity of a field²⁰, identify investigators with a knack for fostering collaboration^21,22, or develop network-level interventions to foster new collaborations²³. We recently used this method to assess how effectively the University of Pennsylvania's Clinical and Translational Science Award (CTSA) has promoted collaborations over time²⁴. We reasoned that a similar approach, with a particular emphasis on how collaborative networks change over time between institutions, could dramatically increase our understanding of how ADCs have performed since their inception. Therefore, we built co-publication networks of ADCs on a year-by-year basis, and measured their growth, productivity, and impact. Based on these analyses, we present evidence supporting the conclusion that ADCs have been effective catalysts of productive collaborations in research on AD and related dementias between universities.

Given the rate at which our population is aging, AD represents a looming public health crisis. The development of novel therapeutics for AD requires the combined expertise and resources of many different research projects. Therefore, we believe that a close analysis of how well the ADC program has fostered collaborations will be a valuable contribution to the field, especially in an increasingly tight funding environment.

Results and Discussion

Basic information is needed to assess the evolution of collaborations in any scientific organization: (1) a year-by-year list of active members, and (2) their resulting publications. To identify active ADC investigators, we manually compiled a year-by-year roster of every ADC using the ADC telephone directories spanning the late 1980s through 2012. We then automatically harvested the publications of active ADC investigators from publically available databases²⁴. These data were manually curated, and the vast majority of harvested papers were found to be genuine products of one or more ADCs. Based on this, we were confident that these data represent an accurate and comprehensive assessment of the scientific output of ADCs since their inception.

A total of 12,170 unique ADC papers were published from 1985 through 2012. We used this resource to generate interaction networks (Figure 1). Every active ADC investigator was represented as a node, and co-authorship of a publication with another ADC member was considered an edge, linking two nodes together. By breaking down these networks on a year-by-year basis, trends in the evolution of ADCs become apparent. In 1985, the first year of the ADC program, there were several dozen active investigators linked together by pre-existing collaborations (Figure 1A). Over the next 10-15 years, the number of active nodes grew rapidly, as additional centers opened and new investigators were recruited into the program (Figures 1B-D). By the end of the last decade, the growth in the number of active investigators had begun to plateau. Nevertheless, the total number of interactions between these nodes continued to increase, resulting in considerably larger and more connected networks (Figures 1E-F). In 2012, for example, there were 857 ADC publications originating from 662 active investigators linked together by 9018 unique collaborative interactions. In contrast, in 1985, there were 88 ADC publications from 113 active investigators and 139 unique edges.

Figure 1. Co-publication networks of ADC investigators over the last 25 years.

Publications from active ADC investigators were harvested from public databases, and networks of co-authorship were generated for every year from 1985 to 2012. Every red circle (i.e., every node) represents a single active ADC investigator. Every blue line linking two nodes (i.e., every edge) represents shared co-authorship of a paper between two investigators. Blue edges with increased width represent co-authorship of more than one publication. Representative networks are shown for 1985 (A), 1990 (B), 1995 (C), 2000 (D), 2005 (E), and 2010 (F). These networks illustrate the growth of ADCs over the last 25 years, both in terms of the number of active investigators, as well as the collaborative interactions between them.

These data are analyzed in more detail in Figure 2. As would be expected from a growing research initiative, the total number of collaborative ADC publications per year has increased nearly linearly from 1985 to the present (Figure 2A). In part, this is due to the recruitment of new centers and investigators. Nevertheless, the total number of active ADC investigators per year has leveled off during the last ten years, suggesting that the average productivity of each investigator has increased during this time (Figure 2B). This possibility is supported by the growth in the total number of “edges per year” between ADC investigators during the last 25 years (Figure 2C). Said differently, these data indicate that the total number of collaborative interactions has grown consistently, regardless of changes in the total number of investigators. These collaborative interactions include co-authorship with members of the same institution (intra-ADC edges) as well as co-authorship with members in different institutions (inter-ADC edges). There has been a modest increase in the total number of intra-ADC edges during the last 25 years. More interesting, however, is the striking increase in the total number of inter-ADC edges during the same time (red bars in Figure 2C). The proportion of inter-ADC edges reached a minimum during the early 90s, roughly corresponding to the peak recruitment of new centers (Figure 2D). Since that time, the percent of inter-ADC edges has grown dramatically, reaching over 80% of collaborative publications in 2011 and 2012 (Figure 2D). Taken together, these data suggest that membership in the ADC program encourages collaborative interactions, particularly between investigators in different institutions.

Figure 2. Network statistics for collaborative ADC publications.

Descriptive statistics were calculated from the co-publication networks shown in Figure 1. Panel (A) shows the total number of unique ADC publications on a year by year basis. These data were fit to a linear regression line with and R² value of 0.9567. Panel (B) shows the total number of linked ADC investigators within each network; i.e., active ADC investigators who published a paper with another ADC investigator per year. These data were fit to a second-order polynomial with an R² value of 0.9845. Panel (C) shows the number of links between ADC investigators per year. Columns in blue represent intra-ADC collaborations; i.e., co-publication links between two investigators within the same ADC. Columns in red represent inter-ADC collaborations; i.e., co-publication between two investigators in different ADCs. These data were fit to a linear regression line with an R² value of 0.5313. Panel (D) shows the percent of edges that come from inter-ADC collaborations. These data were fit to a second-order polynomial with an R² value of 0.8517. Panel (E) shows the main cluster size of co-publication networks; i.e., the percent of nodes connected by one or more edges to the largest sub-network. These data were fit to a second-order polynomial with an R² value of 0.8919. Panel (F) shows the number of edges per node for co-publication networks. The growth of inter-connectivity between ADC investigators is shown by the positive trend in node degree over time (NDT = 0.274). The insert bar graph shows that the majority of this increase in node-degree has originated from the increasing frequency of inter-ADC collaborations.

The percent of nodes connected to the largest cluster, i.e., the main cluster size, of these interaction networks has grown as well (Figure 2E). Judged by this metric, the degree of connectivity within ADC co-publication networks has dramatically increased, to the point that over 90% of ADC investigators are connected either directly or indirectly to other ADC investigators in any given year. Similarly, we calculated the node-degree over time (NDT), a measure of the interconnectivity of a network that is largely independent of the number of nodes²⁴. We found that the ADC network has an NDT of 0.274, which indicates a clear, positive trend in the degree of inter-connectivity (Figure 2F). Most interestingly, NDT can be separated into the individual contributions of inter-ADC and intra-ADC edges. As would be expected based on the growth of inter-ADC edges described above, the major contribution to NDT has come from collaborations established between different institutions (Figure 2F, insert). We cannot formally exclude the possibility that these changes reflect an underlying change in the nature of the AD field without a carefully matched control population of non-ADC investigators. Nevertheless, we can conclude that the growth in collaborative studies during this time is consistent with the goals of the ADC program, especially with respect to the rapid increase in inter-institutional connections.

To extend on these results, we generated interaction networks at the level of institutions, rather than single investigators (Figure 3). When integrated over the entire tenure of the ADC program, there was a remarkable degree of interconnectivity between different institutions (Figure 3A). The vast majority of active ADCs had at least one collaborative connection to at least 26 other ADCs (Figure 3B). In fact, 323 of the 351 (92%) potential connections between active ADCs were represented by at least one collaborative publication. The total number of collaborative publications shared between different ADCs was more variable than the number of unique edges (Figure 3C). Nevertheless, we found that each active ADC has on average 122 collaborative publications shared with its sister ADC institutions. Therefore, we conclude that productive links between different ADCs are the norm rather than the exception.

Figure 3. Interaction networks between different ADCs.

Panel (A) shows the geographical distribution of active ADCs and the collaborative links between them. Every red circle represents a single ADC, and every blue line represents collaborative publications shared between two ADCs. The width of each blue line is proportional to the number of co-publications shared between two centers. This network represents every collaborative link established during the lifetime of the ADCs (1985-2012), rather than the year-by-year networks shown in Figure 1. Panel (B) shows the number of unique connections to other ADCs for each center (average = 29.6). Blue bars represent active centers; red bars represent previously active centers. Panel (C) shows the number of total connections between ADCs; i.e., the total number of collaborative papers spanning different ADCs (average = 122.5). Blue bars represent active centers; red bars represent previously active centers. These data indicate that the majority of active ADCs have established direct collaborative interactions with most other ADCs.

This analysis can also be used to assess the evolution of research interests and methodologies over time. To this end, we harvested Medical Subject Headings (MeSH terms) for every ADC paper and measured the rate at which their usage changed between 1990 and 2012 (Figure 4). Although the vast majority of MeSH terms are used too infrequently to identify meaningful trends, there were 700 terms represented in the ADC literature at least once per year on average. The majority of these MeSH terms showed remarkable consistency over time. Nevertheless, the use of several dozen MeSH terms dramatically changed over the last two decades (Figure 4A). For example, the frequency of papers dealing with human genetics underlying the susceptibility to AD has dramatically increased. This is seen in the increased usage of MeSH terms such as ‘AD genetics’, ‘single nucleotide polymorphisms’, and ‘genetic predisposition to disease’ (Figure 4B). At the same time, there has been a modest decrease in the usage of terms dealing with drug pharmacology and neuronal physiology.

Figure 4. Evolution of the interests and methodologies of ADCs.

MeSH terms were harvested from public databases for every ADC paper. MeSH terms that were used at least once per year on average were selected for further analysis (N = 700). Panel (A) shows the rate of change in MeSH term usage as measured by the slope of a linear regression line during the lifetime of ADCs for all 700 terms. The vast majority of these terms showed remarkably constant usage in the ADC literature, with slopes roughly equal to zero. Red broken lines represent two standard deviations plus and minus the average value of these data. At both extremes, there were several dozen MeSH terms whose usage frequency substantially changed. Examples of these are shown in panels (B-E). Panel (B) shows the increased frequency of publications dealing with the human genetics of AD, including the terms ‘AD genetics’ (blue), ‘single nucleotide polymorphisms’ (red), and ‘genetic predisposition to disease’ (green). Panel (C) shows a shift in model systems during this time, from predominantly rats (blue) to transgenic mice (red). Panel (D) shows an increased awareness of the influence of a patient's sex on AD. Similarly, panel (E) shows a growing focus on how a patient's age affects the diagnosis and progression of AD.

As one would expect, the most common model organism has changed during this time, given the shift in emphasis away from physiology and pharmacology towards molecular studies of disease genetics. While rats were the most common model twenty years ago, the more recent development of transgenic mouse models has allowed sophisticated investigations into molecular genetics (Figure 4C). Moreover, the emphasis on disease genetics has coincided with an increased awareness of patient to patient variability in the susceptibility to and progression of AD. Most notable has been a dramatic increase in the frequency of sex-specific studies (Figure 4D), although patients’ age has taken on a growing significance as well (Figure 4E). Based on these trends, we expect that additional factors governing the interaction between genetics and environment – such as race and socioeconomic status – will take on special importance in the years ahead. The distribution of ADCs in major university hospitals throughout the United States should prove to be uniquely advantageous in recruiting patients for studies with a more refined appreciation for patient to patient variability.

Moreover, the extensive interconnectivity between ADCs suggests that there should be fewer barriers to the successful completion of unusually large or ambitious research initiatives. A critical test of this hypothesis is whether collaborative publications from multiple ADCs have a disproportionately large impact on the field. We therefore compared the number of citations received by collaborative ADC articles versus articles from the AD field as a whole (Figure 5). We found that collaborative, multi-ADC publications had been cited more frequently than would be expected by chance. Moreover, the number of extremely highly cited articles from ADCs has increased over time (Figure 5A). We found that 10.5% of collaborative, multi-ADC publications were among the top 1% most-cited papers within the AD field (Figure 5B). In fact, more than 85% of collaborative ADC articles were cited at least as much as the median article for any given year. Some of this can be explained by the population of investigators which comprise the ADCs, who tend to be among the most productive within the field. For example, a random sampling of all ADC papers shows that 4.6% of papers were among the top 1%, and 67.3% were in the top half of published papers in any given year (Figure 5C). Nevertheless, multi-ADC papers tended to be higher-impact than the population of all ADC papers as a whole, with over two times the likelihood of being among the most highly cited papers in the field. We believe these results support the hypothesis that collaborative, multi-ADC studies have a disproportionately large impact on the AD field.

Figure 5. Scientific impact of multi-center ADC publications.

The total number of citations received for inter-ADC collaborative publications were harvested from public databases. These citation counts were compared to number of citations received for every publication of the topic of AD since 1985. For the purpose of this analysis, only primary research articles were considered. Panel (A) shows the number of citations received per paper as a function of time. The four blue lines represent the 99^th (dark blue), 90^th (blue), 75^th (light blue) and 50^th (lightest blue) percentiles of citations received for the Alzheimer's field as a whole. Each red ‘X’ represents the citations received by a single, multi-center ADC publication. Panel (B) shows the percent of collaborative, multi-ADC publications reaching the 99^th, 90^th, 75^th, or 50^th percentile of citations received in any given year. Panel (C) shows the percent of all randomly sampled ADC publications reaching the 99^th, 90^th, 75^th, or 50^th percentile of citations received in any given year. Collaborative ADC publications are typically higher-impact than the AD field as a whole.

This is the first study to use SNA to assess quantitatively the performance of the NIA ADC program over time by systematically harvesting every paper published by ADC investigators to delineate co-publication networks among the ADCs. The most significant conclusions we draw from this SNA of the ADC program are that: (1) The frequency of collaborations among ADCs and ADC investigators increased dramatically since 1985; (2) Collaborations among the active ADCs increased dramatically between multiple different universities at which the ADCs are based; and (3) Collaborative, multi-center ADC research articles are a signature of inter-ADC publications and they have a consistently higher-impact than papers on AD as a whole. Although direct causality is difficult to demonstrate, these data are consistent with the conclusion that the ADC program has been successful in fostering high-impact, multi-university collaborations, which is a major part of the mission of the NIA funded ADC network. However, this SNA is not the whole story of the success of the ADCs. ADNI, for example, is a spin-off of the social evolution of the collaborative culture of the ADC network, and is one of the most successful ADC-linked programs in the NIA AD research portfolio with a profound impact on driving AD biomarker research forward and especially to improve the conduct and efficacy of AD clinical trials²⁵.

Since the establishment of the NIA ADC network there have been tremendous basic science advances in understanding the neurobiology of the normal aging brain as well as mechanisms of AD and related neurodegenerative diseases. Highlights of these advances are provided each year by the AD Education and Referral (ADEAR) Center (see http://www.niapublications.org/adear/), and many ADC scientists funded by this program have been instrumental in moving the AD research agenda forward¹⁰. Our SNA of the ADC network provides quantitative data illustrating the success and impact of this program, and these findings suggest that the structural and administrative features of the ADC program could be successfully replicated in other fields of patient oriented research funded by the National Institutes of Health or through public-private partnerships as is now taking place for AD research. Even more importantly, we believe that the culture of the ADC program – one that encourages and rewards open sharing of resources, data, and ideas – has been enormously successful in fostering the research accomplishments described above.

Methods

Data collection and validation

Year-by-year rosters for all active (ADCs were manually re-constructed from archived personnel directories from 1989 through 2012. Custom-built Ruby scripts were written to systematically harvest ADC publications and their associated MeSH terms from NCBI's PubMed. Papers were required to be written in English, include an active ADC member as senior author, and have at least two ADC members from the same institution. A keyword filter was used post-hoc to select publications dealing with AD, neurodegenerative disease, aging, or nervous system function. To assess the false-positive rate, 500 randomly selected papers meeting these criteria were manually examined. Based on these data, the false-positive rate (i.e., the number of papers not directly supported by ADC mechanisms) is estimated to be less than 3%. False-negative rates are more difficult to estimate. Nevertheless, during the course of this analysis, we manually examined ADC publications from over a dozen Institutions spanning several hundred individual papers. False-negatives fell into two categories: (1) papers by active ADC members that are unrelated to AD or brain function, and (2) papers with atypical author lists, such as publications by consortiums. We considered the exclusion of the former case to be justifiable; exclusion of the latter case affected less than 1% of ADC publications. Citations were assessed as of January, 2013. All scripts used in this study are freely available on demand.

Network generation, statistics, and visualization

Co-publication was defined as two or more active ADC members authoring the same publication. Individual pairs of co-publishing authors were identified using custom Ruby scripts and weighted according to the number of publications in common. Networks were generated and visualized using Cytoscape version 3.0.0 beta1 (www.cytoscape.org). Network statistics were calculated using Cytoscape and MS Excel. NDT was calculated as previously described ²⁴.