The Pioneering Role of the Vaccine Safety Datalink Project (VSD) to Advance Collaborative Research and Distributed Data Networks

Abstract

Introduction:

Large-scale distributed data networks consisting of diverse stakeholders including providers, patients, and payers are changing health research in terms of methods, speed and efficiency. The Vaccine Safety Datalink (VSD) set the stage for expanded involvement of health plans in collaborative research.

Expanding Surveillance Capacity and Progress Toward a Learning Health System:

From an initial collaboration of four integrated health systems with fewer than 10 million covered lives to 16 diverse health plans with nearly 100 million lives now in the FDA Sentinel, the expanded engagement of health plan researchers has been essential to increase the value and impact of these efforts. The collaborative structure of the VSD established a pathway toward research efforts that successfully engage all stakeholders in a cohesive rather than competitive manner. The scientific expertise and methodology developed through the VSD such as rapid cycle analysis (RCA) to conduct near real-time safety surveillance allowed for the development of the expanded surveillance systems that now exist.

Building on Success and Lessons Learned:

These networks have learned from and built on the knowledge base and infrastructure created by the VSD investigators. This shared technical knowledge and experience expedited the development of systems like the FDA’s Mini-Sentinel and the Patient Centered Outcomes Research Institute (PCORI)’s PCORnet

Conclusion:

This narrative reviews the evolution of the VSD, its contribution to other collaborative research networks, longer-term sustainability of this type of distributed research, and how knowledge gained from the earlier efforts can contribute to a continually learning health system.

Introduction

The potential of “big data” in health care research is being realized with the development of large networks including the Food and Drug Administration’s (FDA) Sentinel and the Patient-Centered Outcomes Research Institute’s (PCORI) National Patient-Centered Clinical Research Network (PCORnet) that connect data for more than 100 million people. Large-scale distributed data networks that bring together diverse stakeholders including providers, patients, and payers are changing health research in terms of methods, speed, efficiency, reach, and impact.

The pioneering accomplishments of the Vaccine Safety Datalink (VSD) helped set the stage to evolve and expand research and surveillance networks. This experience offers lessons for anyone seeking to develop new and larger data networks for improving health care and public health. The VSD’s contributions included the following:

• Bringing together many stakeholders, including health plans, in collaborative research;

• Developing and refining a distributed data model (DDM) for secure analysis of de identified data;

• Advancing breakthrough methods including rapid cycle analysis (RCA) and the use of sequential and self-controlled case series analyses for safety surveillance;

• Setting and maintaining high standards for quality and completeness of data capture; and

• Establishing common standards for data sharing, and organizational and governance structures to support the networks.

This narrative reviews the evolution of the VSD, its contribution to other collaborative research networks, and how the knowledge gained contributes to a learning health system.

A Shared Goal of Safe Vaccines

In 1990 the Centers for Disease Control and Prevention (CDC) established the VSD as part of the effort to monitor the safety of vaccines used in the United States and to strengthen public confidence in vaccines.1 The VSD continues to provide the capacity to identify symptoms or problems relative to the total administered vaccines by using information in data systems of integrated delivery systems providing longitudinal care both before and after vaccination.

Researchers from participating health plans and the CDC work together on the VSD project to monitor the safety of vaccines administered to infants, children, adolescents, and adults. Beginning as a partnership of four integrated health systems2 studying the safety of childhood immunizations, the VSD evolved over 20 years and expanded to nine health insurance plansⁱⁱ with about 10 million members serving as a valuable model for helping to ensure the safety of vaccines for people of all ages. These integrated health systems provide “complete capture” of events through reliable and comprehensive data from electronic health records (EHRs) covering virtually all the patients’ health care encounters along with administrative data from health plans. Collaboration among researchers at participating health plans provides expertise to analyze longitudinal data for potential causal relationships between vaccines and adverse events. The duration of the project required adapting to a changing regulatory and health care environment including new technologies, new privacy laws for human subjects’ research, new vaccines, and the H1N1 influenza pandemic in 2009.

As the VSD has expanded and matured, it has gained recognition and become the proven model for conducting safety surveillance for vaccines and has stimulated new safety surveillance activities in the United States and around the world.3 The FDA Mini-Sentinel was established, based on the VSD model, to conduct safety surveillance of drugs, vaccines, blood products, and medical devices utilizing data from more than one-third of the United States population.4

Evolving from a Centralized to a Distributed Data Model (DDM)

During its first decade, the VSD used a centralized model, with each participating health plan sending de-identified research data files to the CDC once annually so the information could be merged into a centralized database for analyses. These “annual cycle” files included data on vaccinations, hospitalizations, clinic and emergency department (ED) visits, health plan enrollment, and certain demographics, but were static in nature and resulted in data sets that were at least six months old. For a study looking at data from multiple years, researchers would use several years’ cycle files for analyses. For specific studies, more detailed data on diagnoses and laboratory results was accessed from patient charts. When health plans initiated a study, the CDC sent the relevant subset of data to the plan-based researchers.

In 2001, the CDC adopted a DDM for the VSD. This approach, illustrated in Figure 1, allowed each participating health plan to assemble and maintain its data files on its own secure server rather than sending files to the CDC. Each health plan maintained control of its data and study-specific data were securely transferred between the health plans and the CDC via a hub server or via a protocol accessing the individual plan’s server housing their research data set. This distributed model provided the health plans with greater control over their data, assurance of the security of their members’ privacy, and confidence that the data were used only for approved studies. The early development of the VSD has been well documented by Chen et al.5

Figure 1.

VSD DDM Process6

Approaching Near Real-Time Analysis

The DDM allowed the VSD to create “dynamic data files” (DDFs) in addition to cycle files. DDFs allowed for continuous capturing of information with all the essential data required for VSD surveillance studies. More detailed and confirmatory data on diagnoses and laboratory results were accessible from patient charts. Most files were updated weekly, although some updates occurred monthly or quarterly. This resulted in a much tighter time frame, from a vaccine being administered to its data being included in a study. Beginning in 2005 RCA, pioneered by the VSD, enabled the CDC and collaborating investigators to conduct near real-time analysis for ongoing studies. RCA became a standard process for all new vaccines. Another benefit to this approach was a more balanced workload for the health plans over the course of the year, rather than the concentrated labor intensive effort required to create and deliver files once a year. The development of the VSD’s DDM and RCA has been well documented by several authors including Baggs7 and Davis.8,9

The VSD’s DDM became the core of its capacity for near real-time surveillance of vaccine safety. The VSD established a standard for conducting rapid cycle analysis for all newly licensed vaccines for what is usually a two-year period. Each year’s seasonal influenza vaccine is monitored for adverse events, and the VSD conducts studies based on Vaccine Adverse Events Reporting System (VAERS) reports of other suspected vaccine-related adverse events.

Building Capacity

Established as a public-private collaboration, the VSD was a novel approach for the government by funding such an extensive initiative working with health plan data sets for safety surveillance. Previously the government simply purchased data or funded specific research. For more than two decades, the CDC has funded the VSD through a variety of contract mechanisms. Throughout this time, the CDC and the health plans have benefited by sharing data, expertise, infrastructure, and knowledge to improve population health. By creating the VSD, the CDC gained access to timely data and specialized knowledge of local researchers and clinicians who understood the idiosyncrasies and limitations of the data in their systems. Local VSD investigators identified data changes attributable to factors that would not be recognized otherwise such as the implementation of new EHRs that reduced patient encounters, vaccine storage problems, and local events that influenced vaccination patterns. The initial success of the VSD created a sustainable foundation to expand the VSD and to develop additional surveillance efforts using distributed data sets and collaboration between the government and the private sector.

Enhancing Capacity

At the start of the VSD, the participating health plans were implementing electronic health data systems and seeking to improve the depth, quality, and usefulness of their data to provide better care and to better measure performance of quality standards for vaccination. Tracking immunizations, particularly for the pediatric population, was necessary to ensure that children were receiving all the required vaccines on an increasingly complex schedule. At the same time, the needed patient data was in a combination of electronic systems and paper records. The VSD enabled the plans to expedite their efforts to move vaccine data to a standardized electronic format. The same data elements needed for VSD studies were also useful to remind a clinician about a patient’s immunization status, generate documentation for school enrollment, and demonstrate the plan’s progress toward immunization goals. The collaboration on VSD enabled a more uniform approach to building research infrastructure than what had been previously possible.

The VSD provided the CDC with the capacity to monitor the safety of vaccines administered to millions of people in the United States and to conduct longitudinal studies of the potential causal relationship of reported adverse events. This included routine surveillance of newly licensed vaccines using rapid cycle analysis as well as the determining of whether suspected adverse events reported to VAERs, or otherwise hypothesized, were in fact associated with the vaccine. If a safety problem with a vaccine was confirmed, the CDC was able to provide information to change recommendations on its use.

The impact of the VSD on vaccine safety includes the following:

• Identifying the increased risk of intussusception following the rotavirus vaccine;10

• Assessing the risk of febrile seizures following the combined measles, mumps, rubella, and varicella (MMRV) vaccine;11

• Determining the safety of influenza vaccines to support recommendations expanding influenza vaccination in young children12 and pregnant women;13 and

• Contributing data to studies that determined the lack of causal association between mercury content in vaccines and autism.14,15

The introduction of the Menactra meningococcal conjugate vaccine in 2005 resulted in concerns about a potential increased risk of Guillain–Barré Syndrome (GBS) among the mostly adolescent population receiving the vaccine. The rare incidence of GBS in this population necessitated expanding surveillance beyond the VSD. The result was a study run in parallel with the VSD using the same protocol with claims information from several of the nation’s largest health insurers.16 At the same time, leaders at the FDA were recognizing that a similar model could be used for assuring the safety of prescription drugs.17

From 2002 to 2014, several important safety and surveillance efforts were initiated using DDMs. Overlapping participation by a number of investigators and organizations, including the Harvard Pilgrim Health Care Institute (HPHCI)—in its central role—and other members of the HMO Research Network (HMORN), facilitated sharing of knowledge and learning from the experience. The timeline in Figure 2 illustrates the progression of multiple initiatives including key milestones.

Figure 2.

Timeline and Key Milestones of Selected Distributed Data Networks

Expanding Surveillance Capacity and Progress Toward a Learning Health System

The experience and accomplishments of the VSD contributed to the development of other research and surveillance activities such as the FDA Sentinel.18 Trends including advancements in technology, privacy concerns, the need for near real-time data, and urgent public health concerns contributed to greater use of DDMs for safety surveillance. One VSD investigator noted, “the VSD was ‘big data’ before there was anything else that could be called by that name.”19 The VSD demonstrated the success of the distributed network approach to safety surveillance, leading to other networks using a similar approach. The technology, methodology, organizational capacity, and collaborative culture that emerged from the VSD were readily adopted and refined by other research networks, especially given the overlap in people and organizations that were involved. Examples of subsequent research networks that benefitted from and built on the experience of the VSD include the HMORN Virtual Data Warehouse (VDW), Post-Licensure Rapid Immunization Safety Monitoring (PRISM), and the FDA Mini-Sentinel. With the participation of some VSD investigators, efforts such as the Centers for Education and Research on Therapeutics (CERTs), the Observational Medical Outcomes Partnership (OMOP), and the Clinical and Translational Science Awards (CTSA) have all contributed to learning and new infrastructure through shared technical knowledge.

The HMO Research Network and the Expanded Role of Health Systems

Following closely on the footsteps of the VSD collaboration, the HMORN, recently renamed the Health Care Systems Research Network (HCSRN), was established in 1994 and comprises 19 integrated health care systems. In 2002, with support from the National Cancer Institute, the HMORN began developing its VDW, which followed the distributed model of the VSD and built on the capacity and expertise developed with the eight HMORN members that were part of the VSD. For more than a decade, the VDW has served as a resource for a variety of studies supported by government agencies, private foundations, and PCORI. The establishment of the VDW helped lead to the formation of other networks affiliated with the HMORN including the Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) network, which is funded by the Agency for Healthcare Research and Quality (AHRQ), and the National Institutes of Health (NIH) Health Care Systems Research Collaboratory.

The HMORN’s VDW and the institutional knowledge gained from the VSD stimulated expansion of similar collaborations with government researchers. Researchers from the original Kaiser Permanente and Group Health Cooperative VSD sites provided leadership as research efforts were initiated for cancer, cardiovascular disease, diabetes, and mental health. Funding from the NIH created the specialized networks of the Cancer Research Network (CRN), Cardiovascular Research Network (CVRN), Mental Health Research Network (MHRN), and Surveillance Prevention, and Management of Diabetes Mellitus (SUPREME-DM). The AHRQ funded CERTs was established for comparative effectiveness research (CER) and included a governance structure for the network. Many of the same investigators and organizations were involved in these efforts and shared their knowledge and experience. Each of these efforts resulted in enhanced capabilities and methods, along with improved diagnostics, clinical practice, and patient outcomes. Studies across a broad spectrum have included evaluating treatment outcomes based on early detection of cancer and prevention of infection in intensive care units.20,21

The FDA Sentinel Initiative and PRISM

The FDA Amendments Act (FDAAA) of 2007 required the FDA to establish a postmarketing, safety surveillance network for drugs, vaccines, blood products, and medical devices. This mandate, to conduct surveillance for most of the United States population, targeted data for 100 million people by 2012. The FDA’s Sentinel Initiative began its first phase in 2009 with a five-year demonstration project called the “Mini-Sentinel” intended to serve as a small-scale “proof of concept.” For Mini-Sentinel, the FDA recognized the advantages of a surveillance system with a DDM22 with the capacity and flexibility for participation by many diverse data partners including integrated health systems, health plans, and public health. The VSD’s successful development and implementation of methods like sequential analysis increased the enthusiasm of the FDA for using a distributed model for Sentinel. Sequential analysis provided the framework for continuous drug safety surveillance along with propensity scores; both became integral components of Mini-Sentinel. The DDM and RCA propelled Mini-Sentinel. Self-controlled methods and sequential analyses were thoroughly tested and proven by the VSD and adapted for Mini-Sentinel.23 Further, the DDM better protected individual patient information by allowing the data to be housed and maintained by its owners behind their firewalls, as illustrated in Figure 3. Organization-specific knowledge and skilled, experienced programmers were called upon for better understanding differences in data from a variety of sources.24

Figure 3.

Mini-Sentinel DDM25

The Influenza Pandemic and the Post-Licensure Rapid Immunization Monitoring (PRISM) Project

In 2009, as the Mini-Sentinel was being established, an influenza pandemic resulted in the rapid creation of an additional monovalent influenza vaccine for the H1N1 virus. It was very important to monitor the safety of the H1N1 vaccine in the broader population due to its limited clinical testing and the need for the vaccine to be quickly made available to patients. The just completed study of the safety of the Menactra vaccine26 had demonstrated the capacity to adapt a VSD protocol to the use of claims-based data from several large health insurance plans covering more than 10 percent of the United States population. The Department of Health and Human Services (DHHS) requested assistance from America’s Health Insurance Plans (AHIP) and HPHCI to establish a similar initiative in parallel with the VSD using claims data from national health plans to monitor the safety of the H1N1 vaccine. The result was a new public health surveillance activity, the PRISM project.27,28 A unique aspect of PRISM was the linking of claims data with additional data from several state immunization registries, providing access to data on H1N1 vaccines administered through the public health system and other providers outside of those in an insured person’s care network. During 2009 and 2010, PRISM was part of the government’s influenza vaccine safety program, and its data complemented the work done by the long-established VSD. PRISM was classified as a public health surveillance activity rather than as research, with regard to the Health Insurance Portability and Accountability Act (HIPAA) and the Common Rule governing human-subjects research. This fact, along with PRISM’s structure minimizing the exchange of protected health information and proprietary data, accelerated the participation of data partners. The success of PRISM led to the Mini-Sentinel also being considered public health surveillance, to encourage more data partners to participate. PRISM was later merged into Mini-Sentinel as its vaccine safety component and has continued to operate in parallel with and complementary to the VSD.29 With their different populations and variations in data, the VSD and PRISM each play important roles in ensuring vaccine safety and can serve as a check on each other’s validity.

Mini-Sentinel Established on a Proven Model

The Mini-Sentinel evolved beyond proof of concept to study drug and vaccine safety and resulted in dozens of presentations and publications describing its methods and structure. One recent study of the safety of the rotavirus vaccine was conducted in parallel with a similar study by the VSD. With similar results from different populations, the two studies demonstrated the reliability of the methods and systems used by VSD and Mini-Sentinel.30,31 In addition, Mini-Sentinel conducted safety studies that resulted in FDA label changes.

In 2014, the FDA solicited proposals for the Sentinel project, which was designed to build on the success of Mini-Sentinel by expanding the population, types of data, and surveillance of more medical products—including implantable devices. The new contract was awarded to the same team at HPHCI who created the Mini-Sentinel infrastructure and expanded the number of data partners with another regional health insurer, a national hospital chain, and a diverse group of collaborating health systems that also participate in PCORnet’s clinical data research networks (CDRNs). The goal for the next five years is to continue building on the success of the Mini-Sentinel, adding data from other sources such as EHRs to study medical devices with the implementation of unique device identifiers (UDI).

PCORnet as a “Network of Networks”

In 2010, the Patient Protection and Affordable Care Act (ACA) created PCORI. The mission of PCORI is to support clinical CER to develop evidence-based information guided by patient perspectives. PCORI’s scope includes creating a large research network to increase the amount of timely and relevant information for patients and their caregivers to better understand care options and outcomes.

PCORnet includes health systems and patient-led groups, to share information and conduct clinical outcomes research. PCORnet began its first phase at the end of 2013 and has a Coordinating Center led by HPHCI and the Duke University Translational Medicine Institute (DTMI) teams who have extensive experience with other networks including VSD, Sentinel, PRISM, and the NIH Health Care Systems Research Collaboratory.

PCORnet was initially focused on developing a functional network to conduct CER studies using a DDM. The PCORnet DDM as illustrated in Figure 4 is very similar to the Mini-Sentinel DDM, incorporating more diverse networks and integrating data from other sources such as registries and outside networks. The Coordinating Center works with 14 CDRNs, 20 Patient Powered Research Networks (PPRNs), and PCORI to establish a common data model, governing principles, and processes to share data, while protecting security and privacy. The effort is informed by the experience and infrastructure of the Mini-Sentinel and the NIH Collaboratory.

Figure 4.

PCORnet structure and DDM32

Mini-Sentinel and PCORnet have generated new opportunities to leverage the success of distributed data system methods to respond to public health concerns. By sharing standards for data and governing principles, the networks are efficient, and investigators capitalized on lessons learned when the new systems were created. The concept illustrated in Figure 5 is for a “network of networks” enabling participation by some or all members of various networks.

Figure 5.

Network of networks33,34

Building on Success and Lessons Learned

The VSD’s early successful implementation of a DDM to evaluate the safety of vaccines used by millions of Americans demonstrated the value of this approach for expanding research capabilities, protect data privacy and security, and advance public health surveillance. The DDM enables timely analyses of near real-time data with the ability to add data sources such as EHRs and registries to better understand possible causal relationships. A common data model (CDM) and standardized systems to access, share, and analyze data make it easier for others to participate and contribute to a distributed network. The CDM has been a necessary tool for bringing together data from multiple sources that were created for various purposes other than research. Over time, the evolution and widespread adoption of new common data standards may reduce or replace the need for the steps involved with the network-specific CDM.

New protocols that reduce barriers to participation are as important as technology and methodology in facilitating expanded networks. The PRISM H1N1 study demonstrated value for the public health surveillance designation when it reduced administrative barriers and accelerated participation by the collaborating data partners. This public health model for PRISM and Mini-Sentinel made a big difference in engaging key organizations to contribute to improving public health and safety.35 This experience led to a cultural shift as organizations became comfortable sharing data and collaborating in what they considered an important public service.

As noted by one investigator with PRISM and Mini-Sentinel, “Experience facilitated expanded participation for the organizations involved with the Menactra and Guillain Barre study who found it was easier and quicker to participate in Mini-Sentinel, because of their earlier experience.”36 This resulted in a number of the same plans participating in multiple networks.

Other efforts that developed or refined research methods have benefited through sharing experiences and results via publications and the direct involvement of VSD investigators. Among these was the FDA-sponsored Medication Exposure in Pregnancy Risk Evaluation Program (MEPREP), which funded research to study the effects of prescription medications during pregnancy. MEPREP brought together the FDA, the HMORN CERTs, and Vanderbilt University—resulting in a model to define and identify pregnancy through data elements. The VSD later developed its own pregnancy algorithm, which overcame the difficulties in identifying pregnancy episodes using strictly EHRs and claims data, allowing the VSD to conduct studies of immunization safety during pregnancy and any impact on both mother and child.37

Sustainability

The importance of research networks in producing valuable knowledge for public health and the safety of medical products leads to the question of how to ensure their continuity and benefit going forward. Surveillance helps ensure the safety and effectiveness of health care products and services and requires a continuous process, not simply a response to a single event or public health concern. Continuity of systems may yield additional benefits, as noted by an early leader of the VSD: “The longitudinal experience of the VSD contributed to the ability of investigators to understand and respond to situations in the high pressure environment of H1N1/PRISM when the spotlight was on. They could knowledgeably explain things to lay people and policy makers based on the understanding of the methods, data, and idiosyncrasies from years of working with the system.”38

The combined experience of the networks has demonstrated many characteristics that offer potential for long-term success. As explored in a recent article by Steiner and colleagues from the HMO Research Network,39 essential organizational elements for sustaining a research network include the following:

• Governance structure;

• Standards and systems for reliable data;

• Shared knowledge about approaches and methods;

• Funding to support infrastructure;

• Culture of collaboration; and

• Expansion of the stakeholder base to include patients, clinicians, and management operations, since these all offer the potential for long-term support as more find value in the network.

Demonstrating that the network is a valuable, productive resource for stakeholders is critical to ensuring adequate stable funding. The capacity to efficiently answer questions of interest to the stakeholders is a key value proposition for sustaining a research network. Results with solid scientific value with clinical, policy, and regulatory applications are needed on an ongoing basis to sustain support. A number of investigators have noted that systems must be continually used to provide data and answer questions the stakeholder organizations need to improve their operations and report on performance standards.40 The systems must be valuable enough to the organizations that they will support them financially.41

Balancing competition and collaboration may be a challenge to the continued success of a network. Partners may lose interest if they don’t feel they are playing a meaningful role or getting value from participation. Partners want to provide more than just data and want a meaningful level of engagement in protocols, analyses, and publications. The concept of “collabetition” was described by an investigator as partners collaborating while also competing for resources and potential scientific recognition. There can be acknowledgement that one partner’s cooperation with another partner’s research offers the opportunity for gaining mutual support in their efforts.42 A benefit of collaboration is that research developed by a group of experts and stakeholders can result in shared interest in the work. As noted by several investigators with experience in government, academia, and other private sector research, the engagement of researchers from multiple sectors is desirable to provide perspectives, application of the latest methods, and sufficient depth and diversity of experience.43

Initiating and sustaining efforts such as distributed data networks requires funding to cover the costs of labor and materials for developing and maintaining basic capacity to participate with up-to-date data sets. If resources are shared by multiple networks, the burden of participation may be reduced. Dedicated external funding, a value-based case for internal support, or some combination of both is necessary for the data infrastructure to be created and maintained. The government and other external funders play an important role in supporting many of these efforts.

Conclusion

An initial collaboration between the CDC and four integrated systems44 of care evolved and advanced public health surveillance research collaboration with far-reaching implications for improving the quality and safety of health care. The experiences of the VSD and the expanded surveillance activities led to the use of new statistical methodologies and structures. This led to rethinking the research model of the future to answer critical questions in near real time and subsequently to improve public health.45

The result of nearly a quarter century of work has been the development of large-scale distributed data systems for health research. What began as a system to monitor the safety of vaccines administered to a few million children ultimately became research networks with data from multiple sources with as many as 178 million individuals46 to monitor safety, effectiveness, and an almost infinite variety of public health questions.

The VSD’s legacy for Sentinel and PCORnet and other networks, consists of the standards and methods including the use of sequential analyses for safety surveillance. References to these key elements are in descriptions of PRISM, Mini-Sentinel, and other networks.47,48,49 The comprehensive data capturing a majority of health care encounters for enrolled individuals set a high standard for data completeness. Bringing together multiple data sources into common standards is challenging, but VSD, PRISM, Mini-Sentinel, and PCORnet demonstrate the value of this for public health.

Realizing value for stakeholders is important to a network’s success. This may include demonstrating the usefulness of the data in practical applications. For example, the VSD and Mini-Sentinel both point to multiple publications per year based on studies conducted within their networks and on study results that led to changes in practice or instructions that have been implemented. When the VSD began, part of the motivation for the health plans to participate was to develop their own capacity to better track immunizations as a quality measure.50 The ability to use data sets developed for Sentinel, PCORnet, and others to assess quality or comparative effectiveness will be of interest to payers and providers. Additionally, each network may need to measure and assess its performance in meeting its strategic goals such as improving safety or quality.

As Holve notes,51 the ability to incorporate new information, new data sources, new technology, new stakeholders, and perspectives will allow a network to be flexible to adapt and stay relevant in producing knowledge. Research and surveillance methods will continue to evolve, and there will be breakthroughs just as the VSD and rapid cycle analysis set a foundation for subsequent research networks. The NIH Collaboratory, FDA Sentinel, and PCORnet have been cited as major investments in network capacity to analyze data on more than 100 million people, and to conduct surveillance and clinical trials more quickly and efficiently than ever before. To the extent that these networks are able to rapidly provide answers to important health care questions that can have an impact on decisions about care by patients, providers, and payers, they offer the potential to contribute to a learning health care system.52

Much like the initial collaboration in the VSD enabled the CDC and health plans to benefit from each other’s contributions, sustaining engagement and support of any research collaboration requires continued realization of benefits by all participants and stakeholders. The progression toward a learning health system will require, rely on, and help sustain collaborative research efforts and will ensure that what was once unique becomes standard—while continuing to evolve and expand knowledge.