Building a Governance Strategy for CER: The Patient Outcomes Research to Advance Learning (PORTAL) Network Experience

Abstract

Introduction:

The Patient Outcomes Research to Advance Learning (PORTAL) Network was established with funding from the Patient-Centered Outcomes Research Institute (PCORI) in 2014. The PORTAL team adapted governance structures and processes from past research network collaborations. We will review and outline the structures and processes of the PORTAL governance approach and describe how proactively focusing on priority areas helped us to facilitate an ambitious research agenda.

Background:

For years a variety of funders have supported large-scale infrastructure grants to promote the use of clinical datasets to answer important comparative effectiveness research (CER) questions. These awards have provided the impetus for health care systems to join forces in creating clinical data research networks. Often, these scientific networks do not develop governance processes proactively or systematically, and address issues only as problems arise. Even if network leaders and collaborators foresee the need to develop governance approaches, they may underestimate the time and effort required to develop sound processes. The resulting delays can impede research progress.

Innovation:

Because the PORTAL sites had built trust and a foundation of collaboration by participating with one another in past research networks, essential elements of effective governance such as guiding principles, decision making processes, project governance, data governance, and stakeholders in governance were familiar to PORTAL investigators. This trust and familiarity enabled the network to rapidly prioritize areas that required sound governance approaches: responding to new research opportunities, creating a culture of trust and collaboration, conducting individual studies, within the broader network, assigning responsibility and credit to scientific investigators, sharing data while protecting privacy/security, and allocating resources. The PORTAL Governance Document, complete with a Toolkit of Appendices is included for reference and for adaptation by other networks.

Credibility:

As a result of identifying project-based governance priorities (IRB approval, subcontracting, selection of new research including lead PI and participating sites, and authorship) and data governance priorities (reciprocal data use agreement, analytic plan procedures, and other tools for data governance), PORTAL established most of its governance structure by Month 6 of the 18 month project. This allowed science to progress and collaborators to experience first-hand how the structures and procedures functioned in the remaining 12 months of the project, leaving ample time to refine them and to develop new structures or processes as necessary.

Discussion:

The use of procedures and processes with which participating investigators and their home institutions were already familiar allowed project and regulatory requirements to be established quickly to protect patients, their data, and the health care systems that act as stewards for both. As the project progressed, PORTAL was able to test and adjust the structures it put place, and to make substantive revisions by Month 17. As a result, priority processes have been predictable, transparent and effective.

Conclusion/Next steps:

Strong governance practices are a stewardship responsibility of research networks to justify the trust of patients, health plan members, health care delivery organizations, and other stakeholders. Well-planned governance can reduce the time necessary to initiate the scientific activities of a network, a particular concern when the time frame to complete research is short. Effective network and data governance structures protect patient and institutional data as well as the interests of investigators and their institutions, and assures that the network has built an environment to meet the goals of the research.

Introduction

Observational comparative effectiveness research (CER) typically requires large data sets composed of electronic health records (EHRs) along with patient and organizational data to assess the potential benefits and harms of competing treatments or interventions for different patients.1 Findings from CER are fundamental in helping “consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels.”1 For years, infrastructure awards from the National Institutes of Health (NIH), Agency for Healthcare Research and Quality (AHRQ), Centers for Disease Control and Prevention (CDC), Food and Drug Administration (FDA), private foundations, and other entities such as the Patient-Centered Outcomes Research Institute (PCORI) have provided the impetus for health care systems to create clinical data research networks that conduct CER and patient-centered outcomes research (PCOR). In these networks, collaborating institutions agree to share EHR information and other clinical data essential to the conduct of CER and PCOR.

These networks cannot succeed without strategies for governing the relationships among network members, the development and exchange of research data, and the conduct of scientific projects. Strong governance practices reinforce the commitment of research networks to preserve the trust of patients, health plan members, health care delivery organizations, the researchers themselves, and other stakeholders. Effective governance structures also protect patient and institutional data and the interests of investigators and their institutions.

Motivation: Answering Governance Challenges

Governance of scientific networks can address a wide array of issues—such as developing and overseeing procedures to request and use data; setting research priorities; assuring compliance with security, privacy, and human subject research requirements; addressing proprietary concerns of participating organizations; monitoring research activities; ensuring data quality and integrity; addressing conflicts of interest; developing and maintaining transparency of activity and results; and defining guidance related to data access and use, reproducibility, publishing rights, and dispute resolution.2

Too often, scientific networks do not develop governance processes proactively or systematically, and confront issues only as problems arise. Even if network leaders and collaborators foresee the need to develop governance approaches, they may underestimate the time and effort required. The resulting delays can impede research progress. In the current funding environment, this can be especially detrimental when funders expect research projects to be completed within 12–18 months. In any study, time that is spent on governance is unavailable for scientific work, often to the detriment of the scientific endeavor. Thus, effective network governance facilitates collaboration, promotes current and future research, and ultimately contributes to network sustainability.3–5

The Patient Outcomes Research to Advance Learning (PORTAL) Network was established with funding from PCORI in 2014. Members of the PORTAL team adapted governance structures and processes from past research network collaborations within the Kaiser Permanente-funded Center for Effectiveness and Safety Research (CESR) and the Health Care Systems Research Network (HCSRN, formerly known as the HMO Research Network or HMORN). To promote the efficient operation of future research networks, we present a process model of the PORTAL governance approach, and discuss the generalizability of the PORTAL governance model to research networks based in other health care environments.

History of the PORTAL Network

The PORTAL Network comprises researchers from nine integrated health systems and two collaborating organizations. Six of these health care systems are Kaiser Permanente regions: Colorado, Hawaii, Mid-Atlantic States (District of Columbia, Maryland, and Northern Virginia), Northwest (Portland, Oregon, and southwestern Washington state), Northern California, and Southern California. The other three data-contributing partners are Denver Health and Hospital Authority, Group Health Cooperative (Washington), and HealthPartners (Minnesota). Collectively these health care systems enroll over 18 million members. Smart Patients and the University of Colorado are two collaborating organizations that contribute scientific expertise but do not provide data. In addition to participating in the development of the common data model (CDM) defined by PCORI, the goal of PORTAL is to address critical CER questions, including treatment options for three specific cohorts of patients: individuals with colorectal cancer; adolescents and adults with severe congenital heart disease; and adults who are overweight or obese, including those who have prediabetes or diabetes.6

The decision to pattern PORTAL’s governance structures and processes after scientific networks that came before it accelerated the launch of this network and its research. All of the Kaiser Permanente regions in PORTAL are affiliated with CESR, and 9 of the 10 health care systems are members of the HCSRN. PORTAL governance structures and processes benefitted from prior work by the HCSRN; the National Cancer Institute-funded Cancer Research Network (CRN);7 the National Heart, Lung, and Blood Institute (NHLBI)-funded Cardiovascular Research Network (CVRN);8 two AHRQ-funded networks—the Scalable Partnering Network (SPAN)9 for CER and the Surveillance, Prevention, and Management of Diabetes Mellitus (SUPREME-DM)10 network; and CESR.6 The PORTAL governance approach was particularly indebted to SPAN, which had developed a comprehensive governance structure and document as a specific aim of the award.

Challenges Requiring Governance Approaches

Among the many governance decisions a network may confront are the following:

1. Creating a culture of trust and collaboration. Establishing common expectations for all stakeholders in a network helps to create a culture of collaboration.11 These expectations can be codified in a set of guiding principles for the network. Putting these principles in writing reminds all stakeholders why they came together and why the work is important, and reinforces their commitment to accomplish that work. The CRN, an early HCSRN-based research network, adopted a governance structure that “put high value on transparent decision-making processes… and creating opportunities for all collaborators to play an active role in the consortium’s research and operations.”12

2. Responding to new research opportunities. When a research funding opportunity arises, an approach is necessary to assess the opportunity within the network and select a Principal Investigator (PI). An established process for identifying project leadership is particularly valuable to preserve transparency and fairness when decisions need to be made quickly. Once identified, project leaders need to select sites to participate in the project itself. This process must also be objective, fair, and transparent.

3. Conducting individual studies within the broader network. A scientific network may have many ongoing studies supported by different funders, and multiple ongoing analyses within each study that are led by different investigators. As a result, the network must decide how to identify and prioritize these studies, and whether to maintain governance of individual studies centrally or delegate it to individual investigators.

4. Assigning responsibility and credit to scientific investigators. Large scientific networks are composed of many investigators, and authorship is an important metric for their professional advancement. Networks must develop processes to identify lead and participating authors for individual papers, a process that is notoriously contentious. Opportunities to lead and co-author manuscripts reinforce the role of team members as scientists instead of as merely representatives of data contributing organizations. In practice, site PIs must also decide whether or not their site will participate in a network study based on the opportunities for substantive input and authorship.

5. Sharing data while protecting privacy and security. Data governance should address the tensions between health care organizations that have a proprietary interest in the data, individual investigators inside and outside of the network who have research agendas that rely on those data, and funding organizations and society at large, who are the ultimate beneficiaries of the research. Historically, formal data agreements have governed data exchange between a data partner and an investigator at another site. In a multisite network with simultaneous and multiple analyses led by many sites, the proliferation of such data agreements could substantially delay the conduct of the research.

6. Allocating financial resources. In large networks, resource allocation is critical and can also be contentious. The principles of fairness and transparency are particularly important when money is the topic. When the process for budget setting and monitoring of expenditures is clearly described by network leaders, representatives from each site can be more confident that resources are being allocated in a consistent way across sites and projects within the network.

Guiding Principles and Process Model for Project Governance

Because PORTAL sites had built trust and collaboration in prior research networks, essential elements of governance such as guiding principles, decision-making processes, project governance, and data governance were familiar to PORTAL investigators. The PORTAL Network purposefully built on this foundation, correctly anticipating that new elements of governance could be reviewed, adapted, and approved relatively quickly. As a result, PORTAL was able to establish most of its governance structure by Month 6 of the 18-month project, leaving ample time to refine it and develop new structures or processes as necessary.

Guiding Principles

The first step to creating governance structures was to agree on a set of guiding principles for all stakeholders in the PORTAL Network. The PORTAL Guiding Principles were adapted from the SPAN project, which had made extensive efforts to codify existing but previously implicit principles for network collaborations.3 In response to PCORI priorities, PORTAL added new guiding principles to address the role of individual patient stakeholders and patient advocacy organizations, and a commitment to incorporate learnings at the governance and policy levels from the national Patient-Centered Clinical Research Network (PCORnet). These guiding principles are shown in Figure 1, and were formally accepted by all PORTAL site PIs. It is important to note that guiding principles are not binding policies. The PORTAL Network is not a legal entity so it can neither establish nor enforce policies. If a network guiding principle is in conflict with that of a local site, the network’s expectation is that the local site’s policy takes precedence. Such conflicts can provide the opportunity to reevaluate local organizational policies. For this reason, we have described PORTAL governance in this paper using terms such as “structures,” “procedures,” “principles,” and “guidelines” rather than as policies.

Figure 1.

PORTAL Network Guiding Principles

PORTAL Governance Process Model

Table 1 depicts the PORTAL Governance Process Model. It describes each governance structure and process, how it aligns with the guiding principles, and its effects on network operations.

Table 1.

PORTAL Process Model: Governance Components, Guiding Principles, and Effects on PORTAL

GUIDING PRINCIPLES ADDRESSED	DESCRIPTION	PURPOSE	EFFECT ON PORTAL
GOVERNANCE COMPONENT: DECISION-MAKING PROCESSES
I A, II A, IV A	Included site PIs and other stakeholders from the steering committee in decision-making in all areas of governance and guideline setting	Facilitated collaboration and preemptively avoided conflicts that could slow progress	Helped PORTAL to establish a governance structure by Month 6 of an 18-month project
IV D			Allowed time to apply guidelines and adjust when necessary or desired
GOVERNANCE COMPONENT: PROJECT GOVERNANCE
II B–D	IRB structure (lead and cede)	All sites agreed on a lead IRB (usually lead PI’s site), and other sites cede study oversight	IRB approvals and ceding for the CDM across all took a mean of 25.9 days and a median of 18.5 days to complete; about 150 days for CER cohort approvals
II B, IV C	Subcontracting by using prenegotiated templates	Lead site assumes responsibility for sending subcontracts using prenegotiated language; this expedites participating site execution	Subcontracts executed for all but one participating site in just over 2 months
I A, IV A–C	PI selection for new scientific opportunities	Ensured consistent, transparent, and rapid response to new scientific opportunities	Investigators used the process for PI selection; lead PIs for obesity studies identified in 4 days
I C, II C, III B, IV A and B	Authorship and co-authorship for manuscript	Set expectations for lead authors, co-authors, and use of sites’ data in publications	Investigators submitted manuscript proposals; once approved, all site investigators were invited to be co-authors fostering scientific collaboration and maintaining a culture of sharing scientific credit
GOVERNANCE COMPONENT: DATA GOVERNANCE (is part of governance structure but not the only part)
II B–D, III A	Reciprocal Data Use Agreement (DUA)	Create and execute a single data sharing agreement that covered all study activities at all sites	Draft circulated to all sites for comment, incorporated comments into a final draft that was executed by all sites in 12 weeks; by this time sites were poised to begin data sharing
I B, IV B	Analytic Plan Procedures (APP)	Standardizes SAS code development, distribution, and testing, and specifies how participating sites execute, review and return results	Through standardizing the process of writing, testing, and executing distributed code, cohort teams could plan for this process; in addition review procedures enhanced safeguards to protect PHI
I A–B, II B–D	Other data governance tools and processes ○ Standards for Data Exchange and Quality Assurance ○ Data Sharing Matrix ○ PopMedNet Security Specifications ○ Data Incident Response Plan	Created shared expectations for lead and participating sites regarding data sharing, data quality, and response times Specifies what data can be shared with whom, when, and under what circumstances Outlines security measures at the FISMA-compliant hosting site for PopMedNet Outlined a process for notification “cascade” should a data incident (data released or shared outside of the scope of data agreements) occur	Collaboration is made easier through ensuring predictability in data sharing and management processes
GOVERNANCE COMPONENT: ENGAGING STAKEHOLDERS (NONRESEARCHERS) IN GOVERNANCE
III C, IV E	Role of patients, advisory council members in governance	Foster stakeholder support in research activities Contribute to Network sustainability	Increasing over time Difficult to find those with expertise and willingness to contribute in this area Patient stakeholders included in Steering Committee in Phase II

Decision-Making

Establishing processes for decision-making at the outset of the project served PORTAL well.

1. Development of governance guidelines. PORTAL established a Governance Core at the beginning of the project to develop a draft governance document for review by the PORTAL Network PIs and steering committee. The steering committee is composed of PIs from data contributing sites (called “site Pi”), the three cohort leaders, and representatives from Smart Patients and the University of Colorado. The Governance Core was composed of researchers and project managers with experience in governance of prior scientific networks in the HCSRN. The draft governance document was reviewed by site PIs, who provided feedback. After substantial input, consensus was reached on Version 1 of the PORTAL Governance Document by Month 6, and no vote was necessary. When additional governance questions arose, they were directed to the PORTAL Network co-PIs or the Governance Core, which developed proposed procedures or guidelines for review and revision by site PIs using the same process. Each site PI could seek review as necessary from individuals with appropriate expertise in their own organizations to align PORTAL procedures with organizational policies. This process is illustrated in Figure 2. Version 2 of the PORTAL Governance Document was approved in Month 17 (see Appendix A). This model of central decision-making was reserved for issues of data governance and stakeholder engagement, while most scientific decisions were delegated to the three cohort lead investigators, their sites and their scientific teams.

   This decision-making process proved to be rapid and flexible. The Governance Core was generally able to identify existing governance procedures from CESR or other HCSRN networks that could be modified for use by PORTAL, which decreased the time necessary to draft new procedures. These drafts were circulated electronically with tight deadlines for written comments, with implied consent if site PIs did not provide input. Governance was a standing item on the biweekly agenda of PORTAL steering committee calls, and nuanced discussions of many governance issues took place during those calls.

2. Budgeting. At the outset, the PORTAL co-PIs proposed a base budget for each site and an approach to allocating additional resources to sites to fulfill specific roles. Based on these principles, budgets for participating sites in PORTAL were then individually negotiated between PORTAL co-PIs and site PIs with little difficulty. During the negotiation process, the lead investigators also informed the project team that some resources would be held centrally at the lead site and allocated when project goals had been refined. This budgeting process had been used in other network projects so it was familiar and was considered fair and equitable. Once budget negotiations were completed, rapid execution of subawards was facilitated by the use of prenegotiated language based on Kaiser Permanente interregional templates or HCSRN templates.13

3. Voting procedures. Simple voting procedures were developed to address issues that required input from all participating sites when consensus could not easily be reached. Any steering committee member could call for a vote when a quorum (more than 50 percent of the committee) was present. A motion carried when a simple majority (50 percent of those present) had been achieved. Votes could be taken verbally during a steering committee call or using web-based voting software. Individual votes were public rather than anonymous, recognizing the importance of overt dissent and debate in the culture of scientific research. We used this process when voting whether to endorse PCORnet policies and other related PCORnet-level decisions. This process allowed each site to provide an independent vote.

Figure 2.

PORTAL Decision-Making Process

Project Governance

PORTAL addressed elements of project governance, including how to select scientific leadership; how to structure Institutional Review Board (IRB) review; how to establish subcontracts; how to select new research projects; how to identify participating sites; how to determine authorship; as well as establishing procedures for dispute resolution.

1. Selection of scientific leadership. PORTAL co-PIs used the existing HCSRN approach to select lead investigators for each study cohort. This process solicited nominations from all site PIs when a request for proposal was released. The leader was selected by the steering committee, which included all site PIs. Including them in decision-making was critical because each acted as a representative for their own site’s scientific engagement and data stewardship.

2. IRB review. IRB review presents challenges for networks like PORTAL since all sites are required to obtain approval before research can begin. PORTAL used the process developed by the HCSRN in 200814 to accelerate IRB review for minimal risk studies. These IRB review processes were subsequently tested and enhanced through the work of CESR. The lead PI for each cohort submitted an application to their home institution’s IRB asking for approval to conduct the research and requesting the same IRB to assume study oversight for participating sites if those sites agreed. The latter request was made by submitting the HCSRN’s Multi-Site Research Application Cover Sheet with the application. Once the IRB approved both requests, participating sites submitted this same application, using the same materials, to their local IRBs requesting permission to cede study oversight to the lead IRB. Obtaining approvals using this well-established process reduced the substantial delays experienced by prior studies not conducted within CESR or the HCSRN.15,16

3. Subcontracting. Given the 18-month project period, PORTAL faced the urgent need to execute subcontracts so invoicing by sites could begin. Using an existing interregional subcontracting template for Kaiser Permanente sites and the HCSRN Subaward Template for Group Health and HealthPartners, these documents were executed within 2 months after the notice of award (although one site took nearly 6 months, an uncharacteristically long lag time for this site).

4. Selection of new research, lead PI, and participating sites. Similar to the grant application process described above, collaborators in the PORTAL Network developed approaches to respond rapidly to new research opportunities. PORTAL adapted CVRN and HCSRN processes for PI and lead site selection, including the following:

   • All sites had the opportunity to opt in or opt out of each opportunity.
   • All site PIs received the funding announcement and circulated it at their site.
   • Site PIs decided whether or not their site wished to participate and identified any researcher who wished to be considered for the PI role of the new proposal.
   • When more than one investigator was interested in serving as PI, each was invited to submit an abbreviated NIH-style biosketch and statement of interest. The PORTAL Steering Committee then provided input on all candidates to PORTAL Co-PIs.
   • The PORTAL co-PIs, after considering the input from the steering committee, made the final selection of a PI, taking into account the desire to balance leadership opportunities among the sites in the network while ensuring that the expertise of the team selected offered the best chance at being funded.

   This process was tested in February 2015 when PCORI released its Obesity Observational Research Initiative, with two areas of research emphasis. PORTAL used its process to nominate investigators to serve as the PCORnet project PI. Two PIs from PORTAL were nominated, and one was ultimately selected. PORTAL subsequently identified a network PI to participate in each area of research emphasis. PORTAL again employed its procedures, and selected both PIs in four business days. These PIs in turn used the governance process for site selection to identify individual PORTAL sites to participate in the application.
5. Authorship. PORTAL established publication and presentation guidelines using the processes developed by CRN, CVRN, and SUPREME-DM as models. The PORTAL guidelines included the following:

   • A statement of the purpose of authorship guidelines, to foster a high volume of high-quality publications;
   • A publications committee composed of the PORTAL co-PIs, two Site PIs, and a patient stakeholder, committed to reviewing manuscript proposals within 10 business days;
   • A structured manuscript proposal form that enabled the PORTAL co-PIs to remain informed about manuscripts under development and ensure coordination and avoid duplication;
   • Assignment of responsibility to lead authors for coordinating the writing effort, including collecting and incorporating comments from co-authors;
   • A requirement for authors to adhere to the requirements of the International Committee of Medical Journal Editors (ICMJE); and
   • A suggested description of PORTAL, language for recognizing PCORI as the funder, and a way to designate network authorship if required by the journal.

   When the PORTAL co-PIs began receiving manuscript proposals, we realized that the procedures lacked some specificity in certain areas. Subsequent revisions included the following:

   • Lead authors were expected to extend a co-authorship invitation to every site PI from data-contributing sites to provide each site the opportunity to be represented.
   • Site PIs were expected to respond to the invitation by nominating oneself as coauthor, extending the invitation to another investigator at that site, declining the invitation but allowing that site’s data to be used and mentioned in an acknowledgment, or declining the invitation and prohibiting the use of that site’s data.

   These revisions are examples of how early implementation of procedures such as authorship guidelines provided time for the network to experiment and refine them when necessary.

6. Dispute resolution. The underlying principle for dispute resolution was to attempt resolution at the most decentralized level. For example, our network expects that a scientific debate about development of a cohort should be resolved within the cohort investigators wherever possible. If resolution at that level is not possible, the PORTAL co-PIs should be alerted so they can advise and assist in the process. If the dispute still cannot be resolved, the PORTAL co-PIs will make a final decision.

Data Governance

Sound processes for data governance helped ensure appropriate data protections and minimized the risk of unintended data disclosure. We employed the data governance definition proposed by Rosenbaum, “the process by which [data] stewardship responsibilities are conceptualized and carried out. Data governance establishes the broad guidelines for access, management, and permissible uses of data; identifies the methods and procedures necessary to the stewardship process; and establishes the qualifications of those who would use the data and the conditions under which data access can be granted.”17

Among the data governance issues addressed in PORTAL were the following:

1. Reciprocal Data Use Agreement (DUA). As described above, operating a multisite network presents unique challenges to data sharing because simultaneous analyses are led by many sites. Establishing independent agreements to cover each data transfer could have substantially delayed the conduct of the science. In response, PORTAL adopted the use of a “reciprocal” DUA that was based on agreements used in CESR and other HCSRN collaborations (SPAN and SUPREME-DM; HCSRN Reciprocal DUA Examples). In a reciprocal DUA, all sites agreed to disclose limited data sets relevant to the specific aims of the studies with all other participating sites.13 All sites signed this single agreement covering data sharing for all study activities, from building the CDM to conducting the cohort analyses, within three months. By this time, the structure of the CDM was nearly final and the Data Coordinating Center and cohort lead sites had written SAS code for distribution.

2. Analytic Plan Procedure (APP). The APP was based on procedures developed by SUPREME-DM that streamlined the documentation, writing, testing, and execution of distributed SAS code for analytic plans. These procedures outlined processes for participating sites to run the code, check results (for accuracy and to make sure no PHI was included), and return results. In addition, the APP included a tracking and management process for analysis requests to ensure that participating site data managers were notified of all requests and were aware of the timelines for completion. The process helped minimize the potential burden of multiple requests occurring at the same time. PORTAL agreed to a target of three days for participating sites to opt in or opt out, run the code, and return results to the data coordinating center, and this standard has been met in queries to date.

3. Other Data Governance Tools and Processes. The PORTAL Network developed other tools and processes to increase the predictability of network operations for collaborators. These can be found in the Appendix Toolkit of the PORTAL Governance Document Appendix A. They include the following:

   • “Standards for Data Exchange and Quality Assurance”: This document describes internal (within PORTAL) and external (outside of PORTAL) data sharing; it includes standards for data refreshes and standards for data retention.
   • Data Sharing Matrix: This matrix describes what data can be shared, with whom, and under what circumstances.
   • “PopMedNet Security Specifications”: Developed by Lincoln Peak Partners, LLC the developers and Federal Information Security Management Act (FISMA)-compliant hosting site for PopMedNet queries, this document outlines the security structure for storing results from these queries at the hosting site.
   • “Data Incident Response Plan”: This document summarizes procedures in the unlikely event that data are shared in a way that is inconsistent with our data agreements.

Engaging Patient Stakeholders in Governance

During the initial project period, the leaders of the PORTAL Patient Engagement Council discovered that patient stakeholders preferred to engage in scientific activities such as commenting on draft surveys and methods of survey administration, and participating in periodic cohort calls, rather than participating directly in network governance. The Governance Core participated in two webinars designed to educate and enlist patient stakeholders in the governance process of PORTAL. Since patient stakeholders had limited time (most have day jobs) to spend on PORTAL activities, contributing time to governance processes seemed less interesting to them than contributing time to the PORTAL’s scientific activities. It took some time to develop patient stakeholders who were willing to spend time on governance, and this resulted in patient stakeholders who were willing to become members of the steering committee after Phase II of the project was funded.

Implications for Research Networks

Good governance is “preventive medicine” for scientific networks. The guiding principles of a network help establish its core set of values and promote a shared culture. Principles for decision-making facilitate hard choices, such as resolving the inevitable competition among investigators from multiple sites for leadership in projects and papers. At the same time, project governance procedures help overcome predictable delays in completing IRB review and establishing subcontracts. Data governance processes minimize the risk of unauthorized data disclosures and build trust among the patients and organizations that contribute data to the research. These processes also facilitate communication among investigators, programmers, and statisticians that can minimize errors in data extraction, quality assurance, and sharing, and can prevent time-consuming and costly duplication of effort.

Throughout this paper, we have emphasized that the PORTAL governance approach was not developed de novo, but rather is based on over 20 years of experience with externally funded, multisite research networks.5 Further, six of the nine sites in PORTAL are regional research departments within the same organization, Kaiser Permanente. For these reasons, we recognize potential limitations in the generalizability of these governance processes to scientific networks in other organizational environments. We also recognize that not all networks need to develop governance procedures as extensive as those we have included in the Appendix Toolkit of the PORTAL Governance Document (see Appendix A). With these caveats, we believe that the PORTAL experience can address several questions of importance to new and established scientific networks.

What Are the Highest Governance Priorities for New Networks?

As new networks form, their attention is often initially directed toward resolution of technical issues and the governance questions associated with them. Issues such as data management, human subjects review, and contracting take center stage. Efficient solutions to these issues are clearly necessary to achieve scientific goals within time and budget constraints. Further, discussions around the adoption or development of governance in these areas can promote the development of working relationships, or identify potential barriers to establishing a strong network “culture.”

In contrast, governance components that address professional interactions—such as guiding principles, approaches to decision-making, processes for selecting investigators and sites for new projects, authorship for papers, and procedures for dispute resolution—are often postponed if they are addressed at all. Our belief is that strong professional relationships are more critical to network sustainability than is technical infrastructure. Overall, we encourage new networks to use the development of governance procedures for technical issues as an opportunity to reflect on and ultimately define the social norms that will guide the network, and to develop explicit governance procedures in those areas early on.

What Components of PORTAL Governance Can Be Adapted or Adopted by Other Networks?

The most important part of the PORTAL governance framework is its taxonomy of governance issues that should be considered, rather than any specific solution. That said, we believe that both new and established networks commonly make the mistake of developing governance procedures from scratch, rather than taking advantage of existing prototypes. The online compendium of governance materials in Appendix A of this paper is our attempt to make these resources available for use or adaptation.

In developing PORTAL and prior scientific networks, we learned, sometimes painfully, that substantial differences existed among research departments in Kaiser Permanente and other sites in the HCSRN. Each research department had standard operating procedures, which in some cases could be modified, but in other cases simply needed to be accepted. Thus, the materials developed for PORTAL should be viewed as templates for adaptation to local requirements. Their main virtue may be to demonstrate that achieving agreement between institutions is possible, and that they can accelerate scientific work. The governance infrastructure developed over many years by CESR and the HCSRN allowed a new research partner, Denver Health and Hospital Authority, to enter the network and contribute data and scientific expertise to both the SPAN and PORTAL networks. This provided a “proof of concept” that PORTAL governance procedures could be generalized to another research partner with a different organizational model.

In an era when scientific advances are likely to require increasing interorganizational collaboration, it is worth the effort for researchers and administrators to develop standard agreements with frequent collaborators, whether they be academic institutions, community-based delivery systems, or private industries. When such agreements are in place, they can be rapidly modified to meet the needs of new grant applications or collaborative networks.

How Should Network Governance Align with Organizational Policies?

Scientific networks cannot develop governance in a vacuum. Research networks such as PORTAL are not legal entities, and thus are not empowered to set or enforce policies. Each scientist and research department is part of a larger organization such as a health care delivery system, academic institution, or private company. These legal entities have their own governance that is expressed through enforceable policies and procedures for conducting and participating in research. As a result, researchers are subject to legal and regulatory policies that may differ across jurisdictions and are subject to interpretation by attorneys, controllers, compliance officers, and IT departments in the larger organization. For the most part, such policies have not been developed with the needs of multi-institutional networks in mind, however. When network principles or guidelines conflict with organizational governance policies, the latter must take precedence unless they can be revised. New networks must be aware of these constraints, which may take substantial time to resolve within and among participating organizations. In particular, scientific leaders who propose to develop new networks must engage local organizational leaders early in the process to make the case for adaptation of local policies that could constrain network participation. In response to this organizational reality, PORTAL developed a set of principles and guidelines, which research leaders and other representatives from each health care system endorsed after careful consideration to assure they were not in conflict with local policies and procedures.

The Future of Governance for Scientific Networks

The governance of scientific networks needs to advance at the pace of research itself. We highlight three areas where further governance development is necessary.

Interoperability

As we have shown, the first stage of governance development requires agreement on standard guidelines and procedures among the participants within a scientific network. Researchers and research departments are likely to want to participate in multiple scientific networks, however. To achieve this goal efficiently, governance tools and procedures will increasingly need to be standardized between scientific networks.

Metrics

Although we believe that PORTAL has developed reasonable provisional solutions to governance issues such as IRB review, establishment of subcontracts, and implementation of data agreements, further improvements in efficiency are undoubtedly achievable. Such quality improvements would be greatly facilitated by the development of standard metrics for network performance, which would promote comparisons across sites and scientific networks. Important metrics could include the number of days necessary to execute subcontracts with all participating sites; to obtain IRB approval; to make key decisions; to develop, test, execute, and return results from a SAS work plan; and to perform quality checks on elements added to the CDM. Such information would help the research organizations that participate in scientific networks to improve their own performance. They might also be useful in grant applications to demonstrate operational efficiencies to potential funders.

New Content Areas

PORTAL governance procedures were focused on the conduct of observational CER and pragmatic trials, which have been a staple of PORTAL sites for two decades. Other types of studies raise additional governance considerations. Examples include large-scale observational studies that require collection of genetic material or other biospecimens, and interventional trials that collect primary data from participants using biometric devices or surveys. New modalities for remote data collection, such as web-based surveys, interactive voice response (IVR) calls, text messages, and video encounters will also require new approaches to govern data privacy and security as well as data management and use. While these new areas will require innovative technical solutions, we believe that governance principles used to guide the decision-making process and other nontechnical issues will remain critical and can be readily adapted to meet these challenges.

Conclusion and Discussion

Building on prior governance processes developed and implemented by CESR, the HCSRN, and its scientific networks, PORTAL completed its initial Governance Document by Month 6 of an 18-month project. The use of familiar procedures and processes allowed project and regulatory requirements to be established quickly to protect patients, their data, and the health care systems that act as stewards for both. As the project progressed, PORTAL was able to test and adjust the structures it put in place, and to revise them when necessary. As a result, processes such as PI selection for new research opportunities were predictable, transparent, and effective.

The development of governance processes early in the life history of research networks like PORTAL is essential to support ambitious research agendas. Well-planned governance can reduce the time necessary to initiate the scientific activities of a network and engage researchers in transparent decision-making processes to protect and share data needed to conduct CER. With time, strong governance can promote a culture of trust that allows networks to expand into new scientific domains beyond the expectations of their original funders.