Ensuring Support for Research and Quality Improvement (QI) Networks: Four Pillars of Sustainability—An Emerging Framework

Abstract

Multi-institutional research and quality improvement (QI) projects using electronic clinical data (ECD) hold great promise for improving quality of care and patient outcomes but typically require significant infrastructure investments both to initiate and maintain the project over its duration. Consequently, it is important for these projects to think holistically about sustainability to ensure their long-term success. Four “pillars” of sustainability are discussed based on the experiences of EDM Forum grantees and other research and QI networks. These include trust and value, governance, management, and financial and administrative support. Two “foundational considerations,” adaptive capacity and policy levers, are also discussed.

Introduction

Multi-institutional research and quality improvement (QI) projects using electronic clinical information from electronic health record (EHR) data, electronic patient reported outcomes (ePROs), mobile health technologies (mHealth), and other sources have tremendous potential to contribute to scientific discoveries and improved care delivery and patient outcomes. By its nature, however, this type of “big and complex” science requires substantial support for infrastructure (governance, data, methods, and training)1,2,3 as well as ongoing maintenance and management structures to support projects over many years. Because of these complexities, anticipating key issues for sustainability is challenging. Drawing upon the experiences of research networks participating in the Electronic Data Methods (EDM) Forum and well-established research registries and QI networks, this paper proposes an initial conceptual model of sustainability to guide more holistic thinking about the long-term needs of research and QI networks and strategies to prevent potential problems with future execution and sustainability for research and QI networks. The framework may also guide thinking about future business models for research and QI networks using electronic clinical data.

The proposed sustainability framework to support research and QI networks is based on conversations with EDM Forum4 investigators and stakeholders at the frontier of building systems and processes to use electronic clinical data for research and QI, and includes four “pillars” that the community has identified as critical considerations for achieving sustainability: trust & value, governance, management, and support (both financial and administrative). Each of these elements is important to achieving the goals of a research and/or QI network, which is facilitated by a level of flexibility, or adaptive capacity, and may be influenced or enhanced by external policy levers.

Pillar 1: Trust & Value

Developing approaches for research and QI networks to successfully engage community members (patients, providers, their families, and others5) is a cornerstone of effective comparative effectiveness research (CER), patient-centered outcomes research (PCOR), and QI. Trust requires an explicit effort to work with multiple stake-holders, including those who contribute to and produce science— such as delivery systems contributing data; technologists facilitating access to the data; privacy officers, and investigators, among others—as well as the users of the science, including patients, providers, and policymakers. Research and QI network leaders and investigators must understand the values and goals of participants in the research and QI effort. CER, PCOR, and QI provide unique opportunities for discovery and meaningful improvement in outcomes precisely because the philosophy of the approach to inquiry is grounded in a commitment to facilitate trust with key community members. Gathering broad input to ask (and answer) relevant questions that have perceived value to stakeholders should be an explicit step in all of these efforts. Continually assessing whether the resulting information or “answers” are perceived as useful by end users is crucial to the sustainability of infrastructure not only for ethical reasons, but to vet and refine the value proposition for research over time.

In addition, when scaling up to incorporate new partners (data holders) in a network, trust among the healthcare delivery organizations that share information and personnel must be evaluated. Partners may be concerned with protecting proprietary data and business interests or liability issues; these concerns can undermine research and QI efforts very quickly if not addressed. Furthermore, ongoing validation of the time, effort, and resources to contribute to network-based research and QI projects and programs is essential to ensure all partners continue to see value in participating.

Pillar 2: Governance

A suite of issues on privacy and security; data access and use; roles and responsibilities of partners; legal issues and requirements; submissions to institutional review boards (IRBs); and accountability considerations (among others) are critical to achieve governance policies capable of sustaining trust.6,7 Scaling the network to include more partners and members can be a significant benefit for sustainability, and can be done smoothly if the data governance structure and network expectations of participants are transparent. In short, governance instantiates the approach to achieving community-based support, stakeholder engagement, and trust—all critical elements of sustainability.

Pillar 3: Management

Models for managing a network and individual projects should support specific goals articulated by the networks’ leadership and community stakeholders. The level of administration and project management must also be appropriate to the aims of the network or research and QI projects at every stage. For example, in the phase of development focused on bringing key partners together, working through roles, responsibilities, and data use agreements requires substantial staff time. For example, among the projects participating in the EDM Forum, as required by the funding announcements for the PROSPECT, DRN, and Enhanced Registry projects,8,9,10 each project has supported 30 percent to 100 percent of one project manager’s time dedicated solely to developing the network for at least the first three years. Twenty percent of the principal investigator’s time is required (reflecting the high management required for these activities), and the level of effort/support allocated to senior investigators may vary depending on the phase of the project. Over time, budgets must evolve to ensure sufficient support for maintenance efforts. The amount of time required of senior leaders may decrease once the network, governance, and research protocols are in place, but good case studies demonstrating successful transitions of this type are currently lacking. For large, multi-site networks, however, substantial project management is likely to be an ongoing and important maintenance effort in order to ensure all partners and community stakeholders are informed and up-to-date on opportunities within the network and the value of the network, as well as roles and responsibilities of ongoing participation.

Pillar 4: Financial and Administrative Support

A variety of models for supporting networks have evolved over time. Diverse sources of support are necessary to provide the level of staffing to continue to make information and network resources available to users, which in turn demonstrates value to partners. For example, in-kind support such as shared personnel or expertise; core technical needs; and database maintenance to ensure high data quality, manage data flows, and monitor evolving guidelines, requirements, and evidence should be valued appropriately as important network assets. In this discussion it is also important to consider that “in kind” support is largely a matter of perspective. On some level, the resources that support the network must be perceived to have value to the organization or individual who is contributing these resources. And the network’s requirements to justify a continued level of “in kind” support should be considered.

In addition, there is an undeniable need for targeted financial support to help provide the technology and personnel required to maximize the potential for discovery, quality improvement, and improvements in patient outcomes. Though a deep exploration of potential models is beyond the scope of this discussion, many financial models are possible. For example, supporting core infrastructure, including data resources and maintenance of statistical code may be achieved by charging dues, royalties, and/or user fees to access the information. Traditional grants and contracts may also be available to provide support, but investigators have commented that relying on grant support constitutes an uneven and potentially inconsistent approach. Early thinking about how to develop a diversified approach to funding long-term projects and networks is recommended.

Foundational Considerations

In addition to the pillars, there are important foundational and contextual considerations related to adaptive capacity of the network and network leadership, as well as policy levers that can be crucial to a network’s sustainability.

Adaptive Capacity

As the policy environment and marketplace for health care delivery evolves at an increasing pace, there is a corresponding expectation to rapidly integrate new technologies into clinical practice and care processes. Early experiences demonstrate that practice-based research networks must be flexible with respect to both capabilities and capacity. And perhaps most importantly, QI and research networks must be able to institutionalize elements of innovation and continuous learning in the network’s organization in order to meet new needs and create new opportunities. For example, networks greatly benefit from serving multiple purposes, which may include PCOR and CER, surveillance and safety reporting, QI and support of routine clinical care processes. Adapting to new methods of collecting data is another consideration. As technology evolves, networks need to be flexible to collect data via mobile devices, biosensors, etc., and include new types of information such as patient-reported outcomes or care-giver reported input. The ability to incorporate new information; establish linkages to new data sources; anticipate how new technologies may interface with care delivery, workflow, and processes; and consider how new technologies may impact patient outcomes are other considerations that are dimensions of flexibility. This capacity will allow the network to take advantage of advances in technology and in patient engagement, which is needed to maintain relevance in a rapidly changing environment.”

Policy Levers

Considering the important policy levers that may impact sustainability is also key. These may include regulatory, financial, or legal opportunities, which may be “carrots” or “sticks.” Carrots may drive participation in order to achieve a beneficial outcome—such as incentive payments for participation (e.g. those enacted in the Office of the National Coordinator’s meaningful use stage incentives); and sticks levy a penalty for not participating—such as accreditation or payment practices requiring network participation to ensure licensure, etc. Creating value for the network so that stakeholders see the importance of participating in research and QI that drives improvement and desired outcomes is an explicit goal of these efforts, but the benefit may be sufficiently diffuse that external drivers must be contemplated to optimize participation. Other external considerations are the concepts of marketplace pressures and competition to maintain value and demand.

Discussion

As these types of networks mature, it is critical for investigators and clinical and research leaders to think about the long-term sustainability of research and QI networks, and to do so early in the process. In future, a formal analysis of successful business models will be beneficial to understand and learn from promising approaches. Proposed additions and modifications to this proposed model are welcome, particularly with respect to successfulbusiness or pricing models to ensure sustainability over time. We hope the EDM Forum’s multi-disciplinary community will contribute to this model and to building a shared knowledge base of cases that demonstrate promising practices for sustaining research and QI networks that are generating evidence to improve patient outcomes.

Figure 1.

Sustainability Framework to Support Research and QI Networks