Harnessing the Collective Expertise of Patients, Care Partners, Clinical Teams, and Researchers through a Coproduction Learning Health System: A Case Study of the Dartmouth Health Promise Partnership

Abstract

The Coproduction Learning Health System (CLHS) model extends the definition of a learning health system to explicitly bring together patients and care partners, health care teams, administrators, and scientists to share the work of optimizing health outcomes, improving care value, and generating new knowledge. The CLHS model highlights a partnership for coproduction that is supported by data that can be used to support individual patient care, quality improvement, and research. We provide a case study that describes the application of this model to transform care within an oncology program at an academic medical center.

INTRODUCTION

Despite best efforts by dedicated clinical teams, support staff, and senior leaders, academic health systems that serve the US population often fail to deliver high quality care. Access to care (including essential primary, specialty, and behavioral health care) is limited and not distributed equitably, costs are high and often unaffordable, and clinical and support staff are in short supply and many are suffering from burnout (Shrank et al, 2021). Consequently, many citizens, health professionals, policy makers, and politicians recognize the need for fundamental change – genuine transformation – in the regional health systems that serve the nation.

Dartmouth Health (DH) is one of many large academic health systems faced with myriad challenges as it pursues excellence in clinical care, teaching, and research. Like most health systems, it has an ambitious strategic plan that features a bold statement that is, in fact, a “Promise” to the people it serves:

“Together, we bring the full power of our collective expertise to provide the best possible care to our patients, our people, and our communities.”

In 2019, recognizing the challenge of delivering consistently on this promise, DH senior leaders partnered with researchers at The Dartmouth Institute for Health Policy and Clinical Practice (TDI) to test a model for implementing the strategic Promise. The resulting “Promise Partnership,” built into the strategic plans of both organizations, calls for using a coproduction learning health system (CLHS) model (Nelson et al, 2016) to transform the care delivery system, beginning with an ambitious pilot test within a large clinical program - the regional oncology program based at the Dartmouth Cancer Center.

The term “coproduction,” was first used in the 1970’s by economist and Nobel Laureate Elinor Ostrom (Ostrom, 1996) to describe an effective strategy for managing scarce essential resources. Dartmouth researchers adopted and adapted the term, to health and healthcare, to fundamentally reconceptualize the patient-clinician relationships at the center of health service delivery (Batalden, 2018). The CLHS is built on the belief that the collective expertise of patients, care partners, healthcare teams, and scientists is necessary to fully implement person-centered, evidence-informed care.

In this paper, we describe: (1) the CLHS model and implementation methods, (2) initial measured results of the Promise Partnership’s pilot test in oncology, and (3) lessons learned, future plans, and the potential of the CLHS to transform Dartmouth Health.

METHODS

Coproduction Learning Health System (CLHS) Model

To achieve the strategic “Promise”, we applied the CLHS model within the DH academic health system. The CLHS aligns with the premise of the Institute of Medicine’s (2013) learning health system model in which science, informatics, incentives, and culture are aligned for continuous improvement and innovation, with best practices embedded in the care process, and where new knowledge is generated as an integral by-product of care. The CLHS model extends this definition by explicitly bringing together patients, care partners, clinicians, healthcare administrators, and scientists, including those embedded in leadership, operational, and quality improvement roles, to share the work of optimizing health outcomes, improving care value, and generating new knowledge. The CLHS model highlights a partnership for coproduction that is supported by data that can be used to support individual patient care, quality improvement, and research.

Figure 1 illustrates the aim of the Promise Partnership CLHS framework – to achieve optimal health, high value care and research – and the core components that work together to reach the aim. In developing the Promise Partnership, we considered which tools would facilitate the creation of a sustainable CLHS, focusing on each component of the model. Two initiatives were designed to support coproduction between clinicians and patients: 1) Serious Illness Conversation (SIC) training (Bernacki et al, 2015) in which clinicians provide tailored information and patients share their values and priorities to guide the road ahead, and 2) development of tools to facilitate pre-visit planning and shared decision-making. An online peer-facilitated support network was designed to enable active and bereaved care partners to support one another and share knowledge. A collaborative learning network brought together clinical teams, patients, scientists, and system leaders to learn from one another, drive improvement, and spur research. Finally, a data capture and reporting system was prototyped to enable population-specific registries and a balanced set of measures for tracking CLHS outcomes for patients and populations.

Figure 1.

Promise Partnership Coproduction Learning Health System

Setting

The DH CLHS focuses on cancer patients, their care partners, and interdisciplinary teams delivering cancer care in the Dartmouth Cancer Center (DCC). DCC is the only NCI-designated comprehensive cancer center in Northern New England, serving a largely rural population of 1.9 million people across NH and VT, and with patients from nearby states of ME, MA, and NY. DCC includes care teams in surgical oncology, radiation oncology, medical oncology, and hematology and has sixteen clinical oncology groups (COGs). COGs are interdisciplinary teams structured to provide coordinated care to patients with distinct diagnoses or types of cancer. COGs were selected to participate in working groups based on readiness to engage, identification of clinical champions, adequate resources (i.e., team time), and alignment of initiatives with COG goals. COGs were onboarded in a stepwise fashion for controlled expansion. Due to their role in providing concurrent support for many people with cancer, the CLHS also engaged the palliative care outpatient program.

Leadership

Leadership of the Promise Partnership CLHS is shared between clinical, academic, and operational leaders at DH, DCC, and TDI. The work is guided by a steering committee which meets quarterly and includes eight health system leaders and six academic leaders. Members have expertise in health services research, implementation science, quality improvement, collaborative learning networks, health system operations, and healthcare measurement and evaluation. An operational leadership team meets bimonthly and includes leaders of each working group, DCC leaders, and a patient advisor.

Implementation Framework

We adopted the Quality Implementation Framework (Myers et al, 2012) to guide working groups through phases of implementation. Project Year One activities included project planning; regulatory activities; team formation; identifying process, outcome, and balancing metrics; small-scale testing of interventions; and deploying the collaborative learning network. Year Two activities included expansion of interventions to new COGs, deploying new system-level measurements, and hosting two learning sessions. Year Three activities included spread to additional COGs; ongoing evaluation of interventions; hosting a learning session focused on research; and identifying future funding opportunities.

Working Groups

Six working groups were established to design, deliver, and test CLHS components. Figure 1 illustrates the relationship of each initiative to the CLHS. Each working group applied quality improvement methodologies and an interdisciplinary team approach which includes patients and care partners. Table 1 summarizes working group aims and methods applied to achieve goals.

Table 1.

Working Groups: Challenges, Aims and Methodologies

Working Groups	CLHS Domain	The Challenge	Aims	Methods
Serious Illness Conversation (SIC)	Partnership for coproduction	Conversations to elicit patients’ priorities are infrequent, late and limited.	To improve frequency and documentation of SIC to improve communication about goals of care.	Clinician training and coaching; EHR documentation of SIC; QI coaching.
Pre-visit planning & point-of-care dashboards	Partnership for coproduction	Ineffective communication often leads to services that fail to meet the needs and goals of patients.	To build PVQ and dashboard into EHR (and integrate tools into clinical workflows) to address concerns and goals.	Design, prototype, build, and test PVQ and dashboard. Align data collection with registries.
Peer-facilitated support network	Patient & family support network	Care partners of people with serious illness need to cope with circumstances that arise as disease progresses and in bereavement.	To build an online peer-facilitated support network for care partners of people with serious illness and to develop a strategy to recruit local care partners.	Co-design a peer-facilitated support network with local stakeholders and enable the network with a vendor partner.
Collaborative learning network	Care and quality improvement network	The health system has a limited set of mechanisms to learn from excellence, test innovative practices, and drive clinical transformation.	To build a learning network where team members from different COGs come together to improve outcomes for patients and interdisciplinary teams.	Host biannual learning sessions. QI coaching to develop charters, SMART Aims, key driver diagrams, tables of measures, and show change over time.
Data, measurement, and scholarship	Research registry	There is no standardized measurement and reporting system to assess the progress and impact of the CLHS.	To identify a balanced set of measures; provide timely data reports to leaders and teams; and demonstrate interdisciplinary scholarship.	78-member Delphi panel to identify measures. Align measures with strategic plans and provide monthly data reports.

Abbreviations: COG: clinical oncology group, EHR: electronic health record, PVQ: pre-visit questionnaire, QI: quality improvement, SIC: Serious illness conversation

Evaluation

A formative evaluation of the Promise Partnership is assessing uptake of initiatives, with a focus on end-user engagement, experiences, and changes in healthcare processes. The formative evaluation provides insight into mechanisms and contextual factors that facilitate or impede implementation. A summative evaluation will assess impact on measures in eight domains (Van Citters et al, 2022) we hope to influence with the CLHS.

RESULTS

Key findings from our formative evaluation demonstrate the uptake of initiatives to support the CLHS model (Table 2).

Table 2:

Reach, Challenges, and Milestones of Promise Partnership Initiatives

Initiative	Reach	Challenges	Milestones
Serious Illness Conversation	Approximately 900 patients screened, 500 with a positive screen, and 200 received a SIC.	Staff shortages, high clinical volumes, lack of time for longer visits.	Pilot test in 2 COGs, expanded to 5 COGs. Goal to expand to 11 COGs. Assess sustainability.
Pre-visit planning and point-of-care dashboard	Approximately 1,800 PVQ in palliative care and 200 PVQ in transplant & cellular therapy.	Low PVQ completion rates without multiple methods to support patient completion.	Pilot test in palliative care; Expand to one COG.
Peer-facilitated support network	Approximately 200 members enrolled in the network.	Limited use of health system resources due to status as a research pilot.	Pilot test with 30 people. New target to achieve 250 members by June 2023.
Collaborative learning network	4 bi-annual collaborative oncology learning sessions held, with 124 to 286 attendees each.	COVID-19 impacted ability to convene in person, planned QI trainings not deployed due to limited staff bandwidth.	Deploy and test an online QI curriculum, host collaborative learning events.
Data, measurement, & scholarship	Balanced set of measures to guide reporting. 62 people involved in scholarship activities (4 proposals funded; 2 articles; 8 conference presentations).	Difficult for clinical team members to access data for QI or research; Limited time for clinicians to participate in research.	Streamline data collection and distribution mechanisms to support QI and research.

Abbreviations: COG: clinical oncology group; QI: quality improvement

Serious Illness Conversation (SIC)

The Serious Illness Conversation (SIC) initiative was designed to support patient-clinician communication regarding goals, wishes, and planning in the face of serious illness. Interdisciplinary oncology team members received initial and ongoing training in SIC communication skills from one of three palliative care clinicians. Team members were trained to identify high-priority patients and conduct SICs or refer more complex patients to palliative care. To support uptake, the electronic health record (EHR) was modified to allow documentation of core elements of the SIC, including eligibility to receive a SIC (Wasp et al, 2022). Structured EHR documentation improved the ability to provide person-centered care by allowing patient and care partner goals and care plans to be available to any clinician with access to the patient’s medical record, and creating a field-defined method to capture the occurrence and frequency of SICs. Over 21-months, oncologists screened more than 900 patients and found approximately 500 eligible for a SIC. Among these, more than 200 completed a SIC. Data are reported in a novel EHR application that shows conversations by clinician, involvement of care partners in the conversation, and core SIC components addressed. Spread to multiple COGs has shown that implementation challenges are linked to clinician staffing shortages, competing clinical demands, and time constraints.

Pre-visit planning and point-of-care dashboard

Pre-visit planning questionnaires and point-of-care dashboards were co-designed by patients, care partners, clinicians, and researchers to enhance communication and tailor healthcare services to meet the individual needs and goals of people with a serious illness. The co-design team identified and pilot-tested the specifications of the tools, worked with Epic application analysts to build tools within the EHR, and developed clinical workflows to effectively use data to guide clinical conversations. Following a successful pilot test in outpatient palliative care, the core components were adapted to meet the needs of clinicians and patients within the transplant and cellular therapy COG. More than 1,800 palliative care PVQs were completed over 20-months and nearly 200 transplant and cellular therapy PVQs were completed over 11-months, allowing patients and care partners to provide information to guide conversations around what matters most (Wilson et al, 2022). Patient generated data are now available to form the basis of a research registry and have been used to develop population-level analyses that depict the expected trajectory of recovery following a transplant (Hayes et al, 2022).

Peer-facilitated support network

A peer-facilitated support network was co-designed by people with serious illness, care partners, clinicians, and scientists with the aim of preparing active and bereaved care partners to cope with unexpected surprises. The co-design team identified the specifications of the network, tested prototypes, and worked with an external vendor to build an online network for care partners of people with serious illness. The network has grown to nearly 200 members who have generated nearly 3,500 posts over a 20-month period. Most members report the network helps them find meaning and purpose by supporting others, and most were satisfied with the support and information received (O’Donnell et al, 2022). The most frequently used discussion threads focus on overall well-being, emotions, and sources of joy and hope. The network is poised to grow further but will require further investment to grow beyond its pilot phase, owing to the need for additional network facilitation and a systematic approach to marketing and referrals.

Collaborative learning network

A key accelerator to the CLHS is the collaborative learning network that is designed to engage frontline teams, patients, and care partners in using quality improvement methods to drive improvement. Teams learn, measure, and share generalizable knowledge, creating a cycle of peer-to-peer learning. To date, four learning sessions have been held within DCC, focusing on topics identified by an interdisciplinary group of clinicians, patients, scientists, and health system leaders. Learning session attendance ranged from 124 to 286 (average of 182 attendees/session) over four half-day sessions; more than one-third of attendees at the most recent session were clinicians. Network leaders built and deployed a web-based quality improvement curriculum to provide the tools and content to support frontline teams to efficiently learn and apply new knowledge to drive continuous quality improvement. Challenges faced by the collaborative learning network include: 1) the inability to convene stakeholders in person for biannual learning sessions, and 2) forgoing quality improvement and other “all-teach, all-learn” seminars between the biannual sessions due to limited staff bandwidth during the pandemic. Despite these challenges, the collaborative learning network provided an effective forum to bring together the DCC community during the pandemic. Most attendees rated learning network sessions as excellent.

Data, measurement, and scholarship

Early during the Promise partnership pilot, we convened a Delphi panel of clinicians, scientists, health system leaders, patients, care partners, and community members to identify a balanced set of measures that could monitor the overall functioning of the CLHS. The panel identified eight domains including: functional health and quality of life; experience of care; cost and resource utilization; clinical outcomes; team well-being and joy-in-work; learning culture and community; scholarly engagement and productivity; and diversity, equity, inclusion, and belonging (Van Citters et al, 2022). Ongoing work has refined the measure set and integrated a subset of measures into existing DCC and health system measurement reporting systems. The measures have served to guide research studies and quality improvement initiatives, have supported the development of strategic initiatives, and continue to provide feedback to system leaders on overall progress on metrics that matter.

Lessons Learned: Insights

Pilot testing the CLHS has generated a number of lessons learned, based on insights of clinical champions and patient advisors. Table 3 identifies facilitators and barriers to implementation of the CLHS.

Table 3:

Lessons learned regarding facilitators and barriers to implementation of a CLHS

	Facilitators	Barriers
Collective expertise	• Co-design is central to the CLHS model. “Shared expertise” was echoed in interdisciplinary teamwork, where members of implementation teams (including patients and care partners) share expertise on how to improve a service. • Empowerment of interdisciplinary team members spreads change. Support for non-physician team members to complete projects can spread interest and buy-in for similar projects. • Value of people learning from and supporting one another. Learning network meetings recognized teams’ successes and fostered CLHS engagement. • Trust is important to effective teamwork, especially when new learning creates vulnerability. • Coproduction can improve experiences of care team members. Care team members feel better when they can address patients’ concerns.	• Priorities may differ across stakeholders. Clinicians tend to focus on medical issues. Patients not only care about having symptoms addressed, but also want care teams to put symptoms into the context of their broader life. Health system leaders often prioritize population health, service utilization, and expenditures.
Resources & support	• Organization and project management is critical. Supporting structures of the CLHS helped working groups and clinicians to change care processes. • Embed quality improvement skills/tools within working groups. Having an expert in QI partner with interdisciplinary front-line teams, including patients and care partners, is an effective way to test and refine improvements. Process improvement tools, such as failure modes and effects analysis, can help when rolling out changes to identify potential failure points.	• Priorities of leaders may change over time. Activities within the CLHS need to be seen as the “way we do things” and not a stand-alone, siloed project. Engaging existing cancer center structures (i.e., COGS) into the structure of QI requires providing resources (QI coaching, data and analytics) to COGs. • Meaningful transformation is required. Projects must align with what matters to clinical teams and patients. A sustainable CLHS requires changing how care is delivered. • Competing demands impact the ability to participate. Resources and time to co-design and test new improvements and innovations may not be adequate.
Data	• Make data accessible and interpretable. Data analysts can help frontline teams interpret large amounts of data and use it to inform meaningful changes. • Sharing stories creates buy-in from clinical staff and leaders. Inviting patients and care partners to share stories and reflect on their experiences creates buy-in from clinical staff who may have been early resisters. Translating lived experiences into meaningful change is important to clinical staff, patients, and care partners.	• Timely and actionable data are needed to support change. Ready access to data is important to provide feedback on uptake and impact of innovations. What (and how) the system is already measuring is often not aligned with what is important to the work. • Measures that matter. Engagement with clinicians would be enhanced by showing evidence of linkage between process and clinical outcomes of patients to make the case for why a process intervention is prioritized.

DISCUSSION

Academic health systems serve a large share of the US population and face daunting challenges to achieve their tripartite mission involving service, teaching, and research, while sustaining the well-being of their workforce. We believe that academic health systems will need to innovate – to transform themselves – if they are going to achieve their mission in today’s world, which is moving relentlessly towards value-based payment, public reporting and accountability for quality and costs, and achieving equity in service access and health outcomes.

In developing its strategic plan for a successful future, Dartmouth Health drew on longstanding values of patient- and family-centered care and academic and community partnerships, making a bold promise to those it serves: “Together, we bring the full power of our collective expertise to provide the best possible care to our patients, our people, and our communities.” In taking the first steps to deliver on that promise, the health system, with its partners at TDI and DCC, piloted a CLHS for oncology that was designed to generate better outcomes and experiences for people with serious illness and their health care teams, improve quality of care, and promote learning and research. While work remains before a fully-implemented CLHS is in place, this case study demonstrates the feasibility of creating a CLHS that can be continuously improved to meet an ever-changing environment. Progress in each domain of the CLHS, despite pressures of a pandemic, speaks to the power of the CLHS model, and builds on other literature describing organizational changes that support transitioning to a learning health system (Davis et al, 2020; Easterling et al, 2022; Enticott et al, 2020).

The path towards establishing a viable CLHS began with several critical building blocks:

• Organizational Leadership: A strategic mandate to start the transformation at DH through a pilot in oncology sponsored by the Dartmouth Hitchcock Medical Center Executive Vice President, the DCC Director, and TDI Director.

• Previously Tested Theory: A framework for implementing a CLHS based on successful forerunners (i.e., cystic fibrosis (Marshall & Nelson, 2014), inflammatory bowel disease (Crandall et al, 2011)).

• Implementation Team: A dedicated and effective implementation support and operations team with quality improvement experts and staff from DH, DCC, and TDI.

• Promising Initiatives: Initial initiatives designed to enable the ongoing work of a CLHS by integrating tools necessary for coproduction of care and a system to measure its impact.

• Clinical Champions: Respected clinical champions motivated frontline clinical oncology teams.

• Measurement & Feedback: A commitment to using data to track and improve health outcomes, patient and staff experiences, healthcare value, and research and scholarly activity.

• Flexibility and Patience: Continuous adaptation to address the effects of the pandemic on our workforce, community, and patient population.

Moving beyond these building blocks, a number of facilitators support the CLHS. First, the power of sharing collective expertise is at the heart of the CLHS. Everyone involved in healthcare has and must be given the opportunity to share their own expertise, and in turn stands to benefit from the expertise of others. This pilot project brought together the expertise of patients and care partners (values, priorities, and care experiences) with the expertise of interdisciplinary clinicians (deep knowledge of evidence and their own practice experiences), administrators (who understand operations and finances), and scientists (at the edge of new knowledge) to co-design system-level interventions and individual care plans. While it was critical that each COG designed their own implementation to fit their particular context, all came together regularly in learning collaboratives designed to promote multi-directional learning.

Second, coproduction has the potential to improve outcomes for everyone involved, but takes dedicated time, resources, and support. Everyone may benefit when the most appropriate, evidence-based care is delivered to patients who wish to receive it. However, hearing the voices of multiple stake-holders, and working through conflicting values, priorities, and contexts can take longer, especially during the co-design phase. In the long run, co-design and coproduction save time of reworking solutions that do not meet the needs of all stakeholders.

Third, data measurement and feedback are essential to the CLHS and technology must be embraced. Teams need to see data regularly that reflects progress toward goals. Measures should be balanced and reflect what matters most to stakeholders, not simply what is easiest to measure or already available. While daunting, integrating new technology into the EHR is more durable than creating parallel processes. Data analysts and specialists in data access and presentation are key within a CLHS team.

Finally, creating a CLHS is not a project, it’s a transformation. Sustainability requires changing how care is delivered—replacing the old system with a new way of working, learning new ways of doing things, and redesigning systems that are inefficient but familiar. This work is stressful and creates vulnerability and the people doing it need to be supported during this period of transition. Establishing a CLHS as an add-on to the current healthcare delivery system will add burdens and dilute the benefits. Successful implementation of a CLHS can reveal the limitations of many other deeply entrenched systems (e.g. quality measurement) that are not well-aligned with the goals of the CLHS. Resistance to changing these old systems must be overcome.

Future Actions

The aim to implement a major strategic program to transform the DH academic regional health system started with DH’s oncology system as a three-year pilot test. The goal was to learn if adopting a CLHS framework to begin systemwide transformation could be successful in this region and health system. The senior leaders who sponsored the pilot and the professionals, patients, and care partners that participated in the planning, organizing, and “starter” workgroups believe the pilot CLHS work in oncology has been a success, although several challenges and shortcomings have surfaced and will need to be addressed as we continue to learn and improve the system.

Future actions will focus on critical decisions and activities including: (1) deciding what population(s) should be selected next; (2) improving the capture, display and use of multiple, critical data streams to support learning, improvement, research, and care; (3) enhancing existing patient-centered registries by including patient-generated data, genetic information, and health service utilization data to complement existing clinical data on patient characteristics, treatments, and outcomes; (4) determining what CLHS adaptations are necessary in the context of workforce shortages; and (5) and scaling initiatives that show positive results in improving patient experience and outcomes.

Additional research is needed to address several questions: (1) Are the CLHS components adequate to support a functional learning health system? (2) What is the impact of the CLHS on patients, care partners, healthcare teams, and other stakeholders? (3) What facilitators and barriers influence the ability to successfully implement initiatives and achieve desired outcomes, and how do these factors differ across initiatives? (4) What is necessary to expand the CLHS to launch new learning collaboratives, such as one focused on clinical and translational science to accelerate the transition of evidence into practice to benefit patients, care partners, and clinicians?

Conclusion

The world is changing, reshaped by a global pandemic, economic inequities, and workforce shortages, as well as tremendous scientific breakthroughs and cutting edge technologies. Drawing on the expertise of everyone – clinicians, scientists, educators, administrators, patients, care partners, and communities – will be critical to achieving the best health possible for our communities, and will require meaningful transformation of how we design and deliver healthcare. The CLHS model is designed to facilitate the continuous incorporation of expertise from all sources to power an environment of continuous learning and improvement that drives better health outcomes, better healthcare value, and new knowledge. The Promise Partnership CLHS in oncology described here represents a potential pathway for academic health systems to undergo the kind of meaningful transformation that will be necessary to achieve their mission of delivering high-quality, high-value care to the communities and populations they serve.