Abstract
Introduction
Coproduction learning health system models clearly define the use of clinical and patient-reported data for system learning and quality improvement, but less is known about how to document formative learning about coproduction value creation over the course of a quality improvement initiative. The authors aimed to 1) assess the feasibility, utility, and acceptability of novel self-assessment tools for coproduction value creation and 2) identify domains of coproduction value creation.
Methods
The authors conducted 4 focus groups with quality improvement teams from 4 health systems in the United States and Sweden between June 2021 and September 2023. A single analyst coded transcripts and proposed themes, with investigator triangulation validating results.
Results
Participants found the self-assessment tools acceptable and useful. The improvement passport was seen as more feasible for routine use than the full self-assessment guide. Peer learning within the community of practice, diverse multidisciplinary improvement teams, and leadership support facilitated teams’ work. Domains of coproduction value creation included communication, self-efficacy, interconnectedness, direct and indirect costs of health care utilization, health professional experience, and access to the right care.
Discussion
Peer learning and camaraderie within the community of practice maintained momentum among participating teams during a challenging time of limited resources and mounting responsibilities in health care settings, suggesting enhanced resiliency through approaching difficult tasks in community.
Conclusion
The authors identified themes of coproduction value creation and drivers of engagement. Future research will draw on the measurement domains established in this study to inform the development of measures of coproduction value creation. Those measures could then be incorporated into the data-rich environments of coproduction learning health systems to enhance focus on value from service user and professional perspectives.
Keywords: assessment, doctor-patient relationship, quality, whole person
Introduction
Coproduction is increasingly recognized as a core component of health care service.1–4 However, effective coproduction that improves the value of care for all involved parties, particularly service users, can be difficult to achieve in health care practice. Competing priorities in clinical settings limit the time spent between health professionals and people seeking health services (service users). Clinical training sometimes neglects person-centered goal-setting in favor of established clinical targets. Additionally, it can be difficult to gauge whether services are optimally coproduced in any given setting.
Valid, reliable, and feasible assessment tools can support health care settings to achieve higher-value services through coproduction. The CO-VALUE study seeks to facilitate assessment through a coproduction model of value creation. This model comprises the following: 1) value chain, involving standard processes; 2) value shop, involving customized solutions; 3) facilitated value network, involving groups of people facing similar challenges5,6; 4) interrelation and travel between these value architectures; and 5) leadership.7
Value in health care has been conceptualized as “bringing the right people together with the right information, with the right technology, in the right way, and at the right time, in response to a patient’s needs.”2,5 Recent scholarship is clear about the roles of feedback and feedforward data in promoting value within a coproduction learning health system.8,9 However, methods to document the formative (ie, applied and developmental) learning process within a coproduction learning health system are less established in the literature. To meet this need for methods and resources that facilitate learning and action toward coproduction value creation, phase 1 of the CO-VALUE study focused on developing self-assessment tools for coproduction quality improvement (QI) teams that prompt reflection, learning, and action.10 These tools included a full self-assessment guide supporting teams’ self-evaluation of their competencies, knowledge, and skills related to improving value through coproduction (Supplemental Appendix 1) and an improvement passport supporting teams’ self-assessment of progress on their specific coproduction QI projects (Supplemental Appendix 2).
In this report on phase 2 of the CO-VALUE study, the authors aim to 1) assess the feasibility, utility, and acceptability of these novel self-assessment tools for coproduction value creation based on QI projects conducted in 4 participating health systems and 2) identify domains of coproduction value creation.
Methods
Detailed methods for the CO-VALUE study are published elsewhere.7 Study procedures were reviewed and approved by the Dartmouth Health Institutional Review Board, and participants provided written informed consent at the beginning of the study.
Self-assessment tools
The full self-assessment guide was designed for project planning and design. In response to preliminary user testing, the authors iteratively refined the self-assessment guide by focusing on specific constructs related to coproduction value creation and eliminating superfluous items. Additionally, the authors created the brief, applied improvement passport to support the day-to-day work and learning of coproduction QI teams.
Procedure
Over this second phase of the study (2021-2023), the authors conducted 4 focus groups with health systems carrying out QI projects focused on improving value through coproduction of health services (Table 1). Focus groups were facilitated by 2 team members. Due to varied COVID-19 travel restrictions over the course of the study, focus groups were held virtually using Microsoft Teams videoconferencing software (3 focus groups) or in person at Dartmouth Hitchcock Health, Lebanon, New Hampshire (1 focus group).
Table 1:
Quality improvement project descriptions
| Health System | Location | Quality Improvement Project Focus |
|---|---|---|
| Site 1 | Sweden | Increasing engagement in care among people with multiple sclerosis |
| Site 2 | United States | Increasing engagement in care among people with multiple sclerosis through group educational opportunities |
| Site 3 | United States | Improving nutrition services for older adult patients with multimorbidity through home visits |
| Site 4 | United States | Delivering person-centered, “whole health” care to people with chronic conditions engaged in domiciliary services |
Data collection and analysis
Focus group guides are presented in Supplemental Appendix 3. Each focus group was audio recorded and transcribed by a member of the research team (BCJ). The authors conducted thematic analysis,11 with a single analyst applying codes derived from the study aims to each focus group transcript and all analysts (RCF, BCJ, JA) reading the focus group transcripts in detail. The analysts met to identify and discuss emerging patterns after each focus group transcript was coded. After all focus groups were conducted, coded, and discussed, the full study team then met to discuss and reach consensus on themes and validate findings through investigator triangulation.
Results
Focus group participants
The 8 participants in focus group 1 (June 2021) included 4 clinicians, 2 health system administrators, 1 service user, and 1 improvement scientist. The 8 participants in focus group 2 (February 2022) included 4 clinicians, 2 improvement scientists, 1 service user, and 1 health system administrator. The 6 participants in focus group 3 (February 2023) included 3 clinicians, 2 improvement scientists, and 1 service user. The 10 participants in focus group 4 (September 2023) included 4 clinicians, 3 improvement scientists, 2 health system administrators, and 1 service user.
Feasibility, utility, and acceptability of self-assessment process
The self-assessment process was generally acceptable, useful, and feasible to all participants, particularly when using the improvement passport (Table 2). Most agreed that the passport was a more feasible approach to capturing and distilling information on their progress than the full self-assessment guide. One participant noted, “[the passport is] an easier way to narrow it down and to get the sense of it” (site 1). Another described the passport as “a good frame of reference in identifying our clinical scenario/frame, and the goal for the project and goals of improvement” (site 2).
Table 2:
Feasibility, utility, and acceptability of self-assessment process
| Theme | Illustrative Quote |
|---|---|
| Passport was more feasible to use than the full self-assessment guide. | “It’s [the passport] an easier way to narrow it down and to get the sense of it.” |
| Peer learning and camaraderie supported progress in coproduction quality improvement projects. | “So it has been kind of a space to breathe, to share your own experiences and get inspiration from other groups, and when you are tired and feel kind of hopeless, then you hear that you get inspiration from other groups and you get the power you need to move on. That has been a very important issue, especially during COVID when everything was so difficult.” |
| Support from institutional leadership facilitated coproduction quality improvement work. | “Our medical center director is part of our team so she has immediate communication as far as what’s going on, and that’s been a huge plus for us because she’s very excited about the project.” |
Comparatively, many found the full self-assessment guide too long. One participant explained that “it is overwhelming . . . we’ll get through like three parts of one question in our 30 min talk and I—so we don’t feel much accomplishment if that makes sense because there’s still so much left to do” (site 3).
A key facilitator of QI teams’ progress using the self-assessment guide was peer learning within the community of practice. The QI teams found it helpful to share progress and setbacks with peers from other institutions and felt that “this group creates a psychologically safe environment” to do so (site 4). The inclusion of diverse perspectives within QI teams further facilitated their work. Referring to use of the self-assessment guide within a multidisciplinary QI team, 1 participant explained that “it’s interesting because the answer is so obvious [to me], but when you actually write it down there’s so many different perspectives from all the team members that it’s not as obvious as you thought, and so it actually helps you eliminate assumptions when you're forced to write something down that seems like an easy answer” (site 3). Interaction with people with lived experience of the context in which the team worked was particularly helpful. Sites found that they “had to coproduce it with the staff and the [service users] on the program, and we had a lot of input from [service users] on the project. But, it is a coproduction project. To create lasting change, it has to be coproduced with those folks” (site 4).
Teams also agreed that leadership support and communication facilitated their work with the self-assessment guide. Support from health system leadership, both material support and encouragement, “has energized us and basically shaped this work” (site 4). Where teams lacked explicit leadership support and resources, progress slowed. “Open, clear communication” within teams and between teams and the settings in which they worked was needed for successful execution of teams’ QI projects (site 4). However, the time required for meaningful coproduction through clear communication was also sometimes described as a barrier to immediate progress.
In the resource-constrained health care environment surrounding the COVID-19 pandemic, including substantial staffing shortages paired with increased demand, participants found the CO-VALUE community of practice particularly engaging. One participant explained that “we’ve developed a camaraderie and we’re very supportive of each other” (site 4). The meaningful nature of coproduction-oriented work also allowed them to remain committed to the QI project in the face of many competing demands: “Getting into this coproduction team, it made me realize that there was something we were missing, an unmet need, and I think that brings meaning and brought us here” (site 4).
Domains of coproduction value creation
Participants identified 6 domains of coproduction value creation (detailed in Table 3), including communication, self-efficacy, interconnectedness, direct and indirect costs of health care utilization, health professional experience, and access to the right care.
Table 3:
Domains of coproduction value creation
| Domain | Description | Illustrative Quote |
|---|---|---|
| Communication | Bidirectional information exchange that provides service users and professionals clarity on priorities, goals, and next steps | “Patients are partners, and part of that process is focused on helping them figure out what they want in their lives and some steps they can take to get there, and those are skills that are learned.” |
| Self-efficacy | Service users’ belief in their own ability to execute actions needed to achieve health and well-being goals; sometimes involved health professionals using motivational interviewing to support self-efficacy in coproduction | Use of motivational interviewing “to find out what the patient’s interest is” and “teach them the way we are thinking about coproduction.” |
| Interconnectedness | Strength of service users’ integration and/or connection within their social networks; peer learning among both professionals and service users | “Wondering about whether we can assess strength of network as opposed to the patient feeling more connected to their health care team, and potentially, maybe their social networks, too, as being an outcome measure? . . . Strength of network as being linked to improved health care outcomes.” |
| Health care utilization/direct and indirect costs | Costs of health care services in relation to their value for improving health and well-being | “So if I do this well and it results in fewer hospital stays or fewer emergency room visits, then somebody’s going to listen.” |
| Health professional experience | Levels of staff satisfaction, burnout, and wellness | “I was just thinking about staff burnout or like the impact on employee wellness from the provider side.” |
| Access to the right care | Movement between chain, shop, and network configurations as appropriate to optimize value of the service; ensuring the right team members are involved, including both professionals and service users, family members, and other caregivers | “The value architectures are not just one solution but we need more services based on network, shop, and chain. It’s not either/or but we need to work with all of them. It’s in earlier QI that was one solution that let us be happy, but now we realize that we need a broader spectrum of services.” |
QI, quality improvement.
Discussion
Key findings
Participants found the self-assessment tools acceptable and useful. Feasibility was improved when using the improvement passport format over the full self-assessment guide. Peer learning, multidisciplinary study teams with diverse perspectives, health system leadership support, and active and clear communication facilitated teams’ progress in QI initiatives focused on coproduction value creation. The authors identified 6 domains of coproduction value creation from the focus groups, including communication, self-efficacy, interconnectedness, health care utilization, health professional experience, and access to the right care.
Results in context
This study builds on expansive prior research describing the infrastructure of learning health systems and the role of data and measurement in facilitating learning and promoting value.8,12–15 Specifically, the authors pilot tested a new method for documenting formative learning and defined value creation measurement domains for future measure development and eventual use within coproduction learning health systems.
Participating QI teams recognized the respective roles of Stabell and Fjeldstad’s value chain, shop, and network configurations in contributing to high-value health care services.6 Participants also emphasized the importance of ready travel between value chain, shop, and network configurations, recognizing that different scenarios and contexts require different strategies and actors to achieve optimal health service design. This was especially salient as teams recognized that much of health is achieved outside of health systems, emphasizing the critical role of a value network configuration that invites contributions from within and outside of health care in connecting service users to required information and resources.5
Peer learning within and between the participating quality improvement teams was a key facilitator of progress in the face of extremely limited resources. This finding aligns with Wenger’s communities of practice model that argues that learning is achieved through “a social process of negotiating competence in a domain over time.”16 Peer learning is also an integral component of the positive deviance approach, in which the practices of high performers are studied and disseminated to peer organizations.17
Participants cited peer interaction within the community of practice as a facilitator of their continued commitment and involvement in the face of resource scarcity. It energized teams and facilitated their QI work despite ever-growing responsibilities within their professional and personal lives. However, although the impact of the community was substantial, there was a limit at which resources, particularly staffing, were required to make continued progress, which sometimes stalled in their absence.
Strengths and limitations
A key strength of this study was its inclusion of QI teams from diverse health systems and clinical settings working on varied projects to advance health care quality and value through coproduction, providing a diverse range of insights from varied contexts. The study’s timing coincided with the COVID-19 pandemic that limited face-to-face interaction between participating QI teams; however, a nimble telecommunications strategy facilitated interaction and partnership throughout the study period. A single analyst coded focus group transcripts and proposed initial themes. This analytic approach was paired with investigator triangulation and detailed discussion between analysts who were familiar with the focus group content to reduce the likelihood of misinterpretation and bias.
Conclusion
The authors’ results identified themes of value creation for coproduction of health care service and drivers of engagement in a community of practice. Findings suggest that relational and humanistic elements central to coproduction are value-creating elements that are important inclusions in the developing future of learning health systems and are currently underrepresented in comparison to technical and structural aspects in the development of learning health systems.18 This study’s approach to documenting and enhancing formative learning about coproduction provides generalizable tools to support diverse quality improvement efforts. Future research will draw on the measurement domains established in this study to inform the development of measures of coproduction value creation. These measures can then be incorporated into the data-rich environments of coproduction learning health systems to enhance focus on value from service user and professional perspectives.
Supplementary Material
online supplementary file 1
online supplementary file 2
online supplementary file 3
Acknowledgments
The authors are grateful to the people who contributed expertise in their lived experiences as patients through their participation as active members of sites' quality improvement project teams.
Footnotes
Author Contributions: Rachel C Forcino, PhD, MSc: methodology, formal analysis, investigation, writing – original draft. Bruce C Jobse, MPH: methodology, formal analysis, investigation, writing – review & editing. Jabeen Ahmad, PhD, MPH: formal analysis, writing – review & editing. Brant J Oliver, PhD, MS, MPH, FNP-BC, PMHNP-BC: conceptualization, methodology, investigation, writing – review & editing, supervision
Conflict of Interests: None declared
Funding: None declared
Data-Sharing Statement: Deidentified data are available upon request. Readers may contact the corresponding author to request underlying data.
References
- 1. Masterson D, Areskoug Josefsson K, Robert G, Nylander E, Kjellström S. Mapping definitions of co-production and co-design in health and social care: a systematic scoping review providing lessons for the future. Health Expect. 2022;25(3):902–913. 10.1111/hex.13470 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Batalden P. Getting more health from healthcare: quality improvement must acknowledge patient coproduction—an essay by Paul Batalden. BMJ. 2018;362. 10.1136/bmj.k3617 [DOI] [Google Scholar]
- 3. Baim-Lance A, Tietz D, Lever H, Swart M, Agins B. Everyday and unavoidable coproduction: exploring patient participation in the delivery of healthcare services. Sociol Health Illn. 2019;41(1):128–142. 10.1111/1467-9566.12801 [DOI] [PubMed] [Google Scholar]
- 4. Elwyn G, Nelson E, Hager A, Price A. Coproduction: when users define quality. BMJ Qual Saf. 2020;29(9):711–716. 10.1136/bmjqs-2019-009830 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Fjeldstad ØD, Johnson JK, Margolis PA, Seid M, Höglund P, Batalden PB. Networked health care: rethinking value creation in learning health care systems. Learn Health Syst. 2020;4(2). 10.1002/lrh2.10212 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Stabell CB, Fjeldstad ØD. Configuring value for competitive advantage: on chains, shops, and networks. Strat Mgmt J. 1998;19(5):413–437. [DOI] [Google Scholar]
- 7. Oliver BJ, Batalden PB, DiMilia PR, et al. COproduction VALUE creation in healthcare service (CO-VALUE): an international multicentre protocol to describe the application of a model of VALUE creation for use in systems of coproduced healthcare services and to evaluate the initial feasibility, utility and acceptability of associated system-level VALUE creation assessment approaches. BMJ Open. 2020;10(10). 10.1136/bmjopen-2020-037578 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Nelson EC, Dixon-Woods M, Batalden PB, et al. Patient focused registries can improve health, care, and science. BMJ. 2016;354. 10.1136/bmj.i3319 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Oliver BJ, Nelson EC, Kerrigan CL. Turning feed-forward and feedback processes on patient-reported data into intelligent action and informed decision-making: case studies and principles. Med Care. 2019;57:S31–S37. 10.1097/MLR.0000000000001088 [DOI] [PubMed] [Google Scholar]
- 10. Oliver BJ, Forcino RC, Batalden PB. Initial development of a self-assessment approach for coproduction value creation by an international community of practice. Int J Qual Health Care. 2021;33(suppl 2):ii48–ii54. 10.1093/intqhc/mzab077 [DOI] [PubMed] [Google Scholar]
- 11. Fife ST, Gossner JD. Deductive qualitative analysis: evaluating, expanding, and refining theory. Int J Qual Methods. 2024;23. 10.1177/16094069241244856 [DOI] [Google Scholar]
- 12. Kamal AH, Kirkland KB, Meier DE, Morgan TS, Nelson EC, Pantilat SZ. A person-centered, registry-based learning health system for palliative care: a path to coproducing better outcomes, experience, value, and science. J Palliat Med. 2018;21(S2):S61–S67. 10.1089/jpm.2017.0354 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Gremyr A, Andersson Gäre B, Thor J, Elwyn G, Batalden P, Andersson A-C. The role of co-production in learning health systems. Int J Qual Health Care. 2021;33(suppl 2):ii26–ii32. 10.1093/intqhc/mzab072 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. McGinnis JM, Fineberg HV, Dzau VJ. Advancing the learning health system. N Engl J Med. 2021;385(1):1–5. 10.1056/NEJMp2103872 [DOI] [PubMed] [Google Scholar]
- 15. Friedman C, Rubin J, Brown J, et al. Toward a science of learning systems: a research agenda for the high-functioning learning health system. J Am Med Inform Assoc. 2015;22(1):43–50. 10.1136/amiajnl-2014-002977 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Farnsworth V, Kleanthous I, Wenger-Trayner E. Communities of practice as a social theory of learning: a conversation with Etienne Wenger. Br J Educ Stud. 2016;64(2):139–160. 10.1080/00071005.2015.1133799 [DOI] [Google Scholar]
- 17. Baxter R, Taylor N, Kellar I, Lawton R. What methods are used to apply positive deviance within healthcare organisations? A systematic review. BMJ Qual Saf. 2016;25(3):190–201. 10.1136/bmjqs-2015-004386 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Pomare C, Mahmoud Z, Vedovi A, et al. Learning health systems: a review of key topic areas and bibliometric trends. Learn Health Syst. 2021;6(1). 10.1002/lrh2.10265 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
online supplementary file 1
online supplementary file 2
online supplementary file 3