Abstract
Background
Engaging patients as partners can influence research, with rewards and deterrents. The authors are researchers and patient co-investigators who collaborated on a comparative effectiveness, randomized controlled study of a structured quality improvement (QI) process to improve behavioral health and primary care integration for people managing multiple chronic conditions (MCC). Patient co-investigators responded to a gap in available resources to support study clinics in partnering with their own patients in QI and co-created the Patient Partner Guide (PPG).
Objective
Describe the development of the PPG, its use by clinics undertaking the QI project, and research team partnerships.
Design
Observational report of study intervention component.
Participants
Diverse patients and family members managing MCC and members of their primary care clinics.
Intervention
The PPG component of the study intervention is a five-step workbook providing practical tools and resources to sustain partnerships across clinic QI team members, including patient partners. The process of developing the PPG relied on relationship-building tools that were iteratively assessed, practiced, improved, and incorporated into the PPG under the leadership of patient co-investigators.
Main Measures
Observations related to PPG use and patient partner inclusion in clinic QI; impact on the research team.
Key Results
Of 20 clinics, 6 engaged patients as full partners on QI teams. Clinics found resistance in partnering and challenges in using the PPG but valued the material and their partners’ contributions. Similarly, engagement of patient co-investigators in research brought a shift in perspective to team members. The PPG is available and was adapted for use by research teams.
Conclusions
Engagement of patients and other stakeholders in research can be transformative and productive. Building relationships through meaningful work benefits others, and in turn, the research process. This approach can enhance clinical care QI and may result in substantial contributions to the conduct of research.
Clinical Trial Registration
Supplementary Information
The online version contains supplementary material available at 10.1007/s11606-021-07308-0.
KEY WORDS: Stakeholder participation, Patient engagement, Multiple chronic conditions, Behavioral health integration, Research team development
INTRODUCTION
Engaging patients and stakeholders in research; a practice formalized in 2010 by the Patient-Centered Outcomes Research Institute (PCORI);1 offers multiple benefits such as greater relevance in research topics;1, 2 improved credibility of proposals and research instruments,2 increased patient enrollment; decreased attrition;2, 3 and faster approval of evidence by regulators.2 However, deterrents to engagement are real and may include increased time and costs of the research team,2, 3 tension related to poor clarity about roles,1 unmet expectations,2 lack of confidence,2 the risk of tokenism,2, 3 and the burden of responsibility.2, 3 Acquiring these benefits and managing the deterrents require a shared goal, a thoughtful structure, clear expectations,3 reciprocal relationships,4 and an open and trusting working atmosphere.1, 5 Achieving these prerequisites depends in part on how researchers engage stakeholders, particularly regarding the use of collaborative, rather than consultive, methods that correspond to greater patient influence.6
The PCORI Engagement Rubric1 offers principles for engaging patients throughout the research process from the generation of research questions to the dissemination of results. For example, patients in PCORI-funded research have assisted in developing patient-centered study websites and selecting patient-reported outcome measures.7 A key role for patient partners is identifying the topics, measures, procedures, outcomes, and communication strategies most relevant to them.8 Engagement may be influenced by patient partners’ motivations to participate in research: dissatisfaction with previous or current healthcare experiences,9 strong opinions on improvement for the benefit of others,10 beliefs in the value of the health intervention,10 or to fulfill their own desire to learn.10
The authors are collaborators on a large, pragmatic, comparative effectiveness randomized controlled trial designed to improve care and outcomes for people managing multiple chronic conditions (MCC). The study tested whether a structured quality improvement (QI) process undertaken at primary care practices seeking to integrate behavioral health and primary care (IBH-PC) would produce superior patient health and clinical outcomes when compared to practices undergoing no structured QI process.11 The research team included three patient co-investigators (co-authors JL, PR, and DP) from the inception of the study’s aims in 2015 through dissemination of its results in 2021. Each patient co-investigator had ongoing experiences with diverse presentations of MCCs and responded to invitations from providers to participate. The research team used a “study-focused” framework for involving patients in research,12 practicing thoughtful engagement processes tied to study activities to support collaborative work. Patient co-investigators, considered study key personnel, influenced study activities across three domains of engagement,1 including the following: (1) planning the study, by identifying meaningful patient-reported outcomes, inclusion criteria, and patient-centeredness themes and tools, and developing the Patient Partner Guide (PPG, described below); (2) conducting the study, by developing a common glossary of terms (Table 1 is an example for this report), contributing to the online learning community, and participating actively on the research team; and (3) disseminating results, by planning and delivering presentations, publications, and stakeholder convenings; creating peer-to-peer guidance for patients and clinical teams working together (Appendix: “Would you like to feel better?”); and publishing the PPG on the Internet.
Table 1.
Glossary of Terms and Abbreviations
Behavioral health | Mental health, health behavior, and lifestyle that affect mood, stress, pain, sleep, medications, alcohol and substance use, weight, nutrition, and physical fitness |
IBH-PC | Integrating Behavioral Health and Primary Care research study testing an intervention to create a collaborative system of care with behavioral health providers as full members of primary care health teams |
MCC | Multiple chronic conditions, such as combinations of medical, mental, and behavioral health diagnoses |
PCORI | Patient-Centered Outcomes Research Institute |
PROMIS-29 | 29 items of the Patient-Reported Outcomes Measurement Information System, a survey completed by patients |
PPG | Patient Partner Guide, a structured method for engaging patients in primary care quality improvement |
QI | Quality improvement process for improving health care delivery |
TIDieR | Format for reporting research interventions: Template for Intervention Description and Replication |
This observational report highlights one of many productive patient partner contributions to the study: the development and use of the PPG as part of the intervention. Early in the design of the structured QI process, patient co-investigators identified the need for clinics to include their own patients in QI activities, echoing published recommendations.13–20 The goal of the PPG was to help clinics gain essential, patient-based insight to achieve patient-centered integrated BH services for patients managing MCC. Our patient co-investigators led the design of the PPG, bringing together diverse research team members. This report uses the Template for Intervention Description and Replication (TIDieR)21 format to describe the development of the PPG and its use by clinics undertaking the QI project.
METHOD
PPG Development
Following the PCORI engagement rubric,1 also included in the PPG, co-investigators started by creating an Engagement Team, co-led by a patient partner (JL) and project director (CvE). The Engagement Team included additional patient co-investigators (PR, DP) and worked with stakeholders with personal or professional MCC experiences to learn about engagement. This team drew from demonstrations of healthcare organizations that engaged stakeholders in advisory groups, shared governance, and internal quality improvement teams.20, 22–24 The Engagement Team was guided by sources that emphasized shared leadership as the peak of a graduated scale of partnerships built on a foundation of co-learning, trust, transparency, and honesty.14, 25–27
Engagement Team members assessed, practiced, and improved on methods to engage as multi-stakeholder partners, creating and using an approach to identify common values and goals. Patient co-investigators and stakeholders provided guidance on what matters most when including diverse stakeholders with MCC in QI efforts. Throughout the PPG’s development, team members created affinity diagrams,28 shared lived experiences, and used an appreciative response process to receive and provide feedback (see Appendix: “Offering Feedback”). These processes became part of the PPG to create a healthy, equitable, inclusive, and cooperative environment for all members of clinic QI teams.
In developing the PPG outline, the Engagement Team benefited from existing resources from the Robert Wood Johnson Foundation initiative, “Aligning Forces for Quality,” and others that explore patient involvement18, 29–35 as well as personal conversations with their authors. The Engagement Team heard the value of patience and persistence from these sources. From the study’s Stakeholder Advisory Group (SAG) members with experiences in MCC, the team learned the importance of positivity, appreciation, kindness, compassion, and, above all, the high value in the patient’s lived experience—the expertise of the patient. Engagement Team members revised each step of the PPG in cycles, engaging stakeholders in writing, review, and revising again. In its final presentation, the PPG speaks to two audiences: (1) clinic leadership and clinicians, to support the benefits and challenges of patient engagement in QI and (2) clinic patients and family members, to be welcomed and oriented to a QI team in simple, appreciative language.
PPG Content
The PPG, one component of the IBH-PC intervention, is a printable online document structured as a workbook. It includes five steps that can be followed sequentially or in the order most relevant to the practice (Fig. 1). PPG materials were specifically intended for use with patients who are managing MCC, providing a QI project Welcome Package and Overview to focus on improving care for patients who experience chronic, complex, and multiple health conditions. The PPG includes testimonials, practical guidance, patient-centered scripts and messaging, templates for contracts, spreadsheets to document progress, handbooks, instructions on compensation and confidentiality, and images to enhance the learning experience while sustaining partnerships with all team members. The PPG is framed by simple mnemonics (see Appendix: “The 6Ss” and “6Rs of a Clinical Team”) for successful teamwork and to reinforce the reciprocal relationships needed for equitable partnerships.
Figure 1.
Overview of the patient partner guide.
PPG Delivery
The PPG was located on a private website available to QI teams to download step by step as needed while forming team relationships and following the QI project. QI teams included the role of “patient partner liaison,” a member from the clinic to assist in outreach and support, for example, by flexibly scheduling QI team time to include patient partners.
The PPG encouraged QI teams to adapt the materials to their local environment. In fact, templates, scripts, and handbooks were specifically created for customization, such as organizational rules regarding training in confidentiality and privacy. As recommended by the PCORI Engagement Rubric,1 the PPG included guidance on the value of and methods for compensating patient partners in contributing to clinic QI.
Data Collection and Analysis
During the study’s active phase, remote coaches working with each QI team collected feedback on the PPG from team members, including their initial intention to include patient partners, modifications to patient partner inclusion as the project progressed, the timing of patient involvement related to when partnerships began, and open comments about the PPG. Data were recorded in a shared log by coaches, and reviewed, discussed, and verified at bi-weekly meetings with the lead patient co-investigator (JL). Coaches and patient co-investigators reviewed PPG data again after the end of the observation period to ensure data completeness and accuracy.
Categorical responses were analyzed with descriptive statistics, such as number of sites including patient partners and number of inclusion methods used. Coaches and patient partners, as an analytical team, reviewed and categorized qualitative feedback about the use of the PPG, identifying themes in both appreciative comments and suggestions for improvements. Themes were updated as new data emerged, grounded with direct quotations and re-evaluated to ensure that the meaning of the comment was accurately captured from the perspective of all team members. All categorical and qualitative results were considered in revising the PPG in repeated cycles of improvement.
The IBH-PC study was reviewed and approved by the University of Vermont Research Protections Office Institutional Review Board’s Committee on Human Research (CHRMS#16-554).
RESULTS
PPG Use and Feedback
The active arm of the IBH-PC study for which the PPG was developed included 20 diverse primary care clinics across the USA described in detail elsewhere.11 Of the 20 clinics, 13 elected to download the PPG during the project period and nine created a plan to engage patients in QI. As guided by the PPG, six clinics engaged patient partners on their QI teams and three modified this direct engagement approach by collecting patient input via surveys. Of the six, two teams began their patient engagement at the start of their QI project, one in the middle of the project, and the remaining three engaged patients toward the end of their projects, as they tested new workflows in their clinics (Fig. 2).
Figure 2.
Use of the patient partner guide by clinic QI teams.
Feedback from QI team members highlighted that inclusion of a patient partner on a QI team was simultaneously novel, beneficial, and initially daunting. Clinics noted that finding a patient partner with MCC required discussion among many clinic members, some of whom resisted the idea or expressed doubts about whether a patient would commit to regular meeting attendance. Although some teams worked diligently to recruit two patient partners, per the PPG, all six clinics engaged a single patient on a long-term basis. Typically, the patient who joined the team was someone well known to the clinic, who had previously volunteered, or who served as an advisor to the clinic or health system.
QI team members commented on the length of the PPG, technical challenges in downloading from a private website, and the positive value of the materials. Templates, for example, provided useful starting points for contracts with patients joining a QI team. In all six sites, patients operated as full participants in the QI teams. They attended regular meetings and, in four clinics, were paid for their efforts. They shared their own and family members’ experiences as patients at the clinic, asked clarifying questions, and expressed opinions about decisions the teams faced. Across all six clinics, QI patient partner input and feedback were valued, with one team reporting: “We changed our plan because of our Patient Partner.”
Impact on the Engagement Team
The effect of collaborating on this aspect of the research study as patient co-investigators and academic researchers was new learning for the authors, especially as the act of engaging patients as partners required a shift in understanding that affected everyone. In the words of one of the patient co-investigators (PR),
If you want transformation then you have to transform. Your organization, your process and your thinking must transform. The traditions and beliefs you may have will only be barriers if they are not allowing the patient to the table and to the meaningful transformation work that must be done.
Collaboration continues among research team members and patient co-investigators. For example, co-leaders of the Engagement Team and other stakeholders received an Engagement Award from PCORI to adapt the PPG for use by researchers to engage patients and other stakeholders in research, supporting mutually respectful relationships and processes among team members from varied backgrounds and perspectives (see Appendix for the link to the Partnering for Research website; PCORI EATR-18361).
DISCUSSION
This report described the PPG and its development as part of a structured QI project in a study to improve the care and health of patients managing MCC. Patient co-investigators led the design of the PPG to help clinics engage their own patients as partners in QI. Almost a third of the QI teams included patients as full team members and reported value from patient partnering. Comments from QI team members indicated the PPG was influential in their decision and approach to engaging in a partnership with a patient.
Patient co-investigator leadership in and development of the PPG is a unique example of partner engagement in research. The PPG was conceived by patient co-investigators who saw an unmet need that they were ready to learn how to fill. They wrote, assembled, and curated the contents collaboratively based on stakeholder input, with suggestions from research team members.
Throughout the development and use of the PPG, research team members continued to learn about the importance of meaningful engagement in partnering36 and careful listening to the needs of patients and other stakeholders in their engagement as research partners.37 The bi-directional learning that comes from realizing that each team member has something important to offer the others was a satisfying aspect of this partnering.38 Work on the PPG depended on building reciprocal relationships and supporting co-production of meaningful outcomes.4 The intangible outcomes of this work have included greater equity, trust, and recognition of each other’s capacity to contribute value. These relationships have helped IBH-PC team members grow and develop as partners in health improvement and research.
Our experiences included some of the barriers identified in other studies.3 For example, the time needed to achieve a level of competency with communications technology was greater than expected. Stigmatizing medical terminology was an ongoing source of dismay for patient co-investigators. Also, time constraints limited researchers’ capacities to engage with patients as much as desired for full stakeholder participation. This was the first researcher-patient partnership for this team, and over time, sensitivity to dynamics needing attention, such as symptoms of disengagement or imbalances in power, increased. Regular check-ins and needs assessments helped prevent and address these symptoms and have informed the PPG. Taking time at the start or close of meetings to acknowledge each person was essential to the development of reciprocal relationships. The team depended upon individuals having the confidence to bring their best whole person health to the work. Checking in with each team member regularly became a norm for the Engagement Team. Additionally, patient co-investigators assigned to represent the identity of the MCC population discovered they needed to work doubly hard to maintain their healthy self-image and address their life/work balance throughout the study.
In addition to the team development needed to support a research study, engagement is an iterative process that begins before and continues over the course of and after the study. This process is an important part of team maintenance.39 Research constantly builds on past discoveries and identifies new opportunities that appear on the horizon. To act on those opportunities nimbly, stakeholder relationships are best if continually nourished. Periodic updates, whether through correspondence or virtual team visits, to evaluate future research opportunities, complete surveys on topics of interest, or share questions with peers are all part of nurturing the team and preparing the community for future work. One-on-one crucial conversations,40 if practiced regularly and used when needed, can resolve conflict and enhance team members’ commitment to project goals and support of one another through challenges. Engaging new partners in each of these opportunities builds a sustainable research stakeholder community.
This observational report has several limitations. Those QI teams that chose to use the PPG were self-selected and represent a small group of primary care practices. Results from this study are not generalizable to other practice teams. Also, the results reported here are limited to descriptions of the benefits and challenges of PPG use and do not include results of the QI teams’ work or its impact on patient health.
This study observed whether patient partners could engage in a research project by adding to a part of the intervention, thereby bringing patient engagement into primary care practices. Future research examining the impact of such engagement may strengthen commitment to such diverse partnerships. Future studies that engage patient and other stakeholder partners will likely need to assess their access to synchronous and asynchronous Internet platforms. Technology is a prerequisite to engagement for partners whose management of their MCC prevents travel for on-site meetings. While orienting patients to technology was a substantial hurdle in 2016, the near-universal transformation in communication practices resulting from COVID-19 dramatically shrank the technology gap for engaging new partners. Although facility with communication technology in the general population has grown, it cannot be assumed for all partner populations.
In addition, attention is needed regarding the degree to which partners in research are treated equitably by system structures, such as payments (salaried vs hourly employees or as unpaid volunteers). All authors agree that the communication methods used and compensation provided in the IBH-PC study were supportive of commitment, a culture of sharing, and the practice of dialogue that created the PPG. Understanding and improving key processes of patient engagement in research will continue to lead to new opportunities to standardize research practice. For example, developing a standard process to engage patient partners in examining a proposed intervention and its interface with study participants is a potential step that can build on comparative effectiveness research standards, as proposed by Esmail et al.41
The PPG is freely available as part of the study intervention to help QI teams in primary care clinics engage diverse stakeholders at https://sites.google.com/view/ibhpc/workbooks/patient-partnering.
CONCLUSIONS
Engagement of patients and other stakeholders in research is dependent on a deliberate process to support reciprocal relationships in meaningful work. This example of a multi-stakeholder research team’s development of an intervention component to engage clinic patients in QI demonstrates that patients can do more than contribute to research. They can take leadership in identifying gaps in research concepts and tools and in responding to them, thereby making engagement successful and rewarding for all team partners involved.
Supplementary information
(DOCX 3491 kb)
Acknowledgements
The authors would like to thank all those who contributed to this project: Andrea Auxier, Dana Basiliere, Laura-Mae Baldwin, Andrew Bertagnoli, Levi Bonnell, Georgia Brown, Beverly Canin, Jessica Clifton, Abigail Crocker, Anne Donohue, Florence Fee, Sylvie Frisbie, Chet Fox, Jennifer Jewiss, Rodger Kessler, Parinda Khatri, Benjamin Littenberg, CR Macchi, Katie MacDonald, Susan McDaniel, Ricka McNaughton, Brenda Mollis, John Muench, Lisa W. Natkin, Wilson Pace, Richard Pinkney, Andrew Pomerantz, Alex Pond, Richard Reynolds, Sarah Scholle, Jeni Soucie, Kari Stephens, and Lucy Wolski.
Funding
Research reported in this publication was funded through a Patient-Centered Outcomes Research Institute (PCORI) Award (PCS-1409-24372) and Patient-Centered Outcomes Research Institute (PCORI) Award (EATR-18361). The views, statements, and opinions presented in this report are solely the responsibility of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors, or its Methodology Committee. The Patient-Centered Outcomes Research Institute (PCORI) is an independent, nonprofit organization authorized by Congress in 2010. Its mission is to fund research that will provide patients, their caregivers, and clinicians with the evidence-based information needed to make better-informed healthcare decisions. PCORI is committed to continually seeking input from a broad range of stakeholders to guide its work.
Declarations
Conflict of interest
All authors received funding support through the study on Integrating Behavioral Health and Primary Care funded by the Patient-Centered Outcomes Research Institute. No other conflicts of interest were identified.
Footnotes
Prior Presentations:
Portions of this paper were previously presented at the Patient-Centered Outcomes Research Institute Annual Conference of 2019.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Sheridan S, Schrandt S, Forsythe L, Hilliard TS, Paez KA. The PCORI Engagement Rubric: Promising Practices for Partnering in Research. Ann Fam Med. 2017;15(2):165–170. doi: 10.1370/afm.2042. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Vat LE, Finlay T, Jan Schuitmaker-Warnaar T, Fahy N, Robinson P, Boudes M, et al. Evaluating the "return on patient engagement initiatives" in medicines research and development: A literature review. Health expectations: an international journal of public participation in health care and health policy. 2020;23(1):5–18. doi: 10.1111/hex.12951. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Domecq JP, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, et al. Patient engagement in research: a systematic review. BMC Health Services Research. 2014;14(1):89. doi: 10.1186/1472-6963-14-89. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Shippee ND, Domecq Garces JP, Prutsky Lopez GJ, Wang Z, Elraiyah TA, Nabhan M, et al. Patient and service user engagement in research: a systematic review and synthesized framework. Health Expect. 2015;18(5):1151–66. doi: 10.1111/hex.12090. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Hovén E, Eriksson L, Månsson D'Souza Å, Sörensen J, Hill D, Viklund C, et al. What makes it work? Exploring experiences of patient research partners and researchers involved in a long-term co-creative research collaboration. Research involvement and engagement. 2020;6:33–33. doi: 10.1186/s40900-020-00207-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Ellis LE, Kass NE. How are PCORI-funded researchers engaging patients in research and what are the ethical implications? AJOB Empirical Bioethics. 2017;8(1):1–10. doi: 10.1080/23294515.2016.1206045. [DOI] [PubMed] [Google Scholar]
- 7.Poger JM, Mayer V, Duru OK, Nauman B, Holderness H, Warren N, et al. Network Engagement in Action: Stakeholder Engagement Activities to Enhance Patient-centeredness of Research. Med Care. 2020;58:S66–s74. doi: 10.1097/MLR.0000000000001264. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Forsythe L, Heckert A, Margolis MK, Schrandt S, Frank L. Methods and impact of engagement in research, from theory to practice and back again: early findings from the Patient-Centered Outcomes Research Institute. Quality of Life Research. 2018;27(1):17–31. doi: 10.1007/s11136-017-1581-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Beneciuk JM, Verstandig D, Taylor C, Scott D, Levin J, Osborne R, et al. Musculoskeletal pain stakeholder engagement and partnership development: determining patient-centered research priorities. Research Involvement and Engagement. 2020;6(1):28. doi: 10.1186/s40900-020-00192-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Hemphill R, Forsythe LP, Heckert AL, Amolegbe A, Maurer M, Carman KL, et al. What motivates patients and caregivers to engage in health research and how engagement affects their lives: Qualitative survey findings. Health Expect. 2020;23(2):328–336. doi: 10.1111/hex.12979. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Crocker AM, Kessler R, van Eeghen C, Bonnell LN, Breshears RE, Callas P, et al. Integrating Behavioral Health and Primary Care (IBH-PC) to improve patient-centered outcomes in adults with multiple chronic medical and behavioral health conditions: study protocol for a pragmatic cluster-randomized control trial. Trials. 2021;22(1):200. doi: 10.1186/s13063-021-05133-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Greenhalgh T, Hinton L, Finlay T, Macfarlane A, Fahy N, Clyde B, et al. Frameworks for supporting patient and public involvement in research: Systematic review and co-design pilot. Health Expectations. 2019;22(4):785–801. doi: 10.1111/hex.12888. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Hibbard JH. Engaging health care consumers to improve the quality of care. Med Care. 2003;41(1 Suppl):I61–70. doi: 10.1097/00005650-200301001-00007. [DOI] [PubMed] [Google Scholar]
- 14.Carman KL, Dardess P, Maurer M, Sofaer S, Adams K, Bechtel C, et al. Patient and family engagement: a framework for understanding the elements and developing interventions and policies. Health Aff (Millwood) 2013;32(2):223–31. doi: 10.1377/hlthaff.2012.1133. [DOI] [PubMed] [Google Scholar]
- 15.Han E, Scholle S, Morton S, et al. Survey Shows that Fewer than a Third of Patient-Centered Medical Home Practices Engage Patients in Quality Improvement. Health affairs (Millwood, Va.) 2013;32(2):1–9. doi: 10.1377/hlthaff.2012.1183. [DOI] [PubMed] [Google Scholar]
- 16.Patel N, Rajasingam D. User engagement in the delivery and design of maternity services. Best Pract Res Clin Obstet Gynaecol. 2013;27(4):597–608. doi: 10.1016/j.bpobgyn.2013.04.006. [DOI] [PubMed] [Google Scholar]
- 17.Dickinson WP, Nease DE, Rhyne RL, Knierim KE, Fernald DH, de la Cerda DR, et al. Practice Transformation Support and Patient Engagement to Improve Cardiovascular Care: From EvidenceNOW Southwest (ENSW) The Journal of the American Board of Family Medicine. 2020;33(5):675–686. doi: 10.3122/jabfm.2020.05.190395. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Roseman D, Osborne-Stafsnes J, Amy CH, Boslaugh S, Slate-Miller K. Early lessons from four 'aligning forces for quality' communities bolster the case for patient-centered care. Health Aff (Millwood) 2013;32(2):232–41. doi: 10.1377/hlthaff.2012.1085. [DOI] [PubMed] [Google Scholar]
- 19.Mende S, Roseman D. The Aligning Forces For Quality Experience: Lessons On Getting Consumers Involved In Health Care Improvements. Health Affairs. 2013;32(6):1092–1100. doi: 10.1377/hlthaff.2012.1079. [DOI] [PubMed] [Google Scholar]
- 20.Kemper C, Blackburn C, Doyle JA, Hyman D. Engaging patients and families in system-level improvement: a safety imperative. Nurs Adm Q. 2013;37(3):203–15. doi: 10.1097/NAQ.0b013e318295f61e. [DOI] [PubMed] [Google Scholar]
- 21.Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. Bmj. 2014;348:g1687. doi: 10.1136/bmj.g1687. [DOI] [PubMed] [Google Scholar]
- 22.Kielmann T, Huby G, Powell A, Sheikh A, Price D, Williams S, et al. From awareness to involvement? A qualitative study of respiratory patients’ awareness of health service change. Health Expectations. 2011;14(3):321–333. doi: 10.1111/j.1369-7625.2010.00631.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Maurer, M., Dardess, P., Carman K.L., et al., Guide to Patient and Family Engagement: Environmental Scan Report, in AHRQ Publication No. 12-0042-EF, A.I.f. Research, Editor. 2012, Agency for Healthcare Research and Quality: Rockville, MD.
- 24.Simmons LB, Baker NJ, Schaefer J, Miller D, Anders S. Activation of Patients for Successful Self-Management. Journal of Ambulatory Care Management. 2009;32(1):16–23. doi: 10.1097/01.JAC.0000343120.07844.a9. [DOI] [PubMed] [Google Scholar]
- 25.Blueprint for the CMS Measures Management System, Version 17.0. 2021 [cited 2021 Dec 16]; Available from: https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/MMS/MMS-Blueprint.
- 26.Sheridan NF, Kenealy TW, Kidd JD, Schmidt-Busby JIG, Hand JE, Raphael DL, et al. Patients’ engagement in primary care: powerlessness and compounding jeopardy. A qualitative study. Health Expectations. 2012;18(1):32–43. doi: 10.1111/hex.12006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.PCORI Engagement Rubric. 2014 [cited 2021 Dec 16]; Available from: https://www.pcori.org/document/engagement-rubric.
- 28.Harboe, G. and E.M. Huang. Real-World Affinity Diagramming Practices: Bridging the Paper-Digital Gap. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems 2015; pp. 95–104. Available from: 10.1145/2702123.2702561.
- 29.Pomey, M.P., D.P. Ghadiri, P. Karazivan, N. Fernandez, and N. Clavel, Patients as partners: a qualitative study of patients' engagement in their health care. PLoS One, 2015. 10(4): p. e0122499. [DOI] [PMC free article] [PubMed]
- 30.Karazivan P, Dumez V, Flora L, Pomey MP, Del Grande C, Ghadiri DP, et al. The patient-as-partner approach in health care: a conceptual framework for a necessary transition. Acad Med. 2015;90(4):437–41. doi: 10.1097/ACM.0000000000000603. [DOI] [PubMed] [Google Scholar]
- 31.Peikes, D., Genevro, J, Scholle, SH, Torda, P, The Patient-Centered Medical Home: Strategies to Put Patients at the Center of Primary Care, AHRQ, Editor. 2011: Rockville, MD.
- 32.Scholle SH, Torda P, Peikes D, Han E, Genevro J, Engaging Patients and Families in the Medical Home, A.f.H.R.a. Quality, Editor. 2010: Rockville, MD.
- 33.Johnson BH. Promoting Patient- and Family-Centered Care Through Personal Stories. Acad Med. 2016;91(3):297–300. doi: 10.1097/ACM.0000000000001086. [DOI] [PubMed] [Google Scholar]
- 34.Wiig S, Storm M, Aase K, Gjestsen MT, Solheim M, Harthug S, et al. Investigating the use of patient involvement and patient experience in quality improvement in Norway: rhetoric or reality? BMC Health Serv Res. 2013;13:206. doi: 10.1186/1472-6963-13-206. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Sharma AE, Willard-Grace R, Willis A, Zieve O, Dubé K, Parker C, et al. "How Can We Talk about Patient-centered Care without Patients at the Table?" Lessons Learned from Patient Advisory Councils. J Am Board Fam Med. 2016;29(6):775–784. doi: 10.3122/jabfm.2016.06.150380. [DOI] [PubMed] [Google Scholar]
- 36.Black A, Strain K, Wallsworth C, Charlton S-G, Chang W, McNamee K, et al. What constitutes meaningful engagement for patients and families as partners on research teams? Journal of Health Services Research & Policy. 2018;23(3):158–167. doi: 10.1177/1355819618762960. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Stephens KA, KP Osterhage, Fiore-Gartland B, Lovins TL, Keppel GA, Kim KK. Examining the Needs of Patient Stakeholders as Research Partners in Health Data Networks for Translational Research. in AMIA Jt Summits Transl Sci Proc. 2019: 363–369. [PMC free article] [PubMed]
- 38.Young K, Kaminstein D, Olivos A, Burroughs C, Castillo-Lee C, Kullman J, et al. Patient involvement in medical research: what patients and physicians learn from each other. Orphanet journal of rare diseases. 2019;14(1):21–21. doi: 10.1186/s13023-018-0969-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Katzenbach, J., Smith, DK, The wisdom of teams: Creating the high-performance organization. 1993, Boston, MA: Harvard Business School Press.
- 40.Patterson K. Crucial Conversations: Tools for talking when stakes are high. New York: McGraw-Hill; 2012. [Google Scholar]
- 41.Esmail LC, Barasky R, Mittman BS, Hickam DH. Improving Comparative Effectiveness Research of Complex Health Interventions: Standards from the Patient-Centered Outcomes Research Institute (PCORI) J Gen Intern Med. 2020;35(Suppl 2):875–881. doi: 10.1007/s11606-020-06093-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
(DOCX 3491 kb)