Implementing Community-Created Self-Management Support Tools in Primary Care Practices: Multimethod Analysis From the INSTTEPP Study

Abstract

Purpose

With one-half of Americans projected to be living with at least one chronic condition before 2020, enhancing patient self-management support (SMS) may improve health-related behaviors and clinical outcomes. Routine SMS implementation in primary care settings is difficult. Little is known about the practice conditions required for successful implementation of SMS tools.

Methods

Four primary care practice-based research networks (PBRNs) recruited 16 practices to participate in a boot camp translation process to adapt patient-centered SMS tools. Boot camp translation sessions were held over a 2-month period with 2 patients, a clinician, and a care manager from each practice. Qualitative case comparison and qualitative comparative analysis were used to examine practice conditions needed to implement SMS tools. The Consolidated Framework for Implementation Research guided data collection and analysis.

Results

Four different practice conditions affected the implementation of new SMS tools: functional practice organization; system that enables innovation and change; presence of a visible, activated champion; and synergy and alignment of SMS changes with other work. Qualitative comparative analysis suggested that it was necessary to have an enabling system, a visible champion, and synergy for a practice to at least minimally implement the SMS tools. Sufficiency testing, however, failed to show robust consistency to satisfactorily explain conditions required to implement new SMS tools.

Conclusions

To implement tailored self-management support tools relatively rapidly, the minimum necessary conditions include a system that enables innovation and change, presence of a visible champion, and alignment of SMS changes with other work; yet, these alone are insufficient to ensure successful implementation.

One-half of Americans are projected to be living with at least one chronic condition before 2020.¹ For that reason, the health care system must work toward reducing the burden on primary care for disease management by facilitating the development of activated, informed patients who are proficient in self-management skills.

Self-management support (SMS) helps enable people to manage their conditions day-to-day. SMS is defined as the efforts of the health care team to promote patient engagement in behaviors that positively impact their illness.² A portfolio of different SMS tools and techniques exists that can assist patients in becoming more active participants in illness management, transforming the patient-caregiver relationship into a collaborative partnership.³ Collaborative partnership represents a significant cultural shift in health management, and clinicians and staff may lack confidence introducing and promoting SMS. SMS is a key component of patient-centered medical homes, yet it is least often implemented in the area of disease management because it is challenging to integrate into usual care.^4,5

The Agency for Healthcare Research and Quality (AHRQ) maintains a library of SMS evidence, resources, and tools.⁶ AHRQ sought to improve the practical application of this library in frontline primary care practices. The burden of chronic illness calls for team-based primary care working in partnership with activated, informed patients. Few patients have proficient self-management skills, lacking knowledge and confidence. Primary care practices are dedicating more resources to care coordination, identifying and training care managers within the practice to help manage an increasingly complex panel of patients.

The Implementing Networks’ Self-management Tools Through Engaging Patients and Practices (INSTTEPP) study aimed to assist small to medium-sized primary care practices that were beginning to perform care management. A modified boot camp translation (BCT) model was used by primary care practice-based research networks (PBRNs) to facilitate SMS tool design, refinement, and implementation. BCT is a community-based participatory approach that engages community members in a process that translates evidence-based medical care into locally relevant and culturally appropriate language and constructs.^7,8

The main purpose of the INSTTEPP study was to use the BCT method to choose from or adapt AHRQ’s SMS toolkit for relevant patient- and practice-centered tools, then test selected tool implementation in 16 practices across 4 PBRNs. Main outcomes assessed the impact on patient activation, clinician support for patient activation, and patient self-reported health and process of care.^9–12 Patients in the study practices showed significant improvements in ratings of their health and in their perceptions of the processes of care provided by their practices.¹³

While patient- and practice-level outcomes are essential steps to establish the evidence, it is also important to develop the evidence for how to implement successful SMS tools.^14,15 This manuscript describes the results of the multimethod qualitative assessment of critical implementation conditions and factors for SMS tools.

METHODS

INSTTEPP was an 18-month long, AHRQ-funded trial conducted in 4 U.S.-based PBRNs: the State Networks of Colorado Ambulatory Practices and Partners (SNOCAP), Oregon Rural Practice-based Research Network, Iowa Research Network, and Wisconsin Research and Education Network. A total of 16 primary care practices (4 from each PBRN) participated in this project. There were 9 rural, 4 urban, and 3 suburban practices, with a range of 6 to 126 total clinicians and staff (mean: 35.5 clinicians and staff). Each PBRN conducted a BCT consisting of a minimum of 8 patients, 4 clinicians, 4 care managers, and the local PBRN’s research team. Because INSTTEPP focused on translating SMS tools for implementation in primary care practices, we explicitly recruited clinicians and primary care team members (along with patients), seeking equal representation and participation from practice staff, clinicians, and patients.

A stepped-wedge study design¹⁶ meant that BCT was sequentially rolled out in a randomized order across the 4 PBRNs between March and September of 2014. Each practice participated in the BCT implementation for 2 months. The INSTTEPP BCTs included an all-day, in-person kickoff retreat, followed by 3 conference calls over a 2-month period. The Colorado Multiple Institutional Review Board was the IRB of record — accepting oversight for the participating PBRNs and practices — and approved this study.

The multimethod qualitative approach to assess BCT implementation relied on the Consolidated Framework for Implementation Research (CFIR)¹⁴ to guide qualitative data collection. CFIR consists of 5 major domains: intervention characteristics, outer setting, inner setting, characteristics of the individuals involved, and the process of implementation. These domains are helpful guides for evaluations and for building an implementation knowledge base across multiple studies and settings. Secondarily, the qualitative analysis was paired with fuzzy-set qualitative comparative analysis (fsQCA) to identify conditions/sets important for successful implementation.^17–19 Qualitative comparative analysis (QCA) maintains complexity in the analytical process and allows for multiple pathways to achieve the outcome.¹⁷ Because of the expected diversity in BCT processes and practice types, we anticipated diversity in the conditions necessary to achieve the outcome of implementing SMS tools.

Observations and Interviews

On-site interviews and guided observations were conducted by each network’s PBRN coordinator in each of the 16 practices after the initial BCT kickoff retreat and at the conclusion of the BCT process. The first site visit included a 1-hour meeting, attended by each practice’s care manager/health coach (who also attended the BCT kickoff) and a clinician, with focus on discussion of the actual plans for implementation of the SMS toolkit and a description of current office processes for patients with diabetes or other chronic illnesses. A follow-up on-site or telephone interview was conducted (also with the care manager/health coach and clinician) within 4 weeks of the conclusion of the BCT implementation period to assess SMS tool implementation efforts and to discuss what was working and potential modification(s) of the practice’s implementations.

The interview guide included questions to elicit information about practices that aligned with the major CFIR domains. For example: “What is the planning process your practice will use to implement the SMS library/toolkit?” “What do you think about the evidence or reasons supporting the use of the SMS library and toolkit?” “What external policies or incentives encourage the selection and use of this toolkit?” “How ready is your practice for the implementation of this toolkit?” and “How easily was the toolkit adapted to meet your practice’s needs?” To minimize burden on practices’ time, we divided open-ended questions related to the CFIR constructs across the baseline and follow-up interviews.

Master’s- or PhD-trained researchers in each PBRN collected the observation and interview data. The lead qualitative analyst (D.H.F.) from the SNOCAP PBRN met with each data collector to review the purpose of observations and interviews, discuss the CFIR framework, and discuss interview and observation guide and data collection procedures. For the Master’s-level researchers, the lead analyst attended the baseline observations and interviews in-person to assist with data collection as needed. In addition to notes maintained by the interviewers, interviews were audio-recorded and then transcribed for analysis.

Qualitative Data Analysis

Qualitative data (interview transcripts and observation notes) were analyzed by two members of the Colorado research team (M.S., D.H.F.) using a case-ordered matrix approach. Data were first segmented into 5 CFIR domains — inner setting, outer setting, process, intervention characteristics, and individual characteristics — to begin synthesizing specifics about conditions affecting implementation in this set of practices. Data from each of the 16 practices were then added in summary form to the case-based matrix and reviewed by the Colorado research team for framework inconsistencies, negative cases, and other data questions. The data were re-reviewed and the case matrix was updated as needed. The case matrix was then sent to each of the PBRN’s study team for review and clarification and then for revisions as needed. Study teams comprised each local PBRN principal investigator and research coordinators who helped collect interview and observation data, plus the overall principal investigator and the qualitative data analysts.

Qualitative Comparative Analysis and fsQCA

QCA is a method to analyze case-oriented data which recognizes that different paths or conditions may lead to the same or similar outcomes. QCA is based on set theory, using the strength of membership of each case’s particular group of conditions in sets that are defined by the investigators. For example, a set could be how fully a practice embraces quality improvement. A practice that has fully embraced quality improvement is fully in that set while a practice that has rejected quality improvement is not at all in that set. Another practice may have partially embraced quality improvement and is partially in the set. The level of set membership is based on analysis of the qualitative data. QCA analyzes each case or practice’s membership in each set, identifies necessary and sufficient conditions for a particular outcome, and is suitable for studies with small or medium-size numbers of cases.¹⁷ There are 4 main steps in the analysis: set identification, dataset calibration, necessity testing, and sufficiency testing. Sets are the conditions — analogous to variables (eg, a large, medium, or small practice) — into which each case is assigned membership.

Fuzzy-set QCA refers to the calibration step in which each case (a practice in this analysis) is scored as fully (1.0), partially (>0.0 and <1.0), or not at all (0.0) a member of a set. A membership score of 0.5 indicates ambiguous membership in a set.^17,20 Necessity analysis tests for conditions that must be present all or most of the time for the outcome to occur. Sufficiency analysis assesses which conditions, when present, ensure that the outcome occurs all or almost all of the time.

Set Identification and Calibration

The study teams from the 4 PBRNs met in person over 2 half-days to 1) review the case matrix analysis, and 2) discuss and define key conditions, derived from the qualitative data analysis, that likely affected implementation and adoption of BCT-developed SMS tools. After the in-person meeting, each PBRN study team received the set definitions and instructions for assigning membership scores (from 0.0 [ie, not at all a member] to 1.0 [ie, a full member]) to each practice for each condition, including the outcome condition, based on their further review of the qualitative data and the matrix analysis reviewed at the in-person meeting. After preliminary truth table analysis (a matrix of all possible combinations of conditions, including membership in the outcome set), each team was asked to recalibrate their membership assignments to help clarify membership scores, if possible, based on the data available for each of the conditions defined, especially for those that were ambiguous (ie, membership score of 0.5). Seven (of 96 possible) scores were modified in this recalibration step. All membership scores were combined into a single spreadsheet for review and clarification before beginning analysis. In the analysis, each practice is considered a case. Kirq software (Version 2.1.12, Christopher Reichert and Claude Rubinson, Houston, TX) was used for all fsQCA analyses.

RESULTS

Matrix Analysis (“Traditional” Qualitative Method): Summary of Each CFIR Domain

When organized within the CFIR major domains, the qualitative analysis yielded variation across practices — including within networks — in their relative strengths and threats to implementing a new SMS tool adapted by their community of patients, clinicians, and staff.

Inner Setting (Culture, Readiness, Implementation Climate, Structural Characteristics, Communication)

Among the strengths observed in many of the practices were collaborative practice environments and teams, team-based approaches and care techniques, supportive practice leadership, existing processes for some SMS techniques, and the relatively high priority of delivering more SMS to their patients. However, in most practices, threats to implementation of a new SMS tool included busy clinicians and staff, concurrent initiatives already in progress, and competing priorities encroaching on staff ability to try out new tools and techniques. Among a few practices, there appeared to be relatively low readiness to implement a new SMS tool.

Outer Setting (Patient Needs, Health System, Peer Pressure, External Policies/Incentives)

Few practices appeared to enjoy supporting characteristics external to their clinic. Several practices noted there was some alignment with other priorities and incentive programs (such as health plan incentives or patient-centered medical home [PCMH] recognition) coming from outside the practice. Yet, system pressures or constraints and lack of congruence with other initiatives and system priorities, such as PCMH recognition and “meaningful use” compliance, endangered attempts to fully implement SMS tools. For example, one practice’s implementation was severely hampered because the SMS tool did not carry the approval of the larger health care system, which was required prior to any local adoption of clinic materials.

Process (Planning, Engaging, Executing, Reflecting/Evaluating)

How practices implemented new care processes varied considerably. Some practices appeared to have well-established, standardized vetting and testing processes for new care processes. In one health system, a single practice site was designated as the “test site” for trying new processes and working out implementation issues before spreading to other sites in the system. In the context of INSTTEPP, some practices focused implementation on a pilot population — patients with a particular chronic condition, patients seen by a specific care team, or patients seen by a specific care team member (eg, one particular clinician, health coach). Most practices benefited from a management structure that did not interfere or hinder attempts by individual practices to test new ideas or processes. In most practices, existing quality improvement committees or recent experience with change processes supported implementation efforts for SMS. In contrast, there were several practices that appeared to lack formal systematic processes for implementing new ideas or faced system approval barriers that threatened momentum for changes.

Intervention Characteristics (Source, Evidence Strength, Advantage, Adaptability, Trialability, Complexity, Design Quality, Cost)

Implementation of the SMS tools was variable; there were few observational or interview data about the characteristics of tool implementation itself. However, clinicians and staff offered strong opinions about their general preferences for new interventions: low costs (eg, printing in black and white vs color), integration with electronic health record, and “must work alongside what’s already in place” (ie, existing workflows). Several clinicians commented that they often gave higher consideration to interventions that had been validated and were shown to be reliable in primary care experience as compared to hard scientific evidence of efficacy. The BCT process results are consistent with this preference for SMS tools that are simple and can be tailored to specific populations or communities.²¹

Individual Characteristics (Intervention Knowledge/Beliefs, Self-Efficacy, Stage of Change, Organizational Identification)

Observational findings in some clinics pointed to individuals who helped to promote the use of SMS. These key agents included one clinician with a strong interest in SMS, multiple physicians expressing openness and readiness to trying something new, and an activated health coach and nurse care manager who sought additional information and tools on their own.

fsQCA Analysis (Qualitative and Quantitative)

Grounded in the data and summary results above, the in-person study team meeting and QCA data discussion resulted in 5 key conditions (sets) that likely affected implementation and adoption of BCT-developed SMS tools (Table 1):

Table 1.

QCA Set Descriptions

A. Functional practice organization: A functional practice can manage chaos, prioritize competing demands or projects, is realistic in what it aims to accomplish, has internal alignment among staff/providers or high “groupness.”

B. System enables innovation and change: The system or other known external influences for a practice may stifle a practice’s ability or willingness to innovate, transform, or test important changes around SMS.

C. Presence of a visible, activated champion: A champion (physician or health coach/nurse manager) for changes around SMS can be identified and named, can act with some autonomy, and is a dynamic change agent in a practice.

D. Synergy and alignment of SMS changes with other work: The SMS changes practices are making or attempting to make are relevant, fit with other practice goals, and help “connect the dots” to the bigger picture for practice goals.

E. Patient push: There is energy and interest among patients in the practice that are pushing the practice to make changes around SMS.

Outcome Set. Implementation of an SMS tool: The practice fully implemented a new or substantially new SMS tool that emerged from or was derived from the boot camp translation process. A fully implemented SMS tool is used routinely for a population of patients.

QCA, qualitative comparative analysis; SMS, self-management support.

1. Functional practice organization.

2. System enables innovation and change.

3. Presence of a visible, activated champion.

4. Synergy and alignment of SMS changes with other work.

5. Patient push.

The group defined the outcome condition as “implementation of the BCT-developed SMS tool.” To be a full member of the outcome set, a practice had to have: “fully implemented a new or substantially new SMS tool that emerged from or was derived from the BCT process. A fully implemented SMS tool is used routinely for a population of patients.” The “patient push” condition was dropped from the analysis because there were too few data available across all PBRNs to adequately describe and calibrate, with confidence, both the condition and set membership among their participating practices.

Exploratory Analysis With Truth Tables

Initial exploration of the data using fsQCA examined all of the possible combinations of conditions across all of the cases. The resulting truth table (Table 2) showed that there was just 1 combination of conditions (or pathway) to the outcome of implementing the SMS tool in a practice. Three of 16 cases had the combination of all 4 conditions and also were in the outcome set (ie, consistent observations); 5 cases had the same combination of conditions but were not in the outcome set (ie, inconsistent or contradictory observations).

Table 2.

Initial Truth Table of All Possible Configurations

A Functional Organization	B Enabling System	C Visible Champion	D Synergy/Alignment	Cases, n	Consistency	Outcome	Consistent Observations, n	Contradictory Observations, n
True	True	True	True	8	0.53	True	3	5
True	True	False	False	2	0.29	False	0	2
True	False	True	True	2	0.34	False	0	2
False	True	True	True	1	0.47	False	0	1
False	True	True	False	1	0.38	False	0	1
True	True	True	False	0	n/a	Rem.*	–	–
True	True	False	True	0	n/a	Rem.*	–	–
True	False	True	False	0	n/a	Rem.*	–	–
True	False	False	True	0	n/a	Rem.*	–	–
True	False	False	False	0	n/a	Rem.*	–	–
False	True	False	True	0	n/a	Rem.*	–	–
False	True	False	False	0	n/a	Rem.*	–	–
False	False	True	True	0	n/a	Rem.*	–	–
False	False	True	False	0	n/a	Rem.*	–	–
False	False	False	True	0	n/a	Rem.*	–	–
False	False	False	False	0	n/a	Rem.*	–	–

^*Rem., logical remainders represent logically possible configurations but do not appear in the data.

The consistency was relatively low (0.53), and there were 5 contradictory observations. Consistency below 0.80 suggests that the combination of conditions does not lead to outcome often enough to be considered a robust finding. After considering the relatively short project timeline, we defined a less restrictive outcome by recalibrating the outcome to a crisp set (0 or 1) “of implementing the SMS tool at least minimally” (1.0) versus “not at all implemented” (0.0). The resulting truth table (Table 3) was the same, but with just 2 contradictory cases and a higher consistency score (0.68). We attempted to resolve the contradictory cases through recalibration based on our knowledge of the cases; however, the available case data were insufficient to enable resolving the contradictions.

Table 3.

Truth Table With Recalibrated Outcome Set^*

A Functional Organization	B Enabling System	C Visible Champion	D Synergy/Alignment	Cases, n	Consistency	Outcome	Consistent Observations, n	Contradictory Observations, n
True	True	True	True	8	0.68	True	6	2
True	True	False	False	2	0.29	False	0	2
True	False	True	True	2	0.34	False	0	2
False	True	True	True	1	0.47	False	0	1
False	True	True	False	1	0.38	False	0	1

^*Logical remainders were identical to those shown in Table 2 (outcome = Rem.) and were removed from this table for brevity.

Necessity Analysis

Necessity analysis tests for conditions that must be present for the outcome to occur. The initial necessity test used the dichotomous outcome with consistency threshold of 0.80 and coverage threshold of 0.50. There were 3 separate terms of single conditions necessary for the outcome to occur:

• B. Enabling system (consistency: 0.86, coverage: 0.56).

• C. Visible champion (consistency: 0.86, coverage: 0.56).

• D. Synergy (consistency: 0.89, coverage: 0.55).

In other words, when a practice was able to implement the SMS tools at least minimally, the presence of an enabling system or a visible champion or synergy was necessary for the outcome to occur. An “AND’ed” combination of the 3 conditions together (ie, enabling system AND visible champion AND synergy) yielded slightly higher coverage of cases (0.65) with slightly lower consistency (0.81). While these appeared to be necessary conditions, they may not be sufficient to ensure that the outcome occurs.

Sufficiency Analysis

Sufficiency analysis assesses which conditions, when present, ensure that the outcome occurs all or almost all of the time. Truth tables were reduced to a complex solution, including the 3 necessary conditions from necessity analysis plus the “functional organization” condition. The sufficiency analysis was performed with a consistency threshold of 0.80 and a proportion threshold of 1.00, which resulted in a single “pathway” or configuration of conditions (consistency: 0.68, coverage: 0.76): functional organization AND enabling system AND visible champion AND synergy.

The presence of a functional organization, an enabling system, a visible champion, and synergy led to the outcome. These conditions were needed in combination to be able to implement at least some degree of the BCT-developed SMS tools. The relatively low consistency score (0.68) and the contradictory observations suggest that this combination does not lead to outcome often enough to fully explain the conditions needed to reliably achieve SMS implementation.

DISCUSSION

This multimethod qualitative assessment elaborated important supporting elements from the CFIR for implementing newly adapted, community-created SMS tools in primary care. Among the strengths supporting implementation were collaborative teams and team-based care, supportive practice leadership, a supportive system, established change and improvement processes, and integration within existing clinic workflows. Unfortunately, there were observed threats to implementation, including lack of systematic processes for implementing new ideas, system approval barriers, lack of congruence with other initiatives and system priorities, and insufficient staff or time to implement new tools.

The QCA extended the qualitative thematic analysis by identifying and assessing which combinations of supportive implementation conditions might lead to at least minimal implementation of SMS tools. Yet, the analysis suggests that presence of an enabling system, a visible and active champion, and synergy with existing priorities, while necessary, were insufficient to fully describe how primary care practices can effectively implement their newly created SMS tools.

For primary care practices that want to successfully implement new SMS tools, our analysis pointed to logical necessary conditions, which corroborate previously reported evidence for successfully making practice improvements,^22–24 such as change alignment with clinical priorities, influence of the larger system, and motivation of key practice members. However, our results indicate that there are unexamined or unknown conditions that may more fully explain how SMS tool implementation happens. Sufficient time may be one of those conditions, but we were unable to evaluate this. Our results also suggest that the tools should be low cost, integrate with the electronic health record, have system approval, be tailorable to each practice, and “work alongside” rather than against other priorities. From implementation literature, other conditions like adaptive reserve, facilitation support, and use of multiple effective change strategies^22,25 may be required. Allowing more time for SMS tool implementation and additional data collection may have identified additional conditions required for successful implementation.

Study Limitations

One potential limitation is that a quantitative assessment of tool implementation or clinic readiness was not performed. Our observation window for implementation (about 9–12 weeks for most INSTTEPP PBRNs) was likely unrealistic given the short timeline from SMS tool adaptation using the BCT process, distribution to the practices, and getting the tool into the change process. While our data generated sufficient conditions for this analysis, there may be other pathways with other conditions that we did not identify. Pertaining to this analysis, 2 contradictory cases were removed from the analysis because we were unable to resolve the contradiction. These 2 cases both had relatively high membership scores (0.60–1.0) in all 4 of the necessary conditions; however, both had very low membership scores in the outcome condition (0.0). Primary care practices are complex systems;^26,27 there is likely another condition (and likely another pathway or configuration) that we did not define that explains their failure to implement SMS tools at least minimally or that measurements of the conditions were not valid. For others attempting QCA analysis on implementation, more detailed practice-level data collected over a longer period of time may help overcome some of the analytical limitations we encountered.

CONCLUSIONS

This is the first attempt to analyze what it takes to implement self-management support tools across 16 practices from 4 practice-based research networks across the United States. Despite a short timeline and no project-supported facilitation assistance, 38% of the practices were still able to implement the boot camp translation-developed SMS tools to some extent. While not definitive, primary care practices that hope to implement new SMS tools require at least a system that enables innovation and change, the presence of a visible champion, and the alignment of SMS changes with other work.

Patient-Friendly Recap.

• Self-management support tools offered in a primary care practice may help patients manage chronic conditions like diabetes.

• Practices struggle to regularly implement tools designed to include direct input from patients.

• Practices wishing to implement these self-management tools need to ensure they have a visible clinic champion for this effort and a supportive health system, and that tools align with existing clinic workflows.

• While these factors may smooth the way, they do not guarantee successful implementation of patient support tools.