Core implementation strategies for improving cirrhosis care in the Veterans Health Administration

Abstract

Background & Aims:

The Veterans Health Administration (VHA) provides care for more than 80,000 Veterans with cirrhosis. This longitudinal, multi-method evaluation of a novel cirrhosis care quality improvement program aimed to 1) identify implementation strategies associated with evidence-based, guideline-concordant cirrhosis care over time, and 2) use qualitative interviews to operationalize strategies for a manualized intervention.

Approach & Results:

VHA providers were surveyed annually about the use of 73 implementation strategies to improve cirrhosis care in fiscal years 2018 (FY18) and 2019 (FY19). Implementation strategies linked to guideline-concordant cirrhosis care were identified using bivariate statistics and comparative configurational methods. Semi-structured interviews were conducted with 12 facilities in the highest quartile of cirrhosis care to specify the successful implementation strategies and their mechanisms of change. A total of 106 VHA facilities (82%) responded at least once over the 2-year period (FY18: n=63, FY19: n=100). Facilities reported using a median of 12 (IQR 20) implementation strategies in FY18 and 10 (IQR 19) in FY19. Of 73 strategies, 35 (48%) were positively correlated with provision of evidence-based cirrhosis care. Configurational analysis identified multiple strategy pathways directly linked to more guideline-concordant cirrhosis care. Across both methods, a subset of eight strategies was determined to be core to cirrhosis care improvement and specified using qualitative interviews.

Conclusions:

In a national cirrhosis care improvement initiative, a multi-method approach identified a core subset of successful implementation strategy combinations. This process of empirically identifying and specifying implementation strategies may be applicable to other implementation challenges in hepatology.

Graphical Abstract

Cirrhosis is a leading and increasing cause of US morbidity and mortality.(1) Several evidence-based practices (EBPs) can reduce morbidity and mortality in this population, including surveillance for HCC and surveillance for and prevention of variceal bleeding using endoscopy or non-selective beta-blockers.(2-4) Despite guidelines defining high-quality cirrhosis care, only one third of persons in the US (both Veterans and non-Veterans) receive guideline-concordant care.(3, 5-9)

Each year the Veterans Health Administration (VHA) provides care for >80,000 Veterans with cirrhosis.(10) To improve the quality of this care , VHA established the Hepatic Innovation Team (HIT) Collaborative(11), a national learning collaborative consisting of a leadership team and ~400 members organized into 18 regional teams at 130 facilities.(12) Learning collaboratives aim to build relationships and “hasten the diffusion of knowledge” to improve health care.(12-16) Within this national learning collaborative, VHA facilities can select implementation strategies, or activities used at the patient, provider and system level to improve cirrhosis care.(17)

While implementation experts have classified implementation strategies and developed recommendations for their specification, empirically selecting and specifying combinations of successful implementation strategies remains challenging.(17) Moreover, “analytic transparency” about how implementation strategies are selected is often lacking. As more emphasis is placed on advancing methods of causal inference, configurational comparative methods (CCMs), offer potential solutions.(18) CCMs provide a systematic way to assess all factors and cases in a dataset at once, analyze how specific combinations of conditions relate empirically to an outcome of interest, and then identify the key set of difference-makers that distinguish patients receiving high- vs. low-quality care.(18-21) Because CCMs differ from traditional probabilistic regression analytic methods, combining these methods with qualitative data enables convergent validation and mechanistic understanding.(19)This evaluation aimed to 1) identify implementation strategies associated implementation of EBPs for Veterans with cirrhosis, and 2) specify these core strategies for replication in facilities with opportunities to improve cirrhosis care.

MATERIALS AND METHODS

Study Design and Survey Development

The present study included an implementation strategy survey and qualitative interviews. Details of the study protocol have been previously published.(22) The survey was based on the Expert Recommendations for Implementing Change (ERIC) taxonomy of 73 implementation strategies organized into nine clusters.(23-26) Survey respondents were asked to report on whether their facility used each strategy to improve cirrhosis care in the prior fiscal year (FY) and to provide information about their specialty, degree, and years with VHA.

These data were collected to evaluate the HIT Collaborative for VHA’s HIV, Hepatitis, and Related Conditions Program Office. Per regulations outlined in VHA Program Guide 1200.21, this project was deemed a non-research operations activity. Participation in the evaluation was voluntary.

Setting & Participants

First, surveys were sent to clinicians and system redesign experts from across the 130 VHA facilities (per VHA population health definitions) after the end of FY 2018 (FY18) and FY19. Non-responders received two group and one individual email, following a modified Dillman approach(27), and the survey was promoted on national calls. Participants were asked to collaborate with or forward the survey to others knowledgeable about the facility’s processes, following previously published methods.(25, 26) Then, key informants from 15 facilities with the highest performance on national cirrhosis measures (see below) were invited to complete phone interviews.

Data Collection

Assessments of Facility Cirrhosis Care

Veterans’ data were extracted from VHA’s Corporate Data Warehouse, defining cirrhosis as two outpatient or one inpatient International Classification of Disease codes for cirrhosis or its complication or the inclusion of cirrhosis on a Veteran’s problem list, following validated approaches.(28, 29) Veterans with a VHA encounter in the prior 18 months were included and assigned to the facility of their most recent primary care encounter.

We evaluated two EBPs at the end of each FY: HCC surveillance and esophageal variceal surveillance or treatment (EVST). Definitions for these process outcomes were based upon literature and finalized by a multidisciplinary committee called the VA Hepatology Technical Advisory Group. HCC surveillance was thus defined as receipt of abdominal imaging with ultrasound or contrast-enhanced cross-sectional imaging within six months.(4, 30) EVST was evaluated for patients with cirrhosis and clinically significant portal hypertension (operationalized in VHA as platelets<150,000/mm³) and considered as met if Veterans had received an upper endoscopy within three years or were prescribed non-selective beta-blockers.(31)

Covariates

VHA classifies facilities into five complexity levels, ranging from 1a (most complex) to 3 (least complex), based on a composite measure of patient load and acuity, availability of complex services, research funding, and rurality.(32) Onsite availability of specialty gastroenterology/hepatology services was defined as present or absent.

Interviews with High-Performing Facilities

Clinicians from facilities in the highest quartile of care on both HCC surveillance and EVST were invited by email to complete a phone interview. The interview asked about use of implementation strategies, following recommended strategy specification guidance, including actor, action, target of the action, temporality, dose, implementation outcome, and justification.(17) The evaluation team conducted interviews with 1-5 clinicians from each facility in April 2020. Interviews were recorded and transcribed verbatim.

Data Analysis

Respondent characteristics and the frequency of strategy use in each year were summarized. Implementation strategy associations with HCC surveillance and EVST were analyzed using two distinct mathematical approaches: traditional statistical analysis and CCMs. Configurational analyses were conducted using the Coincidence Analysis package in R (“cna”)(33) R and RStudio Version 1.2.1335.

Identifying Strategies

First, relationships between individual implementation strategies (either used or not) and facility performance on HCC surveillance and EVST (continuous measures) were assessed using Pearson correlation tests. We then applied CCMs to assess potential combinations of implementation strategies that consistently distinguished facilities with high vs. low cirrhosis care, defined as above vs. below the median performance across VHA on each measure.(21, 34) CCMs draw upon Boolean algebra and set theory to identify combinations of strategies (configurations) to high performance.(18, 19, 35) Factors analyzed included facility and respondent characteristics and implementation strategies. Following a previously-published method, we used the “minimally sufficient condition” function within cna to analyze the complete dataset and exhaustively search all1-, 2- and 3-condition configurations instantiated in the dataset, retaining configurations meeting pre-established consistency (i.e., cases with the outcome and the solution divided by all cases with the solution) and coverage (i.e., cases with the outcome and combination divided by all cases with the outcome) thresholds, and then identify particular combinations of strategies with the strongest connections to the outcome. (20)

Specifying Strategies

Results from both regression and configurational analyses were used to identify a core subset of strategies consistently linked to evidence-based cirrhosis care in FY18 and FY19. Strategies that were positively significantly associated with evidence-based cirrhosis care via traditional regression analyses were compared to the set of strategies identified using CCMs. In the configurational analyses, we prioritized strategies and strategy combinations with a minimum of 80% consistency and 20% coverage. The consistency threshold ensured that facilities applying any given strategy or strategy combination regularly had the outcome of interest, whereas the coverage threshold was used to avoid overfitting by establishing that at least 20% of facilities with the outcome present were accounted for by any given strategy.(20) The strategies identified using each method were reviewed by the evaluation team to reach consensus for inclusion based on strength of connection to the outcome, overlap between methods, and anticipated implementation outcomes (i.e., acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, penetration, and sustainability).(36) For example, we have previously shown ERIC strategies focused on finances were both rarely used and unlikely to have strong connection to evidence-based care and were therefore deprioritized.(25, 26) The resulting core subset of strategies then served as the focus of the qualitative interviews with higher-performing facilities. Interviews were coded following the Proctor et al. recommendations for specifying implementation strategies.(17) Using a matrix synthesis analytic approach, we extracted detailed specifications from each facility, then integrated across facilities to establish a core operationalization for each strategy.(37)

RESULTS

Participant Characteristics

Of 130 facilities, 106 (82%) responded at least once; 63 responded in FY18 and 100 responded in FY19, with 57 of these facilities responding in both years. Respondents represented a range of facility complexities, specialties, and provider/staff types (Table 1). While facilities with and without survey responses had similar numbers of Veterans with cirrhosis (553±340 vs. 483±385 in FY18 (p=0.272), 577±410 vs. 515±397 in FY19 (p=.467), participating facilities had higher HCC surveillance and EVST rates than non-responders (HCC: 49% vs. 42%, respectively, in FY18 (p=.001), 53% vs. 44% in FY19 (p=.009); EVST: 55% vs. 49% in FY18, p=.003; 58% vs. 52% in FY19, p=.037).

Table 1.

Survey Participant Characteristics

	FY18 n (%)	FY19 n (%)
VISNs (of 18)	17 (94)	18 (100)
VAMCs (of 130)	63 (48)	100 (77)
1a	23 (37)	31 (31)
1b	11 (17)	20 (20)
1c	19 (30)	25 (25)
2	5 (8)	9 (9)
3	5 (8)	15 (15)
Degree
MD	16 (25)	27 (27)
PA/NP	15 (24)	33 (33)
RN	15 (24)	18 (18)
PharmD	14 (22)	20 (20)
Other	3 (5)	2 (20)
Specialty
GI	30 (48)	60 (60)
Pharmacy	14 (22)	20 (20)
Other (PC, ID, transplant)	19 (30)	20 (20)
Years in VA
0 to 3	8 (13)	18 (18)
4 to 9	14 (22)	32 (32)
10 to 19	30 (48)	38 (38)
over 20	11 (17)	12 (12)

FY=fiscal year; VISN=Veterans Integrated Service Network; VAMC=VA Medical Center; GI=Gastroenterology; PC=Primary Care; ID=Infectious Disease

Implementation Strategy and Cluster Use

Of 73 strategies, facilities reported using a median of 12 strategies in FY18 and 10 in FY19 (Table 2). The most popular strategies were from the Support Clinicians, Train and Educate Stakeholders, and Adapt and Tailor to the Context clusters. Strategies from the Financial and Evaluative & Iterative clusters were the least used. Some strategies were unused (e.g., “capitated payments”) while others were nearly universally used (e.g., “data warehousing techniques”). FY18 and FY19 strategy use patterns were similar, with the greatest between-year decrease in Building networks to promote problem solving (49% to 24%, p=.037). No strategies significantly increased in use over time.

Table 2.

Implementation Strategy Pathways Determined by Configurational Comparative Methods

	Path 1	Path 2	Path 3	Path 4
Model 1 (FY18)	S73: Promote demand for cirrhosis care among patients through any other means	S12*S28: Respond to proposals to deliver cirrhosis care (e.g., submit a HIT proposal to obtain money for your center specifically) AND Use data warehousing techniques (e.g., dashboard, integrating records across facilities)	S33*S37: Provide ongoing training in cirrhosis care AND Provide ongoing consultation with one or more cirrhosis treatment experts
Model 2 (FY19, with GI)	S06: Create or change credentialing and/or licensure standards (e.g., change scopes of practice or service agreements)	S05*S63: Mandate changes to cirrhosis care (i.e., have leadership declare that cirrhosis care is a priority) AND Collect and summarize clinical performance data and give it to clinicians and administrators to implement changes in a cyclical fashion using small tests of change before making system-wide changes	S18*S69: Create new clinical teams (e.g., interdisciplinary clinical working groups) AND Involve patients/consumers and family members	S09: Access new funding for cirrhosis care (this can include receiving funds from the HIT Collaborative to your center) AND Path 2 OR Path 3
Model 3 (FY19, no GI)	S54: Partner with a university to share ideas	S36*S70: Vary the information delivery methods to cater to different learning styles when presenting new information AND Engage in efforts to prepare patients to be active participants in cirrhosis care (e.g., conduct education sessions to teach patients about what questions to ask about cirrhosis)	S18, S47, S48, S52, S67 • Create new clinical teams (e.g., interdisciplinary clinical working groups) • Share the knowledge gained from quality improvement efforts with other sites outside your medical center • Identify and prepare champions (i.e., select key individuals who will dedicate themselves to promoting cirrhosis care) • Build on existing high-quality working relationships and networks to promote information sharing and problem solving related to implementing cirrhosis care • Intentionally examine the efforts to promote cirrhosis care

FY=fiscal year; S=strategy; GI=Gastroenterology

Individual Strategies Associated with Process Measures

Supplemental Table 1 illustrates correlations between individual strategies and facility-level care measures; number of strategies used was weakly correlated with HCC and EVST (r=.25-.43).

In FY18, six strategies were significantly associated with both measures, 18 with HCC only, and two with EVST only. The strategy with highest correlation with both measures was “Provide clinical supervision around evidence-based cirrhosis care” (r=.42-.50, p<.001).

In FY19, five strategies were significantly associated with both measures, 12 strategies with HCC only, and two with EVST only. The strongest association with HCC was “Tailor strategies to deliver cirrhosis care to address specific barriers in your center” (r=.39, p<.001), and EVST was most strongly associated with: “Revise professional roles” (r=.23, p=.022), “Intervene with patients to promote uptake and adherence” (r=.23, p=.023), and “Have someone from inside the clinic or center tasked with assisting the clinic in addressing implementation issues” (r=.23, p=.022).

Eight strategies were associated with higher HCC performance in both years: “data warehousing,” “information sharing and problem solving,” “conduct small tests of change,” “provide ongoing consultation with cirrhosis treatment experts”, “use data to tailor strategies to address barriers”, and “Identify the ways cirrhosis care can be tailored to meet local needs and while still maintaining the core components of evidence-based care” Two patient-oriented strategies “intervene with patients to promote uptake and adherence,” and “prepare patients to be active participants in their care.” Given fewer strategies were linked to EVST, none overlapped between years for EVST.

Several commonly used strategies were not associated with either clinical outcome (e.g., “conduct educational meetings”; “facilitate the relay of clinical data to providers”; “change physical structure and equipment”).

Strategy Pathways Linked to Process Measures

In FY18, three strategy pathways (representing five discrete strategies) collectively explained 75% of higher HCC surveillance at a consistency of 96%. These combinations also had 70% coverage and 88% consistency for EVST. The first pathway included a single strategy: “promote demand for care among patients”. The second included two strategies: “work with the national learning collaborative” together with “data warehousing techniques.” The third was likewise a conjunct of two strategies: “provide cirrhosis expert consultation” and “offer ongoing training in cirrhosis care.”

In FY19, the presence vs. absence of gastroenterology (GI) specialty care was an important covariate. Therefore, we secondarily examined strategy combinations for facilities with and without specialty GI care separately. For facilities with GI specialty care, five discrete strategies explained 65% of higher HCC surveillance with 100% consistency (Table 2). For facilities without specialty care in FY19, “partner with a university to share ideas” alone explained almost a quarter (100% consistency, 23% coverage) of higher HCC surveillance. In addition, five three-strategy combinations (Table 2) anchored by the pair of “Vary the information delivery methods to cater to different learning styles when presenting new information” and “prepare patients to be active participants in their care” each had 100% consistency and 23% coverage.

Identifying Core Strategies

Correlation analyses identified 39 strategies positively associated with cirrhosis care: 13 strategies were associated with both HCC and EVST, and 26 strategies independently associated with HCC or EVST alone. CNA identified 47 strategies in 207 combinations. Strategies associated with both measures in both correlational and CCMs were appraised. To identify the eight final core strategies (Figure 1), we evaluated correlation strength, CCM parameters of fit, and anticipated implementation feasibility/acceptability and excluded duplicative strategies and those that were delivered nationally, rather than by the local QI teams (e.g., national policy changes). Four patient-oriented strategies that were highly intercorrelated and overlapping, were collapsed into “engaging patients”.

Figure 1.

Example Patient Screen from VHA Cirrhosis Dashboard [Figure adapted with permission from Rogal et al. (16)]

Facilities in the highest quartile of performance used significantly more core strategies than did lowest quartile facilities (Supplemental Table 2). Table 3 illustrates HCC surveillance rates in FY18 and FY19 based on use of each core strategy either in no years, FY18 alone, FY19 alone, or both years. This shows that the largest absolute increase in HCC surveillance between FY18 and FY19 was among facilities newly adding core strategies in FY19.

Table 3.

HCC surveillance and core strategy use

	Not Used			Only FY18			Only FY19			Both FY18 + FY19
Core Strategy	HCC18	HCC19	Δ	HCC18	HCC19	Δ	HCC18	HCC19	Δ	HCC18	HCC19	Δ
1. Work with the National Learning Collaborative	48	54	6	58	59	1	49	57	8	54	52	−2
2. Use the Cirrhosis Dashboard	40	45	5	55	53	−2	45	51	6	52	58	6
3. Reach Out Directly to Veterans	50	54	4	45	45	0	48	55	7	54	59	5
4. Consultation with Cirrhosis Experts	47	53	6	51	55	4	49	57	8	55	58	3
5. Provide Ongoing Training to Local Clinicians	47	52	5	57	61	4	44	53	9	54	58	4
6. Use the HCC Clinical Reminder	51	52	1	45	55	10	44	54	10	52	57	5
7. Tailor Strategies to Deliver Care	48	51	3	50	51	1	48	58	10	54	60	6
8. Conduct Small Tests of Change	48	53	5	58	59	1	51	58	7	55	59	4
Average	47	52	4	52	55	2	47	55	8	54	58	4

Specifying Core Strategies

The resulting eight core strategies represented six of nine clusters. Two were technology-oriented (dashboard, clinical reminder), two patient-focused (patient outreach, tailoring care), two education-based (ongoing consultation, training), and two lean quality improvement-oriented (work with the national learning collaborative, small tests of change). These eight strategies were the focus of specification in interviews with higher performing facilities.

Of 15 invited facilities, 12 (80%) agreed to participate in interviews. Interviews were conducted with nine MDs, six NPs, five PharmDs, and four RNs from geographically diverse facilities across the VA with 2-31 years of VA experience. Following Proctor et al.’s guidance for individual strategy specifications, we identified several overarching themes. Actors, or people engaging in the implementation strategies, were diverse and included nurse practitioners, physicians, and system redesign staff, with overlapping roles, regardless of training background. Actions were similarly diverse, targeting patients, providers, and systems. There were differences in temporality (i.e., early vs. later implementation) and dose (i.e., occurred once vs. ongoing use). Most strategies were multi-component with pragmatic justifications. Table 4 details the eight strategies.

Table 4.

Feedback on High-Value (“Core”) Implementation Strategies from High-Performing Facilities

Strategy	High Performer Quote
Work with the HIT (1)	“I think it was a good way, especially early on, to exchange ideas and what was working…You know, if we say, “Well, [VA Medical Center 1] does this, why are we not doing this?…[W]e’re in the same VISN.” And some people say, “Well, this has never been done.” And through the HIT networking, you can say, “Well, actually, [VA Medical Center 1] does it, [VA Medical Center 2] does it, [VA Medical Center 3] does it, and why are we not doing it?” (Hepatologist)
Use the ALD Dashboard (data management tool) to track Veterans due for surveillance (2)	“I work with the dashboard and, in fact that’s what I’ve really been working on, is trying to clean up our dashboard and keep on top of things. So, we’re really good about our Hepatology team. I’d have to say that our [Advanced Practice Nurses] are pretty good about following up with their ultrasounds, but I go behind them and with the dashboard, and check to make sure that they’re ordering, if not, then order them and make sure that they’re getting their screening and their EGDs. But that’s all according to the dashboard.” (Nurse)
Reach out directly to Veterans (3)	“I think a lot of it has to do with our team approach of just kind of following, watching, being a team with primary care, getting the imaging, following them, having multiple conversations and phone calls to these patients. Having kind of a non-judgmental approach at the same time can be super helpful in getting their commitment on their end as well.” (Nurse)
Engage in ongoing consultation with cirrhosis treatment experts about Veteran care (4)	“We don’t have a hepatologist and so we’re just sharing with [hepatologist] in [VA Medical Center]. So, he has a liver [VA]-ECHO that I attend, and he does a lot of case discussion and anything that we need to reach out pertaining to consults…we utilize him.” (Nurse Practitioner)
Provide ongoing training to local clinicians about cirrhosis care (5)	“We have regular hepatology meetings. We go over didactic things with all of the providers at the VA. And then we do have case conferences at [Affiliated University] that includes the VA and [Affiliated University] providers, as well as fellows, and we try to go over guidances and guidelines through case-based discussions in those talks. There’s GI Grand Rounds, there’s lots of opportunities for [providers] to be up to date with AASLD and EASL and other society guidelines.” (Physician)
Use the HCC Clinical Reminder (6)	“[T]he other thing we did, which I think a lot of other VAs might have it, is the HCC clinical reminder, and then we link that to the imaging that is required for them. And now a lot of our patients who have been lost from follow-up are now being circled back, because my guess is that primary care sees that there is this screening due on their reminder page, and they have actually graciously ordered the one-time screening and then they have circled them back to our care.” (Physician)
Tailor strategies to deliver care (7)	“They’ll put in the orders, and they’ll request everything on the same date, especially when it’s someone who’s travelling…we’ve got a lot of rural patients who come up to three, four hours for their care and so in those patients, we do try…[T]hat might be an opportunity where they’ll tag [LPN] on it to say, ‘[LPN], can you just help put all the moving pieces into place.’” (Pharmacist)
Conduct small tests of change, measure outcomes, and then refine these tests (8)	“Like if something isn’t working, we will change it. An example is paper consults. I hate them. I hated them for a long time and one day I threw up my hands and said, ‘I’m done. I’m not touching paper anymore. I’m not running paper all over the building. This is asinine and I want to change it.’ So, I asked the few people I thought needed to be asked and I just started doing it different, and guess what? It worked really well.” (Nurse Practitioner)

HIT=Hepatic Innovation Team; VISN=Veterans Integrated Service Network; ALD=advanced liver disease; EGD=esophagogastroduodenoscopy; ECHO=Extension for Community Healthcare Outcomes; LPN=Licensed Practical Nurse

Core Strategy 1: Work with the national learning collaborative

The national learning collaborative is open to all in VHA, however participation varied. Respondents described participation that included attending national calls, sharing lessons learned with other local providers, engaging in ongoing education/training and networking with other clinicians. Facilities typically appointed ≥1 representative to join calls and relay information back to local/regional colleagues. Motivations for engaging included team building, skill building, and infrastructure expansion. For example, one provider described national face-to-face collaborative meetings as “amazing if for nothing but team building…when you have a feeling of a strong team, you feel a sense of responsibility and ownership.”

Core Strategy 2: Use the National Cirrhosis Dashboard to Track Veterans Due for Surveillance

The Cirrhosis Dashboard is a population management tool developed to identify Veterans who are overdue for cirrhosis care measures. All interviewees described using the dashboard, so it is outlined in detail (Figure 1, Table 5). While the person engaging with the dashboard varied, facilities with nurse practitioners “working the dashboard” had more success. Frequency of use varied, yet facilities generally described an initial push to review all patients followed by a slower, sustained effort.

Table 5.

Dashboard Strategy Operationalization

Specification	Operationalization	Quote
Strategy	Use the ALD Dashboard (population health management tool) to track Veterans due for HCC and EGD surveillance. Develop a system to track local patients and communicate information to clinicians to promote the innovation.	“[T]he Dashboard’s our total strategy.” (Pharmacist)
Actor (who)	Designated care team member who has: Computer skills Communication skills Permissions to write notes in the medical record Order tests (optional).	“I (RN) need the providers to tell me what they want, as far as scheduled stuff like every six months ultrasound, probably I can manage that if I can get a grip on who needs what. And I’m working on that right now, as a matter of fact with the ALD dashboard. I am finding quite a few that are dashboard managed in the community, and I’ve found quite a few that have never even seen hepatology for whatever reason. And I’m writing notes to those providers, and I co-signed Dr. NAME on one this morning, so that she can kind of see what I do with that. And I’m also creating a spreadsheet which is a little simpler to look at than the dashboard, for our providers so that they can see when we’re sort of running behind.” (Nurse)
Action (what)	Using population health management tool Identifying patients with cirrhosis (“scrubbing”) Prioritizing patients by disease stage Ordering HCC and EGD surveillance Team care management-informing clinicians of patient needs and assigning roles (who will order, who will educate patient, who will follow up with missed appointments.)	“So, I (NP) look at the dashboard basically trying to clean it up and get some people into care who are out of care, but I don’t, I don’t use it regularly and I don’t use it as a surveillance tool. So, I don’t use it to help me improve my surveillance numbers. I’m constantly, I actually use it to try to identify higher-risk patients and bring them back into care if they’re not in care.” (Nurse Practitioner)
Action Target (unit of analysis)	1) Care teams delivering liver care 2) Patients with cirrhosis	“And like, for our patients now, I put them on the dashboard. I put them on like the six-month suspension, cause we can’t do anything, and then what I do on that, the advanced liver disease note, I go ahead and put an addendum that the ultrasound is ordered and we’re waiting or that…the provider’s going to put the note, so that I can go back and look at that note, that suspension note, and see what I need to follow-up with.” (Nurse)
Temporality (when)	Early implementation to establish baseline need and ongoing to track patients over time.	“[W]e were able to actually clean up that dashboard, so that was, I think, our first project, and since then I think we have been able to keep up with the rate of variceal surveillance because we have been proactive.” (Physician)
Dose (frequency, intensity)	Ongoing, dependent on patient load (weekly, monthly, or quarterly). Average per-patient review is 15-30 minutes. Daily or weekly report to clinical team on patients due for screening.	“I have not been good about it…I mean in a perfect world I go in there; I think about it all the time…How often do I go in there? Not as much as I should. I did a project on it last year…and was in there all the time, of course. But I haven’t done so much since then.” (Nurse Practitioner)
Implementation Outcomes	Adoption: uptake Fidelity: adherence Penetration: spread Sustainability: institutionalization Primary: adoption, fidelity, penetration, sustainability Secondary: Acceptability, appropriateness, costs, feasibility	“[S]o if I noticed that the patient had a primary care appointment I would actually, if I was reviewing them on the Dashboard, I would put a note to myself to alert the provider like the day before or two days before their appointment. And that way it was fresh in their mind, ‘Hey, this patient’s coming in,’ and then, you know, ‘Please send them to imaging to schedule,’ …and it didn’t always work out that way, but you know I tried to make as few steps as possible.” (Nurse)
Justification (mechanisms of action)	Theoretical, empirical, and pragmatic justifications for population health management tools	“Orders get cancelled, patients don’t respond, no show, and that’s still an area that we haven’t quite figured out how to manage effectively, I guess. We’re, we’ve been using the Dashboard to help with that, so kind of identifying who’s overdue or who’s approaching their due date. (Nurse Practitioner) would then, if it was somebody who was already in the clinic, she would identify those patients and put a note in, like an addendum on the last progress note to the hepatology provider, and just ask that they reorder the imaging.” (Pharmacist)

ALD=advanced liver disease; EGD=esophagogastroduodenoscopy

After identifying patients overdue for surveillance, facilities either communicated with treating clinicians or reached out directly to Veterans (Strategy 3). The dashboard was also used to advocate for staff and engage primary care: “There was another project that spun out from this, and…an educational team meeting with primary care providers as well.”

Core Strategy 3: Reach Out Directly to Veterans

Most facilities developed methods for contacting Veterans directly to engage them in cirrhosis care, most often via phone or mail. Specifics depended on capacity and staffing, with outreach done by either primary care, gastroenterology/hepatology, or radiology, often with nurses taking on a central role and often following dashboard review (Strategy 2) : “[I]f they’re within our guidelines that I can schedule them and get them back into the clinic, I call them and just get them back in and tell them the importance of needing to follow up for their care for their cirrhosis. I’m very convincing.”

Contacting patients after a missed appointment was also viewed as critical in high-performing facilities. This often required cross-communication between departments and care teams. One nurse described how schedulers in radiology would “message me too if they are having trouble or difficulty getting a hold of someone and I have a little bit more time that I can call and be a little bit more persistent with the patients, too, before they discontinue the order.” Another team described using educational letters to ensure that patients were aware of the recommendation for twice-yearly HCC surveillance, while others communicated with Veterans at specific timepoints, such as after a hospital discharge or a non-VA hepatology appointment.

Core Strategy 4: Engage in Ongoing Consultation with Cirrhosis Treatment Experts about Veteran Care

High-performing facilities applied this strategy in two ways, including VA Extension for Community Healthcare Outcomes (VA-ECHO) and interdisciplinary case conferences (e.g., tumor board). VA-ECHO is a program through which specialists provide virtual consultation to primary care clinicians caring for Veterans in areas without specialty care.(38) Tumor boards are interdisciplinary forums for determining treatment and follow-up plans for patients with suspected HCC.(12, 39) While some facilities have individual tumor boards, others rely on regional boards to review cases. One transplant hepatologist summarized, “I try to diffuse as much knowledge as I can and it works both ways.”

Core Strategy 5: Provide Ongoing Training to Local Clinicians about Cirrhosis Care

Training and education activities were described as targeting local clinicians to disseminate cirrhosis guidelines. Clinicians either engaged with training opportunities at affiliated Universities, national online resources, local VHA education or developed targeted trainings to fill gaps or reach non-hepatology providers (e.g., primary care). Many revealed a “very academically motivated” culture where “people want to learn, people want to get better.“

Core Strategy 6: Use the HCC Clinical Reminder

The HCC clinical reminder is a tool available in the VHA electronic medical record system that alerts clinicians to order HCC surveillance imaging for eligible Veterans.(12, 40) Approximately half of high-performing facilities activated this reminder, reporting that it decreased the need for in-person specialty care visits for patients with compensated cirrhosis, increased continuity of care, and helped to institutionalize HCC surveillance. The facilities that used this strategy reported that educating primary care about the reminder and HCC surveillance (Strategy 5) contributed to successful implementation.

Core Strategy 7: Tailor Strategies to Deliver Care

High-performing facilities recognized that typical processes may not be effective for all patients. Facilities devised ways to address the needs of Veterans with specific access or clinical issues (e.g., rural Veterans, or those with mental illness or substance use) by tailoring care delivery through: 1) bundling or coordinating appointments to minimize travel for rural Veterans, 2) obtaining ordering privileges at outlying clinics to extend care, 3) introducing Saturday clinics, and 4) instituted collocated, multidisciplinary clinics for complex patients (e.g., homeless Veterans).

Core Strategy 8: Conduct Small Tests of Change, Measure Outcomes, and Then Refine These Tests

Several teams used the Plan-Do-Study-Act (PDSA) model of improvement. PDSA cycles involve iterative tests of change, measuring outcomes, and refining the approach. Such cycles enabled teams to preserve staff time and effort while maintaining a culture of continuous quality improvement. For example, clinical fellows at one facility developed a patient cirrhosis education letter (Strategy 3) then continuously improved based on pilot tests.

DISCUSSION

This multi-method evaluation identified, specified, and operationalized implementation strategies associated with improved cirrhosis care. Our application of correlational analyses, configurational analysis, and stakeholder interviews along with our triangulation of these results provides a novel approach for identifying and specifying implementation strategies. While we identified a core subset of eight strategies associated with evidence-based, guideline-concordant cirrhosis care, facilities reported using up to 52 of 73 implementation strategies annually. This multi-method approach offers a guide for identifying pathways to high-quality care, by selecting efficient and effective implementation strategies and strategy combinations.

These longitudinal data provided information about effectiveness over the first two implementation years of a national effort. Certain strategies worked better in one year than the other (e.g., “foster a collaborative learning environment,” “inform local opinion leaders”), while others were uniquely effective in the second year (e.g., “assess for readiness,” “local technical assistance”). Still others were institutionalized and maintained over time (e.g., Dashboard and engaging patients). Successful strategies were consistent with the literature describing successful collaboratives as those that reward participation, provide time for participation, set clear goals, and dedicate champion.(13, 16)

While some strategies were associated with improvements in both measures, others were associated with only one. This was expected, since the frequency, equipment, personnel, patient burden, and perceived evidence vary between HCC and EVST. For example, HCC surveillance was required every six months, while EVST was only every three years. Although every VA had on-site ultrasound capacity for HCC, only half of the facilities had on-site endoscopy. Therefore, it follows that “develop resource sharing agreements” to partner with endoscopy sites worked for EVST and not HCC. Conversely, “provide ongoing consultation with one or more cirrhosis treatment experts” worked for HCC not EVST since tumor board consultation was required to develop HCC screening processes.

In addition to identifying individually effective strategies, we also found combinations of such strategies. CCMs helped to further refine the bundle of data-driven strategies and qualitative interviews allowed us to specify the “form” and “function” of the strategies. For example, identifying patients with the Dashboard was more successful when followed by direct patient outreach. Strategies within combinations can target different stakeholders, as in this case, and work in concert. In a sense, the outreach to patients is a secondary strategy borne out of changing the provider behavior through the dashboard. Thus, interviews added to the CCMs output, providing granular information about the sequencing and pairing of strategies. Still, authors often fail to report on the definition, justification, temporality, and outcome of implementation strategies, leading to challenges with replicationand evaluation.(41, 42) While interviews helped to address this gap, some strategies were challenging to define (e.g., “tailoring care”) compared to others (e.g., “data warehousing”), with definitions that only emerged after several interviews.

While mechanistic research is in its infancy, our multi-method approach to strategy identification and definition, and subsequent testing of strategies, aims to address this gap. Michie et al. propose that behavior change occurs when three interrelated conditions are met: capability, opportunity, and motivation.(43, 44) These conditions may be understood as functions which may take a number of forms. Because form and function exist on two different dimensions, one strategy form may address several functions. For example, the Cirrhosis Dashboard strategy addresses capability through increasing knowledge of the patient population and local performance measures, provides opportunities to identify higher-risk patients, and may motivate providers based on low current performance and competition with other facilities. Conversely, several unique implementation strategies can address a single function. Configurational methods appear to be particularly well suited for studying mechanisms and causal complexity, in that minimally necessary and sufficient configurations can “signal” potential underlying causal mechanisms.

Though CCMs can reveal putative causal pathways, causal inference is challenging in retrospective evaluations. Facilities that employed “core” strategies had significantly increased HCC and variceal surveillance compared to other facilities. However, because the strategies were often employed at baseline, we cannot “prove” that adding these strategies improved performance. Therefore, observed associations between strategies and performance could conceivably result from confounding by organizational characteristics (e.g., larger facilities with stronger leadership were more able to use more strategies). However, there are several reasons why we believe that this is not the case. First, simply using more strategies was only weakly correlated with higher HCC and EGD surveillance, suggesting that it was selection of the “right” strategies, rather than total effort, that distinguished high performance. Second, facilities that newly used the core strategies in FY19 had the largest increase in FY19 performance. While it is possible that the facilities with contextual barriers cannot support implementation strategies used by higher performers, prior data suggest that barriers to cirrhosis care are unrelated to fixed facility characteristics (e.g., complexity or rurality).(45) Whether the strategies identified through this approach can be applied prescriptively to other settings is unclear, necessitating further testing of these strategies through a trial. Our ongoing work thus aims to “prescribe” successful implementation strategies to lower-performing facilities and evaluate the changes over time prospectively, while evaluating the impact of contextual factors.(22) (45)

More work is needed to evaluate the applicability of these findings to non-VHA settings. While VHA has notable advantages, including a unified electronic medical record system and data tools, the described tools were not immediately implementable at local levels. In fact, each facility independently installed the clinical reminder, analogous to non-VHA implementation. Likewise, tracking systems were initially developed regionally before the national dashboard was developed. Thus, while universal/national tools and medical record systems may not be feasible across smaller healthcare systems, other strategies and overarching principles likely apply outside of VHA.

Limitations

While this novel study advances knowledge about implementation strategy selection and specification over time, there were several limitations to this approach. First, strategy data were collected from key informants rather than direct observation. The findings have face validity, but self-reporting is prone to positive reporting bias. Also, because strategies were reported annually, within-year temporality could not be assessed. An additional limitation relates to the ERIC strategy taxonomy itself, which contains overlapping and double-barreled strategies and cannot feasibly address form or mechanism in detail, due to length. Many of these limitations were overcome through interviews and descriptions of strategy staging and sequencing. Future work will aim to improve the survey and develop pragmatic approaches for specification.

In conclusion, this study presents a multi-method approach for identifying and specifying data-driven implementation strategies linked to higher cirrhosis care and surveillance. This work builds upon our prior efforts to identify successful implementation strategies using national surveys and objective process outcomes. The general approach of using a broad survey and targeted interviews to define implementation strategies can address a wide range of implementation challenges.

Abbreviations:

EBP

evidence-based practice

VHA

Veterans Health Administration

HIT

Hepatic Innovation Team

CCM

configurational comparative method

ERIC

Expert Recommendations for Implementing Change

FY

fiscal year

VA

Department of Veterans Affairs

EMR

electronic medical record

EVST

esophageal variceal surveillance or treatment

GI

gastroenterology

VA-ECHO

VA Extension for Community Healthcare Outcomes

PDSA

Plan-Do-Study-Act