Feasibility and acceptability of patient- and clinician-level antithrombotic stewardship interventions to reduce gastrointestinal bleeding risk in patients using warfarin (Anticoagulation with Enhanced Gastrointestinal Safety): a factorial randomized controlled pilot trial

Abstract

Background

Overuse of antiplatelet therapy and underuse of gastroprotection contribute to preventable bleeding in patients taking anticoagulants.

Objectives

(1) Determine the feasibility of a factorial trial testing patient activation and clinician outreach to reduce gastrointestinal (GI) bleeding risk in patients prescribed warfarin–antiplatelet therapy without proton pump inhibitor gastroprotection and (2) assess intervention acceptability.

Methods

Pragmatic 2 × 2 factorial cluster-randomized controlled pilot comparing (1) a patient activation booklet vs usual care and (2) clinician notification vs clinician notification plus nurse facilitation was performed. The primary feasibility outcome was percentage of patients completing a structured telephone assessment after 5 weeks. Exploratory outcomes, including effectiveness, were evaluated using chart review, surveys, and semistructured interviews.

Results

Among 47 eligible patients, 35/47 (74.5%; 95% CI, 58.6%-85.7%) met the feasibility outcome. In the subset confirmed to be high risk for upper GI bleeding, 11/29 (37.9%; 95% CI, 16.9%-64.7%) made a medication change, without differences between intervention arms. In interviews, few patients reported reviewing the activation booklet; barriers included underestimating GI bleeding risk, misunderstanding the booklet’s purpose, and receiving excessive health communication materials. Clinicians responded to notification messages for 24/47 patients (51.1%; 95% CI, 26.4%-75.4%), which was lower for surgeons than nonsurgeons (22.7% vs 76.0%). Medical specialists but not surgeons viewed clinician notification as acceptable.

Conclusion

The proposed trial design and outcome ascertainment strategy were feasible, but the patient activation intervention is unlikely to be effective as designed. While clinician notification appears promising, it may not be acceptable to surgeons, findings which support further refinement and testing of a clinician notification intervention.

Essentials

• •Many anticoagulated patients would benefit from medication changes to reduce the risk of bleeding.
• •We tested interventions to facilitate safer prescribing practices in an anticoagulation clinic.
• •Few patients engaged with an activation booklet about medication changes to improve safety.
• •Clinician notification was acceptable and had promising results, warranting a larger trial.

1. Introduction

Anticoagulants are among the most dangerous medications because of the risk of hemorrhage, with major bleeding most often occurring in the gastrointestinal (GI) tract [1,2]. Two evidence-based practices can reduce the risk of bleeding in patients receiving therapeutic-intensity anticoagulation. First, for patients without recent thrombotic events or vascular interventions, guidelines recommend deprescribing concomitant antiplatelet medications [3,4]; the combination of anticoagulant and antiplatelet therapy increases the risk of major bleeding without reducing thrombosis [5]. Second, for anticoagulated patients in whom antiplatelet therapy is indicated, guidelines recommend proton pump inhibitors (PPIs) to reduce the risk of upper GI bleeding [4,6,7]. Both evidence-based practices are underused. Nearly one-third of anticoagulated patients are either prescribed unnecessary antiplatelet medications or prescribed antiplatelet medications appropriately but without a PPI [8].

Scalable and effective interventions are needed to address these gaps for patients at increased bleeding risk because of antiplatelet medications and/or nonsteroidal anti-inflammatory drugs [9]. An electronic health record (EHR)–based clinical decision support intervention had minimal effect on guideline-concordant nonsteroidal anti-inflammatory drug use, including PPI coprescribing [10]. Complex multicomponent interventions, most tested in Europe, using strategies such as education, incentive payments, clinician feedback, and pharmacist support, have had greater success [[11], [12], [13]], but their feasibility in the fragmented US healthcare system is unclear. Few studies have tested patient activation, in which patients are equipped with the knowledge, skills, and confidence to help ensure appropriate medication optimization [14,15].

Herein, we report the results of a randomized factorial feasibility pilot trial of antithrombotic stewardship interventions directed at both patients and clinicians to reduce the use of combination anticoagulation–antiplatelet therapy without PPI gastroprotection, as well as a qualitative evaluation.

2. Methods

The development of the intervention components and study protocol have been previously published and are summarized here [16].

2.1. Study design, setting, and participants

The Anticoagulation with Enhanced Gastrointestinal Safety (AEGIS) pilot study was a single-institution, pragmatic, multilevel, cluster-randomized controlled pilot 2 × 2 factorial trial to assess the feasibility of conducting a larger study in the future with a similar design and interventions. The setting was a tertiary academic medical center with a nurse-staffed anticoagulation monitoring service (AMS).

Patients were eligible if enrolled with the AMS and prescribed warfarin (with anticipated use for >90 days) and an antiplatelet medication (aspirin or P2Y₁₂ inhibitor) without a PPI, per the EHR. Patients were excluded if aged <18 years, intolerant or allergic to PPIs, or status post left ventricular assist device or heart transplant. Clinicians were eligible if designated as the “responsible provider” (recipient of AMS communications) for an eligible patient or a cardiologist who had seen an eligible patient in the prior year (with the exception of electrophysiologists).

2.2. Participant screening, selection, randomization, and dropout

We identified eligible patients and clinicians using an EHR reporting tool and assigned each patient to a single clinician cluster; patients were assigned preferentially to cardiologists. We then selected a stratified random sample of clinicians (6 cardiologists and 6 from other specialties). Clinicians (cluster level) were randomized 1:1 (K.M.K.) to 1 of 2 types of clinician notification (CN). Additionally, patients were randomized 1:1 (independent of clinician) to either a patient activation booklet or usual care education. As such, the design had 2 crossed factors with 4 cells—CN (2 levels) and patient activation (2 levels).

Since this was a pragmatic trial, there was no formal recruitment process for clinicians or patients, and patients discharged from the anticoagulation clinic were treated as dropouts thereafter, without further patient contact. There was no blinding.

2.3. Interventions

Two clinician-level interventions were tested, each delivered by 1 of 2 nurses:

• (1)CN consisted of a templated message sent through the EHR. It identified the patient as high risk for upper GI bleeding and recommended considering either discontinuation of antiplatelet medication(s) or initiation of a PPI, with a link to a clinical guidance summary on appropriate use of antiplatelet therapy (Supplementary File S1).
• (2)CN + nurse facilitation (NF) consisted of a similar message, but in addition, the nurse indicated each patient’s indication for antiplatelet therapy (after doing a chart review) and embedded the portion of the antiplatelet clinical guidance summary relevant to the patient (Supplementary File S1). The nurse also offered to order a PPI for the clinician’s signature and to communicate any medication changes to the patient.

Two patient-level interventions were also tested:

• (1)Usual care education.
• (2)An 8-page illustrated activation booklet that educated patients on GI bleeding and encouraged discussion of either antiplatelet discontinuation or PPI initiation with their physicians (Supplementary File S2). Nurses sent it by mail or electronic portal synchronized with the clinician intervention.

2.4. Quantitative endpoints

While we evaluated multiple dimensions of feasibility, we selected as the primary feasibility endpoint the percentage of patients completing a telephone assessment between weeks 5 and 8 after up to 3 attempts. This outcome is critical to feasibility since a similar strategy would be used to measure effectiveness in a larger future trial. The secondary feasibility endpoint was percentage of patients who received the assigned interventions, according to chart review at week 5.

In exploratory fashion, we assessed potential for effectiveness, defined as percentage of patients who self-reported either discontinuing antiplatelet therapy or initiating a PPI at weeks 5 to 8. We assessed protocol fidelity by chart review at week 5 to determine whether (1) the correct interventions were sent, (2) in the appropriate timeframe, and (3) to the correct recipients. Similarly, we ascertained clinicians’ response rate and time till response, and for CN + NF, frequency of facilitation and accuracy of antiplatelet indication identified by the nurse.

A physician (H.S. or J.K.) performed retrospective chart review for each patient to determine indication(s) for antiplatelet therapy at study entry and to categorize all recommended and completed medication changes as either concordant or nonconcordant with the antiplatelet clinical guidance summary sent to clinicians. When unclear, a cardiovascular medicine specialist (G.B.) made the final determination.

2.5. Power and statistical analysis

We planned to include 12 clinicians and ∼50 patients, which was judged to be feasible and provide acceptable CIs for estimates of effectiveness using generalized estimating equations (GEE) [17]. The primary endpoint was analyzed using GEE (logit link), including effects for patient-level intervention, clinician-level intervention, and the interaction term, accounting for clustering. A similar model was used to assess the secondary outcome. Exploratory outcomes were analyzed using descriptive statistics with CIs that accounted for clustering using an intention-to-treat approach; for proportions with very small denominators (<12), CIs were estimated using the Fay–Grabuard correction for the sandwich estimator from the GEEs. Only patients eligible for the week 5 call (ie, who received the interventions and continued in the AMS) were included in the primary analysis. Analysis of effectiveness was restricted to patients who completed the week 5 phone call and confirmed use of antiplatelet medications at baseline without a PPI.

2.6. Qualitative evaluation and analysis

Patients, clinicians, and nurses were invited for semistructured interviews about their experiences. Interview content and methods of analysis have previously been described [16]. During the week 5 telephone assessment, patients either randomized to the activation guide or who communicated with a clinician about medication optimization were invited to participate. Participating clinicians and anticoagulation nurses were invited by email. A member of the study team (J.E.K. and D.H.) conducted the interviews over Zoom or by phone. Detailed notes were kept, and interviews were recorded with permission.

Clinician interviews were analyzed using a rapid deductive analysis approach [[18], [19], [20], [21]] in a spreadsheet (D.H. and J.E.K.), with major categories informed by the theoretical framework of acceptability (TFA) [16,18,22]. We intended to analyze patient interviews using the TFA [18]; however, many patients lacked an understanding of the intervention components (as distinct from routine care), which limited their ability to speak to many dimensions of acceptability. Therefore, we utilized a rapid inductive analysis approach to identify the main barriers to effectiveness of the interventions. For the same reason, we also omitted reporting results of acceptability scale testing.

Because no primary care providers (PCPs) in the trial agreed to be interviewed, we interviewed a convenience sample of 5 PCPs who received the interventions after the conclusion of the randomized trial; they were not otherwise included in the pilot study.

2.7. Ethics and reporting

The study was approved by the University of Michigan Medical School Institutional Review Board, with waiver of informed consent for the interventions and week 5 phone calls. Verbal consent was obtained for patient semistructured interviews. Written informed consent was obtained for clinician and nurse interviews. We adhered to the Consolidated Standards for Reporting Trials statement extension for pilot and feasibility trials [23] and the COREQ criteria for qualitative research [24]. The study is registered with ClinicalTrials.gov (NCT05085405).

3. Results

3.1. Patient and clinician enrollment

Twelve clinicians and 51 patients were randomized, of whom 47 patients were eligible for week 5 phone calls and included in the primary analysis (Figure). Mean age was 61.7 years (SD, 13.6), 63.8% were male, and 83.0% were non-Hispanic White (Table 1). The 2 most common indications for anticoagulation were heart valve replacement (48.9%) and atrial fibrillation/flutter (38.3%). Based on physician chart review, 25% of patients had an indication for ongoing antiplatelet therapy, most commonly an On-X valve in the aortic or pulmonary position (12.8%) or peripheral artery disease with prior intervention (6.4%).

Figure.

CONSORT diagram. CN, clinician notification; NF, nurse facilitation; PPI, proton pump inhibitor.

Table 1.

Characteristics of patient participants.

Patient characteristics	Overall (n = 47)	CN & usual care (n = 13)	CN + NF & usual care (n = 13)	CN & patient activation (n = 11)	CN + NF & patient activation (n = 10)
Age – mean (SD), years	61.7 (13.6)	62.5 (12.8)	56.5 (15.2)	67.3 (10.9)	61.6 (14.6)
Age group – no. (%)
Age 18-59 – no. (%)	20 (42.6%)	5 (38.5%)	7 (53.8%)	3 (27.3%)	5 (50.0%)
Age 60-69 – no. (%)	10 (21.3%)	3 (23.1%)	2 (15.4%)	4 (36.4%)	1 (10.0%)
Age 70-79 – no. (%)	14 (29.8%)	4 (30.8%)	4 (30.8%)	3 (27.8%)	3 (30.0%)
Age 80+ – no. (%)	3 (6.4%)	1 (7.7%)	0 (0.0%)	1 (9.1%)	1 (10.0%)
Gender – no. (%)
Male sex	30 (63.8%)	6 (46.2%)	10 (76.9%)	7 (63.6%)	7 (70.0%)
Race/ethnic group – no. (%)
White, Not Hispanic	39 (83.0%)	11 (84.6%)	9 (69.2%)	11 (100.0%)	8 (80.0%)
White, Hispanic	1 (2.1%)	0 (0.0%)	1 (7.7%)	0 (0.0%)	0 (0.0%)
Black, Not Hispanic	3 (6.4%)	1 (7.7%)	1 (7.7%)	0 (0.0%)	1 (10.0%)
Asian, Not Hispanic	2 (4.2%)	0 (0.0%)	1 (7.7%)	0 (0.0%)	1 (10.0%)
Other, Not Hispanic	2 (4.2%)	1 (7.7%)	1 (7.7%)	0 (0.0%)	0 (0.0%)
Medical historya– no. (%)
Atrial fibrillation/flutter	18 (38.3%)	5 (38.5%)	6 (46.2%)	3 (27.8%)	4 (40.0%)
CAD	19 (40.4%)	7 (53.8%)	3 (23.1%)	3 (27.8%)	6 (60.0%)
VTE	9 (19.2%)	2 (15.4%)	2 (15.4%)	3 (27.8%)	2 (20.0%)
PAD	11 (23.4%)	4 (30.8%)	1 (7.7%)	4 (36.4%)	2 (20.0%)
Valve replacement	23 (48.9%)	4 (30.8%)	9 (69.2%)	2 (18.2%)	8 (80.0%)
Polycythemia vera	1 (2.1%)	1 (7.7%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
Cerebrovascular disease	7 (14.9%)	2 (15.4%)	1 (7.7%)	3 (27.8%)	1 (10.0%)
Indication for ongoing antiplatelet therapy – no. (%)
None	35 (74.5%)	10 (76.9%)	9 (69.2%)	9 (81.8%)	7 (70.0%)
CAD with intervention <12 months ago	1 (2.1%)	0 (0.0%)	0 (0.0%)	0 (0.0%)	1 (10.0%)
On-X valve in aortic or pulmonary position	6 (12.8%)	1 (7.7%)	3 (23.1%)	0 (0.0%)	2 (20.0%)
PAD with prior intervention	3 (6.4%)	0 (0.0%)	1 (7.7%)	2 (18.2%)	0 (0.0%)
Refractory/extensive vascular disease	2 (4.3%)	2 (15.4%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
Antiplatelet drugsa– no. (%)
Aspirin	43 (91.5%)	11 (84.6%)	13 (100.0%)	10 (90.9%)	9 (90.0%)
Clopidogrel	3 (6.4%)	1 (7.7%)	0 (0.0%)	1 (9.1%)	1 (10.0%)
Prasugrel	1 (2.1%)	1 (7.7%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
Ticagrelor	0 (0%)	0 (0.0%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
Other drugsa– no. (%)
Oral NSAIDs	1 (2.1%)	1 (7.7%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
SSRI	5 (10.6%)	3 (23.1%)	1 (7.7%)	0 (0.0%)	1 (10.0%)
Aldosterone antagonists	7 (14.9%)	2 (15.4%)	4 (30.8%)	1 (9.1%)	0 (0.0%)
Oral glucocorticoid	7 (14.9%)	4 (30.8%)	0 (0.0%)	1 (9.1%)	2 (20.0%)
Specialty of target clinician – no. (%)
PCP	2 (4.3%)	0 (0.0%)	1 (7.7%)	0 (0.0%)	1 (10.0%)
Cardiologist	22 (46.8%)	3 (23.1%)	7 (53.8%)	4 (36.4%)	5 (50.0%)
Neurologist	1 (2.1%)	1 (7.7%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
Surgeon	22 (46.8%)	9 (69.2%)	5 (38.5%)	7 (63.6%)	4 (40.0%)

CAD, coronary artery disease; CN, clinician notification; FN, nurse facilitation; NSAID, nonsteroidal anti-inflammatory drug; PAD, peripheral arterial disease; PCP, primary care provider; SSRI, selective serotonin reuptake inhibitor; VTE, venous thromboembolism.

^aMedications and medical history were ascertained using the electronic medication and problem lists at the time of screening.

We interviewed 11 patients and 12 clinicians, including 4 cardiologists, 3 surgeons, 5 PCPs (Table 2), and both AMS nurses. Representative quotations from interviews are included in the text and tables to support and inform interpretation of quantitative results.

Table 2.

Characteristics of clinician participants.

Clinician characteristics	Clinician participants in randomized trial			Participants in nonrandomized qualitative substudy
	Overall (n = 12)	CN (n = 6)	CN + NF (n = 6)	Overall (n = 8)	CN (n = 4)	CN + NF (n = 4)
Specialty, no. (%)
Cardiology	6 (50.0%)	3 (50.0%)	3 (50.0%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
Primary Care	1 (8.3%)	0 (0.0%)	1 (16.7%)	8 (100.0%)	4 (100.0%)	4 (100.0%)
Surgery	4 (33.3%)	2 (33.3%)	2 (33.3%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
Neurology	1 (8.3%)	1 (16.7%)	0 (0.0%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
Patients in Cluster, no. (%)
1-3	7 (58.3%)	4 (66.7%)	3 (50.0%)	8 (100.0%)	4 (100.0%)	4 (100.0%)
4-6	2 (16.7%)	1 (16.7%)	1 (16.7%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
7+	3 (25.0%)	1 (16.7%)	2 (33.3%)	0 (0.0%)	0 (0.0%)	0 (0.0%)

CN, clinician notification; FN, nurse facilitation.

3.2. Feasibility of outcome assessment, intervention delivery, and patient selection

Telephone assessments were completed for 35/47 patients (74.5%; 95% CI, 58.6%-85.7%; Table 3) after a mean of 1.81 (SD, 0.90) call attempts. In multilevel regression analysis, CN + NF (vs CN) was independently associated with higher call completion (odds ratio [OR], 4.03; 95% CI, 1.03-15.7); there were no differences by patient intervention arm. All patients received the assigned intervention with 100% fidelity. Among patients completing the week 5 telephone assessment, 29/35 (82.9%; 95% CI, 64.5%-92.8%) reported using antiplatelet therapy without a PPI at baseline, supporting reliability of the EHR for identifying eligible patients (Supplementary File S3).

Table 3.

Quantitative endpoints.

Endpoint	Overall n/N, % (95% CI)	Clinician-level interventions			Patient-level interventions
		CN n/N (%)	CN+NF n/N (%)	Odds ratio (95% CI)a	Usual care n/N (%)	Patient activation n/N (%)	Odds ratio (95% CI)a
Primary endpoint
Completion of week 5 assessment	35/47, 74.5 (58.6, 85.7)	15/24 (62.5)	20/23 (87.0)	4.03 (1.03,15.72)	19/26 (73.1)	16/21 (76.2)	1.24 (0.12, 12.73)
Secondary endpoint
Receipt of both assigned interventions	47/47, 100 (-c)	24/24 (100)	23/23 (100)	-c	26/26 (100)	21/21 (100)	-c
Exploratory endpoints
Initiation of either a PPI or discontinuation of all antiplatelet therapy at week 5 as determined by patient interview (Note: denominator is restricted to patients who completed week 5 assessment and reported using an antiplatelet medication at baseline without a PPI, n = 29)	11/29, 37.9 (16.9, 64.7)	4/11 (36.4)	7/18 (38.9)	0.91 (0.13, 6.3)	7/18 (38.9)	4/11 (36.4)	1.10 (0.24, 5.1)
Intervention fidelity, defined as receipt of intervention strategies as assigned within the appropriate week and with the correct clinician recipient	47/47, 100 (1, 1)	24/24 (100)	23/23 (100)	-c	26/26 (100)	21/21 (100)	-c
Patients reached by phone within 3 attempts at week 5	37/47, 78.7 (64.1, 88.5)	17/24 (70.8)	20/23 (87.0)	2.87 (0.65, 12.7)	19/26 (73.1)	18/21 (85.7)	2.34 (0.24, 23.1)
Documentation of a recommendation by one of the patient’s clinicians to discontinue antiplatelet therapy or initiate a PPI as indicated by a clinical documentation or by a change in the EHR medication list, according to chart review	24/47, 51.1 (26.3, 75.4)	9/24 (37.5)	15b/23 (65.2)	3.16 (0.39, 25.4)	15/26 (57.7)	9/21b (42.9)	0.54 (0.20, 1.42)
Concordance of clinicians’ recommendations for medication changes with clinical guidance summary given to cliniciansf(Note: denominator is restricted to patients who had a documented recommendation from a clinician to discontinue antiplatelet therapy or initiate a PPI according to chart review, n = 24)	15/24, 62.5 (0.30, 0.87)	6/9 (66.7)	9/15 (60.0)	-b	10/15 (66.7)	5/9 (55.6)	-b
The proportion of patients who self-reported communicating about medication optimization with their clinicians based on patient recall at week 5d,e(Note: denominator is restricted to patients who completed week 5 phone assessment, n = 35)	16/35, 45.7 (22.4,71.0)	4/15 (26.7)	12/20 (60.0)	-b	10/19 (52.6)	6/16 (37.5)	-b
Concordance of medication changes with clinical guidance summary given to clinicians among patients who made a changef(Note: denominator is restricted to patients who completed week 5 assessment and reported using an antiplatelet medication at baseline without a PPI and who reported making a medication change, n = 11)	9/11, 81.8 (0.36, 0.97)b	3/4 (75.0)	6/7 (85.7)	-b	5/7 (71.4)	4/4 (100.0)	-b

CN, clinician notification; NF, nurse facilitation; PPI, proton pump inhibitor.

^aEstimates of odds ratios for association between intervention group and primary and secondary outcomes were done using generalized linear mixed effects modeling, including effects for the patient-level intervention, the clinician-level intervention, and the interaction term for the 2, as well as random effects for clinician.

^bFor excessively small sample sizes, CIs and odds ratios were not displayed because of imprecision and failure of model convergence.

^cCI is not estimable.

^dDenominator excluded patients unable to be reached or who declined to participate in the week 5 assessment.

^ePatients who were uncertain about whether they had communicated with their clinician about a medication change (n = 3) were categorized as “no.”

^fSee Supplementary File S1 for clinical guidance summary provided to clinicians.

Clinicians responded to 14/23 (60.9%; 95% CI, 24.0%-88.4%) messages with CN + NF vs 10/24 (41.7%) with CN (Supplementary File S3). NF was requested for only a single patient in CN + NF. In CN + NF, the antiplatelet indication identified by the nurse was incorrect for 11/23 patients (47.8%; 95% CI, 30.3%-65.9%) compared with physician assessment as the reference standard.

In interviews, the nurse who delivered CN messages estimated it took 5 minutes (including the activation booklet) and said this posed little burden. The nurse who delivered CN + NF messages estimated it took 10 to 12 minutes, plus 5 minutes for the activation booklet, which could be challenging during busy weeks. Both nurses favored continuing the interventions.

“I don’t feel like it was taking away [from other responsibilities] because […] I feel this is a very worthwhile project. It’s definitely about patient safety, so, I feel that this is part of my job.” – Nurse who delivered CN + NF messages

The 2 nurses provided an average rating of 5 (SD, 0) for the Acceptability of Intervention Measure and Intervention Appropriateness Measure and of 4.88 (SD, 0.18) for the Feasibility of Intervention Measure (instrument score range, 1-5).

3.3. Potential effectiveness and clinician perceptions of notification interventions

Among patients who confirmed taking antiplatelet therapy without a PPI at baseline, 11/29 (37.9%; 95% CI, 16.9%-64.7%) either discontinued antiplatelet therapy (n = 8) or started a PPI (n = 3). There were no differences between CN vs CN + NF (OR, 0.91; 95% CI, 0.13-6.28) for the effectiveness outcome (Table 3). However, in interviews, many clinicians preferred CN + NF.

I appreciate that effort [in CN+NF], because, as you know, general internists are drowning in messages right now. And, honestly, if I had gotten the first message [CN], it would be like, “I will deal with that the next time I see the patient”…So, I would not have responded to the first message. The second message [CN+NF], no problem. If you’ll take it over, I’m happy to let you run with it. – PCP 5

By week 5, 24/47 (51.1%; 95% CI, 26.3%-75.4%) patients had received a clinician recommendation to either discontinue antiplatelet therapy (n = 11) or initiate a PPI (n = 14; 1 patient received both recommendations), which occurred more often with CN + NF (OR, 3.16; 95% CI, 0.39-25.43). Time to clinician response was also lower with CN + NF vs CN (0.7 vs 4.7 days; Supplementary File S3). Patients with a notification letter directed to a surgeon (vs other specialists) were less often recommended for a medication change (5/22 [22.7%] vs 19/25 [76.0%]) or to make a medication change (1/14 [7.1%] vs 10/15 [66.7%]).

In interviews, we identified differences in multiple dimensions of the TFA between medical and surgical specialists to explain the differential effectiveness (Table 4), including affective attitude, burden, ethicality, perceived effectiveness, and self-efficacy [18]. While responding to the messages could be burdensome to PCPs and cardiologists, they had an overall positive affective attitude and felt that the messages were appropriate. They viewed managing medication risks and preventive care as essential, and often rewarding, parts of their professional roles (“ethicality” per the TFA) and preferred to continue receiving similar notifications.

Table 4.

Representative quotations of clinicians’ perceptions of dimensions of acceptability by specialty.

Dimension of acceptability	Primary care providers	Cardiologists	Surgeons
Affective attitude How an individual feels about the intervention	“I think it’s a really good patient safety initiative. I think the harms, potential harms of antiplatelet therapy are just now being recognized, even within medicine, much less than kind of the lay public, and so this is just kind of one more step away from the reason why we’re not putting aspirin in the water, right, because of the internal bleeding risk, especially for the older folks, many of whom have a. fib or whatever and are on DOACs or warfarin.” – PCP 4 "First of all, I love, you know, if there’s someone identified who would benefit from evidence-based change that I’m not perfectly aware of, it’s hard to find all these little things to manage. It’s really nice when it’s something that can be a high-risk thing, to give me awareness of that. So, I really appreciate that." – PCP 2 “I would be ticked if I got another portal message because you sent them [my patients] something in between appointments. Ugh…There ought to be an email to all of us saying, ‘this is a project over the next 6 months, or the next year, people who fall into this category of being high risk for GI bleed on anticoagulation and not on omeprazole will receive a letter to discuss with their doctor whether to start a PPI or stop one of their medications’. I mean, to hit me, to have no idea where this came from. And you don’t understand how many times people say, ‘oh, I saw this on the internet.’ So, I would have no idea what, and patients are frequently very inarticulate about where things came from.” – PCP 5	“I am [interested in receiving more messages like this]. I consider them my patients and if I make mistakes or it needs to be reviewed, I certainly want to know about it.” – Cardiologist 1 “I mean the inbox is the inbox, you know how that goes. But it’s like I said, this is important and it’s important enough that I certainly didn’t mind receiving messages about it.” – Cardiologist 2 “Yes, I think it makes sense [to continue sending clinician notification messages]. I mean, it’s a good checkup. Sometimes you don’t think about – how many people have those drugs on for years, and you just don’t think about or forget about it or whatever.” – Cardiologist 3	“I recommend that you continue sending them [clinician notification messages] because I think it’s important. The more redundancies we have to the system in this way, I think is better. I think too, I do like the idea of patient outreach because some patients are just super adamant against taking another medication, and I’ve had patients just refuse, because they just don’t want another medication and they don’t see it as necessary. So, getting some good education to the patients I think is important.” – Surgeon 3 “I don’t know if it’s just surgeons. We’re all so damn busy that, you know, this is not as much of a burning platform for me as it is for you, so I’m not going to read this [notification message].” – Surgeon 2
Burden The perceived amount of effort that is required to participate in the intervention	Interviewer: “Did it take much time to sort of like think through whether the patient should stop his aspirin or start the PPI?” Clinician: “A little bit, just because they weren’t on my radar, and I didn’t know them that well. But if it was someone that I knew a little bit better, probably wouldn’t have. And I think that the person that I don’t know very well, it’s even more important to send me that information.” – PCP 2 “It probably took me a half hour of time to take care of this one person. So, that would be the other thing is how do we streamline this because it wasn’t easy. You know, I had to page back years to find a neuro-ophthalmology consult and then they didn’t actually consult hematology, they did a verbal or written consult, it was written but it wasn’t a visit, so I had to figure out does this person actually need to see hematology or, you know how do we get there from here. So, it wasn’t, it’s not something that’s a good add-on into the end-of-your-day workbasket.” – PCP 3	“I just think, you know, if I get a lot of, a lot of those letters, it takes a lot of time and effort to respond to it, and that, I think, is the issue.” – Cardiologist 1 Interviewer: “Did this [responding to the clinician notifications] feel burdensome or that it was taking you away from other things that you needed to do?” Clinician: “Oh yeah. Every time I get an in basket thing I shudder. But, nevertheless, that’s just life. So [LAUGHS]. But, when it comes to patient safety, then that’s what it is. But, yes, I find it burdensome.” – Cardiologist 3	“I worry a little bit about the length of the message and whether or not it’s too onerous for a busy provider. You know [the name of a surgical APP] is juggling, you know, hundreds of patients and it might be laborious to do that sort of stuff.” – Surgeon 2 "There are lots of medications that cause risk and we do not send letters to providers to ask them whether they want to add other protective drugs to prevent adverse outcomes. Sorry, I am acutely hypersensitive to MyChart [EHR] traffic and workload and don’t want to see this become hundreds of patients I have to respond to about this." – Surgeon 1a
Ethicality The extent to which the intervention has good fit with an individual's value system	“So, who does the patient contact when they a have a bleed or who follows-up when they get hospitalized for the bleed? It’s the PCP, it’s not the specialist. And so I kind of feel like the holistic management of multiple medications kind of falls on the PCP in this situation.” – PCP 1 “I freely admit my immense bias toward prevention, that’s why I do what I do for a living. So, I’d much rather, like I said, I’d much rather that projects like this send more messages to PCPs or cardiologists than have even one patient end up with a preventable GI bleed.” – PCP 4 "So, the only people I have that are on antiplatelet [therapy] are people that cardiology, or occasionally neuro, has put on antiplatelet [therapy]. So, then it’s not my call, it’s their call…The bottom line is when it comes to antiplatelet, it’s neuro and cardiology that have to call that, because I’m never the one putting them on that.” – PCP 5	“I think it’s appropriate that it comes to me. I mean, they’re cardiology patients so we’re the ones usually giving the aspirin and the Coumadin. So, yeah, it would make sense we would make that decision. I wouldn’t send it to the PCP, because then we’re just going to get another message from the PCP, ‘Can I stop the aspirin?’ I would just send it directly to us [cardiology].” – Cardiologist 3 “I think if the question is do you want to stop the aspirin, that’s probably more the question for the general cardiologist than it would be for me. If the question is do you want to stop this anticoagulation, then that might be more for somebody like myself.” – Cardiologist 4, who specializes in cardiac procedures	“It [optimizing medical therapy] is an important thing to address, but I am not going to address it because I don’t have time and it’s not my primary objective.” – Surgeon 1b “We follow the patients until their postop visit and then usually we just release them back to somebody else. Because, you know, we’re their short-term doctor, not their long-term doctor…But we are not necessarily the long-term investment for the patient like a cardiologist is. So, there may be some gain from it, we may respond and make changes to it [medical therapy], but long term I think your bigger bang is going to be through cardiology.” – Surgeon 2
Perceived effectiveness The extent to which the intervention is perceived as likely to achieve its purpose	“I think it generally worked well. You know, I appreciate somebody looking out for those interactions, those things that, you know, I hadn’t really thought about. In his [the patient’s] case, he actually needed both, so it didn’t change anything, but it made me document it more clearly and pull it up to where somebody else can see it. Because one day I’m going to retire or he’s going to change doctors or something. And try to keep that into the problem list as much as we can and not buried back into some computer system that we used to use.” – PCP 3 “So particular to your intervention, I think it’s a great intervention, and in fact, I’ve had one patient that was caught by your QI project. And it turned out, just like can happen, his med [PPI] had just fell off of his medication list and it wasn’t something that I was very practically looking for. He’d been off of it for a year, but he actually had been on a PPI for dual antiplatelet therapy or dual anticoagulation, and it just dropped off his medication list and I didn’t catch it when I saw him that year.” – PCP 1	Interviewer: “So, overall, how effective do you think the messages were in helping you reduce bleeding risk in your patients?" Clinician: “At attempting to reduce bleeding risk? Very effective. If I actually have reduced the bleeding risk, I don’t know yet… No, I think it’s good. It’s a second set of eyes to review stuff.” – Cardiologist 1 “I found the notifications helpful, and I did change some of my patients’ regimens based on the recommendations. I think the format in which the recommendations were delivered also made sense.” – Cardiologist 2 “And I think the problem is maybe moreso the way that our practice works as a proceduralist clinic, too, you know. This may be a more effective means of communication for people that are more clinic-based. But for us, you know, if I’m at work for 10 hours, probably 6 to 8 of those hours I’m scrubbed in without access to a computer. And so, to sit down and try to do some of your Epic stuff and to have these complex ways of responding to communications is really difficult.” – Cardiologist 4, who specializes in cardiac procedures	Interviewer: “So, I don’t know if you recall seeing any of these [clinician notification] messages about this? Clinician: Are they in MyChart [the EHR]?” Interviewer: “They’re somewhere in MyChart [the EHR], yeah.” Clinician: “Yeah, I don’t read any of that shit [LAUGHS]. Way too many things in there to read. I can’t keep up.” – Surgeon 2 “I think it’s a great idea to highlight the risk [of GI bleeding]. I think it’s hard as a nonmedical provider whose clinical pathway is not to prescribe meds and our APPs and nursing support are instructed not to refill meds, they [these alert messages] should go back to the PCP or the specialist who ordered them.” – Surgeon 1b
Self-efficacy The participant's confidence that they can perform the behavior(s) required to participate in the intervention	“It was an opportunity to nip something in the bud before there was a problem. Which, as a primary care, preventive medicine doctor, is my absolute, 100% favorite thing to do. And he [the patient] was like, ‘Great, one less pill a day,’ and like thanked us for letting him know…But it was really nice to have that very clear-cut guidance so we could reach out to him and with a simple phone call from my nurse potentially prevent a disastrous situation.” – PCP 4 “So, in family medicine we think we own everything. But we’d like some time to do that… I think it’s right to send it to family medicine… I might not have the answer, but I can find who [does] and it [the answer] comes back to me, and I prescribe it. That works well. It’s just, do we really have the resources to do it, and to do it well? We’re kind of drowning in that stuff now, but it’s what patients need. So, I think it’s the right thing to do, but whether we can really keep up with it, I don’t know.” – PCP 3 "I think the difficulty with PPIs in general is just the extreme variety of opinions you hear about them and applying that individually sometimes gets lost in the shuffle when we have tried to de-escalate PPI use as a generality in the last 10 years, perhaps." – PCP 1	Interviewer: “One barrier we clearly identified is that people don’t want to stop a drug that someone else has started. It’s like this, you know, stepping on someone else’s toes.” Clinician: “I mean, it’s not even sort of like, am I going to offend that person? It’s more like, do they [the antiplatelet prescriber] know something I don’t know about this patient’s care that means this drug is important. And a lot of times it will seem to me like we could stop it [aspirin], but, you know, I don’t know, was there something else that came up that they [the antiplatelet prescriber] know. That’s sort of the main thing. You know, if the question is would I feel comfortable starting a PPI for somebody, yeah, I’d have no problem with that. It’s more about stopping a drug, I think, that there would be more worries.” – Cardiologist 4 “There are still a lot of folks where we can really safely reduce the aspirin, I think. I am seeing myself doing this. And then, like, we have some patients with really specialty needs, and I really have to think carefully or consult some other providers to see if they should stop the aspirin or not. And those are usually related to patients who recently had CT surgery, and I really want the surgeon to weigh in on stopping the aspirin." – Cardiologist 1	“This type of action was really clunky for me to figure out how to do since I don’t know hardly any of the PCPs in the health system, and I don’t know how to get a hold of them.” – Surgeon 1b “Communications are tough. Like, I’ll look through them… I know some people in my group who have not looked through their encounter communications at all, like because that’s where like all the pathology results and lab results and, you know, they fall into results section, and then the communications they get where they’re cc’d, they don’t even look at those things. And I have some of my partners who have, like, I’m not joking, 1,000 communications that are unread.” – Surgeon 3

APP, advanced practice provider; CT, cardiothoracic; DOAC, direct oral anticoagulant; EHR, electronic health record; GI, gastrointestinal; PCP, primary care provider; PPI, proton pump inhibitor; QI, quality improvement.

^aFrom email communication.

^bParaphrased from interview memo notes.

On the other hand, most surgeons felt the notification messages fit poorly with their professional priorities and capacities and posed an excessive burden. Surgeons described spending less time using the EHR and said their relationships with patients were mostly limited to the perioperative setting. Overall, surgeons’ perceived effectiveness of the intervention was low unless the notification was directed at other members of their teams, like advanced practice providers.

While PCPs and cardiologists felt capable of reviewing and responding to the intervention (self-efficacy), some expressed concerns about stopping antiplatelet medications previously started by another clinician, who may have had greater insight into the rationale, especially if the patient had used it long-term without problems. Others noted that increasing emphasis on PPI’s adverse effects and minimizing polypharmacy influenced their decision making.

Clinician suggestions for improvement included sending messages timed with appointments, redesigning workflow to reduce the clicks required to respond to messages, and confirming patients were using antiplatelet therapy prior to CN.

All clinicians who were interviewed completed the Acceptability of Intervention Measure scale, except one, who had not seen any of the messages [25]. The average score was 4.04 (SD, 0.82). Only 2/11 (18.2%) clinicians (both surgeons and proceduralists) scored any scale items as completely disagree or disagree.

3.4. Potential effectiveness and patient perceptions of activation and notification interventions

There were no differences between activation and usual care in the effectiveness outcome (OR, 1.10; 95% CI, 0.24%-5.08%). Among patients randomized to the activation booklet who completed the week 5 telephone call, 6/16 (37.5%; 95% CI, 19.3%-60.2%) reported receiving the booklet, and only a single patient recalled reviewing it; 6/16 recalled communicating with their clinicians about medication optimization, none of whom initiated contact. Patient interviews revealed 4 major barriers to the success of the activation intervention.

First, several patients had a poor understanding of the intervention and its rationale. Patients were not aware of their risk for GI bleeding, although they had been advised of bleeding risks more generally with warfarin.

“They [medical personnel] just say internal bleeding whenever they warn me about it. And that bruises can last twice as long basically. I’m trying to think of any other warnings; I don’t remember anything stomach-wise specifically.” – Patient 3

In addition, some patients had difficulty understanding why they had received the activation booklet after having taken aspirin and warfarin together for years without problems.

Second, many patients reported receiving a glut of health communication materials by portal and by mail and had a tendency to ignore them.

“If it’s got a pamphlet in it, I pretty much just pitch it in the recycling bin. I’ve been on it [warfarin] long enough to not have to read that stuff.” – Patient 6

Third, some patients placed great trust in their clinicians, which they felt obviated the need for any intervention. This belief appeared to apply particularly to surgeons.

“I trust [my surgeon] with anything. And that [outreach] would be good. But I think he knows everything about that. You know… I would think if there’s something I need to be told, he’d tell me.” – Patient 10

By the same token, if a trusted clinician did recommend a medication, patients were likely to agree.

Fourth, some patients believed that the benefits of a medication change might not outweigh the risks. With regard to the possibility of stopping aspirin, one patient commented:

“I’m more worried about the thing [stent] clogging up! Hah, I’m more worried about that, I’m not worried about the bleeding. I’m worried about the thing clogging up, that’s why I’m taking the Coumadin.” – Patient 10

Other patients viewed the small risk of GI bleeding as acceptable.

[My surgeon’s APP] described to me that study and that they had found that there’s an increase in bleeding ulcers. But he also said it was only 2-3%, and I said, “hmmm, I think I can risk it.”… I know that sounds terrible, but like, I already have enough medical issues. If adding a new med, if it’s a possibility of 2-3% chance [of gastrointestinal bleeding]…I’m going to pray I’m in the 97%. It just takes a little more convincing, I guess. – Patient 11

Patients had greater enthusiasm for the anticoagulation clinic directly contacting clinicians rather than patients about medication changes.

“If this [patient activation] was sent to me, I would probably ignore it, but, I mean, having something [a clinician notification] sent to my doctor… I mean, heck, I don’t even know if he would pay attention to it. But I would be alright with the information being offered to him to make sure he has the information.” – Patient 3

3.5. Appropriateness of medication changes

Of 24 patients recommended for a medication change, chart review demonstrated the recommendations were consistent with the guidance summary provided to clinicians in 15 (62.5%; 95% CI, 0.30, 0.87; Table 3). Eight patients who were likely appropriate for antiplatelet discontinuation were recommended for PPI initiation instead, while only 1 patient who was likely appropriate for PPI initiation was recommended for antiplatelet discontinuation (Supplementary Table S2). Of patients who completed a medication change, the change was consistent with guidance provided to clinicians in 9/11 (81.8%; 95% CI, 0.36, 0.97), of whom 8 stopped antiplatelet therapy and 1 started PPI.

Additional exploratory outcomes are reported in Supplementary File S3.

4. Discussion

In this study, we demonstrated the feasibility of a pragmatic, cluster-randomized controlled factorial trial of patient- and clinician-facing implementation strategies to reduce high-risk antiplatelet use in patients taking warfarin and treated by an AMS. We were able to reach a majority (74.5%) of patients by telephone to assess the primary effectiveness outcome, reliably randomize patients and clinicians, and deliver the assigned interventions as part of a pragmatic design. This work also highlights the value of a factorial pilot trial and the inclusion of qualitative methods in developing a multicomponent intervention consistent with the Multiphase Optimization Strategy [26]. The CN interventions showed promising results for reduction in high-risk antiplatelet prescribing in patients prescribed warfarin but were less effective for surgeons, while the patient activation intervention was less encouraging. The qualitative findings allowed us to gain a deeper insight into mechanisms underlying these quantitative findings and point toward potential refinements of the interventions.

Clinicians who received either clinician-facing intervention recommended a medication change for 51% of patients, and 38% of patients ultimately either discontinued their antiplatelet medications or initiated a PPI. While limited by a small sample size, we did not find differences in the effectiveness of CN and CN + NF. However, in interviews, many clinicians expressed a preference for the convenience of CN + NF, and there was a trend toward a higher response rate in shorter time with CN + NF. Unexpectedly, while many clinicians preferred CN + NF, very few took advantage of the nurse’s offer to provide patient education or to order the PPI. We suspect some clinicians may not have understood that the nurse was able to provide these services, which will need to be clearer in future iterations of the intervention.

Regarding our finding of lower effectiveness for notifications sent to surgeons, in interviews, most surgeons reported that they do not view managing long-term medications as their responsibility and spend less time responding to messages in the EHR; indeed, they had lower response rates to the notification messages compared with medical specialists. It has previously been found that surgeons have more limited EHR use and clear-cut boundaries in their work [27]. One broad implication of our findings is that anticoagulation services must carefully consider to whom they address messages seeking to address quality care gaps.

In contrast to the CN messages, the patient activation booklet appears unlikely to be effective as currently designed since few patients reviewed the booklet, and none initiated contact with their clinicians as a result, which was its intended mechanism of action. We identified several reasons for this (eg, low salience of GI bleeding to patients, receiving a glut of health communications). Methodologic differences between our study and others that found patient activation to be effective may also have played a role [28,29]. First, delivering the activation/education tool synchronized with, and ideally shortly before, a clinical encounter, as was done in a successful trial of a low-literacy education tool to improve pneumococcal vaccination, may reduce the barriers to discussion with a clinician [29]. Second, as a pragmatic quality improvement study, we did not recruit or consent patients beforehand, which may serve to select patients predisposed to respond and prime them for the intervention.

The rate of patient participation in our study is similar to other US and Canadian studies in which telephone calls were used to either screen for study participation or deliver an intervention, with noncompletion rates of 22% to 42% [[30], [31], [32]]. Our findings underscore the importance of prespecifying sound strategies for dealing with missing data in a larger subsequent trial, such as multiple imputation. Multiple imputation is feasible when up to 40% of data are missing for the dependent variable in a clinical trial [33].

Several findings have implications for intervention refinement. Our finding of low agreement between the nurse delivering CN + NF and physician reviewers on patients’ indications for antiplatelet therapy does not support having the nurse perform this activity. The nurse providing this information was intended to save clinicians time in determining the appropriateness of antiplatelet therapy. Because only a single nurse delivered the CN + NF intervention, this finding should be interpreted with caution and would benefit from further study. Our finding that clinicians’ recommendations for medication optimization tended to err on the side of initiating a PPI when discontinuing the antiplatelet medication would have been appropriate may reflect lingering uncertainty about when antiplatelet deprescribing is appropriate and concern that doing so could increase the risk of thrombotic events [34]. Clinicians may benefit from additional education on this topic.

This study has some limitations. First, findings about potential effectiveness should be viewed as preliminary since the study was not powered for this. Second, as part of our trial, most clinicians received relatively few notification messages over a short period of time. It is possible that more frequent and sustained CN messages could contribute to alert fatigue and diminishing effectiveness. Third, the study was conducted in a single center, and the study sample was predominantly White and non-Hispanic with a mean age of only 61.7 years; the results may not generalize to other settings, especially those with different anticoagulation management structures or staffing (eg, pharmacists instead of nurses) or differences in patient characteristics. The study also has notable strengths, including the factorial randomized controlled design, the rigorous intervention development process, and the mixed-methods evaluation.

In conclusion, the pragmatic factorial design and outcome ascertainment strategy we used were feasible, and the CN strategy appears promising, but the patient activation strategy, as designed and delivered, may have little effect. CN appears to be a promising strategy but may not be acceptable to surgeons. These conclusions support further refinement and effectiveness testing of a CN intervention prior to widespread adoption as a standard antithrombotic stewardship activity.