Benefits and Sustainability of a Learning Collaborative for Implementation of Treat to Target in Rheumatoid Arthritis: Results of the TRACTION Trial Phase II

Abstract

Background

We conducted a two-phase randomized controlled trial of a Learning Collaborative (LC) to facilitate implementation of treat to target (TTT) to manage rheumatoid arthritis (RA). We found substantial improvement in implementation of TTT in Phase I. Herein, we report on a second 9 months (Phase II) where we examined maintenance of response in Phase I and predictors of greater improvement in TTT adherence.

Methods

We recruited 11 rheumatology sites and randomized them to either receive the LC during Phase I or to a wait-list control group that received the LC intervention during Phase II. The outcome was change in TTT implementation score (0 to 100, 100 is best) from pre- to post-intervention. TTT implementation score is defined as a percent of components documented in visit notes. Analyses examined: 1) the extent that the Phase I intervention teams sustained improvement in TTT; and, 2) predictors of TTT improvement.

Results

The analysis included 636 RA patients. At baseline, mean TTT implementation score was 11% in Phase I intervention sites and 13% in Phase II sites. After the intervention, TTT implementation score improved to 57% in the Phase I intervention sites and to 58% in the Phase II sites. Intervention sites from Phase I sustained the improvement during the Phase II (52%). Predictors of greater TTT improvement included only having rheumatologist providers at the site, academic affiliation of the site, fewer providers per site, and the rheumatologist provider being a trainee.

Conclusions

Improvement in TTT remained relatively stable over a post-intervention period.

Introduction

Treating to target (TTT) has been accepted as a preferred paradigm in rheumatoid arthritis (RA).(1-3) Randomized controlled clinical trials consistently demonstrate that TTT results in better outcomes compared with usual care in RA.(4, 5) The TTT strategy entails consistent use of a disease activity measures with adjustment in therapy until the target disease activity is reached. This process requires selection of a target and shared decision-making between provider and patient. While the target disease activity is remission in many patients, some patients may be unlikely to reach remission and providers and patients determine that low disease activity is a more appropriate target. Most rheumatologists believe that they manage RA using a TTT approach but evidence suggests that this strategy is not consistently practiced. One study from Corrona, the largest US-based RA registry, found that a minority of patients who had moderate or high disease activity at a given visit received treatment changes in the next 6-12 months.(6, 7) Another large Australian cross-sectional study surveyed rheumatologists regarding reasons they do not adjust RA treatments at visits where patients were found to be in moderate or high disease activity. Responding rheumatologists reported irreversible joint damage and patient preferences as common reasons for not escalating treatment.(8)

In the face of strong evidence supporting TTT and some data suggesting poor adherence with this paradigm in managing RA, we developed and tested a Learning Collaborative (LC) intervention to enhance adherence with TTT. Learning Collaboratives have been successfully used in health care (9-11) These interventions typically develop a set of principles that characterize best practice and then work with care teams, including health care providers and staff, using rapid cycle tests of change.(12) These tests must align with the agreed upon principles, and the teams frequently measure key indicators for care improvement. Teams work collaboratively across sites, sharing successes and failures.

We tested an LC in a randomized controlled trial that produced large and significant improvements in adherence to TTT principles.(13) The trial had a second phase where the wait list control sites received the LC and the initial intervention sites were observed post-intervention. Herein, we report on this second phase of the trial, focusing on two objectives: 1) to replicate the original results in a second cohort; 2) to determine whether improvements in TTT observed in Phase I were sustained during the second phase; and, 3) to identify practice-specific predictors of greater improvements in TTT score.

Methods

Study Design and Patients

We conducted a pre-planned secondary analysis of a cluster-randomized wait-list controlled clinical trial. The unit of randomization was the rheumatology practice. The trial (Treat-to-target in RA: Collaboration To Improve Option and adhereNce, TRACTION) tested the effects of a Learning Collaborative for TTT between January 2014 and October 2015. A second nine-month phase of the trial (November 2015 through July 2016) allowed the wait-list control sites to receive the intervention and the original intervention sites to be observed. A total of 11 US rheumatology practice sites were recruited to participate (see Supplemental Table 1). Sites were randomized to active intervention or wait-list control. Rheumatology providers and sites were not blinded to treatment assignment.

These secondary analyses focused on three major objectives: 1) to replicate the original results in a second cohort; 2) to determine whether improvements in TTT observed in Phase I sustained during the second phase; and 3) to identify predictors of sites that improved adherence with TTT.

We asked sites to select at least 40 patients with RA with a minimum of 5 patients for each provider who attended at least one session of the LC and contributed patients to the monthly medical record review. Patients must have had visits between the baseline (phase I: September to December 2014; phase II: July to September 2015) and follow-up periods (phase I: September to December 2015; phase II April to July 2016). Medical records of these patients were reviewed by study staff, not local site personnel. These staff reviewed laboratory results, imaging results, medication lists, and rheumatology provider notes.

Research activities were approved by the Institutional Review Board of Partners Healthcare. The funder had no role in the study except for monitoring progress.

Trial Intervention

We conducted a Learning Collaborative (LC), which we have described elsewhere and in the Supplemental Methods.(14) The LC comprised a set of principles and associated concepts developed by the faculty that describe the goals for implementing TTT principles, described herein as TTT. The LC principles and concepts were used by sites to guide rapid-cycle tests of change. Teams improve their process through testing small changes, observing effects, and adopting what works. The tests of change, usually described as plan-do-study-act (PDSA) cycles, are several day periods with faculty teaching the process and providing regular support and feedback. Teams focus on specific monthly metrics and spreading successful interventions to the broader provider group at both their own site and across sites.

The first learning session was conducted as a face-to-face one-day meeting and all subsequent sessions were conducted via monthly webinar. The learning sessions included: a) sharing agreed upon metrics collected by each site through local medical record review; and b) LC faculty presenting a question and answer session regarding the principles and concepts of TTT. We asked all team members from each site to attend all learning sessions, but this was not always possible. All learning sessions were recorded and made available on a web-based collaborative tool.

Trial Outcomes

The primary trial outcome was the change in a composite TTT implementation score. The score included four items directly stemming from the principles and concepts of TTT(1): 1) specifying a disease activity target; 2) recording RA disease activity, using one of four recommended measures (e.g., Disease Activity Score-28 (DAS28), Simplified Disease Activity Index (SDAI), Clinical Disease Activity Index (CDAI), or Routine Assessment of Patient Index Data 3 (RAPID3)), with results described numerically or by category (i.e., remission, low, moderate, or high) (15); 3) when a decision was being made (i.e., change in target or change in treatment), documenting shared-decision making; and 4) basing treatment decisions on target and disease activity measure. Each item was determined as absent or present. The study team conducted all medical record reviews with excellent inter-rater agreement (kappa = 0.94, 95% confidence interval 0.90-0.99), and intra-rater agreement (kappa = 0.98, 95% confidence interval 0.95-0.99). We calculated the TTT implementation score for each patient as a percentage of TTT items noted in the visit note at baseline and follow-up visits.

Potential Predictors of TTT Score Improvement

We examined several types of variables as potential predictors of improvement with TTT implementation. Site characteristics included: Phase I versus Phase II intervention; academic affiliation versus no affiliation; rheumatology trainees present versus no trainees; the number of providers involved in the LC; Epic electronic medical record used or not (it allows incorporation of easy to use phrases that can facilitate documentation); and non-physician providers caring for patients at a site (e.g., nurse practitioners or physician assistants described as yes/no). We also examined several provider-level factors: the gender of the provider; which disease activity measure s/he used; the provider training (physician versus non-physician); and whether the primary provider was a trainee. These were determined based on information provided by the sites.

Statistical Analysis

After defining the study population for Phase II, we examined the patient characteristics based on information found in the medical record review. The outcome of interest was defined for each visit using the four items noted above. Based on the presence or absence of each item, the composite TTT implementation score was calculated for each visit. We calculated the implementation score for each site during Phase I and Phase II. The five sites receiving the LC during Phase I had their implementation scores considered as a group, and the six sites receiving the LC during Phase II were considered as a group. We compared the mean change in TTT implementation score (pre- versus post-intervention) for the five teams in the Phase I intervention versus the mean change for the six teams in the Phase II intervention at the start and end of Phase I and the start and end of Phase II. We also compared results at the end of Phase I for the five intervention teams to their results at the end of Phase II to assess whether the improvement in TTT implementation was sustained.

We examined potential site and provider predictors of TTT implementation improvement. Visits were associated with providers who worked at sites, and thus all the analyses accounted for the within provider clustering using Generalized Estimating Equations and Generalized linear mixed models. Each variable was tested in univariate analyses. Those associated with the outcome with p < 0.2 were included in multivariable analyses. Among the visits with a disease activity measure, we examined whether the use of CDAI or RAPID3 (these were the only two measures used by sites) predicted better TTT scores. All analyses were conducted using SAS (version 9.4, Cary NC).

Results

The chart review included 636 RA patients seen by providers at baseline: month zero for sites in the Phase I intervention or month nine for Phase II intervention sites. These patients had a mean age of 61 years, 81% were female, and 79% were seropositive (see Table 1). RA duration included 20% with early disease ≤ 2 years and 32% with > 10 years. The use of DMARDs was common: 81% used a traditional DMARD and 48% a biologic DMARD. Providers involved were 48% male, 18% trainees, and 6% non-physicians. Seventy-three percent of sites were academically affiliated. Most sites had 3-5 providers involved in the LC.

Table 1. Baseline Characteristics of Patients, Providers and Sites in the TRACTION Trial.

	N (%) or Mean (± SD)
Patient Characteristics (n = 636)
Age, years	60.56 (±21.71)
BMI*, kg/m2	29.65 (±7.76)
Female sex	514 (80.82)
RA duration*, years
≤ 2	69 (19.01)
2-5	101 (27.82)
6-10	78 (21.49)
>10	115 (31.68)
Serologic status*
Positive	397 (79.24)
Negative	104 (20.76)
Use of synthetic DMARDs	516 (81.13)
Use of biologic DMARDs	308 (48.43)
Comorbidity index	1.29 (±0.60)
Provider Characteristics (n=50)
Training
MD/DO	47 (94.00)
NP/PA	3 (6.00)
Trainee	9 (18.00)
Gender, male	24 (48.00)
Site Characteristics (n=11)
Intervention
Phase I	5 (45.45)
Phase II	6 (54.55)
Academic	8 (72.73)
Epic electronic medical record	8 (72.73)
Number of providers at site in Learning Collaborative
≤ 2	2 (18.18)
3-5	7 (63.64)
>5	2 (18.18)

Notes: Baseline was defined as month zero for the five Phase I intervention teams and month nine for the six Phase II intervention teams.

^*Data were missing for the following variables: body mass index n=99, serologic status n= 135, and RA disease duration n=273. Abbreviations: SD, standard deviation; BMI, body mass index; RA, rheumatoid arthritis; DMARD, disease modifying anti-rheumatic drug.

At baseline, mean TTT implementation score was 11% in Phase I intervention sites and 13% in the Phase II (see Table 2). After the intervention, TTT implementation improved in the Phase I intervention arm to 57% (SD 32%, p < 0.001) and to 58% (SD 36%) in the Phase II intervention arm (p < 0.001). Teams receiving the intervention during Phase I sustained their TTT score improvement through Phase II, declining slightly from 57% to 52% (SD 38%, p = 0.042) during Phase II. Several other component scores changed from Phase I to Phase II (see Supplemental Table 2). The changes by component of TTT implementation score were similar across Phase 1 and Phase II (see Table 2). The use of treatment targets and disease activity measures improved greatly during both interventions.

Table 2. TTT Implementation Score and Absolute Change in Implementation Score (and Score Components) in the TRACTION Trial.

	Phase I Intervention				Phase II Intervention
	Month 0	Month 9	Month 18	Change* (0→9)	Month 0	Month 9	Month 18	Change* (9→18)
Primary outcome
Implementation score	11.1%	57.1%	52.4%	46.0%	11.0%	13.3%	58.0%	44.7%
Visits with components present
Treatment target	0.6%	45.6%	51.0%	45.0%	0%	1.0%	52.1%	51.1%
Disease activity measure	20.0%	89.1%	70.7%	69.1%	30.2%	34.2%	85.1%	50.9%
Shared decision-making†	51.3%	85.9%	43.4%	34.6%	24.5%	29.8%	67.0%	37.2%
Treatment decision‡	0.6%	27.8%	36.7%	27.2%	0%	1.0%	36.5%	35.5%

The number of visits included differed by Phase and by month of assessment. For Phase I, there were 320 visits at month 0 and at month 9, and 300 visits at month 18. For Phase II, there were 321 visits at month 0, and 316 visits at month 9 and 18.

^*Based on generalized mixed models accounting for clustering within providers, all p-values for change were < 0.001.

^†The shared decision-making criteria did not apply to all visits when no decisions were being made about changing targets or changing treatments. The number of visits when shared decision-making applied for the Phase I intervention group: 115 at month 0, 184 at month 9, and 99 at month 18. For the Phase II intervention group: 102 at month 0, 94 at month 9, and 112 at month 18.

^‡Treatment decision based on target and disease activity measure.

We examined the association of provider and site level factors at baseline with change in TTT implementation (see Table 3). Several site factors were associated with improvements in TTT implementation in unadjusted analyses: having only physician providers, academic affiliation, and few providers at a site. Provider level factors included only being a trainee. Having Epic as an electronic medical record was associated with a trend toward more improvement in TTT implementation. These variables were then examined in fully adjusted models that included all factors with p < 0.2 in unadjusted analyses (see Table 3). The same variables remained significantly associated with TTT improvement (see Table 3). In the subgroup of providers who used a disease activity measure, there was no significant difference in improvement based on whether providers used a CDAI or RAPID3

Table 3. Adjusted Mean Improvement in TTT Implementation Score Over 9 Months by Site Level and Provider Level Factors in the TRACTION Trial.

	Univariate*	P-value	Multivariable	P-value
Site level factors
Intervention
Phase I	42.0 (33.8, 50.3)	0.71	---
Phase II	44.6 (34.1, 55.1)		---
Academic affiliation
Yes	47.2 (39.6, 54.8)	0.12	60.8 (52.0, 69.5)	0.009
No	35.7 (23.3, 48.1)		42.2 (24.7, 59.8)
Fellows at site
Yes	47.3 (38.9, 55.6)	0.20	---
No	38.5 (27.8, 49.2)		---
Non-physician providers
Yes	33.9 (25.8, 42.0)	0.002	41.2 (28.8, 53.6)	0.0001
No	53.1 (43.7, 62.4)		61.8 (48.0, 75.6)
Number of providers
≤ 2	56.2 (31.0, 81.4)	0.12	74.7 (52.2, 97.2)	0.001
3-5	46.9 (38.0, 55.7)	0.07	42.4 (29.3, 55.4)	0.47
>5	34.5 (24.6, 44.4)		37.4 (27.4, 47.4)
Epic electronic medical record
Yes	46.9 (38.8, 55.0)	0.10	56.3 (46.8, 65.8)	0.23
No	35.3 (24.0, 46.7)		46.7 (28.8, 64.6)
Provider level factors
Male sex of provider
Yes	39.5 (29.1, 49.8)	0.31	---
No	46.5 (37.8, 55.3)		---
Provider type
Physician	44.3 (37.4, 51.3)	0.46	---
Non-physician	34.7 (10.2, 59.3)		---
Trainee
Yes	58.1 (42.1, 74.0)	0.05	59.6 (42.9, 76.4)	0.02
No	40.3 (33.3, 47.3)		43.4 (33.5, 53.2)

^*Adjusted least square means generated using Generalized Estimating Equation, accounting for clustering of patients within providers. P-values describe two-way comparisons. Multivariable analyses include the five variables with P values < 0.2 on univariate screen and account for clustering. We also tested the effect of different disease activity measures among providers that used them. The univariate results for CDAI (42.0, 95% CI 32.8-51.3) and RAPID3 (47.4, 95% CI 36.3-58.5) were not statistically different.

Discussion

Treat to target (TTT) is a widely embraced paradigm in RA care.(1, 2) However, there is ample evidence that many providers do not adhere to its principles. We developed an LC intervention that proved effective at increasing implementation of TTT.(13) In a second phase, we found that the LC again worked well with a new set of rheumatology sites. Of equal importance, the initial sites sustained a high level of TTT implementation during observation without intervention. Several provider and site level characteristics predicted TTT improvement during the LC intervention, including a site only having physician providers, academic affiliation, and fewer providers at a site, as well as the provider being a rheumatology trainee.

These Phase II results have several implications. First, the duplication of the increase in TTT implementation over the course of the LC with a second group of sites gives us confidence that this intervention can be replicated and would likely benefit other sites. The success of the LC is not surprising as it has been used with similar effects in many different health care settings.(16) Its success hinges on: 1) a recognition that there is a better way to practice; 2) expertise from the LC faculty on how to achieve improvements in care; 3) small tests of change to achieve improvements; and 4) time to consolidate these changes with reinforcement. Second, the sustainability of the improvements in TTT implementation suggests that the sites changed their practice of care for RA, and sites were able to continue to adhere to TTT principles even after the active intervention ended. Anecdotes from Phase I sites lend credence to this observation: we have heard that collecting disease activity measures has become routine and that patients expect to discuss their target disease activity. However, there were some slight differences in the Phase I sites between months 9 and 18; they improved on some components and regressed on others (see Supplemental Table 2). These results were not hypothesized and should not be emphasized.

Finally, the predictors of improvement were not entirely expected. One might have guessed that academic sites and rheumatology trainees might be more open to change than other sites and providers. However, at baseline, we found that trainees implemented TTT less well than attending rheumatologists. Thus, it is heartening to observe that trainees improved more than attending rheumatologists during the LC. Perhaps, most surprising is the finding that sites with non-physician providers (nurse practitioners and physician assistants) improved less than other sites. We had not hypothesized these difference and thus the findings should be considered hypothesis-generating, warranting follow-up. We observed no difference in effect for sites using CDAI or RAPID3. These measures differ in that the RAPID3 only includes patient-derived information and thus might have been less influential on the decision to adjust treatment.

The study presented has several important strengths. The data were derived from the second phase of a randomized controlled trial in which all measurements, outcomes, and analyses were pre-defined. The sites were geographically spread out across the US and they were relatively heterogeneous with respect to size and other practice characteristics noted in Table 1. Importantly, all of the medical record reviews were conducted centrally by trained study staff and not by local staff.

Limitations of the study include a relatively small sample size followed for only 18 months. The outcomes examined were process measures and not clinical outcomes. Finally, the analyses of predictors of improvement were limited to relatively few potential predictors, small numbers of sites with correlated characteristics, and unmeasured variables that may account for the differences observed.

In conclusion, we found that an LC was an effective tool for improving implementation of TTT and that its effects were sustained and replicated across a second cohort of rheumatology sites. We suggest that the LC for implementing TTT could be used more broadly across rheumatology sites. Learning collaborative might be considered for other areas of quality improvement within rheumatology, including prevention of glucocorticoid induced osteoporosis, vaccination of patients using immunomodulatory agents, as well as other site-specific quality improvement needs.

Significance and Innovation.

• - A Learning Collaborative produced consistent and sustained improvement in adherence with treat to target for rheumatoid arthritis.

• - Predictors of improvement included academic affiliation of the site, only having rheumatologist providers at the site, fewer providers at the site, and the provider being a rheumatology trainee. This suggests that a Learning Collaborative approach to process improvement can be integrated into training.

• - Learning Collaboratives could be considered for other areas needing process improvement in rheumatology practice, including vaccinating at-risk patients, and screening and managing osteoporosis and cardiovascular disease.