Traffic Lights Intervention Reduces Therapeutic Inertia: A Randomized Controlled Trial in Multiple Sclerosis Care

Abstract

Background: Therapeutic inertia (TI) is a common phenomenon among physicians who care for patients with chronic conditions. We evaluated the efficacy of the traffic light system (TLS) educational intervention to reduce TI among neurologists with MS expertise. Methods: In this randomised, controlled trial, 90 neurologists who provide care to MS patients were randomly assigned to the TLS intervention (n = 45) or to the control group (n = 45). The educational intervention employed the TLS, a behavioral strategy that facilitates therapeutic choices by facilitating reflective decisions. The TLS consisted in a short, structured, single session intervention of 5-7 min duration. Participants made therapeutic choices of 10 simulated case-scenarios. The primary outcome was a reduction in TI based on a published TI score (case-scenarios in which a participant showed TI divided by the total number of scenarios where TI was possible ranging from 0 to 8). Results: All participants completed the study and were included in the primary analysis. TI was lower in the TLS group (1.47, 95% CI 1.32-1.61) compared to controls (1.93; 95% CI 1.79-2.08). The TLS group had a lower prevalence of TI compared to controls (0.67, 95% CI 0.62-0.71 vs. 0.82, 95% CI 0.78-0.86; p = 0.001). The multivariate analysis, adjusted for age, specialty, years of practice, and risk preference showed a 70% reduction in TI for the TLS intervention compared to controls (OR 0.30; 95% CI 0.10-0.89). Conclusions: In this randomized trial, the TLS strategy decreases the incidence of TI in MS care irrespective of age, expertise, years for training, and risk preference of participants, which would lead to better patient outcomes.

The management of multiple sclerosis (MS) evolves with the availability of new disease-modifying agents, varying dosage forms (oral, injectable, infusion), and different safety and efficacy profiles. As a result, physicians who care for MS patients face more complex therapeutic decisions when considering individual patients, number of relapses, activity on brain imaging, and the need to escalate therapy. Moreover, many MS patients remain undertreated.^1–3

Therapeutic inertia (TI) corresponds to the absence of treatment initiation or intensification when treatment goals are unmet. It affects more than 50% of clinicians caring for patients with chronic conditions.^1,4–6 Insufficient knowledge integration and knowledge-to-action gaps are among the most common explanations for suboptimal therapeutic decisions. However, TI is a complex process also related to other characteristics of the providers such as lack of knowledge about appropriate goals, high patient volume, and time constraints. Some physicians fail to integrate the available information (e.g., severity of the condition, risk of progression, imaging findings affecting outcomes) with best practice recommendations for a given risk-scenario.^7–10 Furthermore, physicians have limited training in risk management and formal learning in medical decision making.^11–13 All of these factors may contribute to TI and undertreatment.

Educational interventions have been designed to optimize knowledge integration and bridge knowledge-to-action gaps for complex medical decisions (e.g., diagnostic challenges, varying risk categories, availability of multiple agents with a broad range of safety/efficacy ratio).¹¹ One such intervention is the traffic lights system (TLS). The TLS emerged as risk categorization and warning strategy to reduce human errors that relies on a relatively automatic, well-established, and cross-cultural concept to increase the chance of an optimal course of action.^14,15 The TLS facilitates the decision-making process using traffic light terminology, which creates a link between a color, representing a risk level, and an action: red light (“high risk”/“stop and think”), yellow light (“intermediate risk”/“reassess soon”), and green light (“low risk”/“continue the same strategy”). For example, studies showed that the TLS facilitates healthier food choices by interrupting automatic behavior and triggering a reevaluation processes.¹⁶ A functional magnetic resonance imaging study showed that TLS labels enhance the coupling between brain regions associated with valuation (i.e., ventromedial prefrontal cortex) and self-control.¹⁷ Evidence from the literature suggests that medical decisions leading to TI are likely related to knowledge-to-action gaps.⁷ The design and application of the TLS as an educational intervention provides a unique opportunity to overcome knowledge-to-action gaps in MS care.¹⁸

In a previous pilot study, we assessed the feasibility and potential efficacy of a TLS educational intervention in 25 neurologists from Spain.¹⁹ TI was present in 72.0% of participants in at least one case scenario. The primary feasibility outcome, the completion rate of the study, was 100% (25/25 participants). While not powered to detect a significant difference between groups, our pilot study demonstrated a nonsignificant reduction in TI for the targeted intervention group relative to the control group (22.6% v. 33.9% postintervention; odds ratio [OR] = 0.57; 95% confidence interval [CI] = 0.26 to 1.22).¹⁹ The TLS also showed a high usability score (74.7; 95% CI = 70.1 to 79.2) when tested in a larger study comprising neurologists from Argentina, Chile, and Canada.²⁰

In the present study, we evaluated the efficacy of our simple and pilot-tested educational TLS intervention for reducing TI among neurologists who routinely provide care of MS patients.

Methods

Study Design and Participants

The overarching goal of the TLS intervention was to facilitate the integration of gaps between risk stratification and treatment decisions (initiation or escalation) in MS care. Specifically, our randomized parallel trial tested the efficacy of an educational TLS intervention (active group) against usual care (control group) for reducing TI in the management of MS (Figure 1). Participants were randomly assigned (1:1 ratio) to the TLS or the control group by Qualtrics (Qualtrics.com; Online Appendix, Figure e1). Allocation concealment was implemented in Qualtrics, so that participants did not know which intervention they were allocated to after completing the initial demographic information. Investigators were also blinded to the treatment allocation. Participants were recruited by automatic e-mail invitation from the study platform. The Institute of Neuroscience Buenos Aires and the Argentinian Neurological Society facilitated the mailing of information to potential participants who met the inclusion criteria.

Figure 1.

Consort flow diagram. Of 117 eligible participants, 90 participants were randomized to the educational intervention (n = 45) and control (n = 45). All participants completed the intervention and contributed a complete set of data to the analysis.

Data Collection

Participants answered questions as follows: 1) demographic and practice-based information, 2) behavioral experiments, and 3) 10 case scenarios that assessed TI.

Behavioral experiments used previously established designs to assess participants’ risk preferences and tolerance to ambiguity in the health and financial domains.^21,22 In brief, ambiguity aversion is defined as dislike for events with unknown probability compared to events with known probability.²¹ For example, in the medical domain, an ambiguity-averse individual would rather choose a treatment where the probability of benefits or side effects is known (even if these are somewhat unfavorable) over one where this probability is unknown (Figure 2A and B). Risk experiments involved determining the subjective value of a risky (50/50) option in terms of a safe (100%) option. By asking participants to indicate the magnitude of the safe option at which they would be indifferent between the two options, we were in a position to determine the point of subjective equivalence of the risky and the safe option. The higher this point, the higher the propensity to take risk. Further details of these experiments were published in previous studies (and appendix).³

Figure 2.

Experiments to assess ambiguity in the financial and health domains. Participants were told to imagine two different types or urns. For urn type A, they knew that 50% of the balls were red and the other 50% were blue. For urn type B, they did not know the exact proportion of blue to red balls, with the grey bar representing the unknown proportion of balls. For the financial domain (Panel A), participants knew that if they drew a blue ball, they would win the full amount of $400. If they drew a red ball, they would win $ 0. For the health domain (Panel B), participants decided between two treatments for a patient. With “Treatment A,” the patient had a 50% probability of survival. With “Treatment B,” the exact probability of survival was unknown, with the grey bar representing the unknown probability. In our tasks, participants were asked to choose between one option (presented as two-colored bar) with known 50/50 probability of winning 400 or 0 American dollars (urn A) versus an option with unknown probability of the same outcomes (urn B). Participants who chose the 50/50 options were classified as averse to ambiguity; the remaining participants were classified as tolerant to ambiguity.¹ A similar approach was used to determine aversion to ambiguity in the health domain (Panel B).

Participants were also asked to select those MS drugs that they use and then rank them from a list including all available agents approved by the local regulatory body in Argentina by March 30, 2018. The purpose of this strategy was to compare how TI derived from case scenarios relates with agents used for treatment escalation (e.g., fingolimod, natalizumab, alemtuzumab) in routine clinical practice. At the time of conducting this trial, cladribine and ocrelizumab were not available and therefore not included in case scenarios.

In line with the learning and education literature, vignettes, clinical case scenarios, or “real-world” encounters are regarded as the best simple strategy to evaluate potential cognitive biases and medical decisions.²³ Case scenarios were designed by our research team and MS experts derived from the most common situations in clinical practice as previously reported in our pilot study.¹⁹ Eight case scenarios were designed to assess appropriate treatment initiation or escalation, whereas the remaining two cases were designed to assess overtreatment (defined as treatment escalation when there was low risk of disease progression and no evidence of disease activity).^24–26 Participants from each randomized group were exposed to the same case scenarios. Inclusion criteria comprised neurologists who were actively involved in managing MS patients. Physicians whose practice was primarily in caring for MS patients or who obtained a subspecialty degree were classified as “MS specialists.” Physicians who were not practicing neurology or seeing less than one MS patient per month were excluded from the study.

Definitions

For the primary analysis, bad prognosis was defined as the combination of a clinical relapse plus the presence of new brain lesions in follow-up magnetic resonance imaging (MRI) scans or at least one gadolinium-enhancing lesion.^27,28 All high-risk cases included a description of an MRI with more than five new T2 lesions or at least one enhancing lesion.²⁹ The use of these definitions combining a clinical relapse and MRI activity is consistent with recent evidence regarding the risk of treatment failure among patients receiving interferon-β.³⁰ Disease progression was defined as at least one point worsening from baseline to 1-year follow-up in the Expanded Disability Status Scale (EDSS) score.³¹ Recent meta-analysis confirmed that alemtuzumab, natalizumab, and fingolimod are the best available choices for preventing clinical relapses in patients with relapsing-remitting MS (RRMS).³² The current treatment option for RRMS include first-line (β-interferons, glatiramer acetate), second-line (fingolimod, cladribine), and third-line (natalizumab, alemtuzumab, ocrelizumab) therapies. For the present analysis, we used the aforementioned scheme according to the current clinical practice.^24–26,33

The Traffic Light System (Figure 3)

Figure 3.

The TLS intervention. Panel A illustrates background information on the TLS and application to therapeutic decisions. Panel B illustrates how the TLS facilitates the decision-making process using traffic light terminology, which creates a link between a color, representing a risk level, and an action: red light (“high risk”/“stop and think”), yellow light (“intermediate risk”/“reassess soon”), and green light (“low risk”/“continue the same strategy”). Panel C provides a case scenario as an example of those given the participants.

In our study, the TLS was applied to help participants identify scenarios with poor prognosis (high risk of disease progression; Figure 3B). Consequently, participants would be able to identify the “red” traffic light as a warning sign for a high-risk situation, and subsequently escalate treatment. The “yellow” traffic light represents caution for scenarios with either a clinical relapse or some degree of activity on brain imaging (but not both), which requires a reassessment within 6 to 12 months. The control group made therapeutic decisions without being exposed to the TLS intervention as part of the current standard practice. Further details of the TLS intervention are described elsewhere.¹⁹ An example of the presented case scenario is represented in Figure 3C.

Outcome Measures

Therapeutic inertia was the primary outcome of interest, measured as a continuous variable (TI score) and as binary. The TI score was defined as the number of case scenarios in which a participant showed TI (ranging from 0 to 8). The TI score was reported in our previous studies to reflect the magnitude of TI.^19,34 A low TI score represents low TI, whereas a higher score represents higher. A 0.5 point difference in the TI score was deemed as clinically meaningful given the impact in clinical practice. In our study, detecting a difference equal to or greater than 0.50 between groups in the TI score would represent a clinically meaningful improvement. The TI score was derived from case scenarios, which were aligned with the current Argentinian, North American, and European practice recommendations.^24–26

A reduction in the TI score reflects that participants appropriately switched from a first-line agent (e.g., glatiramer, interferons) to a high-efficacy treatment (e.g., fingolimod as a second-line therapy or monoclonal antibodies as third-line therapies) when clinical and radiological evidence of disease progression.

TI as a binary outcome was defined as the proportion of participants demonstrating TI in at least one of the eight scenarios (prevalence of TI).

Statistical Analysis

We used parametric tests (t test and Fisher’s exact test) to compare continuous and categorical variables between groups. Linear regression analysis was used to determine the efficacy of the TLS for reducing TI scores in the intervention group versus the control group. Similarly, logistic regression analysis was used to determine the efficacy of the TLS with respect to the proportion of participants with TI in at least one case scenario. We also evaluated the association between restrictions for prescribing DMTs and the number of second- and third-line agents commonly used in clinical practice with the TI score.

For multivariate analysis of individual responses, we constructed multilevel mixed-effects models where participants (n = 90) and individual responses (n = 720; 90 participants each completing 8 case scenarios) entered as random effects. The aim of this analysis was to evaluate the contribution of individual-specific variables to the variation of TI. Given the findings from our previous studies, we included the following a priori variables: participant age, expertise (MS specialist v. general neurologist), years of experience, and aversion to ambiguity.^3,19 All tests were 2-tailed, and P values <0.05 were considered significant. We used STATA 13 (StataCorp LP, College Station, TX) to conduct all analyses.

Further details of the protocol are published in ClinicalTrials.gov (# NCT03134794).

Results

Out of the 117 neurologists with expertise in MS care who were invited to participate in the study, 90 completed the survey (response rate 76.9%) between April and September 2018. There was representation from all provincial territories. The mean (SD) age was 46.4 (±10.3) years; 48 (53%) were male neurologists. Thirty-one (34.4%) participants primarily focused their practice on MS care. They had 20.3 (±10.9) years of practice and were assessing 22 (±6.6) MS patients per week. Table 1 compares baseline characteristics between groups. Groups did not differ in demographics or in risk preferences as measured by the behavioral risk tasks (P = 0.40 for risk preferences and P = 0.63 for aversion to ambiguity). There were no differences in treatment escalation at baseline between groups. On average, participants in the TLS group used 3.09 agents for treatment escalation versus 2.91 in the control group (P = 0.62). There was no association between participants’ restrictions to prescribe MS drugs and TI score (P = 0.44) or the prevalence of TI (P = 0.78).

Table 1.

Baseline Characteristics of Participants

Characteristics	Control (N = 45)	Educational Intervention (N = 45)	P Value
Age in years (mean ± SD)	46.8 ± 10.3	46.0 ± 10.4	0.72
Sex, n (%)			0.67
Female	20 (44.4)	22 (48.9)
Male	25 (55.6)	23 (51.0)
Specialty, n (%)			0.82
General neurologists who care for MS patients	29 (64.4)	30 (66.7)
MS specialists	16 (35.6)	15 (33.3)
Practice setting, n (%)			0.37
Public	17 (37.8)	13 (28.9)
Private	28 (62.2)	32 (71.1)
% time in clinical practice, n (%)
>75%	19 (42.2)	20 (44.4)	0.83
Years in practice (mean ± SD)	21.1 ± 10.5	19.5 ± 11	0.48
MS patients seen per week (mean ± SD)	21.8 ± 4.5	23.0 ± 8.3	0.37
Author of a peer-reviewed publication in the last 1 year, n (%)	20 (44.4)	23 (51.1)	0.53
Restriction to prescribe MS drugs, n (%)			0.83
No restrictions	28 (62.2)	29 (64.4)
Treatment escalation, mean drugs (±SD)	2.91 (1.24)	3.09 (2.05)	0.62

MS, multiple sclerosis

Table 2 summarizes the primary and secondary outcome measures at the participant and individual response levels. TI scores were significantly lower in the TLS intervention group than in the control group (1.36, 95% CI = 1.23 to 1.50, v. 2.04, 95% CI = 1.90 to 2.17) after adjustment for the prespecified variables (age, specialty, years of practice, and risk preferences). The observed 0.68 difference between groups in the adjusted TI scores was greater than the minimal clinically meaningful measure of 0.5 to detect an improvement. Similarly, participants in the TLS intervention group had a lower prevalence of TI compared to controls (63.7%, 95% CI = 58.9% to 68.6%, v. 83.9%, 95% CI = 80.4% to 88.4%) (Figure 4).

Table 2.

Multivariate Analysis for the Primary and Secondary Outcome Measures^a

Outcome Measures	Control	Intervention	Difference Between Groups	Multivariate Regression Analysis (95% CI)
Primary outcome
Participant-level analysis	n = 45	n = 45
TI score, mean (SD)	1.93 (1.42)	1.47 (1.42)	(0.46)	−0.68 (−1.24 to −0.11)b
Secondary outcome measures
TI (present v. absent) in at least one case scenario, n (%)	37 (82.2)	30 (66.7)	(15.5)	0.30 (0.10 to 0.89)c
Individual responses	n = 360	n = 360
TI score, mean (SD)	1.93 (1.41)	1.47 (1.41)	(0.46)	−0.68 (−0.87 to −0.48)d
TI present versus absent, n of individual responses/total	87/360 (24.2)	66/360 (18.3)	(5.9)	0.60 (0.41, 0.88)d

CI, confidence interval; OR, odds ratio; TI, therapeutic inertia.

^aAll models adjusted for age, specialty, years of practice, risk preference, and group (intervention v. control).

^bDerived from linear regression models and expressed in β coefficients (95% CI) with TI score as dependent variable.

^cDerived from multivariate logistic regression analysis with TI (present v. absent) as dependent variable.

^dDerived from multilevel mixed effects models expressed as OR (95% CI) for binary outcomes (TI present v. absent) and β coefficients (95% CI) for the TI score.

Figure 4.

TLS intervention decreased therapeutic inertia (TI). (A) Comparison of adjusted TI scores in the intervention and control groups. This graph was derived from the multivariate linear regression analysis adjusted for age, years of practice, participants risk preference, and specialty (general neurologist v. MS expert). TI scores were significantly higher in the control group compared to the intervention group (*P value <0.001). (B) Comparison of adjusted prevalence of TI between the intervention and control groups. This graph was derived from the multivariate logistic regression analysis adjusted for age, years of practice, participants risk preference, and specialty (general neurologist v. MS expert). The prevalence of TI was significantly higher in the control group compared with the intervention group (*P value <0.001).

The multivariate analysis also revealed a significant reduction in the TI score for the TLS intervention compared to the control group (β = −0.68; 95% CI = −1.24 to −0.11). The multivariate logistic regression analysis showed 70% reduction in the odds of TI for the TLS group (OR = 0.30, 95% CI = 0.10 to 0.89). Specialist status (P = 0.002), higher years of experience (P = 0.007), and tolerance to ambiguity (P = 0.043) were associated with lower TI. The adjusted models showed good discrimination (c-statistic = 0.74) and calibration (goodness-of-fit test P = 0.52).

Results were consistent for the analyses of individual responses (Table 2). There were no significant differences between fixed and random effect models (data not shown). Figure 3 represents the relationship between the observed and predicted TI scores (Figure 5A) and stratifies the data by group (Figure 5B), revealing that TI was consistently lower in the intervention group.

Figure 5.

Adjusted probability of therapeutic inertia (TI). (A) Adjusted probability of TI as a function of TI scores. (B) Adjusted TI score categories stratified by intervention assignment group. The x-axis represents categories of the TI to evaluate whether the intervention had a different effect among participants with low, medium, and high TI scores. The y-axis represents the TI scores to be able to show the lack of overlap of 95% CI between TLS and controls for each TI category (P value TLS v. controls = 0.02). Data derived from multivariate linear regression with TI score as the dependent variable. “I” represent 95% CI error bars.

Participants who commonly used agents for treatment escalation in their daily practice had lower TI scores (1.48 v. 1.8; P < 0.01). Accordingly, participants who do not commonly use agents for treatment escalation in their daily practice benefited from the intervention (for TI score: β = −0.78, 95% CI = −1.03 to −0.52; for TI prevalence: OR = 0.56, 95% CI = 0.36-0.88).

The analysis of individual responses also revealed that for every 100 MS patients with a bad prognosis (e.g., both clinical and radiological evidence of disease activity), there will be over 24 patients who will remain with the same treatment if managed by neurologists without educational intervention (control group). That number would be decreased to 10 patients if treated by neurologists who received the TLS educational intervention.

Discussion

In the present randomized controlled trial (RCT), we evaluated the efficacy of a newly designed pilot-tested¹⁹ educational intervention to overcome TI among practicing neurologist care for MS patients. We found TI in 7 out of 10 participating neurologists in at least one case scenario. The TLS educational intervention was associated with a 68% reduction in the TI score or 70% reduction in the odds of TI. In other words, participants appropriately choose a higher efficacy treatment (e.g., monoclonal antibodies) instead of continuing with the same agents (e.g., glatiramer, interferon) when clinical and radiological evidence of disease progression. The effect of the educational intervention was similar for all categories of TI scores. Specialist status, years of experience, and tolerance to ambiguity were associated with lower TI. Moreover, selection of common agents used for treatment escalation in participants’ routine practice was associated with lower TI scores. More interestingly, the TLS intervention was effective among participants who do not commonly use agents for treatment escalation in their daily practice by showing a significant reduction in TI. Our results were consistent for the analysis at the participant level and individual responses for both categories of TI and TI scores.

There are not many proven effective interventions in medical education associated with improvements in clinically meaningful outcomes.³⁵ A recent systematic review evaluated 302 controlled studies that had investigated the effect of evidence-based educational interventions. Of 85 articles that met the inclusion criteria, 46 (54%) studies were randomized trials, 51 (60%) included postgraduate-level participants. Although the authors evaluated outcomes in multiple domains (e.g., self-efficacy, knowledge, behavior change), none of the studies assessed patients’ benefits.³⁵ In MS care, TI may lead to undertreatment. By extension, the TLS intervention used here may eventually have patient benefits if it reduces TI in MS care.

We used TLS to reduce TI in MS care. Previous authors proposed the TLS to monitor treatment response in patients with RRMS. They included a more sophisticated scoring system (0 to 3) for different categories (clinical relapses, evidence of disease progression, cognitive status, and MRI findings) making practical use in daily practice more difficult. This scoring system leads to a decision model that uses the TLS to facilitate therapeutic choices.³⁶ However, this strategy has not been previously tested in a RCT. The findings of our RCT suggest that the TLS may be a useful medical educational intervention, in keeping with research on the management of obesity, fever in children, blood pressure control, and healthy food choices.^14,15

Our results have limitations that deserve comment. First, our sample size is relatively small. However, our RCT was designed and powered to detect differences in TI following our pilot study.¹⁹ Second, case scenarios may not necessarily reflect participants’ daily practice. It is also possible that general neurologists apply the “first do not harm rule” when not escalating treatment (commonly associated with more severe side effects). Following these arguments, our study underestimates the prevalence of TI in real-life practice given a tightly controlled environment and applicability of treatment recommendations in our RCT design. Third, we cannot rule out the possibility that unmeasured confounders (e.g., health policy, restrictive prescription rules) may play a role for the studied outcome measures. We controlled for this issue by measuring the prevalence of prescription restrictions in the workplace for each participant. No association was found between prescription restrictions and the outcomes of interest. Fourth, we have no information regarding the sustainability of the TLS effect on TI given the design of our study. Future research will be needed to investigate this question. Finally, the definition of TI applied to MS care may not be widely used. Nevertheless, we used a practical and conservative definition of TI (absence of escalation with the concomitant presence of a clinical relapse and evidence of imaging activity) consistent with our previous studies, which is supported by guidelines showing improvements in clinical outcomes when escalating therapies (i.e., blood pressure and diabetes).^6,24,26

In the evolving landscape of MS treatment, new and more effective agents with improvements in safety profiles are becoming available.³³ Despite such advancements, many MS patients remained undertreated.^25,26 Several conditions affect the risk of TI, but physicians’ factors are regarded as the most influential.³³ Our results revealed that an innovative and highly usable educational intervention may revert the incident risk of TI among neurologists who care for MS patients. This is also supported by the following facts: 1) participants who commonly use agents for treatment escalation had lower TI, 2) the significant reduction in TI for the TLS intervention group among neurologist who do not commonly use agents for treatment escalation, and 3) the relevance of the role of MS specialist in making therapeutic decisions given the lower prevalence of TI.

Our findings have practical clinical and health policy implications, which may not only lead to improving outcomes for patients but also to the implementation of educational interventions in physicians managing high-risk and complex patients. Our intervention has the potential to be translated to other highly prevalent medical conditions, including the management of hypertension, diabetes, and dyslipidemia commonly affecting individuals at high risk of cardiovascular and cerebrovascular diseases.