Methods to Limit Attrition in Longitudinal Comparative Effectiveness Trials: Lessons from the Lithium Use for Bipolar Disorder (LiTMUS) Study

Abstract

Background

High attrition rates which occur frequently in longitudinal clinical trials of interventions for bipolar disorder limit the interpretation of results.

Purpose

The aim of this article is to present design approaches that limited attrition in the Lithium Use for Bipolar Disorder (LiTMUS) Study.

Methods

LiTMUS was a 6-month randomized, longitudinal multi-site comparative effectiveness trial that examined bipolar participants who were at least mildly ill. Participants were randomized to either low to moderate doses of lithium or no lithium, in addition to other treatments needed for mood stabilization administered in a guideline-informed, empirically supported, and personalized fashion (N=283).

Results

Components of the study design that may have contributed to the low attrition rate of the study included use of: (1) an intent-to-treat design; (2) a randomized adjunctive single-blind design; (3) participant reimbursement; (4) intent-to-attend the next study visit (includes a discussion of attendance obstacles when intention is low); (5) quality care with limited participant burden; and (6) target windows for study visits.

Limitations

Site differences and the effectiveness and tolerability data have not been analyzed yet.

Conclusions

These components of the LiTMUS study design may have reduced the probability of attrition which would inform the design of future randomized clinical effectiveness trials.

Introduction

Attrition of participants from effectiveness randomized clinical trials occurs frequently and leads to missing visits, premature discontinuation of treatment, and incomplete study procedures, complicating the interpretation of study results [1, 2]. Attrition can bias the estimate of the treatment effect and reduce power, precision, and generalizability [3]. Attrition can also introduce self-selection bias (i.e., treatment assignment is no longer strictly a function of randomization), thereby detracting from a fundamental strength of randomized trials [2].

In longitudinal intervention studies for bipolar disorder, high attrition rates present a primary obstacle to obtaining adequate power and bias the estimate of treatment response [3]. In a recent review of 14 randomized trials for the maintenance treatment of bipolar disorder, the attrition rates ranged from 19% to 98% (Median=68%), with higher study withdrawals for lithium compared to divalproex, lamotrigine, carbamazepine, and olanzapine [4]. These high attrition rates may be due to long follow-up periods (i.e., 6 months to 2.5 years) and restricting treatment to monotherapy without allowing interventions for residual symptoms. Nevertheless, lithium trials have higher attrition rates.

One study found that 25% of bipolar patients receiving prophylactic lithium treatment discontinued within the first six months, due to side effects, failure to adhere to treatment instructions, and missing appointments [5]. In a 12-month randomized double-blind maintenance study, Tohen (2005) reported a 32.7% attrition rate for lithium monotherapy-treated subjects compared to a 26.7% for those treated with olanzapine monotherapy [6]. Randomized subjects were seen biweekly for the first 4 weeks of the study and then completed ten monthly visits as part of a maintenance period. In this study, subjects randomized to the lithium monotherapy group also had significantly earlier time to discontinuation (207 versus 303 days, respectively) [6]. In a randomized, double-blind divalproex monotherapy study, the attrition rate was 62% among patients randomized to divalproex monotherapy (n=197) and 76% among patients treated with lithium (n=91) [7]. For the first six weeks of this study, subjects were seen weekly and then biweekly until week 12. Monthly follow-up visits occurred from weeks 12 to 52.

Attrition rates may be lower for acute depression trials compared to maintenance trials for bipolar disorder. Kemp et al. (2009) reviewed 14 bipolar studies examining atypical antipsychotics (N=5), anticonvulsants (i.e., lamotrigine; N=5), psychostimulant (i.e., modafinil; N=1), and combination therapy (N=3) for the treatment of bipolar depression [8]. The attrition rates for these studies ranged from 17% to 66% (Median=36%). It may be that participants are more motivated for treatment during acute episodes; however, attrition rates were obtained over much shorter study durations (i.e., Range=6 to 24 weeks, Median=8 weeks) compared to the maintenance trials. Nonetheless, the attrition rates from these bipolar pharmacotherapy studies suggest that a conservative estimate of attrition would be nearly one third of study participants, even for studies with short follow-up periods. Thus, the attrition rate in the Lithium Treatment -Moderate dose USe study (LiTMUS) study for bipolar disorder of only 16.25% is noteworthy. This is a comparative effectiveness, 6-month study (i.e., 9 study visits) examining the effectiveness of lithium compared to optimized treatment (OPT; guideline-informed, personalized pharmacologic treatment).

The aim of this paper is to discuss factors related to the LiTMUS study design that may have maximized retention and limited attrition. The discussion focuses on six unique components of the study design that could have contributed to the low attrition rate, and which may inform the development of future randomized comparative effectiveness clinical trials, particularly with patient populations (e.g., bipolar disorder) that are vulnerable to treatment nonadherence and study discontinuation. The identification of such potential modifiable factors could prove to be critical in the design and analysis of novel treatments for bipolar disorder.

The LiTMUS Study

Overview

LiTMUS was a 6-month, multi-site study examining the effectiveness of lithium as a component of OPT. OPT was openly administered, guideline-informed, empirically supported, and personalized pharmacologic treatment based on current symptoms, prior treatment history, and course of disorder. The only requirement for OPT was that subjects were prescribed at least one mood stabilizer, as defined by the Texas Medication Algorithm Project [9]. Participants were either randomized to receive open lithium plus OPT or OPT without lithium. The primary study outcomes were the Clinical Global Illness of Severity for Bipolar Disorder [10] as well as a novel metric assessing the number of Necessary Clinical Adjustments [11]. These adjustments were medication type or dose changes recommended by the study physician to reduce symptoms, optimize response and functioning, or to address intolerable side effects. This metric provided a proxy for both clinical response and tolerability. Secondary and exploratory analyses include symptomatic recovery, quality of life, suicidal behaviors, and moderators of suicidality. The rationale and design for the LiTMUS study have been described in detail elsewhere [11].

Current Attrition

Recruitment (N = 283) was completed for LiTMUS in September, 2009 and the last study participant completed the study on March 11, 2010 (see Table 1). A total of 237 participants (84%) completed the study and thus, 46 (16%) terminated prior to completing the study (see Figure 1). Of the participants that exited the study early, the majority (i.e., 32/46) were lost to follow-up (see Table 2). Other main reasons for withdrawing from the study were participants’ inability to follow the study procedures (6/46) and withdrawing consent due to being dissatisfied with their treatment or not specifying a reason (5/46). Although early terminators left the study at each of the study visits, 45% of them completed at least half of the study or three months of follow-up (see Table 3). From the discussions with the LiTMUS principal investigators for each study site, feedback from study coordinators, and site monitoring trips (by authors L.S. and C.K.), we have distilled the main factors that could have contributed to limiting attrition in the LiTMUS study (see Table 4).

Table 1.

Recruitment and Early Study Termination by Site

Site	Subjects Randomized	Early Study Terminators N (%)	Early Lithium Discontinuation N* (%)
001	44	7 (16%)	1
002	54	6 (11%)	5
003	43	14 (33%)	2
004	44	1 (2 %)	7
005	54	13 (24%)	7
006	44	5 (11%)	2

Note:

^*N= subjects who discontinued lithium before the week 24, or final study visit.

Figure 1.

LiTMUS Cumulative Attrition Rates (%) by Month

Note. Duration of Study is the time from when the first subject was randomized in LiTMUS to the completion of the follow-up phase for the last randomized subject.

Table 2.

Reasons for Early Study Termination

Reasons for early termination	N (% of randomized)
Lost to follow-up	32 (11.3%)
Unable to follow study procedures, visit schedule, etc	6 (2.1%)
Withdrew Consent
Dissatisfied with treatment setting	3 (1.1%)
Reason unspecified	1 (0.4%)
Dissatisfied with clinical response to treatment	1 (0.4%)
Other*	3 (1.1%)

Note:

^*“Other” reasons for early termination included: refusal to comply with MD recommendations (1); lost to follow-up, living arrangement not conducive to study participation (1); military obligations (1).

Table 3.

Last Study Visit Attended by Early Study Terminators (ESTs)

Last visit Attended	Number of ESTs at visit (Cumulative % of participants who completed the study, N=283)
Baseline	7 (2.8%)
Week 2	5 (4.2%)
Week 4	6 (6.4%)
Week 6	4 (7.8%)
Week 8	2 (8.5%)
Week 12	7 (11.0%)
Week 16	6 (13.1%)
Week 20	7 (15.5%)
Week 24	2 (16.3%)

Table 4.

Summary of the LiTMUS Study Design Aspects to Minimize Attrition

Method
Intent-to-treat design
Randomized adjunctive single-blind design
Participant reimbursement
Intent-to-Attend measure and procedures
Quality care with limited participant burden
Target windows for study visits

Strategies to Minimize Attrition

Intent-to-treat Design

The LiTMUS study used an intent-to-treat study design that allowed participants to remain in the study regardless of whether they were adherent to their randomized study treatment (i.e., Li plus OPT or OPT only). This strategy enhances therapist-provider rapport, an important component of treatment adherence, and thus, study retention [12, 13]. Study participants were offered clinical treatment regardless of their adherence to the study treatment, indicating a commitment to their clinical care by the study staff. In contrast, it could be argued that by keeping these participants in the study, we artificially boosted our retention rate. Given that our participants were still required to adhere to the other aspects of the study protocol, such as the visit schedule and assessments, our retention rate is still high by historical standards [4, 8]. It has also been suggested that an intent-to-treat design may increase medication non-adherence. Yet in the LiTMUS study, 11% (N=31) did not take the randomized study treatment (i.e., Li plus OPT or OPT only), whereas typically 20–50% of bipolar patients are not adherent to their study medication [14].

This study design feature also improves our estimation of treatment effect sizes by impacting the statistical analyses. Specifically, the intent-to-treat design reduces self selection through attrition so that it does not influence treatment assignment. To conduct these analyses consistent with the intent-to-treat design, we explicitly distinguished between stopping the study randomized medication (lithium) and study termination (see Table 1). Thus, data were to be analyzed such that each subject would be classified into a treatment group based on randomized assignment, without regard to medication adherence [2]. The analyses of one of the two co-primary variables, the Clinical Global Impression for Bipolar Disorder Severity of Illness scores will involve mixed-effects models which can make valid inferences assuming ignorable attrition, or attrition that is accounted for by measures of covariates or the dependent variable that are measured prior to dropout [15]. The other co-primary, Necessary Clinical Adjustments, was a composite of medication changes per unit time (described above).

Randomized Adjunctive Single-Blind Design

The participants in each randomized group received OPT to manage symptoms or episodes as guided by published treatment algorithms [9]. The OPT plus lithium group received all available treatments plus lithium, or lithium monotherapy. The OPT only group could receive all available treatments with the exception of lithium. Thus, OPT could be tailored for each participant based on their treatment history and was informed by systematic diagnostic assessments, tracking of symptoms and side effects, and therapeutic blood levels of medications (when clinically appropriate). To fully optimize treatment, the study physicians were unblinded not only to OPT but also to participants’ randomized treatment group. This allowed them to make treatment recommendations without any restrictions, a practice that closely resembled treatment in a community setting.

The unblinded, flexible, and personalized treatment embedded in OPT seemed to be favored by LiTMUS participants, as many commented informally that this was a primary reason for entering and staying in the study. Participants perceived themselves as receiving personalized optimized naturalistic treatment rather than being “experimented with”. Participants seemed to prefer having very few limitations imposed on their spectrum of treatment options, particularly in the context of a research study, as they perceived it as a restriction of their care.

A limitation to OPT was that the treatment intensity may have been confounded with one of the two primary outcomes (CGI-BP) given that not only can illness severity influence treatment, but treatment can also influence illness severity. As a result, OPT may not have been equivalent across the two groups given the flexibility of dosing and medications. To minimize this potential confound, we summarized the treatment guidelines for bipolar disorder [9] in the LiTMUS Clinical Operations Manual which were reviewed carefully with each LiTMUS provider. Experts in pharmacotherapy for bipolar disorder (by authors E.F. and M.T.) also reviewed reports each quarter of all participants’ medications to verify that participants were indeed receiving OPT, or being prescribed medications consistent with the published treatment guidelines. In the few instances that participants were not receiving OPT (e.g., re-administering to a patient a treatment that was previous successful in that same individual, despite inconsistency with published treatment guidelines), the National Coordinating Center for LiTMUS contacted the treating study physician. If it was determined that it was not possible to have the participant comply with OPT, a protocol violation was completed and the participant remained in the study to be consistent with the intent-to-treat design.

A second limitation was that group differences may have been due to the study physician’s expertise in optimizing treatment. We accounted for this possibility by stratifying randomization by provider. Each site was also only allowed to have up to three providers to reduce the possible variance accounted for by having more than one study physician. As mentioned, all medication recommendations were reviewed each quarter to ensure that providers were following the treatment guidelines. We also utilized the Necessary Clinical Adjustments to account for the number of medication and dose changes that were due to a worsening course. This was be a primary outcome to examine whether the OPT plus lithium group required more clinical adjustments, as well as a way to account for the flexibility of treatment as part of OPT. A possible drawback for the single-blind design was that study physicians and participants could favor the active arm which would bias outcomes. To minimize bias of the active treatment arm, the primary and secondary outcomes were rated by blinded raters.

Participant Reimbursement

Participants were reimbursed $50 per study visit for time and expenses. Each participant agreed to the nine study visits (see Table 3). The baseline visit, or first study visit, was often split into two sessions to reduce the study burden, each lasting approximately 2 hours. All of the follow-up visits were approximately 60 to 90 minutes, with the exception of the week 12 (mid-treatment) and week 24 (end of treatment) visits which were approximately 90 to 120 minutes. The length of each visit, as well as the procedures for each visit (blood draw, self report forms, meetings with study staff) were outlined at the time subjects consented, so they were aware of the study burden. Thus, participants who completed all of the follow-up visits received $450 for approximately 14 hours of their time. We opted to reimburse participants $50 for each study visit as most of the study sites are located in urban environments associated with substantive travel and parking costs. We also elected to not use gas cards, or other, more restrictive forms of reimbursement, given we wanted to standardize the study procedures yet sites required different forms of reimbursement (e.g., some participants did not drive). Thus, utilizing money as the form of reimbursement ensured that all types of study costs (e.g., childcare, taxis, public transportation, missed meals) were covered for our study participants.

Participants across the sites often stated that the study visit reimbursement was an important motivating factor for scheduling and attending follow up visits. This was keenly observed at one site (i.e., Massachusetts General Hospital) which had to switch their payment procedures halfway through the study. Initially, this LiTMUS site was reimbursing participants with cash at each visit, but then payment changed to a check request system which required that visit payments came several weeks late, resulting in participants’ paying out-of-pocket for their travel time and study-related expenses. After the payment procedures were altered, this site observed a decline in their retention rates and an increase in missed visits, suggesting that subject reimbursement was influential.

A potential limitation to participant reimbursement was that the amount of money could be considered coercive. In discussing the costs associated with the study, such as transportation (highly variable), parking ($10–25), participant time ($10–20), and childcare ($15–40), the amount provided ($50/visit) seemed reasonable. Additionally, each sites’ Internal Review Boards approved this amount for study visit reimbursement. It is also possible that this amount of reimbursement may attract the wrong type of study participants, or individuals with the intent to obtain money opposed to having an interest in the study treatment. Although it is difficult to determine individuals’ intent to participate in a study, we carefully screened individuals, such that, we split the first study visit into two visits (i.e., a screening and a baseline/randomization visit) to determine their dedication to the goal of the study and its procedures.

Intent-to-Attend Procedures

The Intent-to-Attend scale is a one item measure that assesses the likelihood that participants will attend their next study visit [16]. At baseline the intent to complete the trial was also assessed. Scores were rated by participants on a 9-point Likert scale. If participants responded with a 4 or below (i.e., less than “unsure” about attending) on the Intent-to-Attend scale, the research coordinator discussed with the participant the reasons that he or she did not want to attend the next study visit. The coordinator then discussed with the participant ways to problem solve the identified obstacles of future attendance and attempted to accommodate the participant’s needs. This study was the first to implement such intent-to-attend procedures, or a discussion of problem-solving attendance obstacles when participants rate a low likelihood to attend the next study session. Future analyses will examine the effectiveness of these procedures.

A possible limitation of the Intent-to-Attend scale is that it could encourage attrition by introducing the possibility of not returning for the next study visit. This was a concern of the National Institute of Mental Health’s Data and Safety Monitoring Board and thus, we decided to add the novel procedures discussed above for scores equal than or less than a score of a four. It is also possible that subjects may resent talking about their motives to be in a study or their likelihood to return for future study visits. The research coordinators were carefully trained on how to sensitively administer the additional procedure for low scores, but most participants (i.e., those with a moderate to high intent to attend) completed this measure on their own with no follow-up.

Quality Care with Limited Participant Burden

LiTMUS, through the use of OPT, ensured that participants received the best possible personalized treatment for bipolar disorder. This study was also designed to allow for close monitoring of participants as the first five study visits occurred every two weeks, and the remaining four visits occurred monthly. These visits are more frequent than usual as typical pharmacotherapy visits for bipolar disorder are every four to eight weeks. Participants at each study visit also met with at least three study personnel (i.e., a study physician, research coordinator, and a blinded rater). At each study visit, data on adverse events, symptoms, functioning, and suicidality were obtained and carefully reviewed; however, we were mindful to exclude scales and information not critical to the primary aims of the study, thereby limiting participant study burden [11]. We also only required laboratory monitoring at Weeks 0, 2, 14 and 24. We also attempted to minimize this burden on participants by choosing the duration of the study to be only six months, which is shorter than many maintenance trials in bipolar disorder [4]. Our rationale for this study duration was that participants would likely understand the need for an adequate period of follow up to obtain the needed outcome information, but that a 12 to 18 month study would likely have been less appealing, given the effort involved with attending visits and the possibility of lack of adequate benefits, or adverse effects. We also explained to participants that the tolerability of lithium was a priority, so that lithium dosage would be lowered if adverse effects ensued. For these reasons, participants reported receiving very good care and attention which likely affected their motivation to complete the study.

A potential drawback with this type of treatment is that it is expensive and does not accurately reflect “real-world” treatment. A cost-benefit analysis will be conducted to examine this further, although close monitoring is necessary when investigating pharmacotherapy for bipolar disorder given the tendency for mood to vary over time and the risk of side effects of these study medications [17, 18]. OPT could also be considered the gold standard for treatment received in the community. Further training and dissemination of the empirically-supported treatment guidelines for bipolar disorder to community and rural clinics as well as how to diagnose and monitor the disorder are needed.

Target Windows for Study Visits

We utilized “target windows” for study visits in LiTMUS which allowed for some flexibility in scheduling them. For the weekly visits, a window of six days (i.e., ±3 days from the target date) and for the monthly visits, a window of ten days (i.e., ±5 days from the target date) were permitted. We also utilized a target window calculator that determined the appropriate window of the eight follow-up study visits which the site coordinators reported was very helpful. Participants reported that they appreciated the flexibility of having a target window, opposed to a target day, to accommodate their schedules, such as work, other doctor’s appointments, as well as gave them extra time, for example, in the event that they became ill or took a vacation. The use of target windows was also particularly helpful in rescheduling visits that were missed, forgotten, or rescheduled at the last minute by the subject which study coordinator’s believed reduced the number of missed visits. If a study visit during the target window was not possible or failed to occur, then an out of window visit could be held, to ensure continuity of care.

A potential drawback of the flexibility allowed by using target windows is that it may sacrifice scientific rigor by not having the exact same time between each study visit. However, the mixed-effects models will incorporate the time of actual assessment, not the planned assessment time, in models that assume a linear effect of time. It is also possible that participants’ may perceive that the actual target date for the study visit is less important and therefore, be less motivated to attend the follow-up visits on time.

Discussion

The LiTMUS study was a longitudinal, randomized multi-site comparative effectiveness trial that examined lithium in bipolar disorder, a disorder typically associated with high rates of attrition. Despite the rigorous study design, study medication, and patient population, this study yielded a lower attrition rate than that encountered in other longer-term bipolar disorder treatment studies. We highlight six possible components of the LiTMUS design that may have contributed to minimizing attrition rates: a randomized adjunctive single-blind design, reimbursement for study-related costs, intent-to-attend procedures, quality care with limited participant burden, and target windows for study visits (see Table 4).

Despite these substantial strengths of the study, there were also several limitations. First, we have not yet analyzed the efficacy data or the final adverse event data. Thus, it is possible that our participants had very few adverse effects, which is an important reason for attrition in most bipolar studies [6]. It is also possible that we did not recruit a representative bipolar population. Given that participants were required to have an overall Clinical Global Illness Severity score of at least “mild” at study entry, participants must have been experiencing at least some symptoms that prompted them to seek care. We also made several specific efforts to recruit “real-world” patients for this study. We paid for study medication (i.e., lithium) as well as OPT medication, as needed, to recruit participants from low income families and/or those without insurance.

As mentioned, it is possible that the treatment and monitoring details of this study are not feasible in community settings given that the study was conducted at bipolar, or mood disorder, specialty clinics. We believe that following empirically-supported treatment and monitoring guidelines for bipolar disorder should represent the standard of care and that disseminating this information should be an area of continued focus [9, 19]. However, guidelines do not address certain aspects of the treatment program included in this study, such as the bi-monthly pharmacotherapy or staff (i.e., research coordinators) assigned to follow each patient. These features of the study may have contributed to the low attrition rate, but do not mimic treatment received in community settings. Yet, these aspects are typical for pharmacotherapy studies and yet, LiTMUS had a low attrition rate, suggesting that specific components of this study could be useful in reducing attrition and increasing retention of study participants.

LiTMUS also did not utilize a run-in phase prior to randomization which could have further minimized our attrition rates. The BALANCE (Bipolar Affective disorder: Lithium/ANti-Convulsant Evaluation) study, a longitudinal, open-label multi-site trial (N=330) to compare maintenance treatment with lithium, valproate or combined, yielded a relatively low attrition rate, or 21% [20]. This is particularly noteworthy as BALANCE utilized a 24-month follow-up phase compared to 6-months of follow-up in LiTMUS. However, BALANCE utilized a run-in phase, or assessed treatment response over 4 to 8 weeks to ensure that only participants who “tolerated both drugs in the short term” were randomized. Of the 459 participants enrolled in BALANCE, 129 (28%) withdrew from the study during the run-in phase and prior to randomization. Given that nearly half the participants that attrited from LiTMUS, did so before week 8 (see Table 3), such a run in phase would likely have reduced our attrition rate further [20]. Nonetheless, BALANCE utilized several of the strategies summarized in Table 4, such as an intent-to-treat, single-blind design while providing quality clinical care with limited participant burden, which also likely explains their low attrition rate.

We also had site differences in attrition which suggests that other, non-study related factors may have impacted attrition (see Table 1). For example, the site with the highest attrition rate (i.e., site 003) is located in a state that requires all residents to have health insurance. This requirement afforded participants at this site more treatment options, and in particular, the ability to have clinical care without participating in research. Sites with the lower attrition rates (e.g., site 004) noted that often their participants were only able to receive treatment if enrolled in a research study, due to lack of health insurance or availability of clinical care. Thus, further research is warranted to investigate other factors that may impact attrition.

In summary, several design aspects of LiTMUS, a comparative effectiveness study of bipolar disorder could be considered to limit attrition rates. Low attrition rates allow for more precise evaluations of interventions for difficult-to-treat populations and require innovative approaches for optimal study performance.