Certainty of genuine treatment increases drug responses among intellectually disabled patients

Abstract

Objective:

To determine the placebo component of treatment responses in patients with intellectual disability (ID).

Methods:

A statistical meta-analysis comparing bias-corrected effect sizes (Hedges g) of drug responses in open-label vs placebo-controlled clinical trials was performed, as these trial types represent different certainty of receiving genuine treatment (100% vs 50%). Studies in fragile X, Down, Prader-Willi, and Williams syndrome published before June 2015 were considered.

Results:

Seventeen open-label trials (n = 261, 65% male; mean age 23.6 years; mean trial duration 38 weeks) and 22 placebo-controlled trials (n = 721, 62% male; mean age 17.1 years; mean trial duration 35 weeks) were included. The overall effect size from pre to post treatment in open-label studies was g = 0.602 (p = 0.001). The effect of trial type was statistically significant (p = 0.001), and revealed higher effect sizes in studies with 100% likelihood of getting active drug, compared to both the drug and placebo arm of placebo-controlled trials. We thus provide evidence for genuine placebo effects, not explainable by natural history or regression toward the mean, among patients with ID.

Conclusions:

Our data suggest that clinical trials in patients with severe cognitive deficits are influenced by the certainty of receiving genuine medication, and open-label design should thus not be used to evaluate the effect of pharmacologic treatments in ID, as the results will be biased by an enhanced placebo component.

In any pharmacologic treatment, contextual and cognitive factors contribute to the therapeutic response, such as patients' treatment expectations and knowledge about the treatment.¹ These factors, referred to as nonspecific treatment mechanisms, are active ingredients in placebo responses.² To date, little is known about the contribution of nonspecific treatment effects in trials for patients with severe cognitive deficits.

Intellectual disability (ID) is defined by impaired intellectual functions, such as reasoning, abstract thinking, and learning from experience. ID is confirmed by intelligence testing (IQ <70), combined with assessments of mental abilities and adaptive functioning and evidence that the observed limitations were manifested during the developmental period (DSM-V).³

A recent meta-analysis showed that patients with genetically determined ID display significant improvements in the placebo arm in randomized controlled trials, both on subjective and objective outcomes.⁴ Yet few studies included a no-treatment control group, and the placebo response (improvement in the placebo arm) could thus not be separated from a true placebo effect, which controls for natural history, being in a trial, and regression to the mean. In order to determine the placebo effect, which rests on treatment expectations,² studies in other patient populations have developed a method where drug responses in placebo-controlled trials are compared with those in open-label trials, as they represent 50% vs 100% certainty of receiving active drug.^5–7

We assessed the placebo component in ID clinical trials by performing a meta-analysis comparing drug responses in open-label vs placebo-controlled trials. Studies were restricted to genetically determined ID, and the statistical analyses were matched for drug type, so that placebo-controlled and open-label trials compared the same drugs.

METHODS

Data.

We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines for reporting meta-analysis results, and data were obtained from MEDLINE/PubMed and PsycINFO, based on searches for all randomized, placebo-controlled (or open-label), pharmacologic clinical trials in patients with genetically determined ID due to fragile X, Down, Prader-Willi, or Williams syndrome, published before June 2015.

Study selection.

The inclusion criteria for this meta-analysis were open-label (or randomized placebo-controlled) trials in patients with fragile X, Down, Prader-Willi, or Williams syndrome, of any age, from any country, reported in the English language. In line with our previous ID meta-analysis,⁴ studies were excluded if (1) outcomes did not evaluate cognitive–developmental functions or (2) the treatment targeted a peripheral comorbidity rather than core symptoms of ID. Consolidated Standards of Reporting Trials guidelines were employed to ensure adequate quality of the studies included in this meta-analysis (figure 1).

Figure 1. Flow chart for the open-label drug trials included in the meta-analysis.

Data extraction and quality assessment.

Exclusion criteria were (1) studies that failed to report an inference test or enough information to compute an effect size, (2) studies with fewer than 5 patients, (3) studies that did not provide separate reports for drug/placebo (applicable for placebo-controlled trials), (4) not a randomized trial (applicable for placebo-controlled trials), (5) only treatment results from healthy controls reported. Data extraction was performed independently by 2 reviewers. Discrepancies were adjusted in reviewer meetings and confirmed with a third reviewer.

We checked for overlap in participants between studies from the same authors. Based on the patient description including mean age, age range, and date of inclusion, it was possible to rule out the overlap in participants between studies by Erickson, Heller, and Kishnani. More precisely, the mean age and age range did not overlap, especially considering that in all these studies from the same authors, the most recent ones were performed in younger children. In the studies by Berry-Kravis, the majority of patients were not included in both studies. There was some doubt in less than 15% of the patients. It was not possible to rule out some overlap in participants in the studies by Torrioli and Prasher.

Statistical analysis.

Treatment response was defined as the difference in outcome measures from pre to post treatment within each treatment arm. Calculation of effect size was performed by means of bias-corrected standardized mean differences (Hedges g) using Comprehensive Meta-Analysis version 3.0 (meta-analysis.com). In order to adjust for heterogeneity in the included data, all analyses were performed using random-effects instead of fixed-effects models. In studies with multiple outcome measures, a combined within-study outcome was computed as a synthetic effect size (with a variance that takes account of the correlation among the different outcomes). This approach corrects for the problem of more weight being assigned to studies with more outcomes.⁸ For a complete list of all outcomes, listed per study, see table e-1 at Neurology.org.

An overall meta-analysis was performed to assess the effect size of pre-post treatment within the open-label trials (for a corresponding meta-analysis of the placebo-controlled studies, see our previous report⁴). In order to compare the effect size of open-label and placebo-controlled trials, studies were matched on drug category. The only criteria for matching studies was that a drug category must be represented in both open-label and placebo-controlled trials in order to be included. Hence, the principle was to include all studies from each eligible drug category, and we did not exclude any studies based on other criteria. A wide range of treatments were represented (including vitamins, γ-aminobutyric acid agonists, and statins); for a full list, see table e-2. Six different drug categories were represented both in open-label and placebo-controlled studies and were thus used for the comparison between open-label and placebo-controlled studies. Twelve open-label studies and 12 placebo-controlled studies were included in the matched comparison (even if there was no requirement to end up with the same number of studies) (table 1). The overall effect was assessed using a univariate analysis of variance, using trial duration (weeks) as covariate, and Sidak correction for multiple comparisons was applied to the pairwise comparisons.

Table 1.

Open-label and placebo-controlled studies

RESULTS

Overall meta-analysis: open-label.

Seventeen open-label trials met the inclusion criteria, comprising a total of 261 patients. The mean duration of studies was 38 weeks (SD 48, range 8–208 weeks). Sixty-five percent of the participants were male, and the mean age was 23.6 years (SD 17.5, range 2–53) (table 1). Three of the 17 open-label trials, representing 29 (11.1%) participants, did not report participant sex distribution. We initially identified 13 open-label studies with Williams syndrome but none of them was eligible: fewer than 5 patients in the trial (12 cases) and not evaluating cognitive developmental functions (1 case).

The pre-post analysis of outcomes in the open-label trials showed a significant treatment response (g = 0.602, SE = 0.116, p < 0.001; figure 2). In line with our previous reports of ID placebo responses and trial duration,⁴ we found no significant correlation between study effect size and length of open-label clinical trials (r = 0.066, p = 0.802), indicating that long and short trials had comparable treatment results.

Figure 2. Forest plot of treatment responses in open-label intellectual disability (ID) drug trials.

A significant improvement from pre to post treatment was seen across all studies (p < 0.001). If studies included more than one outcome measure they were combined into one value. A random-effects model was used to calculate significance.

Overall meta-analysis: Placebo-controlled.

Twenty-two placebo-controlled studies met the inclusion criteria, comprising 828 patients in the drug arm and 721 in the placebo arm. The mean duration of studies was 35 weeks (SD 37, range 4–156 weeks). Sixty-two percent of the participants were male, and the mean age in the drug arm was 17.8 years (SD 16, range 0–53) and 17.1 years (SD 15.6, range 0–55) in the placebo arm.

As previously reported by our group,⁴ a pre-post analysis of outcomes in placebo-controlled trials (k = 22) showed significant treatment responses, both in the drug arm (g = 0.678, SE = 0.171, p < 0.001) and the placebo arm (g = 0.468, SE = 0.150, p < 0.005).⁴

Open-label vs placebo-controlled trials.

The effect of trial type on treatment outcomes was determined by comparing effect sizes between drug-matched open-label (k = 12) and placebo-controlled (k = 12) trials (table 1), comprising a total of 199 patients (open-label) and 394 patients (placebo-controlled). The effect of trial type was statistically significant (F_2,32 = 5.7, p = 0.008, η² = 0.263), and revealed higher effect sizes in studies with 100% likelihood of getting active drug (open-label, mean g = 0.65, SD = 0.41) compared to the drug arm (p = 0.043) (mean g = 0.31, SD = 0.38) and placebo arm (p = 0.009) (mean g = 0.21, SD = 0.34) in placebo-controlled studies (figure 3). The study effect sizes were not affected by trial duration in any of the 3 groups: open-label drug (r = 0.34, p = 0.287), placebo-controlled drug (r = 0.15, p = 0.645), or placebo-controlled placebo (r = 0.243, p = 0.447), indicating that ID treatment responses were unaffected by differences in trial length.

Figure 3. Drug response from pre to post treatment in drug-matched open-label and placebo-controlled trials.

There was a significant difference between study effect sizes between drug-matched open-label (k = 12), placebo-controlled drug (k = 12), and placebo-controlled placebo (k = 12) treatment groups. The overall effect was assessed using a univariate analysis of variance and Sidak correction for multiple comparisons was applied to the pairwise comparisons. *Significant at p < 0.01; **significant at p < 0.05.

DISCUSSION

Our comparison between drug effects in open-label and placebo-controlled studies validate that treatment expectations play a role in treatment of ID, in spite of severe cognitive deficits. In open-label studies, patients (as well as their doctors and families) are assured that the allocated treatment contains an active drug, and thus the contribution of positive treatment expectations is likely to be high. In placebo-controlled trials, however, there is generally a 50% chance of receiving an active drug. The certainty of getting active medication in placebo-controlled trials is thus lower, and may provide less positive expectations of improvement. The effect of certainty in drug trials is previously reported among individuals with intact cognitive functions,^5–7,9 and evidence suggests that certainty of receiving the real drug maximizes patients' treatment expectations, leading to higher treatment responses.¹⁰ Here we demonstrate, for the first time, that patients with severe cognitive deficits are susceptible to the different treatment contexts of open-label and placebo-controlled drug trials, similar to the effects seen in non-ID patients.^6,7Open-label drug trials in ID patients are therefore biased by the certainty of receiving genuine medication, and should not be used to study pharmacologic effects in ID, as the results will be misleading. Novel therapies for ID that target the underlying pathophysiology are currently being developed, for example for Down syndrome and fragile X syndrome.¹¹ New promising results can lead to a general increase of treatment expectancy among patients' families, which may enhance the bias (as seen in secretin trials for children with autism^12,13) if treatments are not tested in properly blinded placebo-controlled designs. Even if open-label drug trials will not be able to separate the pharmacologic effect from the placebo component of a given treatment, open-label trials may be used for the purpose of assessing side effects. Nonetheless, research in non-ID patients shows that side effects are also shaped by expectations though so-called nocebo effects,¹⁴ and it is important to recognize the possibility that information about side effects (written/oral) may affect the occurrence of adverse effects also in ID patients.

In response to the prevalent notion that ID patients are less susceptible to treatment expectancies, we previously performed a meta-analysis where we were able to demonstrate that patients with genetically determined ID display placebo responses in placebo-controlled clinical trials.⁴ Yet our previous meta-analysis did not compare placebo responses to a no-treatment control, and we could therefore not decidedly link the pre to post improvements to placebo mechanisms (even if natural remission of genetically determined ID would be unlikely). In the present analysis, we employed a balanced design that allowed for interpretation of the role of treatment expectations between open-label and placebo-controlled drug trials. Patients with ID thereby demonstrated genuine placebo effects, which are not explained by spontaneous remission or regression to the mean.

There are several possible mechanisms involved in shaping treatment expectations and placebo effects, including implicit and explicit learning.¹⁵ In addition to verbal information about the benefits of a drug (requiring cognitive learning abilities), treatment cues may be implicitly embedded in the patient–clinician interaction and clinical environment.¹⁶ In patients with ID, it is likely that the implicit social influence of placebo by proxy¹⁷ plays a role, which describes the indirect influence of treatment expectations among clinicians and close relatives, and has been mentioned apropos of placebo effects in young children. In brief, placebo by proxy refers to an alteration in the patient's psychosocial context that may promote placebo effects in a way that is not strictly a learning phenomenon, yet it may have the same profound influence on patients' response, for example through caregiver's altered behavior towards the patient. Even if we studied the effects of active drugs (and not placebos) in this study, the placebo by proxy phenomenon is highly relevant for understanding the placebo component of genuine treatment outcomes. Here, the effect may be described as an expectancy by proxy effect on clinical outcomes, reflecting the expectations of surrounding parents, caretakers, and clinicians.

Limitations to our study include the inability to control for the possibility that treatment responses rated subjectively by parents, teachers, or caregivers were biased by raters' knowledge of trial type, despite no or little effect on patients' actual behavior. We have previously been able to reject the notion that subjectively rated outcomes have larger placebo effects compared to objectively measured outcomes in ID patients,⁴ and predict that the same is true in the present study. In our previous study, we found that the relatively small number of studies (k = 22) led to differences in effect size if outliers were removed. We also demonstrated significant differences in treatment responses based on which drug was used in the trial, both in the drug arm and the placebo arm.⁴ In order to address drug heterogeneity in the present study, we used a drug-matched approach when analyzing the effect of trial type. Yet this reduced the sample size and led to apparent differences in effect size for the overall vs subgroup data. Another limitation is the restriction to include studies that measure treatment effects on core ID symptoms. In the future, it would be valuable to assess the placebo component of treatment for somatic symptoms in patients with ID.

GLOSSARY

DSM-V

Diagnostic and Statistical Manual of Mental Disorders, 5th edition

ID

intellectual disability

AUTHOR CONTRIBUTIONS

Conceived and designed the study: A.C., I.K., T.J.K., K.B.J. Performed the data collection: A.C., K.Y., M.P., K.B.J. Analyzed the data: A.C., K.Y., M.P., K.B.J. Contributed reagents/materials/analysis tools: A.C., I.K., T.J.K., K.B.J. Wrote the paper: A.C., K.Y., M.P., A.R., I.K., R.L.G., V.d.P., T.J.K., K.B.J. Interpreted the data: A.C., A.R., I.K., V.d.P., T.J.K., K.B.J.

STUDY FUNDING

T.J.K. is supported by NCCIH/NIH grant 2K24 AT004095.

DISCLOSURE

The authors report no disclosures relevant to the manuscript. Go to Neurology.org for full disclosures.