Specific-class Skin Side-effects of Drugs Might Compromise Blinding in Randomized Controlled Trials

Randomized controlled trials (RCTs) are the gold standard for high-quality evidence-based medicine. Double-blinding in RCTs allows for avoiding performance bias, whereas absence of blinding exaggerates treatment effects by 14–35% compared with double-blinding (1, 2). Therefore, maintaining blinding is crucial, especially when the endpoints of studies are subjective outcomes (3, 4). The “open” or “blinded” design is usually reported and discussed in the limitation sections of studies, but the risk of unblinding during the trial is rarely reported (3, 4). However, except for new drugs, the side-effects of which are still not known (5, 6), several drugs induce specific-class side-effects (SCSEs), especially skin effects; hence the participants taking the drug(s) during a trial can be detected easily.

To highlight this important issue, we chose 2 classes of molecules (acitretin and isotretinoin for oral drugs, imiquimod for topical drugs) with well-known SCSEs (mainly cutaneous and mucosal xerosis/local inflammatory reaction) and performed a systematic review to deduce the risk of unblinding in RCTs of these drugs.

MATERIALS AND METHODS

Trials comparing acitretin, isotretinoin and imiquimod with placebo or an active comparator were selected; which were included in the electronic databases MEDLINE and CENTRAL during 1982–2016 with at least 1 actor reported to be blinded (patient, care provider or outcome assessor). Extracted data included items related to blinding, the frequency of SCSEs and the primary outcomes of trials that we classified as: (i) objective outcomes (such as complete regression of lesions on imaging), (ii) subjective outcomes (pain or quality of life for example), and (iii) unclear (7).

Statistical analysis

Descriptive statistics are expressed with number (%) for categorical data. Meta-analysis of adverse events was performed by computing odds ratios (ORs) with use of fixed-effects modelling. ORs and 95% confidence intervals (95% CI) were calculated. Statistical analyses involved use of SAS 9.3 (SAS Institute, Cary, NC, USA).

RESULTS

From the 486 articles screened, 75 trials were included (21 for acitretin, 25 for isotretinoin and 29 for imiquimod) that concerned 3 main conditions: inflammatory diseases (n = 54), skin cancers (n = 16), and non-skin cancers (n = 5).

In 61% of RCTs, care providers and patients were reported to be blinded (16/21, 18/25 and 12/29 for acitretin, isotretinoin and imiquimod, respectively), and outcome assessors were reported to be blinded in all trials. In 4 RCTs of imiquimod, the authors only briefly mentioned “difficulties” with unblinding.

Adverse events were reported in 80% of RCTs overall (n = 60). As expected, SCSEs included dryness and cheilitis for retinoids (reported in 67% of RCTs assessing acitretin and 56% of isotretinoin) and local irritation, inflammation and wounds for imiquimod (reported in 83% of RCTs). Forest plot analysis of the 40 (53%) RCTs that provided details on adverse effects showed that the rates of SCSEs were always much higher in the experimental than control groups (Fig. 1).

Fig. 1.

Forest plots describing the occurrence of specific class side-effects (SCSE) in experimental and control groups of randomized controlled trials of acitretin, isotretinoin and imiquimod. (References are available in Appendix S1) “Events” are the number of patients with specific class side-effects, and “Total” represents the number of patients included in the trial. Control includes placebo (for 25/40 trials, 62.5%) or active comparator (for 14/40 trials, 35%) and in one trial (1/40, 2.5%), experimental treatment was compared with both placebo and active comparator. 95% CI: 95% confidence interval.

Subjective primary outcomes were used in 53% of RCTs (16/21 for RCTs of acitretin, 12/25 isotretinoin and 12/29 imiquimod), and objectives outcomes represented 16% of RCTs (1/21, 6/25 and 5/29, respectively); the others were considered unclear. For conditions, subjective and objective outcomes were used in 78.0% (32/41) and 9.8% (4/41), respectively, of RCTs of inflammatory diseases, 25% (4/16) and 56.3% (9/16) of RCTs of skin cancers and 0/4 and 4/4 of RCTs of non-skin cancers; the others were unclear. No guarantee was mentioned to maintain blinding despite the probable unblinding linked to SCSEs of drugs.

DISCUSSION

By focusing on 3 different molecules, this study shows that a high frequency of SCSEs that are visible and that occur very rarely in the control group could lead to unblinding in RCTs. Indeed, when SCSEs occurred, they allowed for the outcome assessors to easily guess to which treatment group patients were assigned. If the endpoint of the trial is subjective, then the outcome assessor might be influenced in the evaluation of treatment efficacy, which leads to a bias, as in open-label RCTs (3, 4, 7). In most cases, primary outcomes were subjective in RCTs of these 3 molecules.

As dermatologists, we considered only drugs with cutaneous SCSEs because they are visible and easily recognizable. For some, such as imiquimod, local irritation is a side-effect and is directly linked to the treatment effect (8). However, several drugs might induce SCSEs that are less visible, such as a decrease in cardiac frequency (e.g. RCTs of propranolol) or biologic side-effects (e.g. increase in liver enzyme levels after methotrexate treatment). To maintain blinding, protocols elaborate specific procedures: assessing efficacy by using photographs as in the RCT on propranolol in infantile haemangiomas (9) or using 2 independent assessors (1 for efficacy and 1 for biologic side-effects) in the trial on methotrexate for spontaneous chronic urticaria (10). Finally, some adverse events, such as stomach pain or headaches, are frequently reported in experimental groups, but also in placebo groups in RCTs (11–13) and therefore are less likely to induce unblinding.

In conclusion, in RCTs assessing drugs with SCSEs, objective outcomes are recommended (as in open-label RCTs) to avoid risk of unblinding; otherwise, measures must be implemented so that outcome assessors can be independent (photographs, active placebo, makeup). When such strategies cannot be implemented, transparency in lack of blinding is mandatory.