Are there really only two kinds of people in the world? Evaluating the distribution of change from baseline in pain clinical trials

Abstract

It is often assumed that there are two types of pain patients: those who respond well to efficacious pain therapies and those who do not respond at all, with few people in the middle. This assumption is based on research that claims that changes in pain intensity have a bimodal distribution. The claim of bimodality has led to calls for a change in how pain clinical trials are designed and analyzed, for example, performing “responder” analyses instead of comparing group means to evaluate the treatment effect. We analyzed data from four clinical trials, two each of duloxetine and pregabalin, for chronic musculoskeletal and neuropathic pain conditions to critically examine the claim of bimodality of the distribution of change in pain intensity. We found that the improper construction of histograms, using unequal bin widths, was the principal flaw leading to the bimodality claim, along with the use of the oft-criticized baseline observation carried forward (BOCF) method for imputing missing data also serving as a contributing factor. Properly constructed histograms of absolute change in pain intensity using equal bin widths, combined with more principled methods for handling missing data, resulted in distributions that had a more unimodal appearance. While our findings neither support nor refute the hypothesis that distinct populations of “responders” and “non-responders” to pain interventions exist, the analyses presented in earlier work do not provide support for this hypothesis, nor for the recommendation that pain clinical trials prioritize “responder” analyses, a less efficient analysis strategy.

1. Introduction

In the past decade, a series of articles have claimed that changes in pain intensity, as measured by a self-report numerical rating scale (NRS) or visual analog scale (VAS), have a bimodal distribution [12–15]. This observation suggests that there exist substantial subgroups of those who respond well to pain interventions and those who do not, with a relatively small percentage of people falling in the middle [17]. The claim of bimodality has been echoed by other researchers [7] and has led to calls for a change in how pain clinical trials are designed and analyzed [9,13].

Traditionally, the effect of treatment has been estimated using between-group differences in means. If the distribution of changes in pain intensity in the treatment group is as strikingly bimodal as claimed, it would support the argument that the between-group difference in means is not a good summary of treatment efficacy because it does not reflect the experience of most patients [3,25,26]. The claim of bimodality – that patients “respond either by a relatively large amount or not at all, so that mean change is not informative” [22] -- has led to recommendations to prioritize “responder” analyses in which treatment groups are compared with respect to the proportion of participants who achieve a pre-specified reduction in pain intensity from baseline [8,12,22]. Common thresholds for “response” include a reduction in pain intensity of at least 30% or 50% [4,6]. The implications of this recommendation are important because trials that use binary outcomes (“responder”, “non-responder”) typically require substantially larger sample sizes than trials that use continuous outcomes.

Closer inspection of the evidence supporting the claim of bimodality reveals substantial flaws, the most significant being the use of unequal histogram bins when presenting the distributions of outcomes [12–15]. Other problems include the use of baseline observation carried forward (BOCF) imputation to handle missing data, which is known to have poor statistical properties [10,16], and the use of percentage change from baseline as a metric. Percentage change is a ratio that tends not to be normally distributed, even when an absolute change might be, and is an inefficient method for incorporating the use of baseline information in the analysis of outcomes [20,24].

In this article, we use data from four randomized trials of pregabalin and duloxetine for chronic musculoskeletal and neuropathic pain conditions to examine the shapes of the distributions of absolute changes in pain intensity using more principled methods for dealing with missing data. By manipulating the cut-points used to define the histogram bins, we recreate the illusion of strong bimodality of these distributions that previous work has promoted. We also illustrate the undesirable effects of BOCF imputation and the use of percentage change from baseline on the distribution of outcomes.

2. Methods

2.1. Data Sources

This work was conducted under the auspices of the Analgesic, Anesthetic, and Addiction Clinical Trial Translations, Innovations, and Networks (ACTTION) public-private partnership with the U.S. Food and Drug Administration (FDA). Through this partnership, ACTTION was granted access to data from clinical trials of various chronic pain conditions. For this article, we included data from two clinical trials of duloxetine, one for chronic low back pain (CLBP) [21] and the other for osteoarthritis joint pain (OA) [2]. For the CLBP study, the 60 mg/day and 120 mg/day groups were separately randomized and analyzed. For the OA study, participants were randomly assigned to either remain on 60 mg/day or to switch to 120 mg/day at Week 7; for the analyses in that paper, these groups were combined for the primary analysis. In our analyses of the data from both of these studies, the 60 mg/day and 120 mg/day groups were combined. Data from these trials have been previously used in analyses that purport to provide support for the claim of bimodality [14].

We also included data from two published clinical trials of pregabalin, one involving participants with painful diabetic peripheral neuropathy (DPN) [19] and another involving participants with postherpetic neuralgia (PHN) [23]. Data from the highest pregabalin dosage in each trial were used for illustration given the established efficacy of these dosages (300 mg/day, 600 mg/day). These four clinical trials were selected because they examined two medications with different mechanisms of action that have been approved by the FDA for the chronic musculoskeletal and neuropathic pain conditions in which they were studied. The study designs and the percentages of participants with missing endpoint data for the four trials are summarized in Table 1. Outcomes summarized at each visit, including baseline, were the ratings of average pain intensity measured on a 0–10 NRS, averaged over the 7 days prior to the visit. If there were fewer than 7 diary entries available prior to a particular visit, the average over the available entries was used unless there were fewer than 4 diary entries, in which case the outcome for that visit was considered to be missing. The primary outcome variable in each trial was the average pain intensity rating over the seven daily ratings provided prior to the final double-blind visit.

Table 1.

Summary of analyzed duloxetine and pregabalin clinical trial data

Protocol		Pain Condition	Placebo Group		Active Group††		Dosages (mg/day)	Follow-up Weeks
			N	Dropout N (%)	N	Dropout N (%)
Duloxetine	Reference 21	CLBP	121	59 (48.8)	115	62 (53.9)	60, 120†	0, 4, 7, 13
	Reference 2	OA	128	17 (13.3)	128	35 (27.3)	60, 120†	0, 4, 7, 13
Pregabalin	Reference 19	DPN	85	14 (16.5)	82	12 (14.6)	600	0, 2, 4, 6
	Reference 22	PHN	93	41 (44.1)	90	34 (37.8)	300/600*	0, 1, 4, 8, 13

CLBP: chronic lower back pain; OA: osteoarthritis; DPN: diabetic peripheral neuropathy; PHN: postherpetic neuralgia

^*Subjects in this arm received either 300 mg/day or 600 mg/day depending on their creatinine clearance at baseline

^†In the CLBP trial, the 60 mg/day and 120 mg/day groups were separately randomized and analyzed. In the OA trial, participants were randomly assigned to either remain on 60 mg/day or to switch to 120 mg/day at Week 7, and the two groups were combined for the primary analysis. The 60 mg/day and 120 mg/day groups were combined in the present analyses of the data from these trials.

^††Only the maximum dosage from each trial was used.

2.2. Handling Missing Data

Simplistic methods for handling missing data such as BOCF are well known to have poor statistical properties, yielding generally biased estimates of treatment effects and underestimating the variance of the treatment effect, possibly leading to an inflation of the Type I error probability [11,16]. Also, it can be argued that in most clinical trials involving participants with pain, imputing outcomes using BOCF is overly pessimistic given the improvement (from baseline) often seen in participants receiving placebo.

Instead, more principled methods have been recommended for dealing with missing data in the analysis of clinical trials [16]. Here, we have used a recently developed method for handling missing data called control-based multiple imputation based on pattern mixture modeling (PMM) [1,18]. This method allows one to make various assumptions concerning the distribution of outcomes after dropout in a participant given the observed outcomes prior to dropout, treatment group, and other characteristics of that participant such as the reason(s) for dropout. We imputed missing data based on two different strategies using this method. One is the “missing at random” (MAR) assumption, whereby a participant’s post-dropout outcomes will be similar to those of other participants who completed the study and had similar characteristics (such as treatment group assignment) and pre-dropout outcomes [18]. This assumption is standard in the analyses of clinical trials with missing data. The imputation model under the MAR assumption included treatment group and the pain intensity ratings at all visits prior to the time of dropout. The other strategy assumed MAR for those in the placebo group. In the active treatment group, the “jump to reference” (J2R) assumption was used for those who dropped out due to adverse events and MAR for those who dropped out for any other reason (and also for intermediate missing data). According to the J2R assumption, after dropout, outcomes from a participant in the active treatment group immediately switch to resemble those from similar participants in the placebo group who completed the trial.

The multiple imputation algorithms were executed using 100 imputations, and the average of the 100 imputed values was used for the missing outcome at the final visit for purposes of constructing histogram summaries of the distributions of interest.

2.3. Statistical Analysis

After imputation of missing data using the BOCF method, the distributions of the percentage change from baseline in the active treatment group were summarized for each of the 4 clinical trials using histograms that were constructed in two different ways:

1. Using the bins < 15%, 15–29%, 30–49% and ≥ 50% reduction in pain intensity from baseline that have been used in previous studies [12–15]. These bins, clearly of unequal width, seem to be selected to correspond to the IMMPACT recommended definitions of a substantial reduction in pain (≥ 50% reduction in pain intensity from baseline), a moderate reduction in pain (≥ 30% reduction in pain intensity from baseline), and a minimal reduction in pain (15–20% reduction in pain intensity from baseline) [4].

2. Using equal-width bins, which is required for proper representation of distributions using histograms. The histograms were drawn using the “hist” function in R with default settings to determine the number of bins and bin widths based on the data.

Given the shortcomings of percentage change from baseline as a metric, we also constructed histograms with equal bin widths for the absolute change from baseline for each trial, after imputation of missing data using the pattern mixture model (assuming MAR and J2R/MAR). Additionally, to illustrate that, with a certain choice of cut-points, it is possible to make the distribution of absolute change from baseline in pain intensity look strongly bimodal, even when the true distribution appears to be unimodal, we chose “bimodalizing” cut-points and constructed the resulting histograms. The cut-points used were: < 1.8, 1.8–2.3, 2.3–3.0 and ≥ 3.0.

Smoothed density curves (kernel density estimates) were also superimposed on the histograms of percentage change and absolute change from baseline. The density curves were estimated using the “density” function in R with all parameters set to default values.

3. Results

Histograms of the percentage change from baseline (with positive values indicating improvement in pain intensity) using BOCF imputation for missing data are shown for the duloxetine and pregabalin trials in Figures 1 and 2, respectively. The histograms on the top row, properly constructed with equal bin widths, do not show the obvious bimodality that is present in the histograms on the bottom row with unequal bin widths. The histograms on the bottom row essentially reproduce the results in previous work claiming bimodality of these distributions [12–15]. The main observations from this figure are that (1) the use of bins of varying width when constructing the histogram produces a misleading depiction of bimodality, (2) the use of BOCF imputation produces spikes at 0% in the distribution of percentage change that may produce the illusion of bimodality, and (3) the distribution of percentage change appears to be far from normal. Similar results are apparent in the pregabalin trials (Figure 2).

Figure 1:

Histograms of percentage change from baseline in pain intensity (with positive values indicating improvement) after BOCF imputation for missing data in duloxetine-treated subjects in two clinical trials for chronic lower back pain (CLBP) and osteoarthritis (OA). The histograms on the top row, with density curves superimposed, were constructed using bins of equal width and the histograms on the bottom row were constructed using bins of unequal width, with cut-points used in [12–15].

Figure 2:

Histograms of percentage change from baseline in pain intensity (with positive values indicating improvement) after BOCF imputation for missing data in pregabalin-treated subjects in two clinical trials for diabetic peripheral neuropathy (DPN) and postherpetic neuralgia (PHN). The histograms on the top row, with density curves superimposed, were constructed using bins of equal width and the histograms on the bottom row were constructed using bins of unequal width, with cut-points used in [12–15].

Histograms of the absolute change from baseline (with positive values indicating improvement in pain intensity) using J2R/MAR imputation for missing data are shown for the duloxetine and pregabalin trials in Figures 3 and 4, respectively. The histograms on the top row use equal bin widths while those on the bottom row use the “bimodalizing” cut-points to create bins of unequal width. As shown by the density curves, the histograms on the top row of each figure reveal distributions that have a much more unimodal appearance, whereas the histograms on the bottom row are indisputably (and artificially) bimodal. Histograms when MAR imputation was used, and also when complete cases were used, reveal a similar pattern of results (see Supplemental Digital Content). Although the histograms and density curves for the OA and DPN trials (Figures 3 and 4) indicate a possible small departure from strict unimodality, they provide compelling evidence that the distribution of change from baseline is not concentrated at the two extremes of minimal or no change and substantial change [12–15].

Figure 3:

Histograms of absolute change from baseline in pain intensity (with positive values indicating improvement) after J2R/MAR imputation for missing data in duloxetine-treated subjects in two clinical trials for chronic lower back pain (CLBP) and osteoarthritis (OA). The histograms on the top row, with density curves superimposed, were constructed using bins of equal width and the “bimodalized” histograms on the bottom row were constructed using bins of unequal width.

Figure 4:

Histograms of absolute change from baseline in pain intensity (with positive values indicating improvement) after J2R/MAR imputation for missing data in pregabalin-treated subjects in two clinical trials for diabetic peripheral neuropathy (DPN) and postherpetic neuralgia (PHN). The histograms on the top row, with density curves superimposed, were constructed using bins of equal width and the “bimodalized” histograms on the bottom row were constructed using bins of unequal width.

4. Discussion

In this study using data from four randomized, double-blind, placebo-controlled clinical trials of efficacious dosages of duloxetine and pregabalin, in four different chronic pain conditions, we found no evidence that changes in pain intensity scores among duloxetine-treated or pregabalin-treated participants have the strikingly bimodal distribution depicted in previous studies. Nevertheless, we were able to reproduce the previously observed bimodality results by using the binning strategy that the authors employed [12–15]. This strategy used much wider bins at the extremes of the distribution than in the middle of the distribution, directly resulting in more participants falling in those extreme bins and the misleading perception of bimodality. Using histogram bins of unequal width can produce a very misleading depiction of a distribution unless one compensates for this by appropriately adjusting the height of the bar so that the relative frequency of subjects falling into that bin is represented by the area of the bar. We believe that this is the overarching explanation for the apparent striking bimodality observed in previous studies.

The unfortunate consequence of the bimodality claim is the perpetuation of the notion that these studies provide evidence that, in response to an efficacious intervention, a minority of individuals experiencing pain will have substantial reductions in pain intensity (“responders”), a majority will have little or no reduction in pain intensity (“nonresponders”), and very few patients will fall in between these two extremes [12–15]. When we used proper methods to construct histograms of changes in pain intensity, we arrived at a very different conclusion. The bimodality claim also perpetuates the notion that a group mean provides a misleading description of the typical patient’s experience in a trial, and that so-called “responder” analyses are more appropriate for analyzing pain intensity data from clinical trials than analyses of between-group differences in means. Such analyses are typically much less efficient than analyses that compare group means, leading to increased clinical trial costs and participant burden. Also, as has been noted by others [5,17,20], the term “responder analysis” conflates outcomes with treatment effects. It is not the case that individual changes in pain intensity are necessarily due to actual responses to the experimental treatment; changes could also be due to factors such as measurement error, regression to the mean, mood, diurnal variation and physical activity, among others. Similarly, one cannot interpret between-subject variation in changes in pain intensity as true heterogeneity in response to treatment using data from trials having either a parallel group design or a standard cross-over design [5,20]. Studies that attempt to identify treatment effect modifiers on the basis of a definition of individual “response” may thus simply be attempting to identify predictors of outcome rather than of response to treatment [17]. We acknowledge that “responder” analyses, when properly presented and interpreted, can yield useful information for clinicians concerning the distributions of individual subject changes in pain intensity by treatment group and we support their inclusion in clinical trial reports.

We also found that the strategy for imputing missing data can contribute to the misleading appearance of a bimodal distribution. The use of BOCF for imputation of missing data is widely criticized in the biostatistics literature. The assumption on which it is based is not realistic, particularly in pain trials in which improvement in pain intensity is often observed, even in the placebo group. There are various reasons for such improvement, including “placebo effects” and the power of expectation, and the increased attention that trial participants receive from clinic staff, among others. It also creates an artificial spike in the distribution of change (imputing zero change for those who do not complete the trial), which can contribute to the appearance of bimodality. One of the original rationales for using BOCF was that it does not assign a favorable outcome to someone who does not complete the trial, which seems attractive in the context of most pain studies in which the pain condition may not be worsening over time. In conditions that can spontaneously improve over time, however, BOCF may be too punitive. In this case, the J2R imputation strategy seems more reasonable in terms of penalizing outcomes post-dropout. It also avoids introduction of an artificial spike in the distribution of change. A comparison of the distributions of percentage change from baseline and absolute change from baseline revealed that the latter seemed to be closer to being normal, as would perhaps be expected based on criticisms of the use of the percentage change metric [20,24].

A limitation of this study is the fact that data from only four trials of two interventions (duloxetine and pregabalin) for four chronic pain conditions (CLBP, OA, DPN, and PHN) were examined. Our results would have been more informative if it had been possible to analyze other trials of additional treatments (e.g., opioids, NSAIDs, and non-pharmacological interventions). Such analyses would make it possible to determine how broadly our conclusions generalize across different pain treatments and conditions or whether they are treatment-specific and/or population-specific. On the other hand, we were able to very closely reproduce the bimodal distributions observed in earlier investigations using similar strategies for constructing the histograms, suggesting that our findings may not be isolated. A recent article [17] examined the distributions of the percentage change from baseline in pain intensity in 10 randomized trials of nonsurgical interventions for spinal pain, with some trials including chronic pain and others including acute or subacute pain. The authors found that while the distributions appeared to be non-normal, there was no evidence of bimodality.

In conclusion, our findings fail to support existing claims of striking bimodality of the distributions of changes in pain intensity in clinical trials of two efficacious medications with different mechanisms of action examined in four different chronic musculoskeletal and neuropathic pain conditions. Although the methods used for imputation of missing data and the use of percentage change from baseline as a metric may contribute to the appearance of striking bimodality, the overarching contributing factor appears to be the inappropriate use of unequal bin widths when constructing histograms depicting these distributions. We emphasize that the evidence presented here neither supports nor refutes the hypothesis that distinct populations of “responders” and “non-responders” to pain interventions exist; however, our findings demonstrate that the evidence presented in earlier work that supports this hypothesis is misleading. The recommendation -- in response to these previous studies -- that clinical trials in pain should prioritize “responder” analyses will lead to less efficient designs. It is possible that the underlying distributions of change in pain intensity in the trials that we examined consist of mixtures of distributions of those who respond somewhat differently to pain medications; however, evidence of the existence of such mixtures is very difficult to identify in parallel group studies [5,20].