Research designs for proof-of-concept chronic pain clinical trials: IMMPACT recommendations

1. Introduction

Clinical trials account for nearly 40% of the research and development budget of major pharmaceutical companies [94], representing a major investment and consequently a significant hurdle to development of new treatments. Increasing efficiency early in treatment development could increase the number of approved pain therapies by allocating available resources to more resource-intensive later phase confirmatory trials of only the most promising new therapies. Proof-of-concept (POC) trials can be considered those trials in which the objective is to obtain an initial evaluation of the potential efficacy of a treatment. Such trials include early dose-finding studies that aim to identify potentially efficacious medication dosages or treatment frequencies for a target disorder and later stage trials in which the goal is to obtain sufficient evidence of efficacy to warrant further study of the treatment in confirmatory trials.

Different objectives exist for clinical trials conducted at different stages in the development and investigation of a treatment, and these objectives determine the trial design. In some cases, the interest is in understanding whether an intervention is efficacious under ideal conditions. In this case, the goal of the trial can be considered “explanatory,” with the effect of the treatment compared to placebo using tightly controlled methods that maximize the internal validity and assay sensitivity (i.e., signal detection ability) of the clinical trial [108]. In contrast, the objectives of “pragmatic” trials include evaluating the effectiveness of an intervention in real-world settings [30,108], for example, in patients who may not be adherent to treatment and who have comorbid conditions that might reduce the treatment’s efficacy [31,102,122]. POC trials typically have explanatory objectives rather than pragmatic aims and generalizability to the population is thus not a main priority. These explanatory aims include early stage exploratory components such as identifying the appropriate disorder (e.g., chronic neuropathic vs. musculoskeletal pain), evaluating different potential primary outcome measures (e.g., spontaneous vs. evoked pain), determining pharmacological dosages likely to have a favorable risk-benefit ratio, and optimizing length, intensity, and/or components of non-pharmacologic treatments as well as the initial evaluation of treatment efficacy.

The objective of the present article is to describe research design considerations for POC chronic pain clinical trials, addressing both exploratory and preliminary efficacy objectives that are often accomplished with separate study designs. We first discuss general considerations regarding POC trials, and then describe the major POC trial designs and their advantages and limitations when used to evaluate chronic pain treatments. We focus on those designs that maximize statistical power by decreasing variability in outcomes, maximizing effect size, or minimizing participant withdrawal. Much of our discussion focuses on pharmacologic studies, but many of the considerations we examine also apply to non-pharmacologic treatments and examples are provided throughout the manuscript.

2. Methods

An IMMPACT consensus meeting was held that included an international group of participants from universities, government agencies, industry, and a patient advocacy organization. Participants were selected on the basis of their research, clinical, or administrative expertise relevant to the design and evaluation of treatments for chronic pain. The meeting was designed to reflect a broad representation of relevant disciplines and perspectives while limiting the size in order to promote productive and efficient discussion.

To facilitate discussion, background lectures were presented at the meeting that examined (1) POC trials: an industry perspective (CR); (2) POC trials: an academic perspective (IG); (3) single-dose and short-term POC trials in neuropathic pain (SNR) and non-neuropathic pain (NPK); (4) statistical considerations in POC trials, including cross-over designs (SS); (5) futility trials and other adaptive designs (CM); (6) dosage determination from preclinical to POC trials (CT); (6) PK-PD modeling and dosage determination for POC trials (MG); and (7) phenotyping chronic pain patients for POC trials (RB). To complement the background presentations, a systematic review was performed to identify recent methodological articles pertaining to POC trials and POC trials with novel designs. Pubmed was searched for articles published between January 2005 and April 2013 using the following criteria: (“pain” AND “proofof-concept”) OR (“pain” AND “phase 2”) OR (“pain” AND “phase II”). The search yielded 1,413 items that were examined by the first author, who identified 130 articles for additional attention based on the titles. Review of the abstracts of these articles led to 27 being identified for review by the second author, 15 of which were then selected for complete review and inclusion in this manuscript where appropriate.

It is important to emphasize that the following discussion has been based on the results of systematic studies to the greatest extent possible. However, the evidence base relevant to the design of chronic pain clinical trials is not extensive [25,26,27,28,59] and our considerations also reflect accumulated experience derived from clinical trials of treatments for chronic pain and other medical and psychiatric disorders. The design considerations for POC clinical trials of chronic pain treatments presented in this article are based on the background presentations and extensive discussions at the consensus meeting, subsequent discussions and review of the literature by participants, circulation of a draft manuscript to all authors, and iterative revision of draft manuscripts until consensus was achieved; the final version of the article was approved by all authors.

3. General Considerations

To ensure internal validity, POC trials should be conducted with techniques to minimize bias, including randomization and blinding [82,83,111]. We propose that POC clinical trials should have no more than two active treatment groups and a placebo group whenever possible, to maximize statistical power with limited sample sizes. A recent meta-analysis examining the relationship between trial characteristics and standardized effect size (SES) in neuropathic pain trials found no relationship between number of study arms and SES [29]. Although evidence to support the benefits on assay sensitivity of limiting the number of treatment arms in analgesic trials is inconclusive, no data suggest that fewer treatment arms leads to lower assay sensitivity. Thus, we suggest limiting the number of study arms whenever possible. Exploration of multiple dosages (or frequency of non-pharmacologic treatments) is an important exception to this consideration, and is often performed separately from POC studies in which the major objective is to evaluate whether there is preliminary evidence of efficacy.

The treatment duration used in a POC trial requires careful consideration. Although a shorter trial is more desirable when resources are limited, in trials of chronic pain it is important to allow adequate time to observe a treatment effect if one exists and determine the duration of treatment required to reach maximal separation of experimental treatment and placebo group responses [40,66]. It is difficult to identify the optimal follow-up duration in a POC trial because the ideal length will depend on what is known or predicted about the time to the treatment’s maximal effect. Regulatory agencies generally require that the maintenance phases of confirmatory trials last for at least 12 weeks to establish the durability of the treatment effect and to provide a reasonable amount of time to assess safety and tolerability. The IASP special interest group on meta-analyses [82] suggests a follow-up duration of at least 7 weeks for a chronic pain trial, although their discussion was not aimed specifically at POC trials. A review by Quessy and Rowbotham [96] revealed a trend toward increased placebo response in trials of longer duration when evaluating studies that lasted between 4 weeks and 22 weeks. These results suggest that a trial with a shorter follow-up duration (e.g., 4 weeks) may be associated with smaller placebo responses and increased assay sensitivity, supporting a general consideration of a 4-week follow-up duration for POC studies. Thus, we suggest that when the primary objective of a POC trial is to conduct an initial evaluation of efficacy and the treatment is expected to achieve its maximal effect within several weeks, treatment duration of at least 4 weeks can be considered generally adequate. Although it is important to recognize that efficacy at 4 weeks may not be maintained in a longer trial, there are few if any examples of such transitory analgesic effects. Furthermore, a confirmatory trial should have sufficient power to compensate for a possible increase in placebo response in a trial of longer duration. In cases in which existing interventions are being evaluated for efficacy in new conditions, the treatment duration in POC trials can often be based on what has been observed in studies of the initial indications.

3.1. Patient population

A POC trial is often the first step in the evaluation of the potential efficacy of a new treatment or an existing treatment in a new condition. As a result, external validity (i.e., generalizability) is not a major goal of most POC trials. Thus, one way these studies are often performed is to enroll a well-defined, uniform group of patients in whom the intervention is considered most likely to be efficacious. In determining which patients will be included, characteristics that should be considered include demographic factors (e.g., age), diagnosis (including the etiology of the chronic pain), pain duration, pain intensity, specific pain symptoms and signs, and medical and psychiatric comorbidities. A comprehensive discussion of factors to consider when enrolling patients in analgesic clinical trials can be found in previous IMMPACT recommendations for confirmatory trials [27] and assay sensitivity [28].

In contrast to studies that enroll a homogenous study population, another objective of POC trials can be identifying the optimal target patient population, and in such cases the treatment could be tested in patients with varying characteristics to determine which patients seem to respond best to the treatment. One limitation of a study in which the treatment is evaluated in different types of patients can be a lack of statistical power to adequately estimate a treatment effect in each subgroup. This would be particularly challenging when resources are limited, as they often are for POC studies in academic settings. One approach to addressing this challenge can be to use an N-of-1 study design, as discussed below.

3.2. Active comparators

An active comparator is a treatment that is known to be efficacious for the condition being studied and its selection should generally be consistent with multinational treatment guidelines or approval by the Food and Drug Administration (FDA) or the European Medicines Agency (EMA). In many circumstances, an active comparator can be recommended for POC trials. Although adding another treatment group to any clinical trial uses important, often limited, resources, it can increase the interpretability of a negative result. Trials that do not demonstrate the efficacy of an active comparator can be considered to lack assay sensitivity. It should, therefore, not be concluded that the investigational treatment in such a trial lacks efficacy if it too failed to separate significantly from placebo. If the active comparator was shown to be efficacious and the investigational treatment was not, it can be more confidently concluded that the investigational treatment is not efficacious for the condition being studied.

Three recent studies demonstrate the value of including active comparators in POC trials of chronic pain treatments. Rowbotham et al. [100] examined the analgesic effects of an α4β2 neuronal nicotinic receptor agonist in painful diabetic peripheral neuropathy (DPN). Although efficacy was not demonstrated for the nicotinic receptor agonist, duloxetine was included as an active comparator and it provided significantly greater pain relief than placebo. The authors were able to conclude that their POC trial was sufficiently sensitive to detect the analgesic efficacy of duloxetine in DPN and that the nicotinic receptor agonist was therefore likely not efficacious.

Huggins et al. [49] used naproxen as an active comparator in a clinical trial of a fatty acid amide hydrolase-1 (FAAH) inhibitor for osteoarthritis knee pain. Patients were randomized to one of two separate two-period cross-over trials comparing placebo to either naproxen or the FAAH inhibitor, rather than examining both active treatments and placebo in a single three-period cross-over trial. Naproxen was found efficacious vs. placebo, whereas no effect was detected for the FAAH inhibitor. This novel POC cross-over design is advantageous because it decreases the burden on participants by using two periods instead of three, although it does require a larger sample size than a three-period cross-over trial. However, to conclude that the positive result for naproxen truly serves as a positive control for the FAAH inhibitor using this design, it must be assumed that the relatively small number of participants randomized to each of the two-period cross-over trials are comparable in measureable and unmeasurable characteristics that might affect assay sensitivity.

In a press release, Xenoport [133] reported the results of an unpublished POC study for painful diabetic peripheral neuropathy in which neither an experimental drug nor pregabalin (i.e., the active comparator) demonstrated significant treatment efficacy. The knowledge that the trial was not sufficiently sensitive to detect efficacy of an FDA-approved treatment allows the sponsor to make a more informed decision about whether to halt development or modify the study design for another POC trial.

Adding an active comparator to a trial is consistent with laboratory standards that require positive controls; however, this approach consumes valuable resources. It is therefore important for investigators to consider, in advance, whether an inability to find an effect of an active comparator will modify how they proceed in the face of a negative trial. It could also be argued that adding an active comparator may not be a cost-efficient strategy if investigators are confident in the assay sensitivity of their planned trial. Table 1 presents active pharmacologic comparators that could be used in POC trials of several commonly studied chronic pain conditions.

Table 1.

Active comparators to consider for proof-of-concept analgesic trials.

Chronic pain condition	Active comparator
Painful diabetic peripheral neuropathy	Duloxetine*, pregabalin*, or tricyclic antidepressants [4,23,24]
Postherpetic neuralgia	Gabapentin* or pregabalin* [4,23,24]
Osteoarthritis joint pain	Duloxetine, NSAIDs*, or tramadol [47,88]
Chronic low back pain	Duloxetine* or NSAIDs* [14]
Fibromyalgia	Duloxetine*, milnacipran*, pregabalin*, or amitriptyline* [12,117]

^*Approved for the indication by either the Food and Drug Administration (FDA) or European Medicines Agency (EMA)

3.3. Active placebo

Active placebos mimic the side effects or other characteristics of the active treatment in a clinical trial and, as a result, can increase the integrity of the double-blind of the study. They are often recommended for chronic pain POC and confirmatory clinical trials with active treatments that have common side effects that can be recognized by patients and investigators. For example, diphenhydramine, an anti-histamine that has sedative properties, has been used as an active comparator for alfentanil, dextromethorphan, ketamine, morphine, pregabalin, systemic lidocaine, and topiramate [64,70,127,128]. Benzodiazepines (e.g., diazepam, lorazepam, midazolam) have also been used as active placebos in clinical trials of gabapentin, fentanyl, ketamine, and morphine due to their sedative effects [20,42,43,90,105] Benztropine, diphenoxylate/atropine, and loperamide can be used to cause mild constipation to mimic the constipation that is a common adverse effect of opioid analgesics [63].

It could be argued that active placebos are unethical because they cause discomfort for participants in the placebo groups of clinical trials without likely potential for benefit to the individual. On the other hand, to the extent that the use of active placebos reduces the probability of false positive results due to patient or investigator unblinding, their use would be associated with more meaningful results and greater contributions to health care knowledge. Whether the advantages outweigh the disadvantages of active placebos is debated. We emphasize that the use of an active placebo in a POC trial should be considered and the benefits be weighed against the risks under specific trial design and intervention circumstances.

3.4. Prioritizing probabilities of false positive and false negative outcomes

The goal of a POC trial is not to confirm efficacy, but to provide initial evidence of efficacy and determine whether additional studies of a treatment should be conducted in a given condition. A common challenge of POC trials is to obtain such information as efficiently as possible. Moving a treatment from the POC phase into trials designed to confirm efficacy exposes many more patients to possibly ineffective investigational treatments with risks that are not fully understood. However, abandoning the development of a truly efficacious treatment can be detrimental to those patients who have not responded to existing treatments. Balancing the risks of moving a treatment forward based on false positive results or stopping investigation prematurely based on false negative results is a major challenge in POC trials. If the goal of a POC trial is to evaluate whether a treatment should be transitioned to a confirmatory trial, the relative costs of abandoning further research on the basis of a false negative result vs. conducting another, typically larger trial on the basis of a false positive result must be carefully evaluated. Designing an informative POC trial with a relatively small number of participants using a conventional significance level of 5% with 90% or greater power is challenging, and in many cases not realistic. Although the research designs proposed here can help increase statistical power for a given sample size, compromises are often necessary at the POC stage in terms of the probabilities of type 2 error (finding a negative result when the treatment is actually efficacious) and type I error (finding a positive result when the treatment is actually not efficacious).

To illustrate the balance between false positives and false negatives, suppose that the trial is testing a novel intervention in patients with painful DPN. In this case, it could be argued that a false negative outcome is more acceptable because several efficacious treatments are already available for this condition. Conversely, for a condition for which there are no or few efficacious pharmacologic treatments, such as painful HIV neuropathy or lumbosacral radiculopathy, it could be argued that a false positive result is more acceptable because it would be more detrimental to fail to identify a potentially efficacious treatment than to conduct further trials that ultimately have negative results. If the balance of such considerations favors accepting an increased probability of type I error to avoid a falsely negative outcome, the pre-specified significance level can be increased above the conventional level of 5%, for example, to 10% or 20% [61]. On the other hand, in circumstances in which a false negative outcome is more acceptable than a false positive result, the conventional 5% significance level could be used accompanied by reduced power, for example, 80% rather than 90%, so that the required sample size is more feasible.

3.5. Enrichment

Treatment response is often highly variable among subjects in analgesic clinical trials. This variability can be caused by multiple factors including: (1) subjects may experience different degrees of improvement due to placebo effects and other non-specific factors; (2) subjects may vary in the extent to which they adhere to the protocol; (3) subjects may have differential ability to rate their pain consistently; (4) subjects may vary in their ability to tolerate the treatment; (5) the treatment may be more efficacious in some subjects than in others. Enrichment designs can be used to decrease the variability from these possible sources in order to increase the chances of detecting an effect if one truly exists in a certain population of subjects [121,125]. Although enrichment strategies may increase assay sensitivity of the double-blind phase, they may not always increase the overall efficiency of the trial. See Brittain and Wittes [9] and Schechtman and Gordon [107] for factors that may influence the efficiency of enriched vs. non-enriched designs.

Use of run-in periods to identify and exclude participants who demonstrate a pre-specified magnitude of placebo response, non-compliance, treatment intolerability, or variability in pain ratings has been discussed in previous IMMPACT articles [27,28]. Placebo-, compliance-, and variability-based enrichment select subjects in whom a treatment effect may be more readily detected. Enriching for subjects based on a pre-specified level of positive response to the investigational treatment or a proven effective treatment can potentially identify a subset of patients for whom the treatment is likely to be efficacious. This type of enrichment can be based on an initial unblinded treatment period or previous clinical experience [119]. When the enrichment is based on the experimental treatment and the subjects are randomized in the double-blind phase to either continue treatment or receive placebo, the design has been called an enriched enrollment randomized withdrawal (EERW) design [60,80]. Previous IMMPACT articles have discussed the advantages and limitations of EERW designs in depth [27,28]. Because of this, we limit our discussion to systematic evidence addressing the relationship between EERW design and assay sensitivity. Although it is reasonable to expect that enrichment based on initial efficacy would increase effect size by identifying a subset of participants for whom the treatment is likely to work, systematic evidence to support this assumption does not exist. A recent meta-analysis by Furlan et al. [36] did not find a statistically significant difference between the mean effect-sizes for EERW vs. non-EERW trials that evaluated opioid treatments for chronic pain. This analysis, however, included only 12 EERW trials and may have lacked sufficient power to detect minimally important differences between EERW and non-EERW trials. Since Furlan et al. [36] is the only study to systematically compare effect sizes for EERW and non-EERW analgesic trials, it is impossible to determine whether the EERW design does in fact increase assay sensitivity (see Hewitt et al. [45] for a comparison of treatment efficacy in enriched and non-enriched samples in a single randomized withdrawal study).

A recent trial by Ho et al. [46] evaluated the concept of enriching for analgesic responders using a novel design with two treatments known to be efficacious for neuropathic pain (gabapentin and tramadol) and placebo (Figure 2). In their study, small fiber neuropathy patients were selected based on a positive response to gabapentin — a medication considered efficacious in this condition — and the efficacy of both gabapentin and tramadol was examined in a placebo-controlled phase. Self-reported gabapentin responders were tapered off gabapentin and those whose pain was ≥ 3 and had increased by ≥ 30% from their pre-taper pain score at the end of the taper phase were randomized in a double-blind, three-period cross-over trial of gabapentin, tramadol, or active placebo (diphenhydramine). With only 18 patients randomized in the cross-over trial, both gabapentin and tramadol showed statistically significant improvements in pain intensity compared to placebo. The results of this POC trial suggest that enriching a trial by selecting patients who are responsive to one efficacious treatment may enhance the sensitivity of the trial to detect the efficacy of another treatment and provide adequate power with a very small number of participants. However, because the authors did not randomize those patients whose pain did not increase upon gabapentin withdrawal, it is difficult to determine whether the enrichment phase actually increased the assay sensitivity of the trial.

Figure 2.

Example later stage POC designs: Sequential parallel comparison design (SPCD) and enriched cross-over design. (A) SPCD incorporates Stage 2 data from Stage 1 placebo non-responders only, potentially decreasing the impact of the placebo response [32]. *Indicates groups that are included in the final analysis. (B) In an enriched crossover design, two single-blind run-in phases (drug with known efficacy followed by placebo) used to identify subjects who appear to respond to the currently indicated drug. The experimental drug is tested only in these “responders” [46].

A unique enrichment strategy by Byas-Smith et al. [11] identified potential responders to transdermal clonidine using an initial 2-period cross-over design rather than an unblinded phase. Forty-one patients completed the initial cross-over trial, in which the mean difference in pain between clonidine and placebo was not significant (p=0.55). Twelve patients defined as apparent responders were then enrolled in the second 4-period cross-over trial, which detected a significant difference between clonidine and placebo (p=0.015), with 8 out of 12 participants reporting lower mean pain intensity during their 2 clonidine periods than in their 2 placebo periods in the second cross-over study. Thus, after starting with an unselected sample of patients with painful DPN, these two cross-over trials conducted in sequence successfully demonstrated efficacy for a treatment that is seemingly efficacious but in only a relatively small subgroup of patients.

Although existing evidence is not sufficient to conclude that enrichment based on initial positive treatment response increases overall assay sensitivity, enrichment may be useful to identify treatments that are beneficial in only a subgroup of patients. However, in many enriched designs, participants become familiar with the side-effects of the active treatment and may recognize the absence of these side effects if they are switched to placebo [69]. This possibility makes unblinding a particular concern in enriched designs and consideration of an active placebo for these designs is therefore encouraged.

Evaluations of blinding have been performed using blinding questionnaires [41,42,105,123] that ask participants to guess their treatment group assignment and indicate the reason for that guess (e.g., improvement, side effects, other). If a high percentage of participants guess their assignment correctly based on side effects, the blinding and thus the internal validity of the study may have been compromised [28], although not necessarily. If, however, a high percentage of participants guess their assignment correctly based on improvement, this is likely an indication of treatment efficacy and would not reflect compromised internal validity [1,104,110].

Finally, enrichment based on initial response may limit the generalizability of trial results to the larger unselected population of patients with the pain condition of interest. The goal in a POC study, however, is usually focused on identifying any potential efficacy and maximizing internal validity, rather than on external validity. Once potential efficacy is indicated at the POC stage, generalizability can be tested at the confirmatory stage. Furthermore, while the FDA has acknowledged [125] that “results from trials using empiric enrichment strategies provide no way to prospectively identify patients with a greater likelihood to respond, or predict the magnitude of response in an unselected patient,” it is suggested that “when the prescriber is reasonably able to gauge the effectiveness of a drug in an individual patient (e.g., pain is relieved, cholesterol is reduced), the pretreatment ability to predict the likelihood of a drug response with accuracy may not be as critical.” Finally, The FDA guidance states that “labeling will reflect limitations and concerns, but it seems clear that a drug shown effective in an enriched study should be available even if the responder population is not identified as precisely as would be desirable.”

3.6. Missing data

Missing data are common in chronic pain trials, with drop-out rates greater than 50% in some trials [59]. Missing data can be associated with poor safety and tolerability, poor treatment effectiveness, and other factors. Statistical methods for missing data generally rely on assumptions that cannot be verified, and large amounts of missing data can introduce bias and obscure interpretation of trial results [72,73]. It is critical to incorporate strategies to increase participant retention and prevent missing data to the greatest extent possible. Strategies that should be considered to reduce missing data include: (1) allowing subjects to continue on current medications, (2) providing rescue treatments, and (3) flexible dosing, in which the double-blinded medication dosage can be titrated up or down to achieve the best balance of pain relief and adverse events [73]. It is possible that allowing concomitant analgesic medications and rescue interventions compromises assay sensitivity, although further research on this issue is required [28]. Other strategies to prevent missing data include, but are not limited to, selecting experienced investigators with established track records, limiting participant burden and particularly the number of study visits, ensuring that contact information is up to date, expressing appreciation for participants’ time, and making sure that participants recognize that they are truly collaborators in research [73]. Investigators should also consider providing encouragement and incentives to participants who continue to provide efficacy and safety data even if they discontinue the study intervention [34]. It is important to note that the trial results in this case may be confounded by participants’ use of other treatments; however, having the data to perform sensitivity analyses can further validate conclusions.

Although these strategies can limit missing data, it is likely that at least some data will be missing. Appropriate statistical methods should be employed to accommodate the missing data [81,86]. Recent recommendations from the National Research Council [86] include (1) utilizing all data available from randomized participants and (2) performing sensitivity analyses using methods that make different and realistic assumptions about the nature of the missing data. Suggested methods to accommodate missing data include multiple imputation (MI), mixed-effects models (MEM), and weighted generalized estimating equations [80]. MI and MEM can be used in cross-over trials to incorporate data from participants who provide information for only one period [54,109]. These approaches have been used in analgesic cross-over trials for neuropathic pain [31,52] and osteoarthritis [62,65]. A recent review of analgesic clinical trials demonstrated that using a last observation carried forward (LOCF) imputation method for missing data yielded lower (better) numbers needed to treat than a responder analysis that defined withdrawn subjects as non-responders, which makes similar assumptions to a baseline observation carried forward (BOCF) approach [84]. However, single imputation methods, such as LOCF and BOCF, are not recommended for primary analyses because both methods make often unrealistic assumptions about the missing data pattern and spuriously decrease the uncertainty inherent in imputation [86]. Decreased uncertainty in imputation decreases the estimated variability of treatment effect estimates, which can increase the likelihood of a false positive conclusion [86].

3.7. Informing subsequent research

In addition to providing early evidence regarding potential treatment efficacy, POC trials can be used to inform multiple aspects of the design of subsequent confirmatory clinical trials. For example, POC trials can include multiple pain assessment methods, which might help identify the most suitable primary efficacy outcome measure to be used in confirmatory trials. This outcome should not only be responsive to treatment in the proof-of-concept trial, but also clinically relevant. Methods to reduce missing data could also be evaluated in POC trials and the most successful then used in later trials. Various questionnaires and clinical examinations can also be implemented for use in hypothesis-generating secondary analyses.

In some therapeutic areas, “biomarkers” are available that can be used as outcomes in POC studies [50]. For example, tumor size is often used as a short-term outcome to suggest preliminary efficacy for cancer treatments, instead of death, the most important clinical outcome. Unfortunately, no validated biomarkers currently exist for studying chronic pain treatments. However, POC studies can be used to identify and evaluate putative biomarkers for chronic pain, for example, quantitative sensory testing, brain and spinal cord imaging, and punch skin biopsy [5,44].

3.8. Pharmacokinetic-pharmacodynamic modeling

In many cases, a goal of POC trials is to examine the dose-response relationship, which can be achieved using pharmacokinetic-pharmacodynamic (PK-PD) modeling. PK-PD modeling can provide relevant insights for POC trials in pain indications with respect to dose selection, informing trial designs and subsequent research, kinetics of onset of action, assessment of biomarker responses, and supportive evidence of efficacy [67,76,126]. Early PK-PD modeling examples in acute dental pain POC trials illustrated the importance of quantifying PK-PD relationships when interpreting the dose/exposure-response and time-effect relationships for onset and duration of action [112,113]. PK-PD models can form the basis for trial simulations to address future trial design and dosage selection questions, and can even provide supportive efficacy evidence for unstudied dosages [74, 87]. In chronic pain trials, PK-PD models that also included the effects of placebo response, biomarkers, and longitudinal pain progression have been useful in understanding the magnitude of drug response and in guiding dosage selection for future studies [16,74,114]. Quantification of PK-PD relationships is, therefore, a recommended strategy for extracting useful information from POC pain trials [67].

4. Designs

4.1. Dose-finding studies

Determining the appropriate dosage for a POC trial can be challenging. Assuming that higher “dosages” (e.g., quantity, intensity, frequency) of most interventions are more effective but also have greater side effects and can be associated with greater participant burden, a dosage must be selected that investigators predict will be efficacious, but that will also be associated with tolerable adverse events and patient burden. In pharmacologic trials, if possible, it is useful to have an activity marker that reflects engagement of the intended pharmacological target by the study medication, and to verify that plasma levels reach a range previously found efficacious in preclinical studies. Multiple ascending dose studies can be used to identify a maximal dosage or a range of dosages to take forward into POC trials focused on efficacy. A variety of approaches, ranging from simple fixed-dosage parallel group designs to PK-PD modeling and more complicated adaptive designs aimed to maximize efficiency while preserving trial validity and integrity, can be employed to identify potentially efficacious dosages in circumstances in which a maximal dosage acceptable to study in a POC trial has not been determined.

The determination of which dosage or dosages to study in subsequent trials is often not straightforward and the pre-specification of various criteria can facilitate these decisions. Comprehensive discussion of study designs to establish safe and preliminarily effective dosages is beyond the scope of this article. More inclusive sources addressing dose-finding studies for pharmacologic treatments are available [7,55,68,93,111,134]. Here, we highlight a few of the simpler techniques that could be considered. The basic parallel group dose-ranging study compares a set number of dosages without use of interim data analyses. The decision of which dosage to use in subsequent trials can be made on the basis of superior efficacy or tolerability or a combination of both.

Adaptive dose-finding designs are either “design focused,” in which a limited number of planned interim analyses alter the trial protocol, or “analysis focused,” in which data are analyzed continuously throughout the trial and dosages are modified frequently [7]. An example of a design-focused approach is the “drop-the-loser” design (Figure 1). This design is similar to the parallel group design except that instead of deciding which dosages to abandon at the end of the trial, a predetermined interim analysis is performed to decide which dosage(s) to drop and the data from before and after the interim analysis are combined to evaluate potential efficacy and tolerability at the end of the trial [7,48,71]. In one type of analysis-focused design, relatively few participants can be randomized to each of many different treatment dosages. Multiple planned interim analyses are then performed to estimate the dose response curve and, on the basis of these analyses, the pattern of randomization is modified to allocate more participants to the dosage range where efficacy and tolerability appear to be dosage dependent. For example, fewer participants are randomized to low dosages that show no efficacy and high dosages in which efficacy plateaus. This approach is more informative than a traditional parallel design because it makes it possible to explore more dosages and eventually allocate more participants to the most promising dosages [51]. All of the designs for POC trials that will be discussed in the following sections assume that generally safe, acceptably tolerated, and potentially efficacious dosages have been established previously.

Figure 1.

Examples of early stage proof-of-concept designs: Drop-the-loser dose finding and single dose studies. (A) In drop-the-loser dose finding trials, the “best” dosage from Stage 1 is selected based on efficacy, or a combination of safety and efficacy. That dosage is then tested in a new set of subjects. Data from Stages 1 and 2 are combined for both the placebo group and the “best” dosage group for final efficacy analysis [71]. (B) In single dose trials, treatment groups can be compared with respect to frequent assessments in a single session using area under the curve (AUC) as an outcome variable.

4.2. Single-dose trials

In single-dose analgesic trials, participants are randomized to receive one dose of an analgesic treatment or placebo and their pain is monitored on a regular basis (e.g., every 30 or 60 minutes) for the next several hours (Figure 1). Single-dose POC trials have been used to examine many treatments for acute pain, including pain from dental procedures [17,35], dysmenorrhea [13,75], and sore throat pain [106].The single session nature of this study makes it very efficient compared with multiple-dose studies lasting weeks or months for those treatments that are expected to have a prompt onset of analgesic effect. Furthermore, with the exception of participants who require rescue interventions when the trial design forbids it, data are much less likely to be missing from these trials, unlike those that require patients to make multiple visits to the study site or mail in forms completed at home.

Single-dose studies often use a cross-over design in which each participant is administered the active medication or placebo at different sessions separated by several days. This approach combines the benefits of the short duration of the single-dose design with the decreased variability and reduced sample size requirements of the cross-over design (see Section 4.4). Single-dose cross-over designs have been used to evaluate cannabis, lidocaine, mexiletine, morphine, clonidine, codeine, and ibuprofen for chronic neuropathic pain [78,101,103,130,131,132] as well as dronabinol for chronic non-neuropathic pain [85]. Given the efficiency of this approach, it is recommended for initial POC trials of treatments that are expected to have a rapid onset of their beneficial effect. It is important to note, however, that the predictive value of single-dose studies for efficacy in long-term use is not clear, especially single-dose studies of IV drugs that will be taken orally long term. Thus, in the context of chronic pain conditions, these studies may be most effective as highly efficient initial screens for subsequent longer-term POC trials. Similarly, this design is not likely to be useful for most nonpharmacologic interventions. Another limitation of single-dose studies is that they do not provide information regarding adverse events associated with continuous exposure to the treatment. Furthermore, single-dose studies are only useful for treatments with rapid onset and not those with which the effect may take several days or doses to become apparent.

4.3. Parallel group designs

The randomized, double-blind, parallel group design is the gold standard for confirmatory clinical trials of chronic pain treatments [27]. This design requires larger numbers of participants to detect treatment differences than the cross-over designs described below. However, for treatments with prolonged or unknown duration of benefit, the parallel group design may be the best or only option for a POC trial. In addition, the design, execution, analysis, and interpretation of parallel group trials can be relatively straightforward compared with other designs. For example, as discussed in the next section, parallel group trials are less than half as long for each participant as two-period crossover trials and complications due to treatment-by-period interaction (e.g., carryover effects) do not need to be considered in parallel group trials. Thus, parallel group studies are likely especially appropriate for nonpharmacologic interventions. However, compared with crossover trials, parallel group designs can require considerably greater sample sizes, financial resources, and investment of personnel which, depending on trial objectives, may make it challenging for academic investigators and small companies to use them for POC trials.

4.4. Cross-over trials

Early studies reported by Watson [129], Max [77,79], and Sindrup [115] each demonstrated that expertly designed and executed cross-over studies could detect analgesic treatment effects with relatively few patients with either painful DPN or postherpetic neuralgia (PHN). Subsequently, cross-over trials have been used to successfully detect analgesic effects in PHN and painful DPN [41,42,89,97,105], painful HIV sensory neuropathy [31], post-traumatic neuropathic pain [52], and osteoarthritis [65]. Cross-over trials involve two or more treatment periods in which patients receive each treatment being examined. These treatment periods are typically separated by washout periods in which patients are administered no medication or occasionally placebo in order to allow any treatment effects from the previous period to dissipate. The response of each participant to an active treatment is compared directly to the participant’s response to placebo or another active treatment, eliminating the between-subject variability that is present in a parallel group study. As a result of the decreased variability, the number of participants required in a cross-over trial to achieve the same power is often considerably fewer than for a parallel group study [109]. The smaller number of participants required for cross-over trials makes this an appealing approach for circumstances in which recruitment is difficult (e.g., rare disorders) or resources are limited.

Multiple issues must be considered when determining whether cross-over designs are appropriate for a particular treatment or chronic pain condition. Cross-over trials are only appropriate for treatments that temporarily decrease or eliminate symptoms, and not for treatments that can cure a disorder or have long-term effects (e.g., cognitive-behavioral therapy, physical therapy). “Period effects” can occur when factors unrelated to the treatment affect the condition. For example, a period effect could occur if the season in which pain is assessed affects participant activity level and thus influences pain levels. Participants are typically randomized to each of the different possible treatment sequences in cross-over trials to limit the impact of any period effects on the estimate of the treatment effect. Statistical analyses can also adjust for the increased variability introduced by period effects [54,109]. If, however, the magnitude of the treatment effect depends on the patient’s degree of pain at the beginning of the treatment period, a treatment by period interaction can occur. Because of possible treatment by period interactions, acute pain conditions are generally not appropriate for cross-over trials. One exception to this is the use of an oral surgery model in which the analgesic effect of a drug is compared after molar extractions on each side of the mouth that occur at two different times (i.e., periods of the crossover study) [18,19]. Such acute pain models are sometimes used as an initial evaluation of efficacy for potential chronic pain treatments.

Another potential cause of treatment-by-period interaction is a carryover effect, which occurs when the treatment given in one period affects the outcomes in a subsequent period. For example, the relatively long-lasting beneficial effects of high-concentration capsaicin in PHN – which appear to involve alterations in peripheral nerve fibers [2] – could affect pain scores in later treatment periods in a cross-over trial; such carryover effects could only be prevented by the use of washout periods with durations sufficiently long to permit return of underlying pathophysiology to its baseline state.

In addition to carryover of treatment effects, patients’ expectations for the second period of a crossover trial may be influenced by how they responded to the treatment in the first period [3,56], and in turn their reported pain levels in the second period may vary based on their expectation [99]. To minimize the effects of patient expectation, investigators could consider blinding patients to the timing of transition between periods. Jenkins et al. [52] reported a trial that used this technique. They incorporated single-blind placebo washout periods. Participants knew they would receive two treatments, but were unaware of when the treatments would be switched. The placebo effect was negligible, suggesting that blinded washout periods may reduce expectations and possibly increase effect sizes in cross-over trials. This strategy of blinding participants to the specific points at which each treatment begins and ends is consistent with the IMMPACT suggestion to blind patients and investigators from as much methodological information as possible about the clinical trials in which they participate [28] and was executed in two recent analgesic trials [45, 98].

Carryover effects decrease the ability to detect a difference between treatments in a cross-over trial, but there is disagreement about how to address carryover when it does occur. Although cross-over designs with more periods than treatments and associated statistical methods have been proposed to adjust for carryover effects [54], we endorse Senn’s [109] arguments against the use of such methods in clinical trials and his recommendation that the statistical analysis plan never be altered based on the presumed presence of carryover. To address the problem of carryover, we recommend including washout periods of sufficient length (based on knowledge of drug pharmacokinetics and persistence of pharmacodynamic effects), accepting the assumption that carryover has not drastically distorted the results, and confirming estimated treatment effects with data from multiple study designs. From this perspective, the use of a cross-over trial at the POC stage and confirmation of the results in a larger parallel group study is an efficient approach.

The knowledge that the participant will certainly receive both treatments in a cross-over trial may allow the participant to think more objectively about pain levels in each period than, for example, in a parallel group trial in which the participant’s hope to receive the treatment may influence the pain rating, possibly leading to a larger placebo effect. A meta-analysis of neuropathic pain trials [58] found that cross-over trials had significantly lower placebo group response rates than parallel group trials. In that study, a low placebo group response rate was associated with a statistically significant treatment effect (positive outcome) in multiple logistic regression analysis. Thus, cross-over trials for neuropathic pain may be associated with lower placebo group response rates and, in turn, increased assay sensitivity.

Considering their noteworthy advantages – particularly regarding sample size – but also recognizing their limitations, the use of cross-over designs should be considered for POC trials of chronic pain treatments. One circumstance in which this may be especially advantageous is when subgroup analyses will be performed to identify the optimal target population because the increase in the number of subjects needed for subgroup analyses can be partially offset by the decrease in the number of subjects needed with a cross-over design compared to a parallel groups design.

4.5. N-of-1 trials

N-of-1 trials evaluate a treatment using multiple periods in a single patient, usually with the intent to determine efficacy for that patient [111]. The fact that the participant receives each treatment more than once enhances the ability to identify true responders to a treatment [22]. Information from multiple N-of-1 trials can be combined using a random-effects model to estimate the percentage of patients that respond to a treatment and the mean effect size [111]. Zucker et al. [135] combined data from 58 patients enrolled in N-of-1 trials testing a treatment of fibromyalgia in a community network setting. The mean change in the primary outcome (Fibromyalgia Impact Questionnaire score [10]) between active and placebo treatment was analyzed using a Bayesian two-level random effects model for the first two periods only (representing a normal two-period cross-over trial) and then with all available data from all six treatment periods. The resulting effect sizes (standard errors) were 8.0 (3.7) and 6.8 (2.1) for the two-period cross-over analysis and combined N-of-1 analysis, respectively, suggesting that variability can be somewhat less with a combined N-of-1 trial versus a conventional two-period cross-over trial.

A recent systematic review of N-of-1 trials published in the general medical literature between 1985 and 2010 showed that only 48% of trials that reported results for more than one patient combined the results across patients to estimate the treatment effect. Furthermore, only 45% of trials reported adequate information to be included in a Bayesian meta-analysis, a method that can be used to combine N-of-1 data from multiple trials reported in the literature [37].

Limitations to consider in N-of-1 trials include the need for short treatment and washout periods and longer overall duration of follow-up for each patient in the trial. Nevertheless, Zucker et al.’s study demonstrates that it is possible to implement an N-of-1 trial in a multicenter community practice setting, an approach that could be especially worthwhile for studying the efficacy of available treatments in new indications or in rare conditions where it is difficult to recruit sufficient numbers of subjects for other study designs. The results of an N-of-1 study in a single patient may not be replicated in subsequent research due to potential lack of generalizability. The reproducibility of treatment effect estimates obtained from combining the results of multiple N-of-1 trials has yet to be evaluated systematically, but would likely provide valuable early stage POC evidence.

4.6. Adaptive designs

Adaptive clinical trial designs incorporate flexibility to change statistical and procedural aspects of the clinical trial including randomization strategy, sample size, primary outcome variable, participant population, treatment dosages, and several others [6,21,38]. Such flexibility can increase the efficiency of clinical trials and can decrease patient exposure to ineffective or unsafe medications or other treatments. However, it is important to consider adaptation as a design feature that is planned in advance in order to preserve the validity and integrity of the trial [21,38,124]. Thus, careful and extensive planning and consideration of often complex statistical issues is essential when designing these trials. Examples include but are not limited to: (1) adaptive dose finding designs, some of which were described in Section 4.1; (2) adaptive allocation designs, in which randomization ratios are altered based on the observed outcomes in each treatment group; (3) group sequential designs, with premature stopping considered for safety, futility, or sufficiently strong evidence of efficacy based on pre-specified boundaries; and (4) sample size re-estimation designs, in which the sample size estimate is revised on the basis of an interim analysis examining the veracity of assumptions concerning nuisance parameters (e.g., the standard deviation of the primary outcome variable) [38] or, more controversially, the observed treatment effect [15,53,95;124]. There are also several logistical and procedural issues that need to be considered in the planning and execution of these designs, many of which are well-summarized in Gaydos [39]. These issues can limit the feasibility of these designs. The potential benefits of adaptive designs are considerable and have been widely discussed in the clinical trials literature, but these designs appear to have been used rarely for POC trials of chronic pain treatments on the basis of published literature [116], a situation that we expect will change in the coming years. However, there can be reluctance to use adaptive design trials because of uncertainties regarding their evaluation by regulatory agencies [124], although this consideration is more relevant to confirmatory trials than to POC trials.

4.7. Sequential parallel comparison design

The sequential parallel comparison design (SPCD) [32] is one of few novel designs that has been specifically developed to address high placebo response rates. There is increasing use of the design in trials of psychotropic medications for mental health disorders. The design involves two stages, the first of which consists of an unbalanced randomization with more participants receiving placebo than active treatment. Placebo non-responders continue to the second stage, where they receive either placebo or active treatment (placebo non-responders are either re-randomized at this point, or were randomly assigned to either a placebo-placebo or a placebo-treatment sequence at the beginning of the trial). Treatment non-responders receive placebo in the second phase to maintain blinding. Responders exit the study, transition to an uncontrolled extension, or can be randomized in the second stage to test additional hypotheses, but their second stage data are not included in the final primary efficacy analysis. All data from stage 1 and data from stage 2 provided by first stage placebo non-responders are combined in the efficacy analysis (Figure 2).

Because the participants identified as placebo non-responders in stage 1 contribute the most data for the analysis, the use of the SPCD could yield a reduction in the magnitude of the placebo response. In fact, all 5 publically reported depression trials using the SPCD reported a decrease in the percentage of placebo responders between stage 1 and stage 2, although the decrease in placebo response was minimal in one of these trials [33,91,92] (Marshall RD, Leigh-Pemberton RA, Memisoglu A, Martin W, DeSomer M, Ehrich E, Fava M. Opioid modulation: a novel mechanism for the treatment of depression: results of the ALKS 5461 phase 2 study. Poster presented at the International Society for Clinical Trials Methodology (ISCTM) 2013). With the exception of that trial [92], all trials defined placebo non-responders as those whose depression score decreased by < 50% and whose depression score remained over a pre-specified cutoff at the end of stage 1. The trial with minimal decrease in the stage 2 placebo response did not include such a cutoff. These results suggest that if an SPCD is used for examining a pain treatment, consideration should be given to including a minimum cutoff for pain at the end of stage 1 as part of the definition of a placebo non-responder.

The SPCD does have some limitations that should be considered. The sample size determination and statistical analyses for trials using SPCD are somewhat complicated. The overall duration of the trial for an individual participant is longer than in a conventional RCT. Because of this, when using the SPCD, investigators may be tempted to shorten the treatment duration of each stage of the design compared to the treatment duration used for a conventional parallel group design to decrease the overall length of the trial for each participant. However, the results of simulation studies indicate that careful attention must be paid to whether the stages of an SPCD can be shorter than what would be implemented in a conventional parallel group design [8,120]. In addition, if placebo non-responders are also treatment non-responders (i.e., a more refractory group), the use of the SPCD may actually decrease power. The SPCD also assumes that placebo non-response is a consistent trait. Although few studies support the presence of consistent placebo responders [57], the fact that the placebo response decreased in the second stage of all of the published SPCD trials suggests that placebo non-response is reliable, at least within psychiatry trials with relatively short-term follow-up. Finally, to our knowledge, SPCD is the only research design that is patented and for which investigators are charged a fee to use it; however, academic investigators with non-profit or government funding may purchase a license for a reduced amount (www.rctlogic.com).

5. Conclusions

Tables 2 and 3 summarize the advantages and disadvantages of the designs we have discussed for POC trials of chronic pain treatments. Although our discussion and examples focus on analgesic drug development, much of the material is also relevant to designing POC trials for other types of treatments. It must be emphasized that many of our considerations are not based on systematic research but are generally based on methods and results of clinical trials in chronic pain and other conditions and discussions among trial design experts. Greater attention to improving designs for POC clinical trials of chronic pain treatments may increase the percentage of truly efficacious treatments that are advanced to confirmatory trials while decreasing the percentage of ineffective treatments that continue to be evaluated rather than abandoned. Improving the transition from POC to confirmation of efficacy would save time and finances, decrease the number of participants that are exposed to ineffective and sometimes unsafe treatments, and thereby decrease the resources needed to discover novel efficacious treatments for pain conditions.

Table 2.

Advantages and disadvantages of various designs.

Design	Situations where design should be considered	Advantages	Disadvantages
Dose finding [7,68,93,134]	Dosages that are expected to be safe, tolerated, and efficacious are unknown for the condition of interest	May increase the probability of finding an analgesic effect at safe and tolerated dosages in subsequent studies For adaptive dose finding designs, see adaptive designs row	For adaptive dose finding designs, see adaptive designs row
Single dose	Limited resources Treatment is known to have prompt onset of analgesic effect	Lower probability of missing data Decreased costs because of single dose of treatment and short duration of follow up	Only applicable to treatments with prompt onset of analgesic effect Single dose and short duration of observation may preclude identification of many adverse events Predictive value for long-term efficacy is not clear
Parallel group	Limitations of other designs make them unsuitable for the treatment and objective of interest	Relatively straightforward design, execution, analysis and interpretation Potentially increased generalizability	Requires larger sample sizes than several other designs
Cross-over [54,109]	Limited resources and patient numbers	Increased power, reduced sample size requirements	Potential for unforeseen treatment-by-period interaction (e.g., carryover) Longer duration and patient burden Greater impact of missing data
Adaptive designs [6,38,116,124]	Trials with a large learning component, for example, sample size estimates, dosage selection, safety	Appreciably increased efficiency Highly informative for planning subsequent studies	Careful and extensive planning is needed to preserve the validity and integrity of the trial Requires unique statistical expertise Several logistical and procedural issues need to be considered in the planning and execution of the trial
Sequential parallel comparison design (SPCD) [32]	Placebo effects are anticipated to be relatively large, possibly decreasing the assay sensitivity of the trial	May decrease the placebo effect in the second stage [33,91], and thus increase power compared to traditional parallel group design	Longer duration of follow-up than in a conventional parallel group design If placebo non-responders are refractory patients who also do not respond to treatment, SPCD could have less power than conventional parallel group designs SPCD is patented and investigators must pay to use it.

Figure 3.