Benefit-risk assessment and reporting in clinical trials of chronic pain treatments: IMMPACT Recommendations

1. Introduction

Approximately 20% of adults report having chronic pain [31; 34; 84; 115]. Unfortunately, response to treatments for chronic pain is often modest and can result in significant side effects including adverse events (AEs) [2; 18; 25; 38; 45; 65; 92; 110]. These realities highlight the need for more effective chronic pain interventions. One challenge in the development of novel treatments is balancing their benefits and risks. An example of this predicament involves the ongoing opioid crisis in the United States, which requires balancing the analgesic benefits of opioid medications with their significant risks, including persisting side effects, dependence potential, and risk of overdose [61; 70; 95; 97; 107; 116; 153]. Prescription opioid analgesics provide a timely example of the need to relieve pain while also protecting patients from the risks of pain interventions.

Benefit and risk data are not reported consistently in many randomized clinical trials (RCTs), including chronic pain trials, making it difficult to combine and compare results across studies [13; 37; 60; 63; 73; 75; 78; 79; 87; 111; 138; 139; 159]. Moreover, the primary outcomes in clinical trials often focus on treatment benefits (efficacy) rather than on risks such as AEs [27; 88]. This is often because studies are designed prospectively to have sufficient power to detect efficacy rather than identify risk [35]. In addition, benefits and risks of treatment are most commonly examined as separate outcomes in clinical trials, which cannot address whether there might be a relationship between the two [42]. For example, patients who benefit from an intervention could also be the same patients who are more (or less) likely to experience harms (i.e., correlated benefit and risk outcomes within the same patients).

Multiple frameworks and methods have been developed to account for benefit and risk outcomes in relation to each other in a combined metric rather than as separate outcomes [13; 21; 29; 30; 40; 54; 63; 87; 111; 112; 120; 122–124; 132; 146; 154]. These methods are diverse and can include qualitative and/or quantitative steps for combining benefits and risks for each treatment condition (group level assessment) [40; 48; 111]. Benefit-risk assessments can also be evaluated at the level of an individual patient and then compared across treatment conditions (individual level assessment) [12; 42; 53; 88; 93]. An additional advantage of benefit-risk assessments is that they can be tailored to best address the demands of a specific trial or other considerations such as patient subgroup differences (e.g., age, multimorbidity, type, and intensity of pain). However, the applicability of these benefit-risk composite measures across chronic pain clinical trials has not been adequately evaluated.

The present article provides an overview of the steps associated with benefit-risk assessments applied to pharmacological and non-pharmacological RCTs across a range of chronic pain conditions. Our aims are based on an Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) consensus meeting and are informed by a review of the benefit-risk assessment tools that have been used in published chronic pain trials and/or highlighted by key stakeholders (i.e., U.S. Food and Drug Administration, the European Medicines Agency, Cochrane, and Outcome Measures in Rheumatology [OMERACT]). Using this information combined with the collective expert opinion of the meeting participants, the present article provides considerations for benefit-risk assessment and reporting in RCTs of chronic pain.

2. Methods

Recommendations presented in this article were informed by a 2011 IMMPACT meeting organized by the Analgesic, Anesthetic, and Addiction Clinical Trial Translations, Innovations, Opportunities, and Networks (ACTTION) public-private partnership with the FDA. The meeting addressed approaches for the assessment and interpretation of benefit-risk in chronic pain clinical trials and other related topics [137] (http://www.immpact.org/meetings/Immpact14/participants14.html). In addition, a review of published clinical trials of chronic pain treatments (pharmacological or non-pharmacological) was completed. A summary of the literature review findings are found in the Supplementary Information. Lastly, an internet search of publicly available documents was completed to identify publications and guidance related to benefit-risk assessments specific to chronic pain treatments. Professional organizations that were searched included the FDA, EMA, National Academies of Science, Engineering, and Medicine (NASEM), Cochrane, and Outcome Measures in Rheumatology (OMERACT; an independent initiative of international stakeholders interested in outcome measurement). The documents included for review comprised reports, publications, and white papers. Presentations, website content, or other informal methods of communication were excluded. Iterative revisions to preliminary drafts of this article were made until co-author consensus on its content was achieved.

3.2. Recommendations for Benefit-Risk Assessment from Regulatory Agencies and Professional Organizations

3.2.1. Cochrane.

The Cochrane Handbook addresses the importance of reporting the desirable and undesirable health outcomes of clinical trials (listed in order of importance) in the ‘Summary of findings’ tables included in each Cochrane Review [118]. In addition, the Handbook provides strategies for assessing benefits and AEs in the same review. For example, owing to differences in coding and categorization of AEs between studies, review authors are instructed to be alert to situations in which the coding of AEs splits data unnecessarily (e.g., pain in leg or arm), which may dilute the signal of a more global effect (e.g., all patients affected by pain). Likewise, authors are warned that combining AEs into a general outcome (e.g., total number of AEs) can only give a broad impression of effects and obscure important differences between the interventions. Lastly, Cochrane authors are instructed to include serious AEs (SAEs) in their reporting and note when safety data have not been adequately reported in the literature.

3.2.2. European Medicines Agency (EMA)

The EMA began a benefit-risk methodology project in 2009 [39; 40] (Supplementary Information). The final report was released in 2012 and recommended the use of the Problem formulation, Objectives, Alternatives, Consequences, Trade-offs, Uncertainties, Risk attitude, and Linked decisions (PrOACT-URL) qualitative framework for evaluating benefit-risk, as well as the inclusion of an ‘effects table’ for conveying benefit-risk information. The EMA also recommended that this qualitative framework be supplemented with a multi-criteria decision analysis (MCDA) quantitative approach in more complex situations [39; 40; 87; 163]. In addition, the EMA provided criteria for evaluating benefit-risk assessment tools and determining their contribution to various types of research [111; 120], including (1) logical soundness, (2) comprehensiveness (e.g., ability to handle uncertainty), (3) acceptability of results (e.g., ability to identify inconsistencies in the data and in people’s judgments, understandable and interpretable output from the analysis), (4) practicality (e.g., analysis is time efficient and can be taught to others easily), and (5) “generativeness” (e.g., the benefit-risk approach provides a clear audit trail and the results can be easily understood).

3.2.4. National Academies of Science, Engineering, and Medicine (NASEM)

Eight NASEM reports or workshop summaries that addressed benefit-risk were located (Supplementary Information). In 2014, the FDA and the Institute of Medicine (now NASEM) convened two public workshops on Characterizing and Communicating Uncertainty in the Assessment of Benefits and Risks of Pharmaceutical Products [77]. The workshops were designed to address uncertainty in pharmaceutical regulatory decision-making related to variability in human biology, drug chemistry, and clinical trial research. A focus of the summary included existing tools and approaches for communicating scientific uncertainties to a range of stakeholders invested in the results of pharmaceutical benefit-risk assessments (e.g., FDA; researchers in academia, government, and regulated industry; policymakers; patient groups; the public).

3.2.5. Outcome Measures in Rheumatology (OMERACT)

OMERACT is an international initiative aimed at improving outcome measurement across rheumatologic conditions, including efforts to simplify the simultaneous assessment of benefits and harms at the individual patient level (Table 1) [5; 11; 12; 134]. The OMERACT method, referred to as a 3X3 methodology, analyzes the benefits and harms simultaneously at the individual patient level (rather than at the group treatment level). This approach can account for the possibility that patients benefiting from the intervention could also be the same patients who are more (or less) likely to experience harms (i.e., correlated benefit and risk outcomes within the same patients). The OMERACT method relies on a contingency table that allows for two or three levels of benefit across two or three levels of harm. The specific benefit and harm levels are uniquely defined depending on the chronic pain condition(s) and treatment(s) being evaluated, and therefore can vary. However, the interpretation of the contingency table is consistent across studies, with an “unqualified success” corresponding to a patient with a good response in the benefit category without any AEs in the harm category. An “unmitigated failure” would involve a patient with no response in the benefit category but at least one AE in the harm category. As represented in Figure 1, the OMERACT method was recently applied to data collected from two separate rheumatoid arthritis clinical trials (The Treatment of Early Aggressive Rheumatoid Arthritis, or TEAR trial; the Rheumatoid Arthritis Comparison of Active Therapies, or RACAT trial) [12]. The primary findings from the trials revealed no significant safety concerns of any treatment and significant beneficial effects of treatment relative to comparators in the TEAR trial, but not in the RACAT trial. However, the secondary analysis of benefit-risk in these trials revealed a more complicated pattern of results not identified in the primary analyses. In the secondary analysis, benefit was defined as good, moderate, or no response depending on the patient’s disease activity, and harms were categorized into three types of AE outcomes (no AEs, non-SAEs, and SAEs). Results of the TEAR trial analysis revealed that treatment response and AE rates were weakly associated with no significant difference between the treatment arms). In the RACAT trial, treatment response and AEs were negatively associated such that the frequency of AEs and SAEs increased as beneficial responses decreased. These findings demonstrate that a combined benefit-risk assessment at the individual level can reveal differences in clinical response that are not obvious when benefit and risk are assessed separately. This method is limited because it classifies benefits and AEs categorically, which could oversimplify these outcomes and the final results of the analysis. For example, the AE category that does not include SAEs is very broad and could include a wide range of potential outcomes. Due to these and other limitations, the OMERACT benefit-risk analysis should be considered a complementary method and should not fully replace current analysis and reporting strategies in clinical trials of chronic pain treatments.

Table 1.

Selected Level Benefit-Risk Assessment Frameworks and Methods for Chronic Pain Clinical Trials

NAME	DESCRIPTION	EXAMPE
GROUP LEVEL ASSESSMENTS
EMA PrOACT-URL	The EMA PrOACT-URL is an eight-step qualitative analysis that provides a generic problem structure for identifying favorable and unfavorable effects, as well as the uncertainty of each, that has been adopted by the EMA [39; 40]. The framework is based on the field of decision analysis and was developed through the public-private partnership, IMI PROTECT.	Rheumatoid Arthritis [157]
FDA BRF	The FDA BRF five-step qualitative framework provides a simple and user-friendly snapshot of benefit-risk assessment that is intended to be broadly applicable [48–50; 87]. It should be updated as new information is received and can be used throughout the regulatory process. The five steps and questions asked include: (1) Analysis of the condition/”what is the problem?”; (2) Unmet Medical Need/”what other potential interventions exist?”; (3) Benefit; (4) Risk/”what am I worried about?”; (5) Risk Management/”what can I do to mitigate/monitor those concerns?”	Chronic pain (general) [114]
PhRMA BRAT	PhRMA BRAT is a six-step qualitative analysis developed to facilitate benefit-risk assessment by pharmaceutical companies and regulators. The method results in a summary table using the following: decision context, outcomes, data sources, framework, outcome importance, and display and interpret key metrics [28; 100]. Benefits and risks are not integrated in this framework, but are assessed separately to reduce complexity.	Migraine [100]
MCDA	Multicriteria decision analysis (MCDA) is a quantitative analysis method based on decision theory that combines evaluations of multiple potential benefits and risks (based on pre-specified criteria) into a weighted benefit-risk assessment [33; 111]. The scoring and weighting process allows the effects of different interventions to be placed on a common scale that allows for comparisons across interventions.	Chronic cancer pain [135]
INHB	Incremental Net Health Benefit (INHB) is a quantitative analysis method that is based on health-outcomes modeling that incorporates a life-expectancy measure adjusted for quality of life (i.e., quality-adjusted life year; QALY) [30; 57; 58]. The QALY represents an adjustment to length of life for the quality of life experienced and can be easily adapted to benefit-risk analysis by separating outcomes into expected health improvements with positive QALYs (benefits) and adverse health impacts with negative QALYs (risks). Benefit-risk differentials can then be expressed as either ratios or differences although the latter is preferred because the difference can be interpreted as healthy days (or months or years) of life gained (lost) since the units of measurement are the same. While standalone use of the INHB in pain populations is rare, clinical benefit-risk (net QALY impact) is the denominator in a range of cost-utility studies that have evaluated pain interventions.	Arthritis [103]
INDIVIDUAL LEVEL ASSESSMENTS
DOOR	The DOOR method is a quantitative analysis that provides a probability of a participant in the active group having a more desirable outcome than a participant in the control group. These probabilities are determined by ranking trial participants based on the desirability of their total experience of benefits and risks, and then the resulting rankings are compared between intervention arms [42; 43]. A key benefit of DOOR is that its calculation and interpretation are straightforward relative to other benefit-risk assessment methods.	Not yet examined in a chronic pain population
ETC Measure	The ETC Measure is a quantitative analysis that Integrates responder criteria for pain reduction (>20%, >30%, or >50% reduction in pain intensity from baseline) and adverse events (no AEs, no or mild AEs, and no or mild drug-related AEs) [88]. The approach assigns a score for both efficacy and tolerability for each day the patient is in the study, thus accounting for incidence, severity, and duration of AEs in one metric. The combination of scores across efficacy and tolerability over time forms a continuous ETC score that generally provides greater statistical power than dichotomous outcomes. The ETC score ranges from 0 to 1 with a clinically intuitive interpretation. For example, a score of 0.45 means the patient’s response was ‘good’ with respect to both efficacy and tolerability 45% of the time.	Chronic low back pain [88]
OARSI	OARSI has provided patient-focused, evidence-based, expert consensus guidelines for the management of knee OA that include the recommendation to perform a quantitative analysis using a composite benefit and risk score [105]. The score is voted on across a panel of expert physicians and calculated as the product of the benefit score (on a scale of 1–10) and the transposed risk score (where 1 = highest and 10 = safety) yielding a range of 1 (worst) to 100 (best). The group’s mean risk and benefit scores [along with 95% confidence intervals (CIs)] for each treatment are then plotted separately as bar graphs.	Knee osteoarthritis [105]
OMERACT	The OMERACT is a quantitative method that relies on a contingency table that allows for two or three levels of benefit across two or three levels of harm (Figure 1) [12]. The specific benefit and harm levels are uniquely defined depending on the chronic pain condition(s) and treatment(s). The interpretation of contingency table is consistent across studies, with an ‘unqualified success’ being a patient with a good response without any AEs and an ‘unmitigated failure’ being a patient having no benefit, but experiencing at least one AE.	Rheumatoid Arthritis [12]

Note: Additional benefit-risk approaches that might be considered, and not highlighted in the present table for brevity, include multiple-criteria decision analysis (MCDA), discrete-event simulation, probabilistic simulation, and Bayesian belief networks [109; 110]. AE (Adverse Event); BRF (Benefit-Risk Framework); DOOR (Desirability of Outcome Ranking Evaluation); EMA (European Medicines Agency); ETC (Efficacy-Tolerability Composite); FDA (United States Food and Drug Administration); IMI PROTECT (Innovative Medicines Initiative Pharmacoepidemiological Research on Outcomes of Therapeutics); INHB (Incremental Net Health Benefit); (MCDA (Multi-Criterion Decision Analysis); OMERACT (Outcome Measures in Rheumatology); OARSI (Osteoarthritis Research Society International); PhRMA BRAT (Pharmaceutical Research and Manufacturers of America Benefit-Risk Action Team); PrOACT-URL (Problem formulation, Objectives, Alternatives, Consequences, Trade-offs, Uncertainties, Risk attitude, and Linked decisions).

Figure 1.

Figure 1 is reproduced from a previous publication [12] and copyright permissions were approved by Figure 1 is reproduced from a previous publication and copyright permissions were approved by Wiley.

The figure illustrates the OMERACT 3×3 Combined Table of Benefits and Risks assessment method [12]. The results represented in the figure are from two randomized controlled trials including the Treatment of Early Aggressive Rheumatoid Arthritis (TEAR) trial (Top Panel) and the Rheumatoid Arthritis Comparison of Active Therapies (RACAT) trial (Bottom Panel). In the panels on the left, results of treatment groups are pooled and categorized according to the combined occurrence of benefit and harm, each in 3 categories. Results are expressed as a percentage of the total group, corrected for rounding. White lines delineate the cutoffs for the 2×2 categorization in the right-hand panels. The panels on the right show the results (percent per treatment group) with the combined occurrence of benefit and harm, each in 2 categories: for benefit, the European League Against Rheumatism (EULAR) good response (yes/no); for harm, the occurrence of any serious adverse event (SAE; yes/no). The length of the diagonal bar in each cell is proportional to the percentage of patients in that cell. The orange/blue (bottom left to top right) diagonal shows the balance between worst and best outcomes. The light grey/purple (top left to bottom right) diagonal shows the balance between 2 types of tradeoff: no benefit + no harm, and benefit + harm. nsAE (non-SAE); MTX (methotrexate); ETN (etanercept); triple (MTX, sulfasalazine, hydroxychloroquine).

3.2.3. U.S. Food and Drug Administration (FDA)

The FDA has released a series of documents focused on benefit-risk assessment, including 10 guidance documents (Supplementary Information). Five of these documents pertain to medical devices, 4 address pharmacological treatments, and one spans multiple FDA centers and addresses benefit-risk reporting on the internet and social media. The FDA currently recommends a structured qualitative benefit-risk framework (BRF) supplemented with quantitative analyses to analyze the benefits and risks associated with medical products [48–50; 87]. The FDA framework addresses four dimensions: (1) the analysis of the condition, (2) current treatment options, (3) benefits, and (4) risk management. The FDA has conducted several public meetings on the topic of benefit-risk assessment in recent years, and draft guidance was scheduled to be published in 2020; however, no updates were located to prepare this article [49; 51; 96]. This guidance is expected to use a case study approach for articulating FDA’s decision-making context for benefit-risk analysis in order to provide stakeholders with a clearer understanding of how considerations of a medication’s benefits versus risks factor into FDA’s regulatory decisions throughout the drug development life-cycle, including pre- and post-market phases. Importantly, this guidance will discuss how relevant patient experience data and related information may be used to inform benefit-risk assessment.

4. Recommendations for Benefit-Risk Assessment and Reporting in Chronic Pain Clinical Trials

4.1. Terminology

Terminology associated with benefit-risk assessment, including operational definitions of key terms, are not standardized and often vary [39; 77]. Opinions vary as to whether the terms “harm” or “tolerability” might be more appropriate than the term “risk” [77; 88]. For this article, we define benefits as the intended favorable effects for the target population associated with an intervention and risks as the unintended clinical and health outcomes or detrimental effects that can be attributed to the intervention [36]. The term risk in the present review includes unwanted side effects, some of which will have an adverse effect on patient functioning, but also includes major safety risks such as myocardial infarction or death. We recommend researchers distinguish between risks attributed to the treatment under study (e.g., chronic nausea or vomiting) relative to those that are most likely not related to the treatment per se (e.g., an injury sustained during a motor vehicle accident). We define benefit-risk assessment as a structured method (qualitative or quantitative) for combining separate benefit and risk outcomes into a composite metric that allows for a clear comparison of benefits and risks in relation to each other at the level of the group or for individual patients. According to our definition, global ratings of patient functioning (e.g., patient global impression of change; PGIC) that do not specifically include harms would not be considered benefit-risk assessment tools. The ratio of the number needed to treat (NNT) and number needed to harm (NNH) could be considered a measure of benefit-risk. We do not consider this approach further because the widely varying definitions used for NNH preclude meaningful treatment comparisons [137].

4.2. Steps Associated with Benefit-Risk Assessment

There are five steps underlying decision-making related to benefits and risks that are common across a range of disciplines(Table 2) [63; 74; 99; 111; 123; 154].

Table 2 –

Steps to consider in benefit and risk assessments in clinical trials of chronic pain treatments

1. Specify

Specify the chronic pain condition(s) under study and the currently available treatments for the condition(s). Unmet clinical needs associated with the condition and contextual information such as common comorbidities associated with the condition should also be addressed.

2. Identify

Identify the key outcomes that will be utilized to assess the benefits (e.g., reductions in pain intensity or severity) and risks (adverse events, reduced quality of life). Patient preference on meaningful benefit and risk outcomes should be incorporated at this level and patient-reported outcomes should be used to gather data.

3. Evaluate

Collect and combine data related to the benefits and risks of an intervention(s) in a way that allows for the ranking or weighting of data. In general, two approaches to benefit-risk analyses can be performed: compare and combine at the level of the intervention or combine and compare at the level of the individual patient.

4. Interpret

The interpretation of data should incorporate value judgments, or trade-offs between the relative importance of benefits and risks in a particular situation, which can vary depending on the type of stakeholder (patient, clinician, regulatory agency). This step should also address the uncertainty associated with the analysis given that benefit-risk assessments are dynamic and evolve as information changes over time.

5. Communicate

Communicate the results of the analysis, including sharing the processes and rationale leading to the final conclusions. Messaging of the findings might need to be tailored depending on the audience and information should be summarized in succinct, transparent, and user-friendly ways (e.g., graphical representations).

4.1.1. Specify.

The first sequential step involves providing a description of the chronic pain condition(s) examined, current treatments for the condition(s), and any other related contextual information specific to the pain condition that could influence relevant risks, including epidemiological information related to patient demographics or comorbid health conditions (e.g., tobacco use, obesity, concurrent medication use). In addition, the collection of patient preference data at the start of the study to determine patient attitudes regarding benefit-risk has been suggested as an important feature of this step [87].

4.1.2. Identify (Outcomes and Assessments).

The second step requires identification of the key outcomes and measures that will be used when combining benefits and risks. As presented in Table 3 and in the Supplementary Information, benefits and risks can be assessed using a variety of outcome measures with the most common being reductions in pain intensity (benefits) and AEs (risks). More nuanced outcomes including health-related quality of life, sleep, physical and cognitive functioning, mental health, type/severity/duration of AEs, and abuse liability might also be of interest [147]. Simply analyzing the frequency of AEs or SAEs or combining different types AEs into one heterogenous outcome can fail to detect important group differences in harms that are revealed when severity and duration of AEs are incorporated into analyses [88; 118]. As discussed in detail elsewhere [127], it is essential to consider the use of standardized language when referring to benefits and risks in order to facilitate the comparison and evaluation of study outcomes (e.g., Medical Dictionary for Regulatory Activities (MedDRA), the Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT) terminologies).

Table 3 –

Benefit-Risk Terminology

	Description or Definition
Benefit	The intended positive or favorable effects of an intervention for the target population (often referred to as “benefits” or “clinical benefits”) that are associated with an intervention [36]. Examples include reduction in pain intensity, increase in number of pain free days, function, and quality of life.
Risk	The unintended negative clinical and health outcomes or detrimental effects that can be attributed to the intervention. The use of the term risk in the present article includes side effects, some of which will have an adverse effect on patient functioning, but also includes safety risks, SAEs such as myocardial infarction, or death. The intensity and duration of all treatment-emergent AEs should be collected (total, severe, and serious), as well as the use of active capture, which includes interviews or questionnaires [36; 73].
Benefit-Risk Assessment	A structured method (qualitative or quantitative) for combining separate benefit and risk outcomes into a composite metric that allows for a clear comparison of benefits and risks in relation to each other at the level of the group or for individual patients.
Clinical Utility	The ability of a clinical test result(s) to inform a decision that positively changes the outcome of a patient [144]
Qualitative Framework	Qualitative or descriptive frameworks provide stepwise instructions for evaluating and balancing benefit and risk, including their frequency and duration, and fully describes how that information weighs into decision making [123]. Examples include: The Problem formulation, Objectives, Alternatives, Consequences, Trade-offs, Uncertainties, Risk attitude, and Linked decisions framework and the United States Food and Drug Administration Benefit-Risk Framework.
Quantitative Framework	Quantitative frameworks provide explicit methods for combining and weighing risks and benefits. A quantitative approach may help to improve the transparency of a review, relative to a qualitative approach, by being explicit about how benefits and harms are estimated and compared (Boyd et al., 2012). While quantitative approaches can be used to examine benefit-risk at the level of the group, there are most commonly used for analyses that begin at the level of the individual patient (Table 1). Examples include multiple-criteria decision analysis (MCDA), discrete-event simulation, probabilistic simulation, and Bayesian belief networks [111; 112].
Patient Preferences	Patient preferences represent patient’s attitudes toward a set of alternatives necessary for decision-making [77]. Collecting data related to a patient’s perspective or preference should be taken into account at all stages of research including planning of the clinical trial design and the identification of patient-relevant outcomes [13; 81; 82; 151].
Standardization and Transparency	A systematic and transparent evaluation process that allows for consistency of reporting, replication, and pooling of data across studies [73].

Note: AE (adverse event); SAE (serious adverse event).

We recommend this step incorporate the needs and preferences of patients into study planning in two ways. First, as noted, the choice of benefit-risk outcomes should be based, at least in part, on feedback from patients, surrogates, or patient advocacy groups, and not simply chosen based on clinician, investigator, or regulatory considerations [113; 130]. While validated measures of patient preferences are currently lacking in the field, we recommend that at least some measures of benefit and risk include patient-reported outcomes (PRO), or data reported by patients without interpretation by someone else [5; 8; 9; 47]. We recommend that such data be collected through active capture using structured interviews or questionnaires, as well as passive capture/general inquiries, which can identify unanticipated outcomes [36]. A detailed discussion and framework for incorporating patient preference data in benefit-risk assessment can be found elsewhere [71].

Medical conditions and associated symptoms and interventions can also influence patient preferences or perceptions of benefit-risk trade-offs [5; 6; 23; 62; 67]. One example includes older patients with knee osteoarthritis who are sometimes willing to forgo greater treatment effectiveness for a lower risk of AEs [52], whereas there is a large body of work demonstrating that individuals with a range of complex, chronic health conditions, including Crohn’s disease, irritable bowel syndrome, low back pain, and osteoarthritis, are willing to accept high levels of risk in return for disease-modifying benefits of treatment [68; 85; 86; 126; 131; 148]. These observations highlight the potential for subgroup differences among chronic pain populations that can influence the weighting of benefits and risks (e.g., age, drug use and dependence history, multimorbidity) [115; 148]. Lastly, this step should include prospective registration of the trial characteristics, including study objectives and hypotheses [44; 136] and benefit-risk assessments that are planned in a public database(s) such as ClinicalTrials.gov.

4.1.3. Evaluate (Endpoints and Analyses).

The third step involves collecting data related to the benefits and risks of an intervention(s) and combining those data in a way that allows for the ranking or weighting of data in a combined metric. A variety of benefit-risk assessments apply to clinical trials of chronic pain treatments (Table 1) [13; 21; 40; 59; 63; 111; 112; 154]. Two approaches to benefit-risk data include those that combine benefit and risk data at the group level and those that first combine such data at the individual level and then analyze differences on the group level [41–43]. The most common approach involves summarizing benefit-risk data at the level of the group or intervention (placebo versus active treatment) and then combining these data in a way that allows comparisons across treatments. This approach has the advantage that it is easy to analyze outcomes and quickly communicate the findings and examples include the FDA’s BRF and the EMA’s PrOACT-URL (Section 4.2). However, this approach does not account for associations between benefits and risks that might occur at the level of the individual patient. For example, a patient who is experiencing the greatest reduction in pain from an intervention could also be more likely to experience SAEs from the same intervention [11; 12].

An alternative approach involves assessing benefit-risk trade-off within each participant [42; 60; 125]. Examples of benefit-risk assessments that focus on the individual rather than group level analysis are represented in Table 1 and include the Desirability of Outcome Ranking (DOOR), Efficacy-Tolerability Composite (ETC), OMERACT, and OARSI methods. In the DOOR method trial participants are first ranked based on the desirability of their total experience of benefits and risks (across multiple dimensions/outcomes), with a focus on the outcomes that are most important from the patient’s perspective [41–43]. The resulting rankings are then compared between intervention arms (Table 4).

Table 4 –

Analysis of Patients by Treatment from Evans and Follmann (2016)

		Treatment A Efficacy			Treatment B Efficacy			Treatment C Efficacy
		+	−		+	−		+	−
Toxicity	+	10	10	+	50	0	+	0	50
	−	40	40	−	0	50	−	50	0

Note: Table is reproduced from a previous publication and copyright permissions were approved by Taylor & Francis [42]. The table represents four patient outcomes as a function of efficacy and toxicity. In all groups, 50% of the patients experience beneficial effects of treatment (efficacy). The interpretation of these outcomes is different when the risks of treatment (toxicity) are combined in the analysis. In treatment A, efficacy and toxicity were uncorrelated resulting in 40 patients that had efficacy without toxicity. In treatment B, efficacy and toxicity were positively correlated resulting in 0 patients that had efficacy without toxicity. In treatment C, efficacy and toxicity were negatively correlated resulting in 50 patients that had efficacy without toxicity..

A last point to consider is that under ideal circumstances, benefit-risk analyses should be compared across different subpopulations that represent different demographic factors and comorbidities [11; 12; 42; 68]. There could be important subgroup differences that can affect the findings from a benefit-risk assessment. For instance, the risks of some pharmacologic treatments can be significantly greater in patients with impaired renal function; thus, the benefit-risk relationship may be quite different in this subgroup of patients relative to the overall study population.

4.1.4. Interpret.

The fourth step incorporates the perspectives of a range of stakeholders (patients, patient advocacy groups, healthcare providers, payers, pharmaceutical and device companies, regulatory agencies) seeking improved treatments for chronic pain, each of whom have a unique perspective on the benefits and risk trade-offs [21; 48; 49; 87; 94; 99; 149]. These various viewpoints add a necessary complexity to benefit-risk assessment [11; 82; 87; 94; 123]. For this reason, we recommend that the interpretation of benefit-risk analyses be as transparent as possible with a clear history of the evaluation process that represents each step taken, including the various stakeholders involved in interpreting the evidence [16; 111; 120]. An additional consideration is the need to account for uncertainty when interpreting benefit-risk findings, including statistical uncertainty, especially for outcomes with low incidence rates such as SAEs. Such uncertainty can also be augmented by accounting for missing data associated with patients who stop their treatment or withdraw early from trials for reasons such as perceived lack of efficacy and adverse side effects [20]. Statistical approaches for addressing intercurrent events and sources of missing data are evolving and are highlighted by the International Council for Harmonisation (ICH) guidance (E9/R1) [22; 76].

4.1.5. Communicate.

The final step includes communicating and reporting the results of the analysis, including sharing the processes and rationale leading to the final conclusions [99]. This step requires that the presentation of the benefit-risk findings can be understood by the target audience (e.g., an individual patient, clinicians, researchers, the public). Basic principles of effective communication apply here, including: 1) providing the information needed for effective decision making which requires an understanding of the patient’s perspective, 2) allowing access to information (e.g., graphical representations), and 3) ensuring that users can comprehend the information (e.g., health literacy) [46]. Composite outcomes such as benefit-risk assessments can be challenging to interpret given that a significant result associated with a composite outcome might not indicate a significantly more beneficial treatment depending how the composite was created [59]. Thus, information should be summarized in succinct, transparent, and user-friendly ways, including graphical representations to the extent possible rather than data heavy text or tables [41; 161].

4.2. Selected Benefit-Risk Assessment Frameworks and Methods

Table 1 describes nine benefit-risk assessment frameworks and methods that are well-suited for clinical trials of chronic pain treatments. The frameworks and methods identified in the table can be complementary and used simultaneously and include tools that combine benefit-risk at the group level (EMA PrOACT-URL, FDA BRF, Incremental Net Health Benefit/INHB and Pharmaceutical Research and Manufacturers of America Benefit-Risk Action Team/PhRMA BRAT), as well as methods that combine benefit-risk at the level of the patient (DOOR, Efficacy-Tolerability Composite/ETC, Measure, Osteoarthritis Research Society International/OARSI Knee Osteoarthritis Model, and the OMERACT method). Few studies have evaluated the various benefit-risk methods described here in clinical trials of chronic pain treatments. For some of these methods, it is possible to use existing clinical trial datasets to evaluate benefits and risks in a combined metric [12; 88].

5. Conclusions

We recommend that benefit-risk assessments be used in chronic pain RCTs to combine benefits and risks at the treatment group level (e.g., FDA BRF or PhRMA BRAT) [28; 48; 49; 100] and at the level of the individual patient (e.g., OMERACT, DOOR) [5; 43] (Table 1). The recommendation to include both types of evaluations is based on the observation that individual differences in clinical response can be obscured when combined at the group level. In many circumstances, it is valuable to include both levels of analysis (group and individual level). It should be emphasized that there is not a “one-size-fits-all” benefit-risk assessment tool for all chronic pain RCTs and that a combination of methods, as represented in Table 1, may be needed depending on the unique circumstances associated with the treatment, chronic pain condition, and clinical trial. Relatedly, given the diversity of benefit-risk assessment tools that can be utilized across clinical trials, researchers should be as transparent as possible when reporting how benefits and risks have been defined, measured, and combined to facilitate the application of study findings to patient care and decision-making.

These recommendations can serve as a starting point for incorporating benefit-risk assessment tools into future chronic pain clinical trials. One important component of a research agenda is evaluating and comparing the properties (e.g., reliability, validity, assay sensitivity) of currently available benefit-risk frameworks and methods to determine if there are approaches that are more informative [12; 88]. There is a need to integrate, to the greatest extent possible, benefit-risk assessment in clinical trials with other types of relevant data such those derived from preclinical and epidemiological studies [15; 121]. This approach could include using health outcomes modeling as a framework, post-approval, epidemiological data regarding the benefits and harms of a particular chronic pain treatment could be combined with individual level data to update earlier benefit-risk assessments, and further guide patient and clinician shared decision making as well as continued drug development and safety monitoring [57]. The systematic assessment of benefit-risk in clinical trials can enhance the clinical meaningfulness of RCT results. We are optimistic that benefit-risk frameworks and methods will be more widely incorporated in future clinical trials of chronic pain treatments.