Responder analyses in randomised controlled trials for chronic low back pain: an overview of currently used methods

Abstract

Purpose

To provide an overview and a critical appraisal of the use of responder analyses in published randomised controlled trials (RCTs) of interventions for chronic low back pain (LBP). The methodology used for the analyses, including the justification, as well as the implications of responder analyses on the conclusions was explored.

Methods

A convenience sample of four systematic reviews evaluating 162 RCTs of interventions for chronic LBP was used to identify individual trials. Randomised trials were screened by two reviewers and included if they performed and reported a responder analysis (i.e. the proportion of participants achieving a pre-defined level of improvement). The cutoff value for responders, the period of follow-up, and the outcome measure used were extracted. Information on how RCT authors justified the methodology of their responder analyses was also appraised.

Results

Twenty-eight articles (17 %) using 20 different definitions of responders were included in this appraisal. Justification for the definition of responders was absent in 80 % of the articles. Pain was the most frequently used domain for the definition of response (50 %), followed by back-specific function (30 %) and a combination of pain and function (20 %). A reduction in pain intensity ≥50 % was the most common threshold used to define responders (IQR 33–60 %).

Conclusions

Few RCTs of interventions for chronic LBP report responder analyses. Where responder analyses are used, the methods are inconsistent. When performing responder analyses authors are encouraged to follow the recommended guidelines, using empirically derived cutoffs, and present results alongside mean differences.

Introduction

Chronic low back pain (LBP) is a common and important cause of functional disability, work absenteeism and high medical expenses [1]. Many studies have been undertaken to search for effective therapies to reduce the burden of chronic LBP complaints and its consequences [2]. However, the effectiveness of many interventions for chronic LBP remains uncertain, as reflected in the findings of numerous randomised controlled trials (RCTs), systematic reviews and meta-analyses [3]. This uncertainty is often attributed to heterogeneous response to treatment among patients. A ‘responder analysis’ has been proposed to offer an additional, clinically meaningful way to facilitate the interpretation of trial outcomes, especially when treatment response is heterogeneous [4, 5].

In the interpretation of clinical trial results, two different aspects of clinically relevant differences must be distinguished [6]. The first aspect deals with establishing the magnitude of difference in outcome between the treatment and control groups that is large enough to define the scientific or therapeutic importance of the results. This is usually achieved through comparison of summary measures (i.e. mean differences, when continuous outcome measures are used). The second aspect deals with determining the proportion of patients who, at an individual level, achieve a change in the outcome measure which represents a clinically important change. This can be explored by analysing the number of individuals who improve beyond a set threshold—a responder analysis. If, as is often the case, continuous outcomes are used, the chosen threshold cutoff must be informed by minimally important change (MIC) for an individual to ‘respond’ to an intervention as well as the minimal detectable change of the outcome measure in question [7, 8].

Reporting of the proportion of participants in a trial who achieve these changes at specific follow-up time points is suggested to facilitate comparison of the results of different clinical trials and may help to identify unique subgroups of patients who respond to certain interventions [6, 9]. Moreover, it allows the calculation of the number needed to treat (NNT) which can further improve interpretation [4]. It is currently unknown to what extent responder analyses are used in chronic LBP trials and what methodology is used to define a responder to an intervention. The aim of the current study was to explore the methodology used to perform responder analyses. The objectives were to describe the current use of responder analysis in terms of: justification for the methodology; the outcome measure; the cutoff values and how they were derived; and the follow-up times at which the proportion of responders were calculated.

Materials and methods

The current review utilised the included studies in four systematic reviews that were commissioned by the Dutch Health Care Insurance Council. These systematic reviews were intended to provide a comprehensive overview of all relevant RCTs and determine the effectiveness of the main non-surgical treatment options for chronic LBP. Four systematic reviews were performed which evaluated physical and rehabilitation interventions (such as exercise and behavioural therapy) [10], complementary and alternative medicine [11], pharmacological therapy [12], and injection and denervation procedures [13] in patients with chronic LBP. The four reviews included all RCTs which evaluated therapeutic interventions in adult subjects (aged 18 years or older) with chronic (>12 weeks duration) non-specific LBP. All of the reviews followed similar methodology which was based on that recommended by the Cochrane Back Review Group [14]. Literature searching was performed on electronic databases such as MEDLINE and EMBASE, as well as by identifying eligible RCTs from previous systematic reviews within the Cochrane Library, published up to December 2008. The risk of bias in all eligible trials had been assessed independently by two authors using an 11-item criteria list [15], with disagreements resolved by discussion or consultation with a third author.

Data collection

Full-text versions of all included RCTs in the four systematic reviews were independently assessed for inclusion in the current study by two authors (NH, AvE). Both authors extracted study-level data from the selected articles. A third reviewer (RO) was consulted when disagreements could not be resolved. Studies were included in the current overview when a responder analysis was presented in “Results” or “Discussion” of the article. A responder analysis was defined as an analysis or presentation of the proportion of participants who achieved a pre-defined level of improvement on one of the main outcomes at a certain time point. Subjects could either have improved by the pre-defined level of improvement and ‘responded’ to therapy, or not achieved the pre-defined improvement, and thus failed to respond to therapy. For the purpose of this review, the response had to be based on either pain intensity (e.g. visual analogue scale (VAS), numerical rating scale (NRS), McGill pain questionnaire), back-specific functional status [e.g. Roland–Morris Disability Questionnaire (RMDQ), Oswestry Disability Index (ODI)], or a combination of both pain and functional improvement. Patient-reported measures of pain and function are the most commonly assessed primary outcomes in LBP trials [16].

A standardised list of items was used for data extraction, considering demographic and methodological data as well as qualitative and quantitative outcomes from the included RCTs. Data included the score for risk of bias, the type of intervention and comparison treatment, the outcome (e.g. RMDQ) and domain of outcome (pain, function, or combined), the criteria or cutoff value to define response, and the reference or source that was used for the definition of response.

Data analysis

A descriptive overview of the methods which are currently employed to perform a responder analysis in all included studies was conducted, focusing on differences between cutoffs chosen and the definition of responders, references or sources for definitions and cutoffs, and the follow-up time points used. Descriptions of the interpretation of the responder analysis, such as whether it was the primary or secondary analysis and its implications were also abstracted.

Results

From the four systematic reviews a total of 162 RCTs were screened (Fig. 1). The review of physical and rehabilitation interventions [10] included the most RCTs (n = 83), followed by the review on complementary and alternative medicine (n = 35), injection therapy and denervation (n = 27), and pharmacological therapy (n = 17). The screening led to the inclusion of 28 RCTs (17 %) which included a responder analysis of the data (Table 1).

Fig. 1.

Flowchart of randomised controlled trials in the review

Table 1.

Characteristics of included studies

Study	Review	Sample size (n)	Groups	Follow-up (weeks)
Atkinson [38]	Pharmacological	83	4	12
Barendse [27]	Injection	28	2	8
Brinkhaus [52]	Complementary	298	3	8
Chrubasik [32]	Complementary	210	3	4
Chrubasik [33]	Complementary	88	2	6
Coats [39]	Pharmacological	293	2	4
Foster [25]	Injection	31	2	3, 8
Freeman [30]	Injection	57	2	26
Frerick [31]	Complementary	319	2	3
Geurts [29]	Injection	80	2	13
Haake [19]	Complementary	1,161	3	26
Katz [40]	Pharmacological	118	2	12
Keitel [34]	Complementary	150	2	3
Leeuw [18]	Non-medical	73	2	26
Manchikanti [35]	Injection	120	2	13, 26, 52
Manchikanti [36]	Injection	75	3	13, 26, 52
Mendelson [41]	Complementary	154	2	18
Pauza [42]	Injection	56	2	26
Peloso [44]	Pharmacological	336	2	13
Ruoff [43]	Pharmacological	318	2	13
Sherman [20]	Non-medical	96	3	12
Smeets [24]	Non-medical	212	3	52
Soriano [37]	Non-medical	71	2	2
van der Roer [21]	Non-medical	102	2	52
van Kleef [26]	Injection	31	2	8
van Wijk [28]	Injection	81	2	13
Witt [22]	Complementary	2,623	2	13
Yelland [23]	Non-medical	88	4	52

Twenty-five (89 %) of the 28 RCTs satisfied 6 or more of the 11 risk of bias assessment criteria and were considered to have a low risk of bias [17]. A responder analysis was performed at more than one follow-up time point in only three RCTs (Table 1).

All included RCTs evaluated at least one intervention group and one comparison group. After classification of all 66 groups in the 28 RCTs, there were 19 placebo groups, 7 injection therapy, and 6 denervation procedure groups. There were nine pharmacological treatment groups and nine complementary and alternative medicine groups (five acupuncture and four herbal medicines). Two behavioural therapy and eight exercise therapy groups were also included in this review. The remaining six groups were control interventions such as usual care or no treatment.

Of the 28 included RCTs, responder analyses were reported using 20 different definitions of response (Table 2). A reference or source to justify the choice of cutoff or definition of response accompanied only 4 of the 20 definitions (20 %). These references are also provided in Table 2. Most responses were defined as a relative decrease of a specific outcome from baseline levels. Pain was the most frequently used domain for the definition of response (50 %), followed by back-specific function (30 %) and a combination of pain and function (20 %). A decrease of at least 50 % pain intensity (from baseline levels) using a VAS was the most common cutoff, which was used in 13 (46 %) of the included RCTs (Fig. 2). All but one study [18] performing a responder analysis for pain intensity used a cutoff equal to or >25 % reduction in pain to define responders.

Table 2.

Definition of “responders” used in randomised controlled trials (RCTs) of interventions for chronic LBP

Outcome	Definition of ‘responder’	Reference or source	RCTs
Back-specific function (Hannover)	12 % improvement	–	[19]
Back-specific function (Hannover)	20 % improvement	–	[22]
Back-specific function (Oswestry)	Two-grade improvement in one or more functional subsets	–	[25]
Back-specific function (RMDQ)	2-point reduction	[53]	[20, 24]
Back-specific function (RMDQ)	30 % reduction	[54]	[21]
Back-specific function (RMDQ)	50 % reduction	–	[20, 23]
Combined	2-point reduction in VAS and 50 % improvement in general perceived effect	–	[26, 27]
Combined	50 % reduction in median VAS-back, without a drop in daily activities or rise in analgesics; or 25 % reduction in median VAS-back, with a simultaneous rise in daily activities of at least 25 % and a drop in analgesic intake of at least 25 %	–	[28]
Combined	50 % reduction in median VAS-leg without a drop in daily physical activities score, a rise in numerical analgesics rating scale, or both	–	[29]
Combined	No neurologic deficit, improvement in LBOS of >7, improvement in SF-36 bodily pain and physical function of >1 standard deviation	–	[30]
Pain intensity	21 mm reduction on a mean of three visual analogue scales of the MPQ	[55]	[18]
Pain intensity	25 % reduction (VAS)	–	[28, 42]
Pain intensity	30 % reduction (VAS)	[47]	[31, 34, 40, 43, 44]
Pain intensity	33 % improvement on three pain-related items on the Chronic Pain Grade scale	–	[19]
Pain intensity	33 % reduction (VAS)	–	[41]
Pain intensity	50 % reduction (VAS)		[23, 25, 28, 31, 34–36, 39, 40, 42–44, 52]
Pain intensity	60 % reduction (VAS)	–	[37]
Pain intensity	67 % reduction (VAS)	–	[41]
Pain intensity	75 % reduction (VAS)	–	[38, 42]
Pain intensity	100 % reduction (VAS)	–	[32, 33, 37]

RMDQ Roland–Morris Disability Questionnaire, VAS visual analogue scale, LBOS low back outcome scale, MPQ McGill Pain Questionnaire

Fig. 2.

Cutoff values for responder analyses using pain intensity (visual analogue scale unless stated otherwise) as the outcome measure

Studies which evaluated responders according to functional improvement used a variety of back-specific function measures and either a relative [19–23] or absolute [20, 24, 25] improvement as the definition for responders. Five studies [26–30] presented a responder analysis using a combination of pain, function, and/or clinical findings to define responders to the intervention. Some of these studies employed a combination of absolute and relative improvements [26, 27], or anchor and distribution-based approaches, [30] to define responders (Table 2).

Fourteen RCTs (50 %) calculated the proportion of responders as the primary analysis for the study [19, 25–37]. Of these, ten RCTs also provided data on continuous outcomes or mean differences between the intervention groups [19, 26–31, 34–36] and four RCTs did not [25, 32, 33, 37]. Twelve RCTs (43 %) calculated the proportion of responders as a secondary analysis (after evaluating group differences) [18, 20, 22–24, 27, 38–43]. In all 12 of these, the responder analysis supported the results of the primary analysis. Two RCTs (7 %) presented a responder analysis in the discussion section of the report [21, 44]. Finally, in three RCTs (11 %) a responder analysis was used to calculate and present the NNT for the intervention and control groups [31, 42, 44].

Discussion

Only 28 out of 162 (17 %) screened RCTs included a responder analysis. Where a responder analysis was included, the cutoff used to define a response varied widely across studies and methodology used to derive the cut point was inconsistent. This is problematic because different cut points can produce different results and eliminate the ability of responder analyses to facilitate comparisons across trials [5, 45]. As currently reported, responder analyses in RCTs of interventions for chronic LBP may offer little to facilitate interpretation as they offer little additional information. Efforts to standardise the methodology of responder analyses along with improvements in reporting may facilitate demonstration of a small mean difference for some interventions coexisting with an attractive number of individual responders to an intervention [4, 5]. This is possible in cases where treatment works well for a proportion of patients and could be especially useful in LBP trials. As responder analyses often require continuous outcomes to be categorised, with consequent loss of information and reduced power for statistical tests [46], they are only recommended as secondary analyses to improve the interpretability of the main analysis [9].

The most commonly reported responder analysis was the proportion of participants with a reduction of 50 % pain intensity from baseline. In a recent consensus statement to improve the interpretation of clinical pain trials, reductions in chronic pain intensity in individuals of at least 10–20 % were suggested to reflect MICs [6]. Reductions of >30 % appear to reflect at least moderate clinically important changes, and consensus on this threshold has also been informed by a panel considering the effects of measurement error from distribution studies. It is recommended that the percentage of patients with this, or greater, changes in pain relief are reported in clinical trials of chronic pain treatments [6]. In addition, the authors could report reductions of >50 % which appear to reflect more substantial improvements [6]. Other research has suggested that for a range of commonly used outcome measures for back pain, a 30 % improvement from baseline may be considered clinically meaningful when comparing before and after measures for individual patients [8, 47]. It should be noted that these recommendations have only recently been published [6] and, while not expected to be reflected in the included studies in this review (as most were published prior to 2008), can be used as a reference point for future RCTs employing a responder analysis.

The presentation of outcomes from a clinical trial can have a considerable effect on the understanding of results and, similarly, there are potentially important variations in the methods used to describe the results of a responder analysis and the interpretation of the NNT [48, 49]. However, very few authors who performed a responder analysis used the findings to provide an estimate of the NNT, which may have limited the interpretation of these findings for patients and clinicians [4, 5]. Froud et al. [5] performed a responder analysis on a large trial (UK BEAM, n = 1,334) of physical treatments for back pain which originally showed small to moderate benefits of spinal manipulation over ‘best care’ in general practice. Using a reduction of five points on the Roland–Morris Disability Questionnaire at 3 months as the cutoff, 62 (24 %) individuals in the best care group and 125 (44 %) in the manipulation group were defined as responders. Calculating the absolute risk reduction and then inverting this gives the NNT [50]. The calculated NNT was 5.2 (95 % CI 3.7–8.8), with corresponding between-group mean difference of 1.4 (95 % CI 0.6–2.1). This means that every five to six patients referred to spinal manipulation instead of ‘best care’, on average, will yield one additional responder to treatment at 3 months. In contrast to the small mean difference originally reported, the NNT was small and could be attractive to clinicians, patients, and policy makers [5].

While these proposed values are not strict definitions of a clinically important change, they do offer a common starting point for future research. Ideally, a cutoff to define ‘responders’ should be backed by empirically derived evidence, decided upon a priori, and set out in the trial analysis plan to avoid the temptation of selecting the best cutoff post hoc. Alternatively, presenting the proportion of responders over a range of cutoff points may allow easier comparisons between treatment groups or applicability to patient care [51]. One note of caution is that responder analyses open up the possibility of reporting outcomes using relative reporting methods (e.g. as percentage of participants who improved) which, in the case of small absolute differences, can make interventions seem more effective. We recommend that if an author chooses to report outcome in relative terms, these are also accompanied by absolute reporting methods.

One limitation of this study is that only four recent systematic reviews were used as the source of potentially eligible RCTs. However, this does cover the majority of conservative treatment, pharmacological therapy, and injection and denervation trials published before 2010 and therefore provide a good overview of all interventions for chronic LBP. Moreover, the four reviews were all performed according to recommended methods of the Cochrane Collaboration and had comparable search strategies and inclusion criteria.

Using summary measures to report outcomes of RCTs for chronic LBP may not give a full picture of the effect of a treatment, especially as individuals respond in different ways. As the methods used to analyse and present the results of RCTs have important implications on how they are interpreted, responder analyses should ideally be performed using empirically derived cutoffs and presented alongside mean differences. In this way, these analyses can aid in interpretation of outcomes as well as the implementation of findings from RCTs.