Gabapentin Versus Tricyclic Antidepressants for Diabetic Neuropathy and Post-Herpetic Neuralgia: Discrepancies Between Direct and Indirect Meta-Analyses of Randomized Controlled Trials

Abstract

Background

Previous systematic reviews concluded that tricyclics antidepressants are superior to gabapentin for neuropathic pain, but were based on indirect comparisons from placebo-controlled trials.

Purpose

To evaluate gabapentin versus tricyclic antidepressants for diabetic neuropathy and post-herpetic neuralgia, using direct and indirect comparisons.

Data Sources

MEDLINE (1966 to March Week 4 2008), the Cochrane central register of controlled trials (1st quarter 2008), and reference lists.

Study Selection

We selected randomized trials directly comparing gabapentin versus tricyclic antidepressants or comparing either of these medications versus placebo.

Data Extraction

Studies were reviewed, abstracted, and quality-rated by two independent investigators using predefined criteria.

Data Synthesis

We performed a meta-analysis of head-to-head trials using a random effects model and compared the results to an adjusted indirect analysis of placebo-controlled trials.

Results

In three head-to-head trials, there was no difference between gabapentin and tricyclic antidepressants for achieving pain relief (RR 0.99, 95% CI 0.76 to 1.29). In adjusted indirect analyses, gabapentin was worse than tricyclic antidepressants for achieving pain relief (RR = 0.41, 95% CI 0.23 to 0.74). The discrepancy between direct and indirect analyses was statistically significant (p = 0.008). Placebo-controlled tricyclic trials were conducted earlier than the gabapentin trials, reported lower placebo response rates, had more methodological shortcomings, and were associated with funnel plot asymmetry.

Conclusions

Though direct evidence is limited, we found no difference in likelihood of achieving pain relief between gabapentin and tricyclic antidepressants for diabetic neuropathy and post-herpetic neuralgia. Indirect analyses that combine data from sets of trials conducted in different eras can be unreliable.

Electronic supplementary material

The online version of this article (doi:10.1007/s11606-008-0877-5) contains supplementary material, which is available to authorized users.

INTRODUCTION

A number of medications, including gabapentin and tricyclic antidepressants, have been suggested as first-line treatment options for neuropathic pain.^1–3 Previously published systematic reviews concluded that tricyclics are more effective than newer medications such as gabapentin for neuropathic pain.^3–5 These findings were not based on head-to-head trials directly comparing these drugs. Rather, the systematic reviews compared how gabapentin and tricyclics each performed versus placebo. Trials that evaluate different drugs versus a common comparator (such as placebo) can provide indirect evidence about comparative effectiveness, while preserving some of the benefits of randomization.^6,7 However, although indirect comparisons usually agree with direct comparisons from head-to-head trials, in some cases large discrepancies have been reported between direct and indirect analyses.^8,9 The validity of indirect comparisons depends on how well the trials meet the critical assumption of consistent treatment effects across all the trials.⁹ This assumption can be violated due to methodological shortcomings in the trials or differences in populations, interventions, measurement of outcomes, or other factors.^7,9

Indirect comparisons in previously published systematic reviews were “informal,” or based on qualitative rank-ordering of pooled estimates for different drugs from placebo-controlled trials.^3–5 Such qualitative comparisons may not be a reliable substitute for formal quantitative analyses.⁷ Disadvantages of informal indirect comparisons are that they do not provide overall summary estimates of effect and do not adjust for additional uncertainty that occurs when comparing evidence indirectly.^7,9 Formal “adjusted” indirect methods, on the other hand, provide a summary combined estimate and incorporate additive variance from both sets of trials.^6,7 Sensitivity and subgroup analyses can also be performed to evaluate effects of methodological shortcomings and differences in study design, population characteristics, and interventions on estimates and conclusions.

The purpose of this study is to evaluate comparative benefits and harms of gabapentin versus tricyclic antidepressants for diabetic neuropathy and post-herpetic neuralgia and to compare results of direct and formal adjusted indirect analyses. We chose gabapentin and tricyclics because they are commonly prescribed for neuropathic pain, and several head-to-head trials^10–12 are available. We chose diabetic neuropathy and post-herpetic neuralgia because they are both peripheral neuropathies and the most common neuropathic pain conditions in clinical trials. Because most trials of tricyclics were undertaken before trials of gabapentin, we postulated that discrepancies between direct and indirect estimates are likely to occur because of differences that have occurred over time in patient characteristics, management of neuropathic pain, and design and conduct of randomized trials.

METHODS

Data Sources and Searches

To identify potentially relevant citations, we searched Ovid MEDLINE® (1966 to March Week 4 2008), the Cochrane Database of Systematic Reviews® (1st Quarter 2008), the Cochrane Central Register of Controlled Trials® (1st Quarter 2008), and the Database of Abstracts of Reviews of Effects (1st Quarter 2008), using terms for gabapentin and tricyclic antidepressants, neuropathic pain and specific neuropathic pain conditions, and randomized trials (the complete search strategy is shown in Appendix 1, available online). We also reviewed reference lists and solicited pharmaceutical companies for additional citations.

Study Selection

All citations were reviewed for inclusion using the criteria shown in Table 1. We applied no language restriction. Two reviewers (RC and SC) independently assessed titles and abstracts and full-text articles of potentially relevant citations for inclusion. Disagreements were resolved by consensus. Trials published only in abstract form (e.g., a conference proceeding) were not included.¹³

Table 1.

Study Inclusion Criteria

Study meets all of the following criteria:
•Randomized controlled trial
•Enrolls adults with diabetic neuropathy and/or postherpetic neuralgia, or at least 75% of enrollees have either diabetic neuropathy or postherpetic neuralgia
•Evaluates gabapentin or a tricyclic antidepressant (versus each other or versus placebo)
•Reports at least one of the following outcomes: pain relief (the proportion of patients with >50% improvement in pain score or at least moderate pain relief or good overall response on a categorical scale), withdrawal due to adverse events, overall adverse events, serious adverse events, somnolence (including sedation, fatigue, tiredness, and lethargy), dizziness or vertigo, ataxia (including gait disturbance or incoordination), or dry mouth

Data Extraction and Quality Assessment

Two independent reviewers (RC and SC) abstracted the following information from included trials: study design, population characteristics; eligibility and exclusion criteria, interventions (dose and duration), numbers lost to follow-up, method of outcome ascertainment, and results. We recorded intention-to-treat results when reported. For crossover trials, we abstracted results from both crossover periods.¹⁴ If this data were not available, we abstracted results from the first intervention period. Two independent reviewers (RC and SC) also assessed internal validity (quality) of controlled clinical trials using predefined criteria for randomization and allocation concealment, blinding of patients and outcomes assessors, and use of intention-to-treat analysis (Appendix 2, available online). Disagreements were resolved by consensus.

Data Synthesis and Analysis

Our primary outcome was the proportion of patients reporting significant pain relief. We defined significant pain relief as at least 50% improvement in pain score compared to baseline (preferred outcome) or the proportion reporting at least moderate or good improvement in pain or global efficacy on a categorical scale. A similar approach for defining pain relief was used in previously published systematic reviews.^3,5,15,16 For adverse events, we evaluated withdrawals due to adverse events, serious adverse events, somnolence (including sedation, tiredness, fatigue, or lethargy), ataxia (including gait disturbance and incoordination), dizziness or vertigo, and dry mouth.

We estimated pooled relative risks and 95% confidence intervals using the DerSimonian-Laird method in a random effects model.¹⁷ We chose the random effects model because trials differed in patient populations, dosing of drugs, and other factors. For all pooled estimates, trials with no events in either group were excluded; trials with events only in one group were analyzed by adding 0.5 to all cells. Statistical heterogeneity was assessed by calculating the percent of the total variance due to between-study variability (I² statistic¹⁸). Higher I² values indicate greater between-study heterogeneity. Relative risks and confidence intervals were calculated using the meta package in R.¹⁹ Forest plots were generated using RevMan 4.2.8 (Review Manager 4.2 for Windows, The Nordic Cochrane Center, Copenhagen, Denmark). When data were available from at least six trials, we constructed L’Abbe plots to identify outlier trials and to assess whether treatment effects vary with differences in underlying risk.²⁰ We assessed funnel plot asymmetry (which can be due to publication bias) with the Egger test.²¹

We performed adjusted indirect comparisons using the method described by Bucher et al.⁶

We calculated indirect relative risks (RR_Ind) for gabapentin versus tricyclic antidepressants for each outcome, adjusted by the results of their comparisons against placebo:

The variance was estimated as:

To test assumptions regarding similarity of treatment effects across trials, we compared mean placebo response rates in trials of gabapentin and tricyclics. We also performed subgroup and sensitivity analyses on study design factors (use of crossover versus parallel-group design, methodological quality criteria, and publication before or after 1997), intervention factors (evaluation of a dose of <2,400 mg/day of gabapentin, evaluation of a secondary versus tertiary amine tricyclic, and use of an active versus inert placebo), and population factors (exclusion of previous non-responders to gabapentin and evaluation of diabetic neuropathy or post-herpetic neuralgia). When funnel plot asymmetry was detected, we performed sensitivity analyses using the trim and fill method, which generates “missing” studies until the plot becomes symmetrical.²²

We measured the discrepancy between direct and indirect estimates by calculating the difference in log relative risks. We deemed a p value of less than 0.05 statistically significant.⁹

RESULTS

Figure 1 shows the flow of studies from initial results of literature searches to final inclusion or exclusion. Literature searches identified 549 citations, and 55 of these appeared potentially relevant. After review of full-text articles, 18 trials met inclusion criteria: 3 head-to-head trials of gabapentin versus tricyclic antidepressants,^10–12 6 trials of gabapentin versus placebo,^23–28 and 9 trials of tricyclic antidepressants versus placebo.^29–37 A list of excluded trials and reasons for exclusion is available from the authors.

Figure 1.

Literature search results.

Direct Meta-analysis

A total of 120 patients were evaluated in three head-to-head trials (n = 25 to 70) of gabapentin versus a tricyclic antidepressant for neuropathic pain (Table 2).^10–12 Two trials compared gabapentin versus amitriptyline for diabetic neuropathy,^11,12 and one trial compared gabapentin versus nortriptyline for post-herpetic neuralgia.¹⁰ The trials were published between 1999 and 2006. Dose of gabapentin varied, with titration up to 1,800 mg/day,¹² 2,400 mg/day,¹¹ or 2,700 mg/day.¹⁰ Maximum doses of tricyclics ranged from 75 to 90 mg/day. One trial used a crossover design.¹² Duration of therapy ranged from 4 to 6 weeks. None of the trials met all quality assessment criteria (Table 2). Two of the three trials did not describe funding source;^11,12 the third¹⁰ reported funding from Pfizer India.

Table 2.

Characteristics of Included Trials and Main Results

Author, year, design	Funding source	Drug, dose, duration (daily dose)	Type of neuropathic pain	Pain relief*	Quality items†
Gabapentin vs tricyclic antidepressanthead-to-head trials
Chandra 200610	Pfizer India Limited	A. Gabapentin up to 2,700 mg	Post-herpetic neuralgia	16/34 (47.1%) vs 17/36 (47.2%)	1)Y
					2)Y
		B. Nortriptyline up to 75 mg			3)Y
Parallel-group					4)Y
		8 weeks			5)N
Dalocchio 200011	Not reported	A. Gabapentin mean 1,785 mg (max 2,400 mg)	Diabetic neuropathy	11/13 (84.6%)	1)NR
					2)NR
		B. Amitriptyline mean 53 mg (max 90 mg)		9/12 (75.0%)	3)N
Parallel-group					4)N
		4 weeks			5)Y
Morello 199912	Not reported	A. Gabapentin 900-1,800 mg	Diabetic neuropathy	11/21 (52.4%)	1)NR
					2)NR
		B. Amitriptyline 25-75 mg		14/21 (66.7%)	3)Y
Crossover					4)N
		6 weeks			5)N
Gabapentin vs placebo
Backonja 1998 23	Parke-Davis Pharmaceuticals	Gabapentin up to 3,600 mg	Diabetic neuropathy	47/79 (59.5%) vs 25/76 (32.9%)	1)Y
					2)NR
					3)Y
Parallel-group		8 weeks			4)U
					5)Y
Gilron 200524	Canadian Institutes of Health Research	Gabapentin up to 3,200 mg (mean 2,207 mg)	Diabetic neuropathy or post-herpetic neuralgia	27/44 (61.4%) vs 13/42 (31.0%)	1)NR
					2)Y
					3)Y
Crossover		5 weeks			4)U
					5)U
Gorson 199925	Parke-Davis Pharma-ceuticals	Gabapentin 900 mg	Diabetic neuropathy	17/19 (89.5%) vs 9/21 (42.9%)	1)NR
					2)NR
					3)U
Crossover		6 weeks			4)U
					5)Y
Rice 200126	Pfizer Ltd.	Gabapentin 1,800 mg or 2,400 mg	Post-herpetic neuralgia	74/223 (33.2%) vs 16/111 (14.4%)	1)Y
					2)Y
					3)Y
Parallel-group		7 weeks			4)U
					5)N
Rowbotham 199827	Parke-Davis Pharmaceuticals	Gabapentin up to 3,600 mg	Post-herpetic neuralgia	47/109 (43.1%) vs 14/116 (12.1%)	1)Y
					2)NR
					3)Y
Parallel-group		8 weeks			4)U
					5)Y
Simpson 200128	Not reported	Gabapentin 900–2,700 mg	Diabetic neuropathy	15/27 (55.6%) vs 7/27 (25.9%)	1)NR
					2)NR
					3)Y
Parallel-group		8 weeks			4)U
					5)N
Tricyclic antidepressant vs placebo
Kishore-Kumar 199029	Not reported	Desipramine 12.5–250 mg (mean 167 mg)	Post-herpetic neuralgia	16/26 (61.5%) vs 3/26 (11.5%)	1)NR
					2)NR
					3)Y
Crossover		6 weeks			4)U
					5)N
Kvinesdal 198430	Not reported	Imipramine 100 mg	Diabetic neuropathy	7/12 (58.3%) vs 0/12 (0%)	1)NR
					2)NR
					3)Y
Crossover		5 weeks			4)U
					5)N
Max 198731	Not reported	Amitriptyline 25–150 mg (mean 90 mg)	Diabetic neuropathy	19/29 (65.5%) vs 1/29 (3.4%)	1)NR
					2)NR
					3)Y
Crossover		12 weeks			4)Y
					5)N
Max 198833	Not reported	Amitriptyline 12.5–150 mg (mean 65 mg)	Post-herpetic neuralgia	16/34 (47.1%) vs 4/25 (16.0%)	1)NR
					2)NR
					3)Y
Crossover		6 weeks			4)U
					5)N
Max 199132	Not reported	Desipramine 12.5–250 mg (mean 201 mg)	Diabetic neuropathy	13/24 (54.2%) vs 3/24 (12.5%)	1)NR
					2)NR
					3)Y
Crossover		6 weeks			4)U
					5)N
Raja 200234	National Institutes of Health	Nortriptyline or desipramine 10–160 mg (mean 89 mg)	Post-herpetic neuralgia	NR	1)Y
					2)Y
					3)Y
Crossover		8 weeks			4)U
					5)U
Sindrup 198935	Research Foundation of Vejle County, Denmark	Imipramine 125–225 mg (mean 178 mg)	Diabetic neuropathy	NR	1)NR
					2)NR
					3)Y
Crossover		3 weeks			4)U
					5)N
Sindrup 199036	Danish Diabetes Association	A: Desipramine 50–200 mg	Diabetic neuropathy	NR	1)NR
		B: Clomipramine 50–75 mg			2)NR
		C: Placebo			3)Y
Crossover		2 weeks			4)U
					5)N
Watson 198237	Not reported	Amitriptyline 25–125 mg (median 75 mg)	Post-herpetic neuralgia	16/24 (66.7%) vs 1/24 (4.2%)	1)NR
					2)NR
					3)Y
Crossover		3 weeks			4)U
					5)U

*≥50% improvement in pain or at least moderate pain relief; †1) Randomization method, 2) allocation concealment, 3) masked patients, 4) masked outcome assessors, 5) intention-to-treat analysis; NR = not reported; N = no; Y = yes; U = unclear; max = maximum

We found no difference between gabapentin and tricyclic antidepressants in likelihood of experiencing pain relief (Fig. 2) (RR = 0.99, 95% CI 0.76 to 1.29, I² = 0%, three trials).^10–12 Estimates were similar when trials were stratified according to whether they evaluated diabetic neuropathy (RR = 0.98, 95% CI 0.69 to 1.38, I² = 26%, two trials^11,12) or post-herpetic neuralgia (RR = 1.00, 95% CI 0.61 to 1.64, one trial¹⁰). Estimates were also similar after excluding the crossover trial¹² and when trials were stratified by methodological quality indicators.

Figure 2.

Relative risk for pain relief from head-to-head trials of gabapentin versus tricyclic antidepressants.

There was no difference between gabapentin versus tricyclics in rates of withdrawal due to adverse events (RR 0.27, 95% CI 0.03 to 2.34), but only three cases were reported in two trials.^10,12 None of the trials reported serious adverse events. There was no significant difference between gabapentin and tricyclics in risk of somnolence (RR 1.22, 95% CI 0.59 to 2.52, two trials^10,12), dry mouth (RR 0.16, 95% CI 0.01 to 2.66, two trials^10,12), or dizziness (RR 3.65, 95% CI 0.85 to 15.78, one trial¹²), but all estimates were imprecise.

Indirect Meta-analysis

Analysis of Placebo-Controlled Trials

Sample sizes in six placebo-controlled trials of gabapentin^23–28 ranged from 40 to 334 (median 112), and in nine placebo-controlled trials of tricyclics^29–37 ranged from 12 to 76 (median 26). All of the gabapentin trials were published in or after 1997. All of the tricyclic trials that reported the primary outcome pain relief were published in or before 1991. Two gabapentin^24,25 and all of the tricyclic trials used a crossover design. Target doses of gabapentin ranged from 900 to 3,600 mg/day (Table 2). Among the tricyclic trials, three evaluated a secondary amine (nortriptyline³⁴ or desipramine^29,32,34), five evaluated a tertiary amine (amitriptyline^31,33,37 or imipramine^30,35), and one evaluated both (clomipramine and desipramine³⁶). Maximum doses of tricyclics ranged from 75 mg/day to 250 mg/day. Three gabapentin trials evaluated patients with diabetic neuropathy,^23,25,28 two postherpetic neuralgia,^26,27 and one included patients with both conditions.²⁴ One gabapentin trial excluded previous non-responders to gabapentin.²⁶ Five tricyclic antidepressant trials evaluated patients with diabetic neuropathy,^{30–32,35,36} and four evaluated patients with postherpetic neuralgia.^29,33,34,37 Only one trial³¹ of either gabapentin or tricyclics evaluated a course of therapy longer than 8 weeks in duration. Five (83%) gabapentin trials^23–27 and three (33%) tricyclic trials^34–36 reported a funding source. Of trials reporting a funding source, four (80%) gabapentin trials^23–27 were funded by pharmaceutical companies, and all tricyclic trials received government or foundation funding.

Gabapentin was superior to placebo for achieving pain relief (RR = 2.18, 95% CI 1.78 to 2.67, I² = 0%, six trials,^23–28 Fig. 3). Tricyclics were also superior to placebo for achieving pain relief (RR = 5.27, 95% CI 3.05 to 9.11, I² = 10%, six trials,^29–33,37 Fig. 4). Funnel plot asymmetry (p = 0.01 by Egger test, Fig. 5) was detected in the tricyclic, but not the gabapentin trials. The L’Abbe plot showed no outlier tricyclic trials (Fig. 6). Adjustment for funnel plot asymmetry using the trim-and-fill method slightly attenuated the estimate of pain relief (RR = 3.99, 95% CI 2.10 to 7.57, I² = 30%).

Figure 3.

Relative risk for pain relief from placebo-controlled trials of gabapentin.

Figure 4.

Relative risk for pain relief from placebo-controlled trials of tricyclic antidepressants.

Figure 5.

Funnel plot, placebo-controlled trials of tricyclic antidepressants, on relative risk for pain relief.

Figure 6.

L’Abbe plot, placebo-controlled trials of tricyclic antidepressants, on relative risk for pain relief.

There was no significant difference in stratified estimates between trials evaluating a secondary amine tricyclic versus a tertiary amine tricyclic (Table 3). For the gabapentin trials, excluding one trial evaluating a dose of 900 mg/day,²⁵ excluding one trial that did not enroll previous non-responders to gabapentin,²⁶ and stratifying trials by use of crossover versus parallel-group design had little effect on estimates. All of the other gabapentin trials titrated patients to a goal dose of at least 2,400 mg/day. For the gabapentin trials, stratifying trials according to whether they met various quality criteria showed no differences in estimates. For the tricyclic trials, we could not assess effects of methodological shortcomings. All tricyclic trials met the quality criterion for masking of patients, but only one small (n = 29) trial met even one other quality criterion.³¹ When stratified according to specific neuropathic pain condition, gabapentin and tricyclics (Table 3) were both superior to placebo for diabetic neuropathy.

Table 3.

Pain Relief in Placebo-Controlled Trials of Tricyclic Antidepressants for Neuropathic Pain

Type of trial	Number of trials	Pooled relative risk (95% CI)*	Test for heterogeneity	p value†
All placebo-controlled trials of tricyclic antidepressants	629–33,37	5.27 (3.05–9.11)	I2 = 10%	Not applicable
Stratified by evaluation of secondary or tertiary amine tricyclic
Secondary amine tricyclic	429–32	6.21 (3.07–12.58)	I2 = 0%	p = 0.65
Tertiary amine tricyclic	233,37	5.52 (1.11–27.46)	I2 = 62%
Specific neuropathic pain condition
Diabetic neuropathy	330–32	6.92 (2.76–17.38)	I2 = 12%	p = 0.58
Post-herpetic neuralgia	329,33,37	4.75 (2.17–10.39)	I2 = 25%
Adjusted for funnel plot asymmetry
Trim and fill method	629–33,37+ 3 trials ‘filled’	3.99 (2.10–7.57)	I2 = 30%	Not applicable

*For ≥50% improvement in pain or at least moderate pain relief, tricyclic antidepressant versus placebo; †for difference in stratified estimates

“Serious” adverse events were reported by three gabapentin trials (range 0% to 1.8%).^25–27 None defined the term “serious,” and two of the three trials reported zero events.^25,27 There was no difference between gabapentin versus placebo for serious adverse events in the remaining trial,²⁶ but the estimate was imprecise (RR = 1.99, 95% CI 0.23 to 17.60, I² = 0%). Estimates were also imprecise for dry mouth and ataxia/gait disorder (Table 4; Appendix 3, available online). No trial of tricyclics reported serious adverse events or ataxia/gait disturbance. Gabapentin and tricyclics were both associated with a relative risk of about 1.70 for withdrawal due to adverse events compared to placebo, but the difference was only statistically significant for gabapentin. Gabapentin and tricyclics were both associated with greater risk of somnolence compared to placebo. Gabapentin was also associated with increased risk of dizziness and tricyclics with increased risk of dry mouth versus placebo. Stratification of tricyclic trials by evaluation of secondary versus tertiary amines or use of active versus inert placebo had little effect on estimates of harms (Appendix 4, available online). All trials of gabapentin used an inert placebo.

Table 4.

Direct and Adjusted Indirect Estimates from Placebo-Controlled Trials of Gabapentin and Tricyclic Antidepressants for Neuropathic Pain

Comparison	Gabapentin versus placebo	Tricyclic antidepressants versus placebo	Gabapentin vs tricyclic antidepressant
	RR (95% CI), test for heterogeneity [number of trials]	RR (95% CI), test for heterogeneity [number of trials]	RR (95% CI): Adjusted indirect estimate
Pain relief*	2.18 (1.78–2.67), I2 = 0% (623–28)	5.27(3.05–9.11), I2 = 10% (629–33,37)	0.41 (0.23–0.74)
Withdrawal due to adverse events	1.69 (1.10–2.60), I2 = 0% (523,25–28)	1.71 (0.68–4.31), I2 = 0% (529,30,32,35,36)	0.99 (0.36–2.74)
Serious adverse events	1.99 (0.23–17.60) (126)	No data [none]	No data
Somnolence, sedation, fatigue, or lethargy	3.92 (2.45–6.27), I2 = 0% (426–28,38)	1.50 (1.09–2.07), I2 = 17% (429,31–33)	2.61 (1.48–4.62)
Dizziness or vertigo	3.92 (2.55–6.02), I2 = 0% (423,26–28)	1.24 (0.64–2.43), I2 = 36% (329,31,33)	3.16 (1.43–6.99)
Ataxia, gait abnormality, or incoordination	17.45 (1.02 to 299) (127)	No data [None]	No data
Dry mouth	5.97 (0.79–45.35) (126)	1.44 (1.13–1.83), I2 = 29% (629–33,35)	4.15 (0.54–31.86)

*≥50% improvement in pain score or at least moderate pain relief; CI = confidence interval; RR = relative risk

Adjusted Indirect Analyses

Pooled mean rates for at least moderate or >50% pain relief in patients randomized to placebo were four times higher in trials of gabapentin (24%, 95% CI 15% to 33%, six trials^23–28) than in trials of tricyclics (6%, 95% CI 2% to 10%, six trials^29–33,37). In adjusted indirect analyses, gabapentin was inferior to tricyclics for achieving pain relief (RR = 0.41, 95% CI 0.23 to 0.74, Table 4). The difference remained statistically significant when we restricted the analysis to diabetic neuropathy trials (RR = 0.28, 95% CI 0.11 to 0.73) or crossover trials (RR = 0.39, 95% CI 0.20 to 0.74) (Table 5). A similar trend was present when the analysis was restricted to post-herpetic neuralgia trials, though the difference was not statistically significant (RR 0.60, 95% CI 0.24 to 1.46).

Table 5.

Sensitivity Analyses on Adjusted Indirect Estimates for Achieving Pain Relief with Gabapentin Versus Tricyclic Antidepressants

Analysis	Gabapentin versus placebo	Tricyclic antidepressants versus placebo	Gabapentin vs tricyclic anti-depressant: Indirect estimate
	RR (95% CI),* test for heterogeneity [number of trials]	RR (95% CI),* test for heterogeneity [number of trials]	RR (95% CI)*
All trials	2.18 (1.78–2.67), I2 = 0% (623–28)	5.27(3.05–9.11), I2 = 10% (629–33,37)	0.41 (0.23–0.74)
Crossover trials only	2.03 (1.41–2.92), I2 = 0% (224,25)	5.27(3.05–9.11), I2 = 10% (629–33,37)	0.39 (0.20 to 0.74)
Diabetic neuropathy	1.93 (1.46–2.55), I2 = 0% (323,25,28)	6.92 (2.76–17.38), I2 = 12% (330–32)	0.28 (0.11–0.73)
Post-herpetic neuralgia only	2.83 (1.84–4.35), I2 = 29% (226,27)	4.75 (2.17–10.39), I2 = 25% (329,33,37)	0.60 (0.24–1.46)
Adjusted for funnel plot asymmetry using the trim and fill method	2.18 (1.78-2.67), I2 = 0% (623–28)	3.99 (2.10-7.57), I2 = 30% (929–33,37) + 3 ‘filled’ trials	0.55 (0.28-1.07)

*For ≥50% improvement in pain score or at least moderate pain relief; CI = confidence interval; RR = relative risk

We found no statistically significant differences between gabapentin versus tricyclics in risk of withdrawal due to adverse events (Table 4). Gabapentin was associated with increased risk of somnolence/sedation (RR = 2.61, 95% CI 1.48 to 4.62) and dizziness (RR = 3.16, 95% CI 1.43 to 6.99) compared to tricyclics.

Discrepancies Between Direct and Indirect Estimates

When all trials were included in the analysis, the discrepancy between direct (RR = 0.99, 95% CI 0.76 to 1.29) and indirect estimates (RR = 0.41, 95% CI 0.23 to 0.74) for gabapentin versus tricyclics for achieving pain relief was statistically significant (p = 0.008, Table 6). The discrepancy was also statistically significant when the analysis was restricted to diabetic neuropathy trials (RR = 0.98, 95% CI 0.69 to 1.38 vs RR = 0.28, 95% CI 0.11 to 0.73, p = 0.016 for discrepancy). When the analysis was restricted to crossover trials, the discrepancy was not statistically significant (p = 0.09), but the direct estimate was based on only one head-to-head trial.¹² There was no discrepancy between direct and indirect estimates for any adverse event, but data from head-to-head trials were sparse.

Table 6.

Discrepancies Between Direct and Indirect Analyses of Gabapentin Versus Tricyclic Antidepressants for Achieving Pain Relief

Analysis	Direct analysis: Number of trials	Direct analysis: Relative risk for pain relief* (95% CI)	Adjusted indirect analysis: Number of trials	Indirect analysis: RR for pain relief* (95% CI)	Discrepancy between direct and indirect estimate: difference in log RR
All trials included	310–12	0.99 (0.76–1.29)	12 (6 gabapentin23–28 and 6 tricyclics29–33,37)	0.41 (0.23-0.74)	Δ = 0.87, p = 0.008
Crossover trials only	112	0.79 (0.47 to 1.31)	8 (2 gabapentin24,25 and 6 tricyclics29–33,37)	0.39 (0.20 to 0.74)	Δ = 0.71, p = 0.09
Adjusted for funnel plot asymmetry using trim-and-fill method	310–12	0.99 (0.76 to 1.29)	15 (6 gabapentin23–28 and 6 tricyclics29–33,37) + 3 ‘filled’ trials	0.55 (0.28 to 1.07)	Δ = 0.59, p = 0.11
Diabetic neuropathy	211–12	0.98 (0.69 to 1.38)	6 (3 gabapentin23,25,28 and 3 tricyclics30–32)	0.28 (0.11 to 0.73)	Δ = 1.25, p = 0.016
Post-herpetic neuralgia	110	1.00 (0.61 to 1.64)	5(2 gabapentin26,27 and 3 tricyclics29–33,37)	0.60 (0.24 to 1.46)	Δ = 0.51, p = 0.32

*≥50% improvement in pain score or at least moderate pain relief; CI = confidence interval; RR = relative risk

DISCUSSION

In a direct meta-analysis of head-to-head trials, we showed no difference between gabapentin and tricyclic antidepressants for achieving pain relief for diabetic neuropathy or postherpetic neuralgia. Although this result is based on only three studies, the estimate is fairly precise and very close to a relative risk of 1.00. Even if statistical power were to be enhanced by the publication of new head-to-head trials, clinically relevant differences would only occur if future estimates of effects are substantially different than currently available evidence. Nonetheless, direct evidence is sparse, and more head-to-head trials are needed to confirm our findings. We found no difference between gabapentin and tricyclics for various adverse events, but analyses of harms are even more limited by sparse data than analyses of benefits.

Indirect meta-analysis of placebo-controlled trials yielded discordant conclusions compared with the direct analysis, showing gabapentin more than 50% less likely to achieve pain relief compared to tricyclics. The discrepancy was more pronounced when we restricted our analysis to diabetic neuropathy trials (gabapentin about one-fourth as likely as tricyclics to achieve pain relief). Our indirect analyses are consistent with previous systematic reviews, also based on indirect analyses, that concluded that tricyclics are superior to newer medications, including gabapentin, for diabetic neuropathy⁵ or peripheral neuropathic pain conditions in general.³ The discrepancy we found is in accordance with previous studies showing that direct and indirect analyses can be associated with discordant estimates of treatment effect.^8,9 In this case, the discrepancy was both statistically and clinically significant (no difference between gabapentin and tricyclics in the direct meta-analysis versus gabapentin inferior to tricyclics in the indirect meta-analysis).

To our knowledge, this is the first study to compare direct meta-analyses to formal adjusted indirect analyses of medications for neuropathic pain. Previous systematic reviews^3–5 comparing neuropathic pain drugs relied on informal indirect comparisons, or the qualitative rank-ordering of pooled estimates from placebo-controlled trials. Apparent differences between drugs in qualitative comparisons may not be statistically significant when formal adjusted indirect analysis is performed.^6,7 In addition, for all indirect analysis, the validity of indirect comparisons depends on how well they meet the critical assumption of consistent treatment effects across all of the trials.^6,7 This assumption can be violated by methodological shortcomings in the trials, differences in populations, interventions (e.g. dosing), assessment of outcomes, or other factors. It is critical to consider such factors when considering whether any indirect comparison—either formal or informal—is valid.

In our study, all placebo-controlled trials of gabapentin were published in or after 1998, while placebo-controlled tricyclic trials were generally published in or before 1991. Indirect analyses may be particularly problematic when trials from different eras are combined, because it is unlikely that patient characteristics, treatment regimens, assessment of outcomes, and design and conduct of randomized trials would remain similar enough to meet the assumption of consistent treatment effects across trials.⁸ Several findings from our study support this hypothesis. First, there was a four-fold difference in placebo response rates between the gabapentin (24%) and tricyclic (6%) trials. Second, the older tricyclic trials did not meet current standards for methodological quality. Only one small trial reporting pain relief met even one quality criterion other than masking of patients.³¹ Finally, all of the tricyclic trials reporting pain relief used a crossover design and evaluated small sample sizes (median 26). The gabapentin trials, on the other hand, mostly used a parallel group design and enrolled larger sample sizes (median 112). Although our results were similar when we restricted our analysis to crossover trials, small crossover trials tend to report higher estimates of effects compared to parallel-group trials.³⁸

Our study has several potential limitations. First, there was clinical diversity in the trials evaluated, including the type of neuropathic pain evaluated and doses of drugs. We therefore used a random effects model. We also found little statistical heterogeneity and performed a number of subgroup and sensitivity analyses showing stable results. Second, as in several previously published systematic reviews,^3,5,15,16 we analyzed a composite dichotomous measure for pain relief. An advantage of using such composite outcomes is that more trials can be entered into analyses. A disadvantage is that it is not certain how valid pooling of disparate methods for measuring pain outcomes is, particularly for poorly validated or described categorical scales.³⁹ We found insufficient data to perform sensitivity analyses on different methods for classifying pain relief outcomes. Finally, we did not include pregabalin in our study, even though it is similar to gabapentin in structure and mechanism of action, because no trials of pregabalin versus tricyclics are available.

We identified several areas related to the conduct and reporting of randomized trials of medications for neuropathic pain that could be improved. First, no trial met all quality criteria. Second, all randomized trials included in our study are “efficacy” studies that applied numerous inclusion and exclusion criteria, were conducted in academic or specialty settings, and were relatively short-term.⁴⁰ Third, assessment and reporting of outcomes were suboptimal. For example, one-third of placebo-controlled trials of tricyclics did not report usable data on the proportion of patients experiencing pain relief.^34–36 Adverse events were reported even less consistently. Fourth, funnel plot asymmetry was present among tricyclic trials, which could be associated with publication bias.²¹ Finally, other factors, such as patient preferences, costs, or risk factors for serious adverse events, such as overdose or cardiac arrhythmias, may be relevant for making treatment choices, but are not well studied in randomized trials.^2,41

Clinicians should be aware that conclusions of systematic reviews that compare different interventions are frequently based on indirect comparisons, even when they don’t use formal indirect methods, and that results based on such methods can be misleading. Our results are consistent with the hypothesis that assumptions underlying indirect comparisons are more likely to be violated when trials from different eras are combined. If direct evidence is sparse or unavailable, indirect comparisons should only be considered when critical underlying assumptions are met, formal adjusted indirect analysis should be performed if possible, and results should be verified against head-to-head evidence as it become available.