Characterization of Internal Validity Threats to Phase III Clinical Trials for Chemotherapy-Induced Peripheral Neuropathy Management: A Systematic Review

Abstract

Objective:

The recent American Society of Clinical Oncology (ASCO) Clinical Guidelines for chemotherapy-induced peripheral neuropathy (CIPN) management (48 Phase III trials reviewed) only recommend duloxetine. However, before concluding that a CIPN intervention is ineffective, scientists and clinicians should consider the risk of Type II error in Phase III studies. The purpose of this systematic review was to characterize internal threats to validity in Phase III CIPN management trials.

Methods:

The PubMed, CINAHL, EMBASE^®, and Scopus databases were searched for Phase III clinical trials testing interventions for CIPN management between 1990 and 2018. The key search terms were neoplasms, cancer, neuropathy, and CIPN. Two independent researchers evaluated 24 studies, using a modified Joanna Briggs Institute Checklist for Randomized Control Trials developed by the authors specific for CIPN intervention trials.

Results:

Two studies exhibited minimal or no design flaws. 22/24 Phase III clinical trials for CIPN have two or greater design flaws due to sample heterogeneity, malapropos mechanism of action, malapropos intervention dose, malapropos timing of the outcome measurement, confounding variables, lack of a valid and reliable measurement, and suboptimal statistical validity.

Conclusions:

Numerous CIPN interventions have been declared ineffective based on the results of Phase III trials. However, internal validity threats to numerous studies may have resulted in Type II error and subsequent dismissal of a potentially effective intervention. Patients may benefit from rigorous retesting of several agents (e.g., alpha-lipoic acid, duloxetine, gabapentin, glutathione, goshajinkigan, lamotrigine, nortriptyline, venlafaxine, and Vitamin E) to expand and validate the evidence regarding ASCO's recommendations for CIPN management.

Introduction

Chemotherapy-induced peripheral neuropathy (CIPN) is one of the most common and debilitating toxicities of cancer treatment that can negatively impact patients’ quality of life and functional status[1,2] and healthcare costs.[3,4] Several agents may cause CIPN, including platinums, taxanes, vinca alkaloids, epothilones, bortezomib, and thalidomides.[5] These neurotoxic drugs cause sensorimotor nerve damage, leading to symptoms of weakness, numbness, tingling, and pain in the hands and feet, which can persist far beyond the completion of chemotherapy. To reduce CIPN progression, oncologists may limit or discontinue patients’ chemotherapy treatment altogether.

Although the negative effects of CIPN on quality of life and chemotherapy administration are well documented, little is known about optimal CIPN prevention and/or treatment strategies. The American Society of Clinical Oncology (ASCO) Clinical Practice Guidelines for CIPN management, informed by a review of over 48 Phase II/III clinical trials of 19 agents for the prevention and six agents for the treatment of CIPN,[5] determined that only duloxetine 60 mg/day can be recommended to treat chronic painful CIPN. No interventions can be currently recommended for CIPN prevention.[5,6] Additional testing was recommended for antidepressants (e.g., nortriptyline HCl and desipramine), gabapentin, and a compounded topical gel with baclofen, amitriptyline HCl, and ketamine (BAK). No further testing was recommended for acetyl-L-carnitine (ALC), amifostine, calcium/magnesium, diethyldithio-carbamate (DDTC), glutathione, nimodipine, Org 2766, all-trans retinoic acid, rhuLIF, or Vitamin E.[5]

While strong evidence demonstrates the inefficacy of some agents (e.g., calcium/magnesium and ALC),[7,8] the abandonment of testing some agents could be premature given the underdeveloped and potentially biased state of the evidence. For example, the recommendations to no longer test DDTC, nimodipine, and retinoic acid were each based on one trial[9,10,11] that were categorized by Hershman et al.[5] as having an intermediate or high risk of bias. Some agents, such as goshajinkigan, were not listed as agents requiring further testing even though at least one trial with a low risk of bias had supported their efficacy. Finally, the ASCO's Clinical Practice Guidelines were informed by one individual's review of the studies’ risks of bias. This individual was not blinded to the study authors and had not done calibration exercises with the research team.[5]

Validity involves the degree to which the study design controls for extraneous variables, thus allowing causal inference to be made between the independent variable (e.g., pharmacological intervention) and the dependent variable (e.g., CIPN severity).[12] Table 1 defines important internal threats to validity to consider when designing and evaluating CIPN management trials. One cannot eliminate the possibility that an extraneous variable influenced the observed results of a study with multiple threats to validity, thus leading to specious conclusions.[12] Thus, the rigorous evaluation of threats to internal validity of previously conducted Phase III CIPN clinical trials is needed to determine the agents that require further testing and to guide the development of future Phase III CIPN intervention trials. The purpose of this systematic review was to describe the internal threats to validity in Phase III CIPN management trials.

Table 1.

Critical appraisal criteria for the assessment of internal validity in Phase III chemotherapy-induced peripheral neuropathy intervention studies

Internal validity threat	Appraisal criteria
Sample heterogeneity	Was the sample homogeneous (or stratified to control for heterogeneity)?
	Did all participants have similar exposure to chemotherapy before study initiation?
	E.g., Were all patients chemotherapy naive at baseline?**
	For trials evaluating chronic painful CIPN, did all participants have stable CIPN for at least 3 months following chemotherapy completion?†
Malapropos intervention mechanism of action and dose	Was the drug and dosage appropriate for the study aims?
	Was the tested drug’s mechanism of action consistent with the pathophysiology of the CIPN under investigation?
	Was the drug administration reasonable?
	Appropriate dose?
	Appropriate titration period?
	Right route of administration?
	No potential interaction with concomitant medications
Malapropos timing of outcome measurement	Were the time points of measurement appropriate?
	Were the outcomes time points appropriate based on the type of trial (e.g., prevention or management)?
	Was the drug administered for a long enough period of time to observe an effect of treatment?
	Were baseline CIPN severity scores high enough to be able to detect a difference in CIPN symptom severity between groups?**
	Was it possible that the effect of coasting or spontaneous CIPN improvement influenced CIPN symptom severity at the time point of measurement?†
	Were the outcome time points well defined and consistent across all participants?
Confounding variables	Was there adequate control for other CIPN influencing factors?
	Did the researchers stratify, exclude participants, or statistically control for covariates such as
	Chemotherapy regimen and dose received**
	Preexisting PN and prior receipt of chemotherapy**
	Conditions associated with PN: Cancer-related PN (e.g., paraneoplastic neuropathic, multiple myeloma-associated neuropathy), diabetes, symptomatic PAD, alcoholic disease, carpal tunnel syndrome, HIV/neurotoxic drugs, Vitamin B deficiencies
	Concomitant analgesic and psychotropic regimens
Lack of valid and reliable measurement	Were valid and reliable CIPN measures used?
	Were psychometrically strong CIPN PRO measures used?
	Were psychometrically strong objective (e.g., TNS) measures used?
	Were the selected CIPN measures aligned with the CIPN symptoms (e.g., sensory CIPN, motor
	CIPN, or painful CIPN) identified in the aims? E.g., if the study focused on treating painful CIPN, was pain measured separately from numbness and tingling?
Lack of statistical validity	JBI: Was appropriate statistical analysis used?
	Was the study adequately powered?
	Were the statistical procedures appropriate, given the aims, number of variables, and study groups?
	Was intent-to-treat analysis used?
	Were appropriate methods used for missing data (e.g., multiple imputation)?
Study design	Was CIPN defined as the primary outcome in the specific aims?
	Was the logical progression of trial research followed: At least two Phase II trials demonstrated efficacy before the Phase III trial?
	Were the design and methods consistent with previous trials’ designs? (e.g., drug/dosage)?

**Applies only to prevention trials, ^†Applies only to treatment trials. CIPN: Chemotherapy-induced peripheral neuropathy, JBI: Joanna Briggs Institute criteria, PAD: Peripheral arterial disease, PN: Peripheral neuropathy, PRO: Patient-reported outcome measure, TNS: Total neuropathy score

Methods

The PubMed, CINAHL, EMBASE^®, and Scopus databases were searched for Phase III clinical trials, published between 1990 and 2018, that tested interventions for CIPN prevention or treatment. The search dates were selected to (1) capture all the Phase III clinical trials referenced in the ASCO recommendations and (2) extend the findings of the ASCO recommendations by including recently conducted Phase III trials. The key search terms were neoplasms, cancer, neuropathy, and CIPN. The reference lists of the included articles and other CIPN treatment reviews were hand-searched to identify additional articles.

Eligibility criteria

To increase the comparability of our findings, the eligibility criteria set forth by the ASCO review[5] were used for this review. Specifically, eligible articles reported the results of a Phase III RCT (2) that tested the efficacy of pharmacological interventions for the prevention and treatment of CIPN.[5] Articles were excluded if they (1) reported the findings of Phase I or II studies, (2) used nonexperimental designs, (3) included nonhuman subjects, (4) did not include cancer patients, (5) were not published in English, or (6) had a sample size of <10 subjects.

Data extraction

Data extraction was conducted based on the PRISMA guidelines.[13] Two authors independently scanned the article titles and abstracts to identify relevant studies that met the inclusion criteria. Questions about article inclusion were resolved through discussion among the co-authors. The following information was extracted from the included trials: design (prevention vs. treatment; single- vs. multi-site), sample size, population of interest, drug dosage, control condition, outcome measurement time points, and CIPN-related outcomes (e.g., CIPN severity and associated physical function, neurophysiological changes).

Data evaluation

The quality of the Phase III studies was evaluated using a modified version of the Joanna Briggs Institute (JBI) Checklist for Randomized Control Trials.[14] Table 1 describes the criteria of the modified JBI checklist that was adapted specifically for CIPN intervention trials. Studies were evaluated as having low risk of bias (<two validity threats) or high risk of bias (>two validity threats). Table 2 identifies the specific threats to validity of each study included. Descriptive statistics were used to quantify the number (n) of prevention and treatment studies that failed to meet each specific internal validity criteria. Recommendations for or against further testing specific agents for CIPN management were based on studies’ risks of bias and findings (the efficacy and safety of the tested agents).

Table 2.

Internal validity threats in Phase III chemotherapy-induced peripheral neuropathy prevention and management trials

Drug	Year	Author	Sample heterogeneity	Malapropos intervention mechanism of action	Malapropos intervention dose	Malapropos timing of outcome measurement	Confounding variables	Lack of valid and reliable measurement	Lack of statistical validity	Study design
Prevention (n=17)
Acetyl-L-carnitine	2013	Hershman						x	x
Alpha lipoic acid	2014	Guo					x		x
Amifostine	1996	Kemp					x	x	x
Amifostine	2003	Lorusso					x	x	x
Calcium/magnesium	2008	Grothey				x	x	x	x
Calcium/magnesium	2010	Ishibashi					x	x	x
Calcium/magnesium	2011	Grothey					x	x	x
Calcium/magnesium	2013	Gobran					x	x
Calcium/magnesium	2013	Loprinzi						x
Glutathione	1995	Cascinu					x	x
Glutathione	1997	Smyth					x	x
Glutathione	2002	Cascinu					x	x
Glutathione	2013	Leal				x		x	x
Goshajinkigan	2015	Oki					x	x	x
Venlafaxine	2015	Zimmerman					x	x	x
Vitamin E	2010	Pace					x		x
Vitamin E	2011	Kottschade						x	x
Management (n=7)
Duloxetine	2013	Smith
Gabapentin	2007	Rao	x		x		x	x	x
Lamotrigine	2008	Rao					x	x	x
Nortriptyline	2002	Hammack		x				x
Topical BAK	2011	Barton	x	x			x
Topical AK	2014	Gewandter	x	x		x
Venlafaxine	2012	Durand							x

BAK: Baclofen amitriptyline ketamine, AK: Amitriptyline ketamine

Results

The database search provided 1199 records. After duplicates were removed and additional records were identified by hand-searching, 1108 abstracts were screened. After full-text review, 24 Phase III trials were selected. Figure 1 presents a diagram of the article selection process.

Figure 1.

PRISMA 2009 Flow Diagram

Table 3 lists the 24 randomized, placebo-controlled, double-blind, Phase III trials (17 prevention and 7 treatment) that had tested 14 different agents for CIPN in adults. The prevention trials tested antioxidants (and an herbal supplement), an ion channel blocker, and a tricyclic antidepressant. The treatment trials tested gabapentinoids, serotonin-norepinephrine reuptake inhibitors, antiepileptics, and topical amitriptyline/ketamine-containing agents. Nine prevention and two treatment trials demonstrated a significant treatment effect on the primary outcome; however, 22 studies (16 prevention and 6 treatment) were considered to have a high risk of bias because of two or more identified threats to validity. Table 4 summarizes the findings and limitations by indication (prevention or treatment), then by agent.

Table 3.

Chemotherapy-induced peripheral neuropathy Phase III prevention and treatment evidence

Year	Author	Design	Type of study	Population	Drug and dosage	Measurement tool	Measurement time points	Results
2013	Hershman	Phase III, randomized, double-blind, placebo-controlled, multicenter	Preventative (n=409)	Stage I-III breast cancer patients receiving taxanes; stratified based on chemotherapy regimen	Acetyl-L-carnitine 3000 mg daily for 24 weeks	FACT-Ntx NCI CTCAE v3.0	Baseline (before taxane); weeks 12, 24, 36, 52, 104	No difference in CIPN at 12 weeks using the 11 item neurotoxicity subscale of the FACT-taxane scale; CIPN was significantly increased at 24 weeks
2014	Guo	Phase III, randomized, double-blind, placebo-controlled, multicenter	Preventative (n=70)	Patients receiving cisplatin or oxaliplatin; stratified according to their exposure to platinums	Alpha-lipoic acid 600 mg daily three times a day for 24 weeks	FACT/GOG-Ntx; BPI score NCI CTCAE v3.0	Baseline, and then at 24, 36, and 48 weeks of treatment	No difference in FACT-NTX, BPI score, pain or functional testing at 24 weeks 71% attrition rate
1996	Kemp	Phase III, randomized, double-blind	Preventative (n=242)	Stage III-IV ovarian cancer patients receiving 100 mg/m2 cisplatin	Amifostine 910 mg/m2 reconstituted with 9.5 mL NS IV over 15 min before each chemotherapy infusion for 6 cycles of chemotherapy (every 3 weeks)	NCI CTCAE	Baseline, before cycles 4, 5, 6, and monthly for 3 months following completion of protocol	A statistically significant difference in the NCI CTCAE was demonstrated between the treatment arm and control arm by cycle 5 (P=0.15)
2003	Lorusso	Phase III, randomized, double-blind	Preventative (n=187)	Stage 1-4 ovarian cancer patients scheduled to receive carboplatinum and paclitaxel	Amifostine 910 mg/m2 reconstituted with 9.5 mL NS IV over 15 min before each chemotherapy infusion for 6 cycles of chemotherapy (every 3 weeks)	NCI CTCAE v2.0	Baseline, weekly, posttreatment	A statistically significant difference in the NCI CTCAE was demonstrated against severe neurotoxicity (Grade 3-4) (P=0.0.02)
2008	Grothey	Phase III 4-arm randomized, double-blind, placebo-controlled	Preventative (n=139)	Metastatic colorectal cancer patients receiving mFOLFOX7 (85 mg/m2 q2 weeks; CO) or mFOLFOX7 with and without oxaliplatin every round of 8 cycles (IO)	Calcium and magnesium 1 g of magnesium and calcium before and after each infusion	Unknown	Unknown	Study aborted due to errant concern regarding detrimental effects of calcium/magnesium. Preliminary results were positive
2010	Ishibashi	Phase III, randomized, double-blind, placebo-controlled	Preventative (n=33)	Metastatic colorectal cancer patients receiving mFOLFOX6 (85 mg/m2 every 2 weeks)	Calcium and magnesium 850 mg calcium gluconate and 720 mg magnesium sulfate in 100 mL dextrose 5% water infused over 15 min before and after oxaliplatin	NCI CTCAE v3.0 DEB-NTS	Base (before oxaliplatin); with each cycle of oxaliplatin and after completion of 6 cycles	There was no difference in the NCI CTCAE v3.0 and DEB Neurotoxicity Scale (DEB-NTS) after the completion of 6 cycles
2011	Grothey	Phase III, randomized, double-blind, placebo-controlled, 4-arm study	Preventative (n=102)	Stage II or III colon cancer patients scheduled to receive FOLFOX4 or mFOLFOX6 (85 mg/m2 q2 weeks) × 6 months; stratified by age, sex, chemotherapy regimen	Calcium and magnesium 1 g of magnesium and calcium in 100 mL dextrose 5% water infused over 30 min before and after chemotherapy compared to before chemotherapy only	NCI CTCAE v3.0	Base; q2 weeks (prior to each cycle); 18 weeks	A statistically significant difference was demonstrated in the percentage of patients with Grade 2 or greater chronic sensory neurotoxicity based on the NCI CTCAE v3.0 (P=0.038) and the oxaliplatin-specific scale (P=0.018) during treatment or at the completion of treatment
2013	Gobran	Phase III, randomized, double-blind, placebo-controlled	Preventative (n=60)	Colorectal cancer patients scheduled to receive an oxaliplatin-based regimen (85 mg/m2)	Calcium and magnesium 1 g of magnesium and calcium in 250 mL IV fluid infused over 30 min before and after oxaliplatin infusion	NCI CTCAE v3.0	Baseline; within 5 days of each chemotherapy cycle; monthly postchemotherapy completion for those who had developed CIPN	No statistically significant difference was demonstrated at the completion of treatment based on the NCI CTCAE v3.0
2013	Loprinzi	Phase III, randomized, double-blind, placebo-controlled, 4-arm study	Preventative (n=353)	Colorectal cancer patients receiving adjuvant FOLFOX or mFOLFOX 85 mg/m2 every 2 weeks for 6 months (12 cycles)	Calcium and magnesium 1 g of magnesium and calcium in 100 mL dextrose 5% water infused over 30 min before and after chemotherapy compared to before chemotherapy only	EORTC QLQ-CIPN20	Baseline (likely prior to first cycle); q2 weeks (before each cycle of chemotherapy); acute symptoms were monitored before each FOLFOX dose and 5 consecutive days after	No statistically significant differences at the completion of 12 cycles using the EORTC QLQ-CIPN 20
2013	Smith	Phase III, randomized, double-blind, placebo-controlled, multicenter, cross-over	Treatment (n=220)	Cancer patients with Grade 1 or higher NCI-CTCAE sensory neuropathy with CIPN pain 4/10 or higher. Patients with diabetes, PVD, and stable analgesic regimens allowed	Duloxetine 60 mg daily×5 weeks (30 mg daily for 1 week - then 30 mg twice daily for 4 weeks) followed by 2 weeks washout period between duloxetine and placebo	BPI-SF; FACT/GOG-Ntx	Baseline; weekly; 6 weeks (end of Phase I), 8 weeks (after wash-out), 13 weeks (after Phase II)	There was a statistically significant decrease in the pain score in the duloxetine group as measured by the brief pain inventory short form compared to those receiving placebo at 6 weeks (P=0.003)
2007	Rao	Phase III, randomized, double-blind, placebo-controlled cross-over	Treatment (n=115)	Cancer patients with average daily pain scores of either (1) >4/10 on NRS or (2) >1 on the 0-3 ENS. Currently receiving neurotoxic chemotherapy (stratified by chemo type) or posttreatment	Gabapentin 300 mg daily increased over 3 weeks to maximum dose of 2700 mg for 3 weeks (6 weeks treatment each phase); 2 weeks washout between study phases	NRS BPI-SF; 24 h average pain on NRS; ENS	Primary - base; weekly. Secondary - base; 6, 8, and 14 weeks	No difference in pain or CIPN scores measured by the NRS and the ENS at 6 and 14 weeks
1995	Cascinu	Phase III, randomized, double-blind, placebo-controlled	Preventative (n=50)	Stage III-IV gastric cancer patients receiving cisplatin (40 mg/m2 weekly)	Glutathione 1.5 g/m2 in 100 mL normal saline IV over 15 min before each weekly chemo infusion and 600 mg intermuscularly on days 2-5 after each infusion	WHO grading scale	Baseline (before cisplatin); after 9 and 15 weeks of cisplatin tx	A statistically significant difference was demonstrated in the glutathione arm at 9 and 15 weeks based on the WHO neurotoxicity scale (P=0.0001)
1997	Smyth	Phase III, randomized, double-blind, placebo-controlled multi-center	Preventative (n=151)	Stage I-IV ovarian cancer patients receiving cisplatin (100 mg/m2 q3 weeks×6 cycles)	Glutathione3 g/m2 in 200 mL normal saline infused over 20 min before chemotherapy infusion every 3 weeks	NCI CTCAE Nerve conduction studies	Baseline (before cisplatin); after 3 and 6 cycles	No difference was demonstrated in CIPN at the completion of 6 cycles based on the NCI CTCAE
2002	Cascinu	Phase III, randomized, double-blind, placebo-controlled	Preventative (n=52)	Colorectal cancer patients receiving 100 mg/m2 (high dose) oxaliplatin every 2 weeks	Glutathione1.5 g/m2 IV over 15 min before each infusion	NCI CTCAE; neurological examination; nerve conduction studies	Base (before oxaliplatin); after cycles 4, 8, and 12	A statistically significant difference was detected in the glutathione arm after 8 and 12 cycles based on the NCI CTCAE. (P=0.003 and P=0.004, respectively)
2013	Leal	Phase III, randomized, double-blind, placebo-controlled multicenter	Preventative (n=122)	Cancer patients receiving paclitaxel (150-200 mg/m2)/carboplatin (AUC 5-7) q3-4 weeks or paclitaxel 80 mg/m2 weekly for 12 weeks (mixed regimens - no stratification but subgroup analyses)	Glutathione1.5 g/m2 IV over 15 min prior to chemotherapy, starting their first or second cycle	EORTC QLQ-CIPN20; NCI CTCAE	Base (before chemotherapy); 1 week after each cycle; within 6 cycles of tx	No difference in CIPN measured by the EORTC QLQ-CIPN20 sensory subscale and the NCI CTCAE v4.0 after 6 cycles; increased time to development of CIPN favored the placebo group
2015	Oki	Phase III, randomized, double-blind, placebo-controlled	Preventative (n=182)	Stage III colorectal cancer patients receiving mFOLFOX6	Goshajinkigan7.5 g/day orally before or in between meals starting on the first day of mFOLFOX6; Stopped after 12 cycles (~26 weeks)	NCI CTCAE v3.0 DEB-NTS	Base (before first chemotherapy cycle); 12th cycle of chemotherapy (24 weeks)	The incidence of Grade 2 or greater neurotoxicity based on the NCI CTCAE v3.0 was statistically significantly higher for the group receiving goshajinkigan (P=0.007)
2008	Rao	Phase III, randomized, double-blind, placebo-controlled	Treatment (n=131)	Cancer patients with CIPN >1 month duration that could be receiving chemotherapy or posttreatment with average daily pain >4/10 NRS or >1 ENS	Lamotrigine Escalating dosing until patient reaches max dose of 300 mg for 2 weeks then tapered off	NRS - average daily pain score; ECOG neuropathy scale	Baseline and weekly	No difference in pain as measured by the NRS and ENS at 10 weeks
2002	Hammack	Phase III, randomizeddouble-blind, placebo-controlled, cross-over	Treatment (n=51)	Cancer patient receiving cisplatin or posttreatment with painful CIPN >1 month; stratified by age, cumulative dose, severity of CIPN, and whether cisplatin administration was ongoing or completed	NortriptylineEscalating doses until patient reaches max dose of 100 mg daily	VAS; VDS	Base; weekly until 9 weeks (end of Phase II)	No significant differences were demonstrated in quality of life measures or symptoms affecting daily life
2011	Barton	Phase III, randomized, double-blind, placebo-controlled	Treatment (n=150)	Cancer patients with >1 month CIPN numbness, tingling, or pain and >4/10 pain severity only in hands or feet, currently or have received neurotoxic chemotherapy	Topical amitriptyline ketamine baclofen Apply gel twice daily for 4 weeks; 10 mg baclofen, 40 mg amitriptyline, 20 mg ketamine	EORTC QLQ-CIPN20; BPI; NCI CTCAE v3.0	Base (before intervention), 4 weeks follow-up	The motor neuropathy subscale had a significant effect size of 0.38 over placebo (P=0.021) measured by the EORTC QLQ-CIPN20. The sensory neuropathy subscale showed a trend favoring the intervention arm (P=0.053)
2014	Gewandter	Phase III, randomized double-blind, placebo-controlled, multicenter	Treatment (n=462)	Cancer patients posttreatment for 1 month with >4/10 pain, numbness, tingling over the past 24 h	Topical 4% amitriptyline and 2% ketamine Apply gel twice daily to areas with pain, numbness, or tingling; 40 mg amitriptyline, 20 mg ketamine	NRS	Daily diary using NRS 11 point scale rating pain numbness tingling starting 1 week before topical AK started and at week 3, 6 after enrollment	No significant treatment effect for numbness, tingling, or pain was noted at 6 weeks measured by the NRS
2011	Durand	Phase III, randomized, double-blind, placebo-controlled, multi-site	Treatment (n=42)	Cancer patients that reported “distressing” CIPN and still receiving oxaliplatin every 2 weeks	Venlafaxine50 mg 1 h before infusion and 37.5 mg extended release twice a day from day 2 to day 11 until the end of chemotherapy treatment	NPSI; oxaliplatin-specific Levi’s scale	Base; days 1-5 after each chemotherapy infusion; completion of chemotherapy; 3 months postchemotherapy completion	A significant treatment effect was noted in the proportion of patients experiencing a complete relief of acute neurotoxicity compared to placebo measured by the NPSI (P=0.03)
2015	Zimmerman	Phase III, randomized, double-blind, placebo-controlled, multisite	Preventative (n=48)	Stage II-IV colorectal cancer patients receiving adjuvant FOLFOX or mFOLFOX for 6 months (12 cycles)	Venlafaxine XR37.5 mg×twice daily started the 1st or 2nd weeks of chemotherapy until 1 week posttreatment	EORTC QLQ-CIPN20; NCI CTCAE v4.0Oxaliplatin acute symptom questionnaire	Base (likely before the 2nd cycle of chemotherapy); before each chemo infusion (oxali-acute sx questionnaire also filled out for 6 consecutive days beginning the day of the infusion); 1, 3, 6, and 12 months postchemotherapy completion	No significant treatment effect was noted between placebo arm and venlafaxine arm for sensory neuropathy measured by the EORTC QLQ-CIPN20
2010	Pace	Phase III, randomized, double-blind, placebo-controlled	Preventative (n=41)	Cancer patients with solid tumor malignancies scheduled to receive cisplatin	Vitamin E400 mg per day orally started 1-8 days before chemotherapy through 3 months after cisplatin completion	TNS; NCS	Base, after 3 cycles; after cisplatin completion; 1 month after cisplatin completion	The incidence of neuropathy was significantly lower in the Vitamin E group compared to the placebo group measured by the TNS after six cycles of cisplatin (P=0.01)
2011	Kottschade	Phase III, randomized, double-blind, placebo-controlled, multi-site	Preventative (n=189)	Cancer patients scheduled to receive taxanes or platinums (stratified by type of chemo, gender, and age)	Vitamin E300 mg twice daily starting within 4 days of first chemotherapy infusion, through 1 month postchemotherapy completion	NCI CTCAE v3.0	Base, before each chemotherapy cycle, and at 1 and 6 months follow-up	No significant treatment effect was noted in sensory neuropathy of the NCI CTCAE v3.0 between the treatment arm and the placebo arm

AUC: Area under the curve, BPI: Brief pain inventory, CIPN: Chemotherapy-induced peripheral neuropathy, EORTC QLQ-CIPN 20: European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire Chemotherapy-Induced Peripheral Neuropathy 20, DEB-NTS: Neurotoxicity criteria of DEBiopharm, ECOG ENS: Eastern Cooperative Oncology Group Neuropathy scale, FACT/GOG-Ntx: Functional assessment of cancer therapy/Gynecologic Oncology Group Neuropathy scale, mFOLFOX: Fluorouracil, leucovorin, and oxaliplatin, NCI CTCAE: National Cancer Institute Common Terminology Criteria Adverse Effects Scale, NCS: Nerve conduction study, NPSI: Neuropathic pain symptom inventory, NRS: Numeric rating scale, PRO: Patient-reported outcome measure, PVD: Peripheral vascular disease, TNS: Total neuropathy score, VAS: Visual analog scale, VDS: Visual descriptive scale, XR: Extended release, WHO: World Health Organization, VDS: Verbal descriptor scale

Table 4.

Summary of findings and limitations by agent

Agent	Results	Limitations
Prevention
Acetyl-L-carnitine (n=1)	The administration of acetyl-L-carnitine had no effect on CIPN severity in comparison to placebo 12 weeks following randomization. At 24 weeks following randomization, CIPN symptoms worsened in the group randomized to receive acetyl-L-carnitine[7]	These results were limited by small sample size, lack of a valid and reliable measurement tool, and heterogeneity of the chemotherapy regimen
Alpha-lipoic acid (n=1)	There were no differences in CIPN severity (FACT/GOG-Ntx and BPI) 24 weeks following study initiation between the group randomized to receive alpha-lipoic acid and the group randomized to receive placebo	The results were limited by small sample size (underpowered) and a high attrition rate in both the control and intervention groups (e.g., 71%). In addition, the statistical methods did not control for imbalances in the amount of neurotoxic chemotherapy received between groups[15]
Amifostine (n=2)	Two studies demonstrated that individuals randomized to receive amifostine experienced reduced CIPN incidence (NCI CTCAE) in comparison to individuals randomized to receive placebo[16,17	Results were limited by lack of a valid and reliable measurement tool, lack of control for confounding variables (i.e., DM, Vitamin B deficiencies, PAD), and small sample size
Calcium and magnesium (n=5)	Three studies demonstrated a neuroprotective effect of calcium and magnesium.[18,19,20] No difference in CIPN was found at the completion of 12 cycles of chemotherapy in two additional studies, using the EORTC QLQ-CIPN 20[8] and the NCI CTCAE.[18] One study[19] was aborted due to concern for detrimental effects of calcium and magnesium on tumor response based on the Concept study;[21] however, preliminary data indicated a neuroprotective effect. Later, this was found not to be the case[8]	Study results were limited by malapropos timing of the outcome measure,[19] heterogeneity of chemotherapy regimens,[19] and lack of control for cumulative oxaliplatin doses.[18] All studies except one[8] lacked control for confounding variables, all studies lacked a valid and reliable measurement tool, and three studies had small sample sizes[19,20,22]
Glutathione (n=4)	Two[23,24] of four[25,26] reviewed studies demonstrated that individuals randomized to receive glutathione experienced less CIPN in comparison to individuals randomized to receive placebo based on the WHO scale[24] and the NCI CTCAE.[23,25,26] Measurement time points varied between 6 and 12 cycles of chemotherapy	Results were limited by a lack of valid and reliable measurement tool in all four studies, lack of control for confounding variables,[23,24,25] suboptimal statistical validity,[26] and malapropos timing of the outcome measurement[26]
Goshajinkigan (n=1)	An interim analysis in a study authored by Oki et al.[27] revealed that randomization to goshajinkigan worsened CIPN incidence (NCI CTCAE) in comparison to randomization to placebo 24 weeks postbaseline	The results are limited by a small sample size, lack of a valid and reliable measurement tool, and lack of control for confounding variables
Venlafaxine XR (n=1)	There were no differences in CIPN incidence (NCI CTCAE and QLQ-CIPN 20) between individuals randomized to receive venlafaxine or placebo after 12 cycles of oxaliplatin[28]	Results were limited by a lack of control for confounding variables, lack of valid and reliable measurement tool, and small sample size (underpowered)
Vitamin E (n=2)	One study found the incidence of neuropathy (TNS) was significantly lower in the intervention group compared to the control group after six cycles of cisplatin.[29] On the other hand, an additional study revealed that there were no differences in CIPN incidence (NCI CTCAE) between the group randomized to receive Vitamin E and the group randomized to receive placebo during taxane/platinum chemotherapy receipt[30]	Results were limited due to underpowered statistical analyses,[29,30] lack of reliable and valid measurement tools,[30] and lack of control for confounding variables
Treatment
Duloxetine (n=1)	A study by Smith et al. demonstrated that duloxetine 60 mg/day was superior to placebo in reducing chronic painful CIPN symptom severity 5 weeks following randomization (BPI)[6]	Changes in concurrent analgesic medications were not assessed throughout the study thus findings could be the result of increased analgesic use. This trial completed an intent-to-treat analysis which statistically provides a conservative estimate of efficacy
Gabapentin (n=1)	Gabapentin (up to 2700 mg/day) was not superior to placebo in reducing CIPN symptom severity (ECOG ENS) in individuals who received taxanes, platinum compounds, and vinca alkaloids 6 and 14 weeks following study initiation, respectively	A major limitation of this trial was that the administered dose of gabapentin may have been inadequate.[31] Gabapentin has been shown to be effective to treat neuropathic pain symptoms in doses up to 3600 mg/day.[32] Another major limitation was that individuals were still receiving neurotoxic chemotherapy during the trial, which may have worsened CIPN severity and subsequently confounded the effect of gabapentin on CIPN severity. Results were also limited by lack of a valid and reliable measurement tool, small sample size, and high attrition rate (25% attrition in the treatment arm)
Lamotrigine (n=1)	Lamotrigine (escalating dose up to 300 mg/day) was not effective in reducing CIPN severity in patients receiving taxanes, platinum compounds, vinca alkaloids, or combination therapy based on the ECOG ENS at 10 weeks[33] for acute and chronic CIPN	Although patients had CIPN for over 1 month at baseline, patients could still receive chemotherapy throughout the study. Additionally, results were limited by a lack of a valid and reliable measurement tool, small sample size, and 46% attrition rate in the treatment arm
Nortriptyline (n=1)	Nortriptyline (escalating dose up to 100 mg/day) was not effective in treating patients for painful CIPN receiving cisplatin measured by a visual analog scale and visual descriptor scale at 4 weeks[34]	Results were limited by a malapropos mechanism of action, lack of a valid and reliable measurement tool, concurrent chemotherapy which may result in unstable CIPN, and insufficient washout period in a crossover design. Evidence suggests nortriptyline should be gradually tapered (decreasing dose) over several weeks to minimize withdrawal symptoms which can include muscle pain.[35] In addition, the eligibility criteria did not specify a minimum baseline pain (at least a four out of 10-pain severity score), which is essential for pain trials[36]
Topical 4% amitriptyline, 2% ketamine, and 1% baclofen (BAK) (n=1)	Randomization to receive topical baclofen 10 mg, amitriptyline 40 mg, and ketamine 20 mg (applied as one spoonful twice daily to affected areas) led to marginally significant improvements in CIPN severity (QLQ-CIPN 20) in comparison to placebo in patients receiving a variety of neurotoxic agents.[37] Study participants included patients who had received or were currently receiving neurotoxic chemotherapy that reported CIPN symptoms	Although BAK targets acute pain mechanisms, the primary outcome was nonspecific to pain, and patients were enrolled if they had numbness, tingling, or pain at baseline for any duration (some participants may have had chronic nonpainful CIPN). Results were limited by a lack of control for confounding variables and concomitant pain medications
Topical 4% amitriptyline and 2% ketamine (AK) (n=1)	Individuals with established CIPN symptoms (1 month postneurotoxic chemotherapy treatment) randomized to receive 6 weeks of topical amitriptyline 40 mg and ketamine 20 mg (applied to affected areas twice daily) experienced similar CIPN symptom severity in comparison to individuals randomized to receive placebo[38]	Results were limited by a malapropos mechanism of action and malapropos intervention dose. The trial focused on all symptoms of CIPN yet measured sensory CIPN over the past week with an NRS of mean pain, numbness, or tingling. Topical AK’s mechanism of action may also only be appropriate for treating acute painful CIPN instead of chronic painful CIPN. In clinical trials for polyneuropathy in diabetic and nondiabetic patients, amitriptyline 75 mg active substance over 4 weeks significantly reduced neuropathic pain.[39] A dose of 40 mg topically twice daily may not have reached a therapeutic level to reduce neuropathic pain
Venlafaxine (n=1)	Venlafaxine immediate (50 mg) and XR (37.5 mg twice daily for 10 days) was superior to placebo for the treatment of oxaliplatin-associated acute sensory CIPN (NPSI).[40] The primary endpoint was the percentage of patients with 100% relief of symptoms during venlafaxine treatment	A limitation of this trial was the poor enrollment rate. Investigators stopped the study before reaching their targeted number of patients because the venlafaxine capsules reached the expiration date

BPI: Brief Pain Inventory, CIPN: Chemotherapy-induced peripheral neuropathy, QLQ-CIPN 20: Quality of Life Questionnaire Chemotherapy-Induced Peripheral Neuropathy 20, EORTC QLQ-CIPN 20: European Organisation for Research and Treatment of Cancer QLQ-CIPN 20, ECOG ENS: Eastern Cooperative Oncology Group Neuropathy scale, FACT/GOG-Ntx: Functional Assessment of Cancer Therapy/Gynecologic Oncology Group Neuropathy scale, NCI CTCAE: National Cancer Institute Common Terminology Criteria Adverse Effects Scale, NPSI: Neuropathic Pain Symptom Inventory, NRS: Numeric rating scale, TNS: Total neuropathy score, XR: Extended release, DM: Diabetes mellitus, PAD: Peripheral arterial disease, WHO: World Health Organization, BAK: Baclofen amitriptyline ketamine, AK: Amitriptyline ketamine

Prevention trials

The most common threats to validity in CIPN prevention trials were lack of valid and reliable measurement (n = 15), confounding variables (n = 13), and suboptimal statistical validity (n = 12). Specifically, only one prevention study utilized both clinical assessment and a patient-reported outcome (PRO) measure with strong psychometric properties.[29] Three studies[18,19,20] used either a CIPN clinical examination or PRO with adequate validity and reliability. Physician-graded (the NCI-CTCAE or WHO) scales were the primary CIPN measure in nine studies.[16,17,18,22,23,24,27,30,41] Eligibility criteria were not reported in four studies,[16,22,25,29] and various studies lacked control for peripheral neuropathy-associated comorbidities, chemotherapy regimen and dose received,[17,24,27,28,30] previous receipt of chemotherapy,[17,19,29,30] and concomitant analgesics/psychotropics/neuroleptics.[16,17,19,22,23,24,25,27,29] Finally, several studies may have utilized an inadequate drug dosage[22,30] or a drug that mechanistically would possibly not lead to meaningful benefits in the outcome.[23,30]

Treatment trials

Three of the CIPN treatment trials may have been biased by lack of valid and reliable measurement,[31,33,34] malapropos intervention's mechanism of action and dose,[31,37,38] confounding variables,[31,33,37] sample heterogeneity,[31,37,38] and/or suboptimal statistical validity.[34,37] The primary threats that could have diluted the observed treatment effects were associated with CIPN instability (coasting effects) and low baseline CIPN severity (lack of room for improvement). Only one study addressed these potential threats.[6] One study may have utilized an inadequate drug dosage,[31] and three studies tested a drug that mechanistically would possibly not lead to meaningful benefits in the outcome.[37,38,40]

Table 5 provides a comparison between the recommendations of the ASCO Clinical Guidelines and of this review based on the evaluation of the Phase III trial threats to validity.

Table 5.

Recommendations for further testing of pharmacological agents for chemotherapy-induced peripheral neuropathy prevention or treatment: Comparison to American Society of Clinical Oncology clinical guidelines

ASCO recommendations	Alternative recommendations
CIPN prevention: No further testing due to lack of efficacy or harmful side effects
Acetyl-L-carnitine	Acetyl-L-carnitine
Amifostine	Amifostine
Calcium/magnesium	Calcium/magnesium
Glutathione
Vitamin E
CIPN prevention: Agents recommended for further testing
Goshajinkigan	Alpha-lipoic acid
Venlafaxine	Glutathione
	Goshajinkigan
	Venlafaxine
	Vitamin E
CIPN treatment: No further testing due to lack of efficacy or harmful side effects
Lamotrigine	Topical AK for acute CIPN
CIPN treatment: Agents recommended for further testing
Amitriptyline	Amitriptyline
Gabapentin	Duloxetine
Nortriptyline	Gabapentin
Topical BAK	Nortriptyline
	Lamotrigine
	Topical BAK
	Venlafaxine

CIPN: Chemotherapy-induced peripheral neuropathy, BAK: Baclofen amitriptyline ketamine, AK: Amitriptyline ketamine, ASCO: American Society of Clinical Oncology

Discussion

This systematic review described the threats to validity of Phase III clinical trials that tested pharmacological agents for CIPN management. Three of the 24 trials reviewed had a low risk of bias.[6,8,40] The remaining studies were compromised by at least two threats to their validity: most commonly, measurement flaws, confounding factors, malapropos intervention's mechanism of action and dosage, inadequate sample size, recruitment, and retention.

Consistent with previous literature, our review suggests that the primary limitation among Phase III CIPN management trials is the use of CIPN measures that lacked sufficient reliability and validity.[42,43,44,45] Specifically, the capability to detect clinically significant changes between groups may have been limited by the use of physician-graded scales – the NCI-CTCAE, WHO, and ECOG scales – as the primary outcome measures (used in 1/3 of reviewed studies). Physician-graded scales are known to lack reliability and sensitivity[46,47,48] and often demonstrate floor effects.[49,50] In treatment trials measuring painful CIPN, the primary outcome measure should assess pain. Consistent with a review by Gewandter et al.,[51] the duloxetine trial by Smith et al.[6] was the only study that used a measure consistent with the primary pain outcome.[51] Further, the lack of a gold standard, reliable, and valid CIPN measure has made comparison among CIPN clinical trials difficult. Ideally, CIPN should be measured using a PRO survey and objective measures of physical findings (e.g., deep tendon reflexes and vibration sensation). The EORTC QLQ-CIPN20 and the Functional Assessment of Cancer Therapy/Gynecologic Oncology Group–Peripheral Neuropathy scale[52] are the examples of PRO measures with demonstrated reliability and validity, sensitivity, and responsiveness that could be used to improve measurement validity in the future CIPN trials. The total neuropathy score (TNS) is an example of an objective measure with demonstrated reliability, validity, sensitivity, and responsiveness that could be used. Moreover, when pain is the primary outcome,[6,31,33,38] a validated pain measure should be used, such as the Brief Pain Inventory-Short Form.[53]

The second most frequent threat to validity was lack of control for confounding factors. Numerous disease processes (e.g., alcoholism, diabetes mellitus, and Vitamin B deficiencies) and pharmacological agents can cause peripheral nerve damage. Heterogeneous chemotherapy regimens also lead to varying CIPN symptoms and severity through varying mechanisms. Even chemotherapies of the same-drug class may vary in presentation. For example, oxaliplatin alone (unique from the other platinum-based chemotherapies: cisplatin and carboplatin) may cause both chronic CIPN and acute transient effects of cold-induced or temperature-evoked dysesthesia.[54] Exclusion criteria or statistical analysis should be used to control for these confounding factors.

The third most common threat to validity was associated with malapropos intervention's mechanism of action and dose. The intervention's mechanism of action did not match that of the nerve cell injury underlying the CIPN manifestations. For example, the pathophysiologic mechanisms underlying acute CIPN are peripheral nerve cell injury, whereas chronic painful CIPN is caused by central nervous system plasticity. Thus, central-acting interventions may treat chronic painful CIPN but would not be expected to prevent CIPN or treat acute CIPN due to peripheral nerve damage.[51] In addition, the intervention must be administered for an adequate amount of time to become efficacious; the primary treatment end points should be measured at the time point when a therapeutic effect would be expected based on previous CIPN trials. Rao[31] evaluated gabapentin for the treatment of CIPN. Patients received gabapentin (300 mg capsules) incrementally over 3 weeks to a maximum dose of 2700 mg daily, which was maintained for 3 weeks. Then, patients had a 2-week washout period before switching to the placebo arm. However, evidence from diabetic neuropathy treatment trials suggest that at least 2 months of gabapentin treatment is required before assessing efficacy.[55] In this example, timing of the primary end point measurement may have been too soon, resulting in insignificant results. In clinical practice, providers prescribe gabapentin for CIPN and titrate the dose to the desired effect.

Many trials exhibited high attrition rates (>50%)[15,20,23,29] which lowers the statistical power of a study. Low power results in effect size estimates being less precise; thus, the researchers may incorrectly conclude that there is no effect demonstrated between the intervention group and the control group. High attrition rates may be the result of poor intervention efficacy, other therapy-related toxicities, or disease progression. Three studies[22,27,40] had low enrollment rates due to restrictive exclusion criteria that attempted to control for confounding factors which can result in increased risk for Type I errors (i.e., failure to detect no difference) and Type II errors (i.e., failure to detect a treatment effect that truly exists). Finally, inadequate sample size may have biased the results of 10 studies.[15,20,22,23,27,28,29,34,37,40]

As presented in Table 4, no further testing is recommended of ALC due to findings of worsening CIPN in the intervention group[7] and of calcium/magnesium based on three clinical trials demonstrating no effect for the prevention of CIPN.[8,20,22] Amifostine is not recommended for further testing due to side effect profile of the drug which includes hypotension.[16,17] The clinical trial evaluating alpha-lipoic acid for the prevention of CIPN would have been strengthened with the addition of an objective measure such as the TNS to identify subclinical findings of CIPN in the control group, thus showing an effect in the prevention of CIPN.[15] Vitamin E was shown to be effective in the prevention of CIPN with a valid and reliable measurement tool.[29] However, a later study investigating Vitamin E showed no effect for the prevention of CIPN but used a less valid and reliable tool; thus, further testing would be beneficial.[30] In addition, glutathione should be retested for the prevention of CIPN using a valid and reliable measurement tool that can identify subclinical CIPN. In agreement with the ASCO Clinical Guideline recommendations, venlafaxine and goshajinkigan should be further tested for the prevention of CIPN.

For the treatment of acute CIPN, topical amitriptyline and ketamine should not be retested based on the mechanism of action. Concordant with the ASCO Clinical Guidelines recommendations, gabapentin, nortriptyline, and topical BAK should be retested for the treatment of CIPN. To date, there are no Phase III clinical studies evaluating oral amitriptyline. As suggested in the ASCO Clinical Guidelines, oral amitriptyline should be evaluated based on its efficacy in the treatment of polyneuropathy in diabetic and nondiabetic patients.[39] The ASCO Clinical Guidelines suggest no further testing of lamotrigine for the treatment and venlafaxine for the prevention of CIPN. However, this review suggests that lamotrigine should be retested for the treatment of painful CIPN using a valid and reliable measurement tool such as the EORTC CIPN20 or the FACT-GOG-NTX. Venlafaxine should be retested for acute painful CIPN using a valid and reliable measurement tool with a study design that can increase enrollment rates to demonstrate statistical validity.

Limitations

We analyzed articles describing the trials for CIPN; thus, our results relied on the detail of the authors’ study documentation. Lack of documentation was interpreted as a negative finding. Although evidence-based, the CIPN-specific critical appraisal criteria were developed by the authors and may not be comprehensive. Finally, the two researchers who evaluated the risks of bias for this review were not blinded to the study authors.

Implications for practice or research

The quality of studies included in a systematic review is important to consider when deciding whether review findings should guide practice and guidelines. This review conveys the complex challenges researchers face when designing Phase III CIPN trials. Despite the rigorous designs of Phase III CIPN clinical trials (e.g., randomization, double-blinding, and placebo-controlling), clinicians should carefully evaluate CIPN intervention trials for threats to validity before implementing changes in protocols or order sets. Only strong and consistent evidence should be used to inform clinical practice. This review can aid clinicians and scholars in identifying design flaws, analysis, or reporting of Phase III CIPN clinical trials.

Financial support and sponsorship

This study was funded by Jonas Nurse Scholar Program 2016–2018 cohort; Jonas Center for Nursing and Veterans Healthcare; Predoctoral Fellowship from the Rita and Alex Hillman Foundation.

Conflicts of interest

There are no conflicts of interest.