CONTENT VALIDITY OF SYMPTOM-BASED MEASURES FOR DIABETIC, CHEMOTHERAPY, AND HIV PERIPHERAL NEUROPATHY

Abstract

Introduction

No treatments for axonal peripheral neuropathy are approved by the United States Food and Drug Administration (FDA). Although patient- and clinician-reported outcomes are central to evaluating neuropathy symptoms, they can be difficult to assess accurately. The inability to identify efficacious treatments for peripheral neuropathies could be due to invalid or inadequate outcome measures.

Methods

This systematic review examined the content validity of symptom-based measures of diabetic peripheral neuropathy, HIV neuropathy, and chemotherapy-induced peripheral neuropathy.

Results

Use of all FDA-recommended methods to establish content validity was only reported for 2 of 18 measures. Multiple sensory and motor symptoms were included in measures for all 3 conditions; these included numbness, tingling, pain, allodynia, difficulty walking, and cramping. Autonomic symptoms were less frequently included.

Conclusions

Given significant overlap in symptoms between neuropathy etiologies, a measure with content validity for multiple neuropathies with supplemental disease-specific modules could be of great value in the development of disease-modifying treatments for peripheral neuropathies.

Even within a single etiology, peripheral neuropathy presents as a diverse array of sensory, motor, and autonomic symptoms of varying severity. The heterogeneity of these signs and symptoms increases when neuropathies of multiple etiologies are considered, therefore, peripheral neuropathy can be difficult to evaluate quantitatively in clinical trials. No treatments for axonal peripheral neuropathy are approved by the U.S. Food and Drug Administration (FDA) or Europe-an Medicines Agency (EMA), with the exception of tafamidis, which is approved for familial amyloid neuropathy, an uncommon polyneuropathy. The failure to identify disease-modifying treatments for peripheral neuropathy may be due in part to limitations of available outcome measures.^1–4

The FDA held a public workshop on “Clinical Development Programs for Disease-Modifying Agents for Peripheral Neuropathy” in February 2013 that included presentations and discussions of outcome measures that could be used to assess the efficacy of disease-modifying agents. There was general agreement that the evaluation of disease modification in peripheral neuropathy can be improved, and this review was undertaken to identify next steps to modify existing peripheral neuropathy measures or develop novel measures. Although many measures of peripheral neuropathy exist, no systematic evaluation of the content validity (i.e., the extent to which an instrument measures the concept of interest)⁵ has been performed for these measures. Published references outline well-specified criteria to demonstrate that a self-report measure has at least achieved minimal standards to be used in research, clinical practice, and clinical trials.⁶ The FDA has published a guidance that describes the process for developing and subsequently qualifying patient- or clinician-reported outcome (PRO or CRO) measures that outlines steps to be taken to satisfy the above-mentioned criteria. Although the guidance outlines a development process that would be used specifically as a basis for regulatory approval and labeling,^5,7 this guidance could also serve as a framework for developing valid and reliable measures for research outside the purview of the regulatory process. To ensure that a measure adequately captures the intended concept of use, the initial stages of development should include a comprehensive assessment of content validity in the intended population of use that is based on, review of available literature, expert input, and contributions from patients.⁵

The goals of this review are to summarize the published evidence of content validity for existing symptom measures in 3 prevalent conditions for which outcome measures have been developed and to compare and contrast the content and format of the items in disease-specific neuropathy measures. Achieving these goals will support the appropriate selection of currently available PRO and CRO measures of patient-reported symptoms. These data can serve as a foundation for development of novel symptom measures and refinement of existing tools for use in clinical trials in patients who have diverse peripheral neuropathies.

MATERIALS AND METHODS

Measures used to evaluate the symptoms of diabetic peripheral neuropathy (DPN), HIV neuropathy, and chemotherapy-induced peripheral neuropathy (CIPN) were identified by searching PubMed (search criteria: [neuropathy] AND [diabetes OR HIV OR chemotherapy] AND [measure OR scale OR patient reported outcome]) and by asking all authors to review the compiled list to determine whether there were any missing measures. The content validity of clinician-based assessments using objective measures of peripheral neuropathy, such as nerve conduction studies, quantitative sensory testing, autonomic testing, and nerve or skin biopsy and of observations made by others in patients who cannot respond for themselves (e.g., parents or caregivers) is beyond the scope of this review. The following 4 searches in PubMed were subsequently performed to identify publications that evaluated content validity for each of the measures identified using the aforementioned methods: (1) “measure name or nickname AND focus group”; (2) “measure name or nickname AND content validity”; (3) “measure name or nickname AND interview”, (4) “measure name or nickname AND development”.

The format used for each measure was identified as: (1) verbal descriptor scales, (2) dichotomous (yes / no) questions, (3) numeric rating scales, or (4) some combination. The body regions assessed by the measures were identified (lower extremity only, upper extremity only, both lower and upper extremities), and a determination was made as to whether they were assessed in the same questions (e.g., numbness in hands and feet is 1 question) or separate questions (e.g., numbness in the hands is 1 question, and numbness in the feet is another). In this review, we refer to the specific questions included in the measures as the “items” and the content that was covered within each item as the “construct.” The constructs of the items from each measure were conceptually categorized into 1 of 3 domains associated with peripheral neuropathy: sensory, motor, or autonomic. The sensory domain included positive and negative symptoms associated with pain, touch, and thermal perception. The motor domain included items that assessed weakness, cramping, or difficulty manipulating objects due to symptoms. The autonomic domain included items that assessed orthostatic intolerance, urogenital dysfunction, and gastrointestinal disturbances.

For the measures that separated the items into sensory, motor, or autonomic domains, the categorization of the original developers of the measure was followed when assigning items to these domains. For measures that did not specify domains, or whose domains were different than the 3 that were adopted for this review, we categorized the items into the 3 organizing domains. Items that were worded differently, but appeared to cover similar constructs, were grouped together. For example, 1 measure⁸ assessed difficulty manipulating small objects and others (e.g., Shimozuma et al.⁹) listed specific activities such as fastening buttons or inserting contact lenses. These 2 sets of items were grouped into the same construct labeled “difficulty with manipulating small objects or fine finger movements”. Conversely, when 2 constructs were compounded in a single item, this item was characterized under both constructs. For example, 1 measure¹⁰ assessed numbness and tingling in a single question. This item was included under both the numbness and tingling constructs.

Some items included in the measures covered social participation or activities of daily living; these items were not included in this summary. These items occurred in 1 DPN measure¹¹ and 4 CIPN measures.^9,12–15 Venn diagrams (Figs. 1–3) were used to depict the overlap of content assessed by measures developed for each of the 3 conditions within each of the 3 domains, respectively. Only items that are assessed in at least 2 measures (regardless of condition) were included in the Venn diagrams. Supplementary Tables S1 and S2, available online, were used to present the individual items that occur in each measure, including those that appeared in only 1 measure.

FIGURE 1.

Sensory neuropathy items. Numbers in parentheses indicate the number of scales that contain the item. * Described as sharp, stabbing, shooting, lancinating, or electric shock-like pain in different scales. Note: difficulty hearing was classified as a sensory item because it is classified in the sensory neuropathy domain in the EORTC-CIPN 20.

FIGURE 3.

Autonomic neuropathy items. Numbers in parentheses indicate the number of scales that contain the item.

RESULTS

Content Validity

A total of 18 measures were identified by the search methods and author input (Table 1). Supplementary Table S1 presents the number of citations that were identified by the content validity literature search, the number subsequently excluded because they did not address content validity, and the number for each measure included in this review. We identified reports of research that investigated and described methods that provided support for the content validity of 11 of the 18 measures (Table 1). Of the 18 measures, 9, 6, and 2 were designed for and primarily used in CIPN, DPN, and HIV-neuropathy, respectively. The Total Neuropathy Score¹⁶ is commonly used for both DPN and CIPN, but it was included in the DPN category for purposes of the Venn diagrams because it was first developed for DPN patients (Figs. 1–3). Of the 6 measures developed for use in DPN, published information for 3 describe at least 1 method to establish content validity. We found no published information regarding content validity for the 2 measures developed specifically for HIV-neuropathy. Eight of the 9 measures developed for CIPN report at least 1 method to establish content validity (Table 1).

Table 1.

Measures

Measure	Method to establish content validity reported in the literature
Diabetic Peripheral Neuropathy
Neuropathy Symptom Score (NSS)28	None reported
Michigan Neuropathy Screening instrument (MNSI)29	None reported
Total Neuropathy Score (TNS)16	None reported
Toronto Clinical Neuropathy Score (TCNS)30	None reported
Neuropathy - and Foot Ulcer-Specific Quality of Life Measurement (NeuroQOL)17	Literature review Discussions with an expert panel Focus groups (4–6 people each) for a total of 47 patients and 15 neuropathy-free controls used to develop the items
Neuropathy Total Symptom Score-6 (NTSS-6)31	Literature review of previous instruments used to evaluate neuropathy and neuropathic pain Items reviewed by neurologists
Norfolk Quality of Life - Diabetic Neuropathy (Norfolk-DN)11	Review of over 1000 semi-structured clinical interviews with patients to develop items Clinical team evaluated the clarity of the questions and modifications were made in an iterative process
HIV-neuropathy
Subjective Peripheral Neuropathy Screen Questionnaire (SPNSQ)32	None reported
The Brief Peripheral Neuropathy Screen (BPNS)33	None reported
Chemotherapy-induced Peripheral Neuropathy
Peripheral Neuropathy Scale (PNS)13	Interviewed 2 cisplatin-induced chemotherapy patients and 2 physicians
Scale for Chemotherapy-Induced Long Term Neurotoxicity (SCIN)34	Literature from long term morbidity and interviews with an unspecified number of patients.
Functional Assessment of Cancer Therapy/Gynecologic Oncology Group -Neurotoxicity (FACT/GOG-Ntx) and - Taxane (FACT/GOG-Taxane)10	Input from 5 expert clinicians Input from 10 patients
National Cancer Institute - Common Toxicity Criteria (NCI-CTC)35	Workshops with experts, including clinicians, to modify the NCI-CTC are reported, but no specifics about modification of the NCI-CTC for neuropathy are provided.
European Organization of Research and Treatment of Cancer-Quality of Life Questionnaire - CIPN208	Literature review Suggestions from 15 health care professionals Structured interviews with 68 patients
Oxaliplatin-Associated Neuropathy Questionnaire (O-ANQ)36	None reported
Patient Neurotoxicity Questionnaire (PNQ)9,37	Discussions with physicians, nurses, and patients in support groups and the clinic
Chemotherapy-induced Peripheral Neuropathy Assessment Tool14,15	In-depth interviews with 14 CIPN patients (saturation of themes was reported) Input obtained on initial draft instrument from 5 experts (a medical oncologist, 2 doctorally trained registered nurses, and an oncology certified registered nurse) Experts were asked to rate individual items and comment on the overall comprehensiveness of the instrument and the clarity of the items Based on expert feedback: (1) wording changes were incorporated to improve item clarity; (2) items pertaining to interference with “relationships”, “enjoyment of life”, and “leisure activities” were added (3) redundant items were removed
Rasch-built Overall Disability Scale for patients with Chemotherapy-induced Peripheral Neuropathy (CIPN-R-ODS)12	Constructs selected from the ‘activity and participation’ section of the International Classification of Functioning, Disability and Health (ICF) Literature review of activity and participation outcome measures used in studies of peripheral neuropathy Consulted various handbooks of neurologic rating scales Rasch modeling to identify final set of items that were well targeted to the sample

If the number of experts or patients consulted is not indicated in the table, it was not reported in the publication.

Measure Format

The majority of the measures examined (67%) use verbal descriptor scale items (Table 2). Neuropathy symptoms are most commonly assessed in separate questions for both extremities (50%). Five measures (28%) assess neuropathy in only the lower extremities, and none focus solely on the upper extremities. The remaining measures either assess symptoms in both extremities within the same questions or do not indicate what extremity should be considered when reporting the symptoms (Table 3). Sixty-seven percent of measures group the items into domains or subscales. All of the measures contain items that were identified as sensory items in this review; all but 1 HIV-neuropathy measure includes items that were identified as motor items. Sixty-one percent of measures include items that were identified as those that address autonomic symptoms.

Table 2.

Summary of the question format used in peripheral neuropathy measures.

Scale	DPN	HIV-neuropathy	CIPN
Verbal Descriptor Scale	5/7	----	7/9
Y/N	1/7	1/2	----
Numeric Rating Scale	----	1/2	----
Combination	1/7	----	2/9

/ indicates “out of”.

Table 3.

Summary of body locations specified in the questions of the peripheral neuropathy measures.

Body location asked about in the questions	DPN	HIV-neuropathy	CIPN
Upper extremity only	----	----	----
Lower extremity only	3/7	2/2	----
Upper and lower extremities in the same items	----	----	1/9
Upper and lower extremities in separate items	3/7	----	6/9
No location specified for any items	1/7	----	2/9

/ indicates “out of”.

Commonalities and Differences in Content of the Measures

Venn diagrams are provided to depict the number of measures that assess each construct and the overlap of the constructs assessed among the 3 conditions [see Figs. 1, 2, and 3 for sensory, motor, and autonomic domains, respectively]. These diagrams also illustrate which constructs are evaluated in measures designed for 1, 2, or all 3 conditions. Numbness, tingling, pain, allodynia (pain from increased sensitivity to light touch from items such as bed covers or activities such as putting on gloves), altered warm and cold perception, difficulty feeling the feet (when walking), burning pain, sharp pain, and burning (not specified as painful) were the most common sensory symptoms and are evaluated in at least 1 measure for each of the 3 conditions. Difficulty or weakness when walking and cramping were the most common motor symptoms evaluated and are included in scales for all 3 conditions. Supplementary Table S2 contains all items included in the individual measures.

FIGURE 2.

Motor neuropathy items. Numbers in parentheses indicate the number of scales that contain the item.

DISCUSSION

We found that researchers reported using at least 1 of the methods for establishing content validity recommended by the FDA (literature reviews or expert or patient input) in 11 of the 18 identified instruments.^5,7 However, patient input, which is emphasized most strongly in the FDA guidance,⁵ was only obtained for 8 measures. The explicit use of all 3 sources of input was only reported for 2 measures.^8,17 Furthermore, although attempts to establish content validity were often mentioned in the Materials and Methods section of articles that described the development of PRO measures for peripheral neuropathy, the available literature does not clearly indicate that patients were consulted on the clarity of the final sets of items included in these measures. Finally, none of the measures were originally developed based on contributions from >1 disease-specific patient population, although several have subsequently been used in studies of additional conditions. This general low level of effort to establish content validity that adheres to current FDA standards suggests that existing measures may not have adequate assay sensitivity to detect modest, yet clinically relevant, disease-modifying treatment effects. Minor problems with outcome measures would not be likely to greatly limit the ability of a trial to detect the effect of a highly effective treatment. However, considering the current dearth of available disease-modifying treatments for axonal peripheral neuropathy, even treatments with modest effects would be valuable.

Our results suggest that it could be advantageous to use available measures that include the highest number of symptom constructs that were most commonly included in measures for all 3 conditions, especially when studying conditions for which no dedicated measures have been developed. Items classified as related to autonomic neuropathy were less common, and none of these items are included in at least 1 measure designed for each of the 3 conditions.

The frequent overlap in content between the measures designed specifically for the 3 different conditions (see Figs. 1–3) suggests that a single “generic” measure of peripheral neuropathy consisting of material in common accompanied by disease-specific modules could be a valuable approach. Combining a general module with disease-specific add-on modules is a potentially efficient method to approach FDA qualification of peripheral neuropathy outcome measures that prioritizes the importance of disease-specific measurements that are validated for the appropriate concepts of interest.

Eventual qualification and inclusion of such a measure in the FDA Clinical Outcome Assessment Compendium¹⁸ will require that the measure be validated as described in FDA guidances on development and qualification of PRO measures⁵ and drug development tools.⁷ For example, the constructs common to scales from all 3 conditions identified in this review could provide a starting point to establish a conceptual framework for peripheral neuropathy. Outlining the exact constructs to be included, and the optimum wording of the questions, in such a “generic” measure would need to be determined using input from focus groups or interviews with demographically and clinically diverse patients with peripheral neuropathy of each etiology for which the measure would be used in future studies. The constructs that were assessed in measures developed for only a single condition could serve as the basis for disease-specific modules; for example, various autonomic items have specifically been included in measures developed for CIPN (e.g., head rush upon standing) and DPN (e.g., problems with vaginal dryness and bloating/vomiting after meals).

The majority of the measures assess symptoms in the lower and upper extremities separately. Evaluation of both upper and lower extremities is important when assessing peripheral neuropathy, because symptoms are often worse in the lower extremities. Any future measure of peripheral neuropathy symptoms should incorporate the separate extremity format. Other formatting issues should be carefully considered, such as the scale used for the questions. The majority of the existing measures use a verbal descriptor scale. In considering the development of a new measure, patient acceptance and understanding of, and satisfaction with verbal descriptor scales should be examined in focus groups and compared with other formats including the 0–10 numeric rating scales that are commonly used to assess pain symptoms.¹⁹

Once a list of potential PRO items has been compiled, the set of questions can be administered in a pilot study of patients who experience peripheral neuropathy and can be evaluated using psychometric methods, such as factor analysis and item response theory (IRT) modeling, including Rasch analyses.^20,21 Content experts and psychometricians can refine the concept of interest by identifying the items that are most pertinent to a set of sample patients and create valid and reliable scales for assessing peripheral neuropathy.⁵ After implementation of these measure-refining methods, the refined content and instructions should be evaluated again by patients and experts for content validity. However, confirmation of content validity is only the first step in demonstrating measure validity. Longitudinal clinical validation studies should be conducted to evaluate the test–retest reliability, construct validity (including convergent and discriminant validity), and the responsiveness (sensitivity to change) of the PRO measure.^5,6 These methods should be considered in the development of new measures of general peripheral neuropathy that would be designed to address limitations of existing approaches and that could include condition-specific modules.

We have focused our systematic review on DPN, HIV-neuropathy, and CIPN, 3 prevalent distal, symmetric, sensorimotor axonal peripheral neuropathies. There are, of course, many other axonal polyneuropathies for which outcome measures have been developed, but we considered them to be beyond the scope of this review. One important example involves transthyretin familial amyloid polyneuropathy (TTR-FAP).²² The results of recent randomized clinical trials of tafamidis²³ and diflunisal²⁴ showed benefits in patients with TTR-FAP on ClinRO and PRO measures, for example, the Norfolk QOL-DN²⁵; in addition, Rasch-built symptom and disability measures have been developed for use in future clinical trials.²⁶ These recent studies of TTR-FAP have the potential to be used as models to address challenging issues in clinical trial design²⁷ and the development of novel PROs⁵ for disease-modifying treatments for the peripheral neuropathies we have discussed.

This study is limited in that we were only able to assess the published material regarding the content validity of the measures we reviewed. Further efforts may have actually been taken to establish content validity but are not reported in the literature. In addition, the content identified by overlap of items in the various measures is dependent on choices made by previous researchers. For this study, we did not conduct focus groups with experts and patients to identify symptoms that should be included in assessments of peripheral neuropathy as a basis for comparing existing measures and evaluating their content validity. However, inclusion of items by multiple independent researchers indicates that the content is likely important and, thus is a valuable starting point upon which to develop a new measure or modify an existing one. The protocol of our study, that is, following the categorization of the original developers of a measure in assigning an item to a domain, may have resulted in inclusion or categorization of items that some investigators might consider outside the concept or domain of interest. Qualitative research based on these results should be performed to determine which of the items should be retained in future, optimized measures. Finally, we did not review other types of validity, such as construct validity or responsiveness, which are important when evaluating the overall value of individual measures.

In summary, we reviewed and summarized published information on the content validity of existing measures of DPN, HIV-neuropathy, and CIPN and compared the specific symptom constructs assessed in each of the measures. By determining the overlap in their content, we identified the set of symptoms that have been considered as most important in the development of 18 different measures. This information can be used to help inform decisions regarding which of the existing measures to use in a particular context, and can also serve as a basis for determining what symptoms should be included in novel measures of peripheral neuropathy that show content validity across multiple etiologies, perhaps with disease specific modules. Publication of the results of future efforts to develop measures should report how the content validity of the instrument was addressed.

Abbreviations

ACTTION

Analgesic, Anesthetic, and Addiction Clinical Trials Translations, Innovations, Opportunities, and Networks

BPNS

Brief Peripheral Neuropathy Screen

CIPN

chemotherapy-induced peripheral neuropathy

CIPN20

European Organization of Research and Treatment of Cancer-Quality of Life Questionnaire

CRO

clinician-reported outcome

DPN

diabetic peripheral neuropathy

EMA

European Medicines Agency

FACT/GOG-Ntx

Functional Assessment of Cancer Therapy/Gynecologic Oncology Group - Neurotoxicity

FACT/GOG-Taxane

Functional Assessment of Cancer Therapy/Gynecologic Oncology Group - Taxane

FDA

Food and Drug Administration

MNSI

Michigan Neuropathy Screening instrument

NCI-CTC

National Cancer Institute - Common Toxicity Criteria

NeuroQOL

Neuropathy - and Foot Ulcer-Specific Quality of Life Measurement

Norfolk-DN

Norfolk Quality of Life - Diabetic Neuropathy

NSS

Neuropathy Symptom Score

NTSS-6

Neuropathy Total Symptom Score-6

O-ANQ

Oxaliplatin-Associated Neuropathy Questionnaire

PNQ

Patient Neurotoxicity Questionnaire

PNS

Peripheral Neuropathy Scale

PRO

patient-reported outcome

CIPN-R-ODS

Rasch-built Overall Disability Scale for patients with Chemotherapy-induced Peripheral Neuropathy

SCIN

Scale for Chemotherapy-Induced Long Term Neurotoxicity

SPNSQ

Subjective Peripheral Neuropathy Screen Questionnaire

TCNS

Toronto Clinical Neuropathy Score

TNS

Total Neuropathy Score

TTR-FAP

transthyretin familial amyloid polyneuropathy