Test-retest and inter-examiner reliability of a novel bedside quantitative sensory testing battery in postherpetic neuralgia patients.

Abstract

In health and disease, the somatosensory system has been interrogated with standardized research techniques, collectively referred to as quantitative sensory testing (QST). In neuropathic pain, QST has been used to characterize multiple sensory derangements. However, the use of QST outside the lab has been limited by several factors, including a lack of standardization, variability in procedural technique, and duration of testing that would be unacceptable for clinic. To address these shortcomings, the Neuropathic Pain Research Consortium (NPRC) designed an easy and low-cost “bedside” QST procedure. To test the hypothesis that this procedure would be clinically reliable over time and across different examiners, a multi-site, blinded study was performed in subjects with postherpetic neuralgia. Generally, agreement between two examiners and over two study visits with one examiner was high. Additionally, intraclass correlation coefficients and Kappa statistics calculated showed that the battery of QST tests included were highly reliable. Interestingly, mechanical modalities (light brush, pinprick, pressure, and vibration) showed the highest reliability. The least reliable modalities were cool (room temperature) and warmth (38°C). These data demonstrate that the NPRC beside QST protocol is reliable across examiner and over time, providing a validated QST tool for use in clinical practice and clinical trials.

Perspective

This blinded, multi-center trial in 32 patients with postherpetic neuralgia demonstrates bedside quantitative sensory testing is reliable and suitable as a clinical trial outcome. The novel bedside battery could be used in clinical trials or in clinical practice over time given the reliability data presented in this article.

Introduction

Quantitative sensory testing (QST) is a well-established method of psychophysical evaluation of the somatosensory system, based on measurements of responses to calibrated, graded innocuous or noxious stimuli^{2, 31, 39}.It has been used traditionally in the research setting to phenotype patients with a variety of pain disorders³⁹. QST yields valuable information about the functional status of the somatosensory system from peripheral receptor to the cortex, and it may be used to quantify and monitor the presence and severity of either positive sensory phenomena (such as amplified pain perception reported in allodynia or hyperalgesia) or negative phenomena (e.g., loss of function as in hypoesthesia or hypoalgesia)^{1, 44}.

The German Research Network on Neuropathic Pain (DFNS) has done pioneering work to standardize and validate a complete QST battery suitable for use in the laboratory setting³¹, and they have reported across a wide range of neuropathic pain syndromes in over 1000 patients that there are substantial abnormalities in sensory profiles measured with the DFNS battery²¹. The DFNS protocol has been used to “deep phenotype” a variety of clinical pain conditions^{5, 15, 27, 30, 43}.

However, despite some recently published studies^{5, 11, 12, 34, 40, 43, 45}, there is a general lack of broad application of systematic, comprehensive QST within the wider pain management community for clinical evaluations or as potential outcome measures in chronic pain analgesic trials. Many reasons have precluded the widespread use of QST, including the lack of standardization, heterogeneity of procedures, expense of equipment, long duration of some QST exams, and lack of availability of “bedside” QST tools which would test the full spectrum of peripheral nerve fibers involved in pain disorders³⁸. Recent efforts to address these issues show promise. A streamlined QST battery described by Freeman and colleagues¹² correlates well with self-reported neuropathic symptoms, although test-retest and inter-examiner reliability have not been reported. Timmerman and colleagues⁴¹ demonstrate agreement of a standardized bedside exam with a clinical assessment of neuropathic quality of pain in a heterogenous group of patients. Despite standardization, measures used were somewhat less quantitative than the DFNS protocol, and reliability metrics could not disentangle examiner and test-retest factors since different examiners tested subjects at different visits. For a bedside QST battery to be used as an outcome in clinical trials or as a clinical tool to assist in diagnosis, prognosis, treatment planning and monitoring treatment response, formal test-retest and inter-examiner reliability data is needed in a homogenous patient population.

To address these practice and research gaps, the Neuropathic Pain Research Consortium (NPRC) was created in North America. Our multicenter consortium sought to create a valid and reliable bedside QST exam for neuropathic pain that could be incorporated into routine neurological examinations and could be used as an outcome measure in neuropathic pain clinical trials. In this study, as a first step towards a validated, bedside QST procedure, we examined the inter-examiner reliability of bedside QST responses in patients with postherpetic neuralgia (PHN) tested by 2 examiners on the same day, as well as the test-retest stability of QST responses over 2 weeks.

Methods:

This was a test-retest reliability study conducted at 6 academic sites (Albany Medical College, Brigham and Women’s Hospital, Swedish Hospital in Seattle, University of California San Diego, University of Minnesota, University of Wisconsin) in 32 subjects with PHN to evaluate a standardized bedside QST assessment. This investigation was an initial step towards fully validating such a process for wider clinical and research use. All study procedures were approved by the Institutional Review Board at each study site and written informed consent was obtained from each subject.

“Bedside QST” Protocol Development:

The NPRC created a standardized testing manual according to testing procedures reported previously^{2, 44}, a set of clinical tools, and a certification process to ensure uniform application of the assessment protocol. A panel consisting of the NPRC investigators used a consensus process to agree upon the modalities of testing and specifics of the protocol. Testing modalities chosen showed relevance to neuropathic pain, as determined by prior studies and literature review^{2, 44}. As described below, this included a comprehensive set of sensory modalities. Developing protocol specifics, e.g. testing equipment, involved a consensus approach. The panel emphasized using low-cost, portable tools that could be employed in a variety of settings. We sought devices that were well manufactured with little variance between units, and that were likely to have reliable and reproducible properties when conducting the testing. Assessments included: symptom questionnaires, light brush, vibration, cool and warm sensation, pinprick, cold and heat pain, and pressure pain threshold. In general, stimuli of each modality were applied to the affected area and to the same (“control”) location on the opposite body side. The panel then conducted training and protocol refinement procedures over several months, using an iterative process in which results from pilot testing in patients with neuropathic pain at each of the investigators’ institutions were used to revise the testing and training procedures. The training process included a mandatory training video, and practice testing of the final protocol in patients, witnessed and critiqued by other NPRC members.

Then at a subsequent meeting, NPRC investigators from six centers participated in the certification process, which entailed conducting the final QST protocol in two patients with neuropathic pain, witnessed by two examiners who were NPRC panel members, who had to verify that the testing procedures were performed correctly. In addition, patients were interviewed after the exams to confirm that they were comfortable with the investigator doing the testing and that the exam technique seemed consistent between different investigators. One examiner candidate required remediation, which involved additional in-person training. Following this, the examiner candidate achieved certification on the same day. Overall, two examiners were certified per site (n=12), who were the examiners at each of the sites in the present study.

Table 1 describes each of the testing tools and the testing protocol. Full descriptions of the required properties of each testing modality for a valid bedside QST examination (such as thermal or punctate stimuli) have been previously published by the NPRC, and incorporated into the testing procedures^{2, 44}. In sum, threshold and supra-threshold testing was conducted using: light brush, pinprick, vibration, thermal (cold, cool, warm, and hot), and pressure. At the conclusion of the testing protocol the examiner rated the quality of the participation of the patient, such as attentiveness, cooperation, and appropriateness of responses. A testing session was considered complete and valid only if it had met each of these criteria. The total time required for the bedside QST battery was approximately 10–15 minutes. Although formal feedback was not obtained from subjects after battery development, an informal review of comments recorded by examiners showed no negative comments.

Table 1:

Description of the testing tools and measures.

Modality	Tool	Testing properties	Units of measure	Units of data analysis
Light brush	Soft brush	Flat tip, 15×5 mm (width × depth), innocuous	Sensation vs. control site	Ordinal - Kappa
Pinprick	Pin	40g sharpness test, suprathreshold	Sensation vs. control site	Ordinal - Kappa
Pressure	Algometer	Pain pressure threshold	Pounds force (lbf)	Continuous - ICC
Vibration	Rydel-Seiffer tuning fork	128 Hz; vibratory detection threshold	Sensation vs. control site & R-S score (0–8)	Ordinal - Kappa & Continuous - ICC
Warm	Heat probe	38°C, innocuous	Sensation vs. control site	Ordinal - Kappa
Hot	Heat probe	47°C, suprathreshold	Sensation vs. control site	Ordinal - Kappa
Cool	Tuning fork	22°C, innocuous	Sensation vs. control site	Ordinal - Kappa
Cold	Tuning fork in ice water	0°C −5°C, suprathreshold	Sensation vs. control site	Ordinal - Kappa

Enrollment and Testing Procedures:

At least 5 subjects with Post Herpetic Neuralgia (PHN) were enrolled at each of the six sites (n=32), after obtaining IRB approval from each institution and obtaining written informed consent from each subject. Subjects were included if a review of medical records indicated that they had a diagnosis of PHN confirmed by a pain medicine physician or a neurologist. The pain syndrome had to affect one side of the body predominately, such as unilateral thoracic pain, so that a control sensory testing site could be designated on the unaffected side of the body. Each study location sought a balanced gender distribution.

Symptom Questionnaires and Physical Assessment:

These were administered at the start of each of the two testing sessions, described below. The history and physical exam were only performed at the first session. The Neuropathic Pain Questionnaire was included to cross-validate with other neuropathic symptom instruments and to measure test-retest reliability. Other symptom questionnaires were administered for descriptive purposes only, and not for cross validation with each other or with the bedside exam protocol. All subjects completed questionnaires for the first visit. Only a subset of subjects completed questionnaires for the second visit (n=22).

Neuropathic Pain Questionnaire (NPQ):

This is a validated 12-item self-report questionnaire that includes symptom descriptors as well as measures of unpleasantness, coping, and provocative factors¹⁹. Following scoring, the NPQ differentiates between neuropathic and non-neuropathic pain based on the cut-off point of 0, where a score above 0 predicts neuropathic pain and below 0 predicts non-neuropathic pain. Test-retest reliability has not been previously reported for the NPQ.

Neuropathic Pain Symptom Inventory (NPSI):

The NPSI is a validated 12-item self-report questionnaire to evaluate the sensory symptoms associated with neuropathic pain, such as burning or tingling⁶. It has been translated into multiple languages and has shown sensitivity to change in treatment studies⁷. NPSI total scores were calculated by summing each pain descriptor severity score (0–10 for each descriptor, 10 descriptors; maximum total score is 100).

PainDETECT (PD):

PD is a 12-item, validated, self-report instrument rating the neuropathic qualities of any pain complaint^{4, 13}. PD was also developed to identify neuropathic pain, and it was determined to classify the likely presence or absence of a neuropathic pain component with a sensitivity and specificity of 80–90%.

Brief Pain Inventory—Short Form (BPI):

This is a widely used and validated patient self-assessment survey in pain medicine research for cancer and noncancer pain^{17, 35}. In addition to a pain body diagram, the other 14 items include ratings of pain intensity (now, average, worst, and least) and activity interference with pain.

Colored Pain Diagram:

This is a descriptive self-report tool for patients to color in the areas of their pain and describe its properties (such as aching, burning, stabbing, or numbness) using different colors for each sensation.

Pain Anxiety Symptom Scale (PASS, long form):

This 40-item scale has been used reliably to assess the level of pain-specific anxiety in pain patients^{22, 23}. It assesses cognitive and physiological symptoms of anxiety common in patients with pain, such as fear of movement or pain catastrophizing symptoms.

Center for Epidemiologic Studies Depression Scale (CESD):

20-item self-report scale designed to measure symptoms of depression including sadness, anhedonia, guilt, suicidal ideation, and somatic symptoms ^{8, 29}. The frequency of symptoms over the past week is scored on a 4-point Likert scale.

Linear Analog Self-Assessment (LASA):

LASA is a widely used, 12-item self-report measure of quality of life covering 11 domains²⁸. It has been used in cancer and non-cancer medical populations^{16, 37}.

History and Physical Exam:

Using a standardized history, examination, and recording form, at the start of the first testing session an NPRC investigator conducted a pain-focused history and review of medical records, a general physical exam, and a neurological exam focusing on the assessment of the pain complaints. As part of the exam, the primary pain site was identified, and the anatomic distribution of sensory abnormality was determined using a common clinical technique of mapping sensitivity to light touch and/or pinprick. The intent of this mapping step was to aid in the determination of whether the primary pain site was classified as neuropathic, musculoskeletal, myofascial, or visceral. In the examiner’s judgment, the primary pain site which was used for testing had to be neuropathic due to PHN (i.e. sensory abnormality in a dermatomal distribution as determined by sensory mapping) for the subject to continue. This is similar to methods used in other studies to select likely cases of neuropathic pain¹³. The primary testing site was the area of worst pain suspected to be neuropathic in nature based on review of the pain body diagram and the pain history. The control testing site was selected based on the following criteria: 1) no pain symptoms or sensory abnormality at that site, 2) diagonally away from most painful area (e.g. for lower extremity pain, the contralateral upper extremity was chosen; for facial pain, the contralateral forearm was chosen), 3) stimuli applied to the control site elicit expected sensations. If a candidate control testing site produced unexpected sensations, a different site was selected. For example, in subjects with unilateral thoracic dermatomal pain, the contralateral side was used as a candidate control testing site. If the candidate control testing site produced unexpected sensations, a site cephalad or caudad to that site would be interrogated until an area without sensory abnormality was found, which would then be used as the control testing site.

Testing Procedures:

Subjects participated in two examination sessions, separated by 2 weeks. At the first session, they completed the symptom questionnaires and history and physical exam. A certified NPRC examiner (Examiner A) then conducted the bedside QST evaluation. (Exam 1A), and within an hour they had a second, identical exam by a second NPRC examiner (Examiner B, Exam 1B). Examiner B was blinded to the results of Exam 1A and subjects were instructed not to discuss the results of the first exam with the second examiner. Prior to Exam 1B subjects had to confirm that their chronic pain had returned to its baseline, before Exam 1A, in case Exam 1A had flared up pain. At the conclusion of both exams, both examiners had to verify that blinding was maintained and that the exams were valid. Subjects then returned in 2 weeks for an identical exam by Examiner A (Exam 2A). Examiner A was instructed not to review the results of Exam 1A prior to Exam 2A. Throughout all 3 exams, both examiners were blinded to the results of the questionnaire assessments; in other words, these were not reviewed prior to the exams.

For each patient tested following protocol development, the following QST protocol was performed. Based on sites determined in the History and Physical portion, the control site was tested first followed by the test site for each modality. For light touch, pin prick, cool, cold, warm, and heat, the instrument was applied to the control site, and the subject was asked whether it felt as expected (e.g. “like a brush”). Then the instrument was applied to the test site, and subjects were asked to rate the perception as more, less, different, or the same as the control site. If rated as more, this was recorded as allodynia or hyperalgesia (depending on stimulus modality). If rated as less this was recorded as a deficit. If rated as different, this was recorded as such. The stimulus order was standardized as follows: 1) light brush, 2) vibration, 3) cool, 4) warm, 5) pinprick, 6) cold pain, 7) heat pain, 8) pressure pain. See below and Supplemental Figure 1 for testing and instrument details. For warm and heat pain testing, a custom, portable heat probe was used. The heat probe is hand-held and consists of a temperature-controlled, ~2.5 cm diameter, circular brass surface which is applied cutaneously (see Supplementary Figure 1). The temperature is set on the handle of the device by the operator and is maintained with a temperature-controlled internal heating element. Although unable to dynamically ramp temperatures like Peltier-thermode devices, the custom heat probe is significantly cheaper, costing approximately $250 per unit. The heat probe was custom developed and made by the University of Wisconsin Department of Bioengineering.

Light brush testing:

A Somedic SENSELab brush-05 (Sösdala, Sweden) was applied to the control and then test sites up to 5 times per site, and responses were classified as more (allodynia), less (deficit), different, or same.

Vibration testing:

A Rydel-Seiffer (R-S) 128 Hz tuning fork was applied to the control and test sites. At the control site, subjects were asked whether it felt “like normal vibration.” The R-S score was determined at the painful testing site by applying the tuning fork to the test site, asking subjects to report when vibration sensations were “completely gone” and then noting the number (0–8) at the intersection of the two triangles formed by the vibrating prongs. Following this test site application, subjects rated this sensation as more (allodynia), less (deficit), different, or the same. The tuning fork was applied with gentle skin contact avoiding boney prominences. For statistical analysis, the R-S score was treated as a continuous variable and classification of sensation intensity as an ordinal variable.

Cool testing:

The tuning fork at room temperature was applied to the control site and then test site. Each application consisted of approximately 3 seconds on the skin alternating between control and test site, with an interval of about 2–5 seconds between control and test applications. Test site sensations were rated as more (allodynia), less (deficit), different, or the same. These steps were repeated three times. The temperature of the tuning fork was maintained by alternating sides of the tuning fork between sites and waiting for the subject to assess the sensations prior to reapplication (>5 seconds). Preliminary testing showed no warming of the tuning fork with these parameters. Room temperature was selected as there is evidence of sensory abnormality in PHN in the room temperature range (20–22°C)^{25, 32}.

Warm testing:

The heat probe was heated to 38°C prior to testing and then applied to the skin at the control site and then the test site. As described above, the heat probe is temperature-controlled, so repeated application to the skin has no effect on the temperature of the heated brass surface that touches the skin. The probe was applied for approximately 2 seconds with 2 seconds between test and control sites. Test site sensations were rated as more (allodynia), less (deficit), different, or the same. Then, the process was repeated three times.

Pinprick testing:

A calibrated pinprick was applied using a Neuropen with a 40 g Neurotip (Owen Mumford; Marietta, Georgia) to control and test sites up to 5 times. Test site sensations were rated as more (hyperalgesia), less (deficit), different, or the same.

Cold pain testing:

A tuning fork that had been immersed in ice water for longer than 5 minutes was applied to the control site and then the test site three times using the same stimulus duration and interval as outlined in the cool testing section above. Test site sensations were rated as more (hyperalgesia), less (deficit), different, or the same.

Heat pain testing:

The custom heat probe described above was heated to 47°C and then applied to the control site and then test site three times. The stimulus was applied for 1 second with a minimum time between test and control site applications of 3 seconds. Test site sensations were rated as more (hyperalgesia), less (deficit), different, or the same.

Pressure pain testing:

An algometer (Wagner “Pain Test FPK20” device with 1 cm² rubber tip; Greenwich, Connecticut) was applied to the thumbnail bed contralateral to the test site (i.e. at a control site). Pain pressure threshold was determined by gradually increasing the pressure applied and having the subject report when the sensation became painful. The rate of rise of pressure was standardized to approximately 3 pounds of force per second. As part of the examiner training process, this rate of rise was practiced until it could be reproduced consistently.

Statistical Analysis:

Data were compiled and analyzed using Excel (Microsoft; Redmond, Washington) and SPSS (IBM; Armonk, New York). Demographic and questionnaire data that were continuous were evaluated for normality graphically with histograms and q-q plots. Shapiro-Wilk tests were also performed. One-way analysis of variance (ANOVA) and Chi square tests were used to summarize the demographic data and descriptive measures, and to compare the results between study sites. Pearson correlation coefficients were calculated for cross-validation of NPQ with NPSI. In exploratory analysis, changes in questionnaire responses between visits 1 and 2 were tested with paired t-test, Wilcoxon signed rank test, or Fischer’s exact test.

We tested inter-rater reliability for Exams 1A vs. 1B, and test-retest reproducibility for Exams 1A vs. 2A. We used intraclass correlation coefficients (ICCs) for the linear outcome measures (e.g., vibration score and pressure threshold)³⁶. To calculate ICC, a one-way random effect model was used, which addresses consistency (rather than absolute agreement) across observations. Coefficients range from 0 to 1, with higher values indicating higher reliability and reproducibility. Generally, coefficients between 0.40 and 0.59 show a fair level of clinical significance; coefficients between 0.60 and 0.74 show a good level of clinical significance; and coefficients between 0.74 and 1.00 show an excellent level of clinical significance⁹. A similar interpretation has been used to assess reliability and reproducibility of repeated QST in volunteers²⁴.

We used the Kappa statistic for the ordinal outcome measures (light brush, vibration sensation, cool, warm, pinprick, cold and heat pain). For ordinal measures, subjects rated the relative magnitude of sensation at the affected site in comparison to the control site (i.e., affected-site sensations were rated as “more,” “less,” “same,” or “different” with respect to sensation intensity at the control site). The Kappa statistic is a measure of the extent of agreement between 2 examiners or 2 different instances of the same exam²⁰. A score ≥.60 is an indicator of good agreement, a modality suitable for use as a standardized measure, similar to the interpretation of interclass correlation coefficients⁹. Unweighted Kappa values were calculated, since possible values were limited to four (“more,” “less,” “same,” or “different”) and an unweighted calculation would provide a more conservative estimate. We also calculated the percent of agreement between findings of Exam 1A vs. 1B and Exam 1A vs. 2A for the ordinal measures. This was done using cross-tabulation tables. The number of subjects with the same ordinal values on the cross-tabulation table (i.e. on the diagonal) was summed and divided by total subject number to yield percent agreement. Since the severity and quality of neuropathic symptoms vary from day to day in any individual, we did not necessarily expect Exams 1A and 2A to be entirely consistent. In addition, we performed comparisons between the six study sites for the reliability and reproducibility statistics using the Wilcoxon rank-sum test to determine the effects of study site.

Results:

The description of the study population and the questionnaire data from visit 1 are displayed in Table 2. Thirty-two subjects enrolled and completed the first visit. One subject did not return for visit 2. Gender was balanced in this study population. Most subjects reported pain in a thoracic dermatome (Table 2). Mean pain intensity was moderate to severe in most subjects, with 65.7% having a BPI pain severity score of greater than 4.0. Additionally, neuropathic symptoms questionnaires showed high scores and sensitivity for neuropathic pain, which is consistent with the patients’ known diagnosis of PHN. All continuous variables except the BPI pain interference score (Shapiro-Wilk p=0.007) and LASA total score (Shapiro-Wilk p=0.002) were normally distributed. PASS data were borderline non-normal by Shapiro-Wilk testing (p=0.039) but appeared normal on graphical analysis using q-q plots.

Table 2:

Patient demographics, pain history, and questionnaire data.

Variable	N=32 (unless noted)
Age (mean years ± sd)	64.2 ± 16.6
Gender (% male)	50%
Pain Location (% thoracic)	72%
Average BPI1 Pain Severity: 0–10 (mean ± sd)	4.9 ± 2.0
Average BPI1 Pain Interference: 0–10 (median, IQR)	2.6, 0.7–4.9
Neuropathic Pain (NPQ2, % yes)	72%
Neuropathic Pain Symptom Inventory (NPSI, mean ± sd)	40.2 ± 20.6
Neuropathic Pain (PainDETECT, % “Very Likely”, n=19)	32%
Neuropathic Pain (PainDETECT, % “Uncertain”, n=19)	47%
Depression (CESD3 scale, mean ± sd, n=31)	16.4 ± 7.5
Pain Anxiety Symptoms Scale (PASS, mean ± sd, n=27)	60.2 ± 37.0
Quality of life (LASA4 total score, median, IQR, n=31)	7.3, 5.3–8.1

¹⁾BPI = Brief Pain Inventory

²⁾NPQ = Neuropathic Pain Questionnaire

³⁾CESD = Center for Epidemiologic Studies Depression

⁴⁾LASA = Linear Analog Self-Assessment

Of note, for none of the descriptive measures was there an asymmetric trend between sites; meaning, there were no statistically significant or clinically meaningful differences between the average values of the measures between the study sites. Age and gender were similar across each study site, as follows: Brigham and Women’s Hospital (n=5) mean 64.2 years ± SD 13.2 years, 60% male; University of Minnesota (n=5) mean 71.0 years ± SD 11.8 years, 80% male; Albany Medical College (n=4) mean 68.0 years ± SD 17.0 years, 25% male; Swedish Hospital in Seattle (n=7) mean 81.1 years ± SD 8.7 years, 29% male; University of Wisconsin (n=6) mean 57.5 years ± SD 19.2 years, 83% male; University of California San Diego (n=5) mean 75.2 years ± SD 14.1 years, 20% male. Similarly, measured QST variables were no different between sites. There was no statistically significant effect of study site using one-way ANOVA or Chi-squared tests. Since there was no effect of study site on descriptive variables or QST measurements, study site was not included in subsequent statistical models.

Table 3 displays the intraclass correlation coefficients for the linear measures (vibration score with a tuning fork, 0–8 analog scale, and pressure threshold 0–10 kgs, analog scale). Comparing across examiners on the same day (Exams 1A vs. 1B) and over time with one examiner (1A vs. 2A), the intraclass correlation coefficients for vibration score and pressure threshold were all >0.60 (p<0.01), indicating good reliability. Inter-examiner pressure threshold ICC was 0.90, suggesting excellent reliability between examiners. ICCs between study sites were not significantly different from each other (Wilcoxon rank-sum, p>0.05).

Table 3:

Test-retest and inter-examiner reliability of continuous measures.

Modality	Exam 1A vs. 1B (n=32)	Exam 1A vs. 2A (n=31)
Vibration Score (CI)	.75* (.51, .87)	.67* (.29, .84)
Pressure Threshold (CI)	.69* (.34, .86)	.90* (.79, .95)

Intraclass correlation coefficients, used for linear measures with the upper and lower bounds of the 95% confidence interval in parentheses.

^*p values <.01.

Table 4 displays the Kappa statistic for the assessment of inter-examiner reliability for the ordinal measures between two examiners on the same test day (Exam 1A vs 1B) and across subsequent test days (Exam 1A vs 2A). The percent of agreement is also summarized. Overall, for the ordinal measures across all exams, the extent of inter-examiner and inter-session agreement for all modalities ranged from 60–87%, with light brush and pinprick having the greatest agreement. For exams 1A vs 1B, levels of agreement ranged from 63–87%. For exam 1A vs 1B, light touch, vibration, and pinprick demonstrate both high levels of agreement and Kappa levels, indicating a high degree of reliability. Kappa levels for painful heat or cold were borderline for reliability, while tests of innocuous thermal sensation (i.e. cool or warmth) had lower Kappa levels and were less reliable.

Table 4:

Test-retest and inter-examiner reliability of ordinal measures

Modality	Exam 1A vs. 1B1 (n=32)	% Inter-Examiner Agreement	Exam 1A vs. 2A1 (n=31)	% Test-Retest Agreement
Light Brush (std err)	.74* (.50, .98)	87%	.93* (.79, 1)	87%
Vibration (std err)	.58* (.33, .83)	74%	.40* (.15, .65)	60%
Cool (std err)	.42* (.17, .67)	66%	.74* (.52, .96)	84%
Warm (std err)	.40* (.13, .67)	63%	.59* (.34, .84)	74%
Pinprick (std err)	.63* (.39, .87)	81%	.55* (.31, .79)	80%
Cold (std err)	.53* (.29, .77)	69%	.52* (.27, .77)	71%
Heat (std err)	.55* (.30, .80)	72%	.47* (.22, .72)	68%
Mean (std err)	.55* (.30, .80)	73%	.60* (.35, .85)	75%

^1.Kappa statistic for extent of agreement for ordinal measures with the upper and lower bounds of the 95% confidence interval in parentheses.

^*p values <.01.

As a measure of test-retest reliability, for Exams 1A vs. 2A conducted by the same examiner, light brush, cool, and warm showed good reliability with Kappa levels ≥0.60. Similarly, pinprick demonstrated 80% agreement between exams with a Kappa=0.55. For exams 1A vs. 2A, levels of agreement ranged from 60–87%, with vibration sensation being the only modality < 70% agreement. Vibration and noxious heat were less reliable. Importantly, there were no significant differences in Kappas for any modality between study sites for Exams 1A vs 1B or Exams 1A vs 2A (Wilcoxon rank-sum tests, p>0.05)

Table 5 shows the test-retest reliability of neuropathic symptom surveys, including for NPQ, which has not previously been published. The NPQ scores from visit 1 and visit 2 are strongly correlated and have good reliability with ICC >0.6. Importantly, when the NPQ score was converted to the binomial variable predicting a neuropathic pain syndrome, there was good agreement and reliability, as measured by kappa statistic. Table 6 displays Pearson correlation coefficients between NPQ and NPSI data from visits 1 and 2. There is a moderate correlation between NPQ and NPSI data across multiple visits, providing cross-validation of the NPQ.

Table 5:

Test-Retest reliability of self-reported neuropathic symptom surveys.

Survey	Correlation (Pearson’s r)	ICC (95% CI)
NPQ (n=21)	0.794	0.802*** (0.570, 0.915)
NPSI (n=22)	0.919	0.916*** (0.811, 0.964)
painDETECT (n=11)	0.827	0.752*** (0.191, 0.931)
	% agreement	kappa (SE)
NPQ, %yes	85.7%	0.63** (0.19)
painDETECT, %very likely	63.6%	0.45* (0.22)

^*p<0.05

^**p<0.01

^***p<0.001

Table 6:

Correlations between NPQ and NPSI at study visits 1 and 2.

		NPQ		NPSI
		visit 1	visit 2	visit 1	visit 2
NPQ	visit 1	1	.794**	.554**	.601**
	visit 2		1	.501*	.697**
NPSI	visit 1			1	.914**
	visit 2				1

Pearson’s r for each pairwise correlation is shown.

^*p<0.05

^**p<0.01

One possible cause for imperfect test-retest reliability for the bedside QST battery is variation in neuropathic symptoms. Table 7 displays an exploratory analysis in which visit 1 and visit 2 questionnaire responses are tested for changes over time. Statistically significant, albeit small magnitude, decreases in painDETECT score and percentage of NPQ-positive patients were detected comparing visit 1 and visit 2. No statistically significant changes were detected in NPSI or NPQ scores. Additional dimensions of pain and factors that may affect pain testing also showed no statistically significant difference between visit 1 and 2.

Table 7:

Self-reported neuropathic symptoms are slightly reduced between study visit.

	visit 1 (mean ± SD)	visit 2 (mean ± SD)	p-value
Neuropathic pain, continuous
NPSI score (n=22)	37.14 ± 21.60	34.59 ± 21.87	0.187a
NPQ score (n=21)	0.62 ± 1.10	0.62 ± 1.10	0.951a
painDETECT score (n=11)	16.45 ± 4.57	14.27 ± 4.52	0.022*a
Neuropathic pain, ordinal
NPQ (n=21), % Yes	76.2%	71.4%	0.011*b
painDETECT (n=11), % Very likely	36.4%	18.2%	0.126b
painDETECT (n=11), % Unsure	45.5%	45.5%
Additional self-report measures, continuous
Pain Severity (BPI, n=22)	4.56 ± 1.85	4.34 ± 1.89	0.300a
Pain Interference (BPI, n=22)	2.90 ± 2.48	2.87 ± 2.41	0.907a
Pain Interference (BPI; median (IQR), n=22)	2.57 (0.82–5.02)	2.07 (0.82–5.18)	0.760c
Depression (CESD, n=22)	17.45 ± 6.88	17.50 ± 8.11	0.974a
Pain catastrophizing (PCS, n=10)	24.10 ± 16.22	24.90 ± 15.52	0.625a
Pain-related anxiety (PASS, n=19)	58.32 ± 36.2	54.11 ± 36.36	0.308a
Quality of life (LASA, n=19)	6.80 ± 1.41	6.82 ± 1.20	0.927a

^*p<0.05.

Statistical tests used include:

^a)paired t-test

^b)Fischer’s exact test

^c)Wilcoxon signed rank test. Fewer subjects completed surveys at visit 2, accounting for reduced numbers with both visit 1 and visit 2 data (subject number in parentheses).

Discussion

This multicenter study investigated whether quantitative sensory testing modalities traditionally used in the research setting can be adapted to general clinical practice or for use in large clinical trials in the assessment and study of neuropathic pain. We used an iterative process of protocol development, high-quality and common clinical examination devices, and a rigorous training and certification process for the examiners. We found that several modalities demonstrate high levels of inter-examiner reliability and test-retest reproducibility in the assessment of painful postherpetic neuralgia. In general, for all modalities tested by two examiners on the same day, or on different days by the same examiner, there was a high rate of agreement between exams. Reliability was greatest for mechanical modalities including light brush, vibration, pressure, and pinprick. Additionally, we provide data for good test-retest reliability of the NPQ and cross-validation of this instrument with NPSI. While replication in a larger sample size is needed, these findings suggest that the NPRC’s bedside QST protocol is suitable for use in clinical examinations or as an outcome measure and tracking tool in clinical trials of neuropathic pain treatments. The availability of an inexpensive, reliable, and temporally stable set of bedside QST tests may lead to broader use of QST across multiple settings.

As part of a thorough clinical examination in patients with neuropathic pain, practitioners often test for loss of sensory function (negative phenomena; e.g. decreased sensation to light touch) or gain of sensory function (positive phenomena; e.g. allodynia, hyperalgesia)². Assessment techniques, however, are extremely variable between practitioners; for example, testing responses to light touch ranges from the use of the handle of a reflex hammer to the practitioner’s fingers to a brush. Often modalities are assessed as being present or absent, without assessing threshold or duration. Some modalities are rarely tested (e.g. pressure threshold). Additionally, protocols for testing (e.g. standard number of repetitions of a test or establishing an order of test modality) are not routinely used in clinical practice. These qualities of routinely used clinical exam techniques likely conspire to decrease reliability across examiners and stability over time, which confounds reliable measures of treatment response (reviewed elsewhere^{1, 2}). In the current study, using tools and protocols known to be reliable in experimental settings, we demonstrate that a bedside QST protocol has high reliability across evaluators and over time, which is a considerable improvement over the current clinical examination.

Beside the variability of the clinical exam, some modalities are simply not routinely tested, such as response to pressure. Importantly, in the current study, pressure threshold was one of the most reliable across study visits of all modalities tested. Light brush, using a standardized brush stimulus, was also highly reliable over time. The use of a pressure algometer or standardized brush is rare in clinical practice. The current study demonstrates that these modalities have test-retest reliability in patients with PHN, laying the groundwork for future studies to include mechanical QST measures longitudinally that may provide additional prognostic or therapeutic information.

Given that incorporation of QST improves on the qualitative nature of a traditional bedside exam by providing quantitative measures, we would predict that inter-examiner reliability would benefit from incorporating quantitative measures. Despite rigorous training, a practitioner’s qualitative evaluation is likely shaped by exposure to patterns of findings within the population treated by the practitioner and order effects such that the last evaluation performed biases the current evaluation. We and others hypothesize that quantitative measures eliminate potential confounds and improve inter-examiner reliability^{1, 2}. Although this hypothesis is not directly tested in the current study, another study reported that inter-examiner bedside exam reliability in a heterogeneous group of chronic neuropathic pain patients was poor to moderate for each bedside exam outcome, despite at least some degree of standardized examiner training⁴¹. This contrasts with the current study and a DFNS protocol study¹⁴, in which most measures showed good to excellent inter-examiner reliability. It seems logical that training and standardization would improve reliability, independent of the use of QST tools. Therefore, the relative importance of standardized training and QST in improving reliability could be further explored by direct comparison between standardized bedside exams and the current bedside QST protocol in the same cohort.

It is likely that bedside QST will provide valuable information for both research and clinical practice. Using bedside tools for mechanical and cold QST, one study found distinct patterns of QST responses that occurred across neuropathic pain syndromes¹². This finding is exciting since these tools could be used to identify clustered phenotypes without requiring an exhaustive “deep phenotyping” used in other protocols, e.g. the DFNS protocol³. This has important ramifications for time-constrained activities such as subject recruitment for clinical trials or patient assessment in the clinic because different phenotypically-defined groups may respond to different treatments or have divergent disease courses. Phenotype-stratified trials which enrich for subjects with specific QST findings using the DFNS protocol have shown that phenotypic profiles determine treatment response^{10, 11}. Another study using algometry and mechanical modalities amenable to bedside testing found that QST responses predicted post-mastectomy pain and opioid use³³. Recently, Kramer and colleagues¹⁸ found that sensory profiling with the DFNS protocol may predict the development of PHN, highlighting the potential prognostic value of QST data in predicting progression from acute herpes zoster to PHN. Using bedside QST protocols such as the protocol described in the current study will be useful in research based on phenotypic stratification and in determining prognosis. The current study provides foundational support for this by demonstrating inter-examiner reliability. Future studies are needed to test whether these bedside QST batteries track with outcomes in clinical trials. The current study lays a groundwork for this by establishing test-retest reliability of bedside QST.

To create a streamlined QST protocol, certain procedures were intentionally not included. For example in PHN patients, the measured surface area of allodynia shows plasticity²⁶ and may track with treatment effects in PHN. Although sensory mapping was done to aid in establishing a clinical diagnosis of neuropathic pain, a measured surface area was not included in the final battery tested due to lack of a reliable method for transferring skin surface area to a digitized data format and due to time constraints. Although numerical rating scales (NRS) of allodynia severity in response to brush stimulation correlate with pain intensity in PHN³², that measure was not included in the current protocol. Instead, brush allodynia was interrogated using ordinal responses. Future studies could include allodynia NRS to track changes over time in addition to ordinal responses, which may be simpler for subjects, to optimize the protocol further. Similar decisions were made regarding temperature testing protocols, which avoided temperature ramping protocols delivered by Peltier-based thermode systems. Although exact temperature thresholds were not obtained, the current study demonstrates that reliable data could be generated to track temperature sensory abnormality with much cheaper instruments. Certainly, this approach has drawbacks, e.g. introducing a confound of concurrent mechanical stimulation by touching the skin with probe rather than warming an already-positioned probe via ramping. To deal with possible sensory contamination, patients were instructed and trained to distinguish touch and temperature sensory experiences. Future studies will clarify whether these drawbacks reduce performance of the battery through cross-validation in larger cohorts.

Moreover, given evidence that surface area quantification and allodynia severity are relevant in PHN^{26, 32}, it is possible that a bedside QST battery tailored to PHN patients should include these measures. This relates to a larger question of whether a single bedside QST battery will perform well across multiple etiologies for neuropathic pain given known heterogeneity within and across diagnoses^{42, 43}.

Despite significant strengths in providing test-retest and inter-examiner reliability data for a cost-effective bedside QST protocol, the current study has several limitations. Although this was a multisite study, limitations include a small subject number with a single diagnosis, PHN. Additional studies of larger groups of patients and with different neuropathic pain diagnoses besides PHN will be important in extending the current findings. Point estimates of agreement for some modalities have low confidence intervals. Future studies of larger groups will provide additional information regarding these point estimates. The current study also does not address whether these bedside QST measures are sensitive markers of treatment effects. The prognostic importance of this protocol is also unclear. Generation of normative data in age-matched controls and inclusion of the protocol in future clinical trials as a secondary outcome measure will be important first steps to address these issues. Although not directly tested, variation in neuropathic symptom severity over time likely contributes to test-retest variability in bedside QST. Exploratory analysis in this study supports this hypothesis, but additional investigation into this question is clearly needed. Similarly, a better understanding of the relationship between self-reported symptoms and bedside QST profiles is an important area of future research.

In conclusion, the current study shows that the NPRC bedside QST protocol is reliable across examiners and over time. This represents an important first step in establishing a QST protocol that is useful in clinical practice and as an outcome measure in clinical research.

Supplementary Material

1. Supplementary Figure 1: Tools utilized in the NPRC bedside QST battery.

From right to left, a brush, tuning fork, pinprick device, and custom-built heat probe were used. See Methods for additional details.

Highlights:

• Development of a comprehensive yet inexpensive and efficient testing battery.

• Battery is reliable in postherpetic neuralgia patients.

• Lays groundwork for use of bedside quantitative sensory testing in the clinic.