Validation of the Composite Autonomic Symptom Scale-31 (COMPASS-31) in patients with and without Small-fiber Polyneuropathy

Abstract

Background

The recently developed composite autonomic symptom score-31 (COMPASS-31) is a questionnaire for assessing symptoms of dysautonomia. It was distilled from the well established autonomic symptom profile questionnaire. COMPASS-31 has not yet been externally validated. To do so, we assessed its psychometric properties and its convergent validity in patients with or without objective diagnosis of small fiber polyneuropathy (SFPN).

Methods

The internal validity and reliability of COMPASS-31 were assessed in participants with or without SFPN spanning the full autonomic symptoms severity. Convergent validity was assessed by comparing results of the COMPASS-31 and the gold standard autonomic function testing (AFT) which measures cardiovagal, adrenergic, and sudomotor functions. Additionally, relationships between COMPASS-31 and the Short Form McGill pain questionnaire, Short Form Health Survey and a 0-10 numeric pain scale were assessed. COMPASS-31 and all other questionnaires results were compared between patients with or without evidence of SFPN, objectively confirmed by distal-leg PGP9.5-immunolabeled skin biopsy.

Results

Among 66 participants (28 SFPN+, 38 SFPN-), COMPASS-31 total scores had excellent internal validity (Cronbach's α =0.919), test-retest reliability (r_s=0.886; p<0.001), and good convergent validity (r_s=0.474; p<0.001). COMPASS-31 scores differed between subjects with or without SFPN (Z=−3.296, p<0.001), and demonstrated fair diagnostic accuracy. Area under the receiver operating characteristic curve was 0.749 (P =0.01, 95% confidence interval 0.627-0.871).

Conclusions

COMPASS-31 has good psychometric properties in the population of patients being evaluated for SFPN and thus it might be useful as an initial screening tool for the more expensive SFPN objective tests.

Introduction

Assessing functioning of the autonomic nervous system is important in evaluation for common neurological disorders, including polyneuropathy and Parkinson's disease. Dysautonomia can cause multiple symptoms, including low blood pressure when standing (orthostatic hypotension), tachycardia, upper and lower gastrointestinal complaints, pupil and sweating abnormalities, and bladder and sexual dysfunction [1].

Objective diagnosis and quantification of dysautonomia is currently best accomplished (the “gold-standard”) by conducting a set of autonomic function tests (AFT), developed and refined primarily at Mayo Clinic, long the epicenter of this field. AFT includes measuring heart-rate variability in response to deep breathing and the valsalva maneuver, hemodynamic responses to 80° head-up tilt test, and 4-site acetylcholine-evoked quantitative sweat production [2]. AFT results are quantified using the composite autonomic severity score scale (CASS), that includes sub-scores for cardiovagal, adrenergic and sudomotor functions adding up to a 0 to 10 total scale score [2].

AFT requires expensive, non-portable equipment, operator training and patient preparations. It is available only at selected university centers, so symptom-based questionnaires are important surrogate. In 1999, Prof. Phillip Low and colleagues at Mayo Clinic developed the 169-item autonomic symptom profile (ASP) questionnaire [3]. To improve clinical utility, 85 questions from ASP were carefully selected to produce the composite autonomic symptom score (COMPASS). Although improved, COMPASS also was not widely adopted; it is time-consuming for patients and has a complex scoring system that requires training. To further increase applicability, Singer and Mayo Clinic colleagues further refined COMPASS to a 31-item, easily scored, questionnaire named COMPASS-31 (4) (available in supplementary material S1, online only). COMPASS-31 quantifies 6 domains: Orthostatic intolerance, vasomotor, secretomotor, gastrointestinal, bladder and pupillomotor. The 6 subscales sum to a total COMPASS-31 score of 0 to 100 (scoring directions are available in supplementary material S2, online only). We are unaware of previous validations of COMPASS-31.

Small-fiber polyneuropathy (SFPN) is a polyneuropathy that exclusively or preferentially affects the small unmyelinated or thinly myelinated peripheral axons that serve nociception and autonomic function [5]. Dysfunction of small fibers causes autonomic and sensory abnormalities and SFPN patients typically report mixed of sensory and autonomic symptoms. Sensory symptoms can include reduced sensation, pain and/or itch. Autonomic symptoms include increased or decreased sweating, perfusion manifest as changed skin color or temperature, sexual dysfunction, orthostatic hypotension and gastrointestinal dysmotility [5]. SFPN is challenging to diagnose since the motor and reflex examinations remain normal and standard nerve-conduction and electromyography testing are insensitive. Neurodiagnostic skin biopsy from the distal leg is the only objective diagnostic test for SFPN recommended by the American Academy of Neurology and the European Federation of Neurological Societies [6,7].

SFPN is a common cause of dysautonomia; it is frequently undiagnosed and disabling illness. Autonomic symptoms are characteristic of small-fiber mediated chronic pain syndromes, since the term “small fiber” encompasses both the sympathetic and nociceptive axons within peripheral nerves. Thus COMPASS-31 might serve as a screening tool for SFPN. We therefore assessed its internal validity, reliability and construct validity plus its ability to discriminate between participants diagnosed with, or without SFPN as confirmed by BIOPSIES .

Methods

Study Design and Subjects

All procedures and protocols had been approved by the hospital's institutional review board. Inclusion criteria comprised age at least 16 years and having completed AFT and biopsy within the last 12 months or being willing to undergo them. Both sexes and all races were eligible for inclusion. Exclusion criteria comprised inability to give informed consent, and any contraindication to AFT or biopsy. A sample size of at least 50 is recommended for validation studies [8], so we planned to recruit 60 subjects.

Participants were recruited from among patients and research subjects of the peripheral-nerve group at Massachusetts General Hospital between June 2013 and April 2014. No specific condition or diagnosis was required for inclusion and all eligible participants were invited to participate by telephone, email, or in person. Participants were paid $10 for completing questionnaires on paper or via the internet and $50 for undergoing AFT or skin biopsy. Data were managed using Harvard Medical School's secure data capture (REDCap) platform [9], and accuracy of data entry was verified.

Primary Outcome

The composite autonomic symptom score (COMPASS 31)

COMPASS-31 was completed twice at two week intervals and scored as recommended [4]. COMPASS-31 total score range between 0 to 100, with high values representing severe symptoms. The first COMPASS-31 score was used in all analyses and the second COMPASS-31score was used only for test re-test reliability assessment.

Comparators

Autonomic Functioning Test

AFT was performed using standard clinical diagnostic methods and equipment (WR Medical Electronics, Stillwater, MN) as previously published [10,11]. Subjects were directed to not wear compressive clothing and not to smoke, eat, or consume alcohol or caffeine for 4 hours preceding testing. Their medications were reviewed and potentially interfering medications were held for 24-48 hours before testing. We measured heart-rate variability to deep breathing (6/minute while supine) and Valsalva maneuver, hemodynamic responses to 80° head-up tilt for 10 minutes, and acetylcholine evoked sweat production. AFT total and sub-scores, with high values representing severe symptoms, were quantified using the standard, validated CASS [2]. This a validated 10-point scale of autonomic function that includes three subscales (adrenergic 4 points; sudomotor and cardiovagal 3 points each). There are different scales for the sexes and different age groups. Scores of 7-10 are interpreted as indicating severe autonomic failure; score ≤3 indicate mild dysfunction and intermediate scores indicated moderate autonomic symptoms.

Numeric pain scale (NPS)

Subjects scored their average pain in the past 24 hours on a 0-10 numeric pain scale (NPS), where 0 represented “no pain” and 100 represented the “worst pain one can imagine” [12].

The short-form McGill pain questionnaire (SF-MPQ-2)

This recommended secondary outcome pain measure [12;13] was completed by all subjects. Total scores and 4 subscores (continuous pain, intermittent pain, neuropathic pain and affective descriptors) were calculated. SF-MPQ-2 score range between 0-10, with high values representing severe symptoms.

Short Form Health Survey (SF-36)

This recommended secondary outcome pain measure [12], was completed by all subjects and total scores plus physical- and mental-health sub-scores were calculated [14]. SF-36 scores range between 0-100, with low values representing low health-related quality of life.

Small fiber polyneuropathy (SFPN) diagnosis by neurodiagnostic skin biopsy

All biopsies were removed from the standard site on the distal leg 10 cm above the lateral malleolus using local anesthesia, as previously published [11]. PGP9.5-immunolabeled epidermal neurites were measured in a blinded manner by experienced personnel in a Joint-Commission accredited clinical diagnostic laboratory using best recommended practices and interpretation. The standard clinical diagnostic criteria for SFPN diagnosis, epidermal nerve fiber (ENF) density <5th centile of predicted laboratory norms were applied to categorize participants as having or not having SFPN [6,7]. Our laboratory's biopsy norms used for interpretation come from a series of 351 screened healthy subjects, 175 males and 176 females, aged 8 through 92 years (mean 40.1 ± 18 years) (See abstract- Klein, MM, et al., Annals of Neurology 76[suppl 18], S69. 2014).

Data acquisition and analysis

Analyses were conducted using the SPSS for Windows version 19 statistical package (SPSS, Inc., Chicago, IL). Data distributions were assessed by the Shapiro-Wilk test and all variables were found to be not normally distributed (p<0.05), therefore non-parametric analyses were used and central tendencies were reported accordingly. COMPASS-31 internal validity was assessed by Cronbach's α. Correlations were assessed by Spearman's correlations. Participants were dichotomized into two groups: ‘participants with SFPN’ (skin biopsy percentile results below the 5^th centile) and ‘participants without SFPN’ (those with neurites densities above the 5^th centile). Differences in all dependent variables between participants with and without SFPN diagnosis were assessed by using the Mann-Whitney U test. Receiver operating characteristic (ROC) analysis was conducted with SFPN diagnosis as the state variable and COMPASS-31 as the test variable. Bonferroni corrections for multiple comparisons were independently applied.

Results

Participants

Sixty-six participants (46 females) met inclusion criteria and completed the first set of questionnaires. Their mean age was 42.2 ± 13.1 years, range 16 to 69. Forty-eight of 66 participants (72.8%) reported somatosensory symptoms and 59/66 (89.4%) reported autonomic symptoms. Two weeks later, 58 completed the second COMPASS-31 for the test/retest reliability assessment. Table 1 presents the COMPASS-31 scores.

Table 1.

COMPASS-31 scores and Cronbach α values of the entire group (n=66)

COMPASS 31	Mean±Std	Median	Interquartile range	Range	Cronbach α
Total score	30.2±23.2	30.2	9.55-42.3	0-95.8	0.919
Orthostatic intolerance	14±12.4	16	0-24	0-36	0.898
Vasomotor	1.6±2.1	0	0-4.17	0-5	0.869
Secretomotor	5.4±6.9	3.21	0-8.57	0-34.3	0.246
Gastrointestinal	6.6±5.4	5.36	1.79-9.82	0-24.1	0.885
Bladder	1±1.6	0	0-1.11	0-6.7	0.598
Pupillomotor	1.7±1.4	1.67	0-2.75	0-4.3	0.910

Reliability of COMPASS-31

There was excellent test-retest reliability (r_s=0.886, p<0.001) and internal validity, with Cronbach's α of 0.919 for the entire questionnaire. The domains for orthostatic intolerance, vasomotor, gastrointestinal and pupillomotor function had excellent to good internal validities (Cronbach's α 0.869-0.910). The bladder domain had low internal validity (0.598) and the secretomotor domain demonstrated unacceptable internal validity (0.246). Deleting question 8 (sweat related item) improved the validity of the secretomotor domain to “good” (0.731), and the Cronbach's α of the total score to 0.942.

Validity of COMPASS-31

Subjects’ AFT results as quantified by the CASS are presented in Table 2. Twenty-seven percent of subjects scored 0, meaning they had no objective evidence of dysautonomia. Forty-three percent had mild (CASS score = 1-3), 24% had moderate (total CASS score = 4-6), and 6% had severe dysautonomia (CASS=7). This cohort thus effectively spanned the full range of symptom severity.

Table 2.

Result of objective autonomic function testing as measured by the AFT and scored by the CASS (n=66)

AFT domains (CASS)	Mean±Std	Median	Interquartile range	Range
Total score	2.6±2.2	3	0-4	0-7
Cardiovagal	0.2±0.5	0	0-0	0-2
Adrenergic	1.2±1.4	0	0-3	0-4
Sudomotor	1.2±1.5	0	0-3	0-4

Table 3 demonstrates correlations between COMPASS-31 total and sub scores and the AFT. Among all AFT domains, the sudomotor had the strongest relations with COMPASS-31 total and subscales scores.

Table 3.

Correlation between COMPASS-31 and results of objective AFT as scored by CASS (n=66)

COMPASS 31	Autonomic function testing (CASS score)
	Total score	Cardiovagal	Adrenergic	Sudomotor
Total score	0.474*	−0.103	0.148	0.608*
Orthostatic intolerance	0.433*	−0.101	0.150	0.558*
Vasomotor	0.379*	−0.024	0.040	0.551*
Secretomotor	0.327	−0.051	0.096	0.392*
Gastrointestinal	0.353	−0.020	0.050	0.476*
Bladder	0.172	−0.073	−0.063	0.350
Pupillomotor	0.246	−0.238	−0.001	0.465*

Spearman's correlations results. P values below 0.0017 are considered significant and marked with * based on Bonferroni corrections for 28 comparisons.

Table 4 summarizes relationships between COMPASS-31 and the SF-MPQ-2, the SF-36, and participant's average pain scores in the past 24 hours. COMPASS-31 (both the total score and subscales) were strongly correlated with them all.

Table 4.

Correlations between COMPASS-31 and measurements of pain and quality of life (n=66)

SF-MPQ-2	COMPASS-31 total score
Total score	0.815*
Continues pain	0.784*
Intermittent pain	0.719*
Neuropathic pain	0.737*
Affective descriptors	0.743*
SF-36
Total score	−0.754*
PCS	−0.734*
MSC	−0.744*
NPS	0.622*

P values below 0.0055 are considered significant and marked with * based on Bonferroni corrections for 9 comparisons. SF-MPQ-2- The short-form McGill pain questionnaire; SF-36- Short Form Health Survey; NPS- Numeric pain scale.

Comparing COMPASS-31 results between participants with or without SFPN

For both groups together, the mean ENF density was 173± 120 neurites per mm² skin surface area (range 0-615). The mean centile was 21.3%±28.3, ranged 0%-95.68%. Twenty eight (42.4%) participants had skin biopsy diagnostic of SFPN and 38 (57.6%) were non diagnostic (see Table 5). Ten participants were identified as borderline SFPN, with neurites count between the 5^th to the 15^th centile. Given their low abundance they were tagged as without SFPN.

Table 5.

Differences in study measures between participants s with and without skin-biopsy diagnosis of SFPN (n=66)

	ENF density < 5% (SFPN)	ENF density > 5% (not SFPN)
COMPASS 31	Median (N=28)	Median (N=38)	U	Z	P value
Total score	38.8	19.6	278	−3.296	0.001*
Orthostatic intolerance	22.0	6.0	265.5	−3.571	<0.001*
Vasomotor	1.7	0.0	395.5	−2.041	0.041
Secretomotor	6.4	2.1	378.5	−2.048	0.041
Gastrointestinal	7.1	5.4	408	−1.613	0.107
Bladder	1.1	0.0	399	−1.93	0.054
Pupillomotor	2.2	1.0	329	−2.662	0.008
SF-36
Total score	33.1	77.9	205.5	−3.997	<0.001*
Physical health	27.5	82.2	184	−4.291	<0.001*
Mental Health	44.5	79.3	217	−3.840	<0.001*
SF-McGill-2
Total score	3.7	0.5	216	−4.104	<0.001*
Continuous pain	4.0	0.7	225	−3.994	<0.001*
Intermittent pain	2.7	0.0	270.5	−3.468	0.001*
Neuropathic pain	3.2	0.5	255.5	−3.610	<0.001*
Affective descriptors	3.0	0.0	137	−3.773	<0.001*
Pain score (NPS)	6.0	1.0	238.5	−3.443	0.001*
Autonomic Function Testing (CASS)
Total score	3	1.5	378.5	−2.031	0.046
Cardiovagal	0	0	475	−0.843	0.399
Adrenergic	0	0	487.5	−0.382	0.703
Sudomotor	2	0	351.5	−2.320	0.020

P values below 0.0025 are considered significant and marked with * (Bonferroni corrections for 20 comparisons). SF-MPQ-2- The short-form McGill pain questionnaire; SF-36- Short Form Health Survey; NPS- Numeric pain scale.

The diagnostic potential of COMPASS-31 for SFPN was evaluated by ROC analysis (figure 1). COMPASS-31 had fair accuracy with area under the curve = 0.749 (P=0.01; 95% confidence interval 0.627-0.871). The optimal cutoff for specificity based on minimal distance from the top-left corner of the ROC curve, is 30, with sensitivity of 0.741 and specificity of 0.730. To optimize screening performance (based on the curve coordinates), a cutoff of 10 in the total COMPASS-31 score had sensitivity of 0.926 and specificity of 0.378 for identifying SFPN skin-biopsy confirmed as defined by the biopsy results.

Figure 1.

Receiver operating characteristic plot of SFPN diagnosis performance by COMPASS-31

Figure 1 to be located near here

Discussion

The current study assessed the psychometric properties of COMPASS 31 and validated it in a cohort of patients and research subjects with a wide range of autonomic symptoms. The results reveal excellent internal validity and test-retest reliability for COMPASS-31, with a medium to strong convergent validity as compared with AFT, the “gold standard” for objective autonomic evaluation [15]. In addition, COMPASS-31 total scores significantly distinguished participants with from without SFPN, suggesting potential diagnostic utility in this population.

COMPASS 31 was published in December 2012 by Sletten et al. [4]. Their study of 405 healthy control subjects found internal consistency comparable to ours, with good to-excellent Cronbach α for the orthostatic, vasomotor, gastrointestinal and pupillomotor domains, but lower values for secretomotor and bladder domains. They did not evaluated how Cronbach α could be improved by item exclusion, but our analysis finds that removing question 8 (sweat changes in the past 5 years) improves internal consistency for both this domain and the total COMPASS-31 score, so we recommend consideration of this modification for other populations. The current study was the first to assess COMPASS-31 test-retest reliability which was excellent.

While good validity was demonstrated as compared with the total AFT score, the AFT sudomotor domain demonstrated the strongest correlation with the total COMPASS-31 score. This may relate to the sudomotor test properties and its scoring. Unlike the other AFT tests, sweat production is measured at four sites. A score calculated out of multiple body sites is likely to better characterize dysautonomia severity, and thus its stronger correlation with the COMPASS-31 is not unexpected. Interestingly, the performance of COMPASS-31 improves when the sweating item (question 8) is deleted, but the sudomotor domain of the AFT correlated best with the results of COMPASS-31. A likely explanation for this seeming discrepancy is that most patients remain unaware if they develop hypohydrosis on their limbs, but interpret the normal compensatory truncal hyperhydrosis as the abnormality.

Strengths of our study include the use of multiple comparators. Not only we compared the COMPASS-31 results with the AFT, we also assessed its relations with other pain related outcomes. The strong correlations between COMPASS-31 (which does not include pain-related items) and questionnaires assessing pain and health related quality of life support use of COMPASS-31 in SFPN evaluation in both research and clinical settings. Further supports for the latter claim are the differences observed between participants with or without skin biopsy diagnostic of SFPN in all studied measures. Another strength is that as far as we know, The SFMPQ-2, developed in 2009 [13], has not yet been validated in the contexts of polyneuropathy, thus our results can be regarded as its validation in SFPN. Limitations of the current study include a relatively small number of participants for the ROC analysis, and bigger studies would more accurately define the diagnostic accuracy of COMPASS-31 in the context of SFPN. In addition, our assumption is that the patients’ SFPN is the underlying cause of their disautonomia, however, we cannot exclude concomitant central causes such as neurodegenerative disorders. Fortunately these are extremely rare in cohorts of this age.

In summary, we report good psychometric properties for COMPASS-31 and it is valid to use in SFPN patients. Most patients with SFPN remain undiagnosed, and the current findings suggest that the COMPASS-31 should be further evaluated as a first, easy-to-use, and inexpensive screening tool for SFPN.