COMPASS 31 – A REFINED AND ABBREVIATED COMPOSITE AUTONOMIC SYMPTOM SCORE

Abstract

Objective

To develop a concise and statistically robust instrument to assess autonomic symptoms that provides clinically relevant scores of autonomic symptom severity, based on the well-established 169-item autonomic symptom profile (ASP) and its validated 85-question scoring instrument, known as composite autonomic symptom score (COMPASS).

Patients and Methods

We assessed the internal consistency of COMPASS using Cronbach alpha coefficients based on the ASP of 405 healthy control subjects recruited and seen in the Mayo Autonomic Disorders Center between March 1, 1995 and March 31, 2010. Applying a simplified scoring algorithm, we then used exploratory factor analysis with orthogonal rotation and Eigenvalue calculations to extract internally consistent domains and to reduce dimensionality. This was followed by expert revisions to eliminate redundant content and to retain clinically important questions, and final assessment of the new instrument.

Results

The new, simplified scoring algorithm alone resulted in higher Cronbach alpha values in all domains. Factor analysis revealed 7 domains with a total of 54 questions retained. Expert revisions resulted in further reduction of questions and domains with a remaining total of 31 questions in 6 domains (COMPASS 31). Measures of internal consistency were much improved compared to COMPASS. Following appropriate weighting, this instrument provides an autonomic symptom score from 0 to 100. Conclusion: COMPASS 31 is a refined, internally consistent, and markedly abbreviated quantitative measure of autonomic symptoms. It is based on the original ASP and COMPASS, applies a much simplified scoring algorithm, and is suitable for widespread use in autonomic research and practice.

The autonomic symptom profile (ASP) is a well-established questionnaire to comprehensively evaluate the severity and distribution of symptoms, and the autonomic functional capacity of patients with autonomic disorders, comprising 169 questions and assessing 11 domains of autonomic function.¹ It was first described by Suarez et al. in 1999, continues to be routinely used at Mayo Clinic and many other institutions around the world for a comprehensive assessment of autonomic symptoms and functions, and continues to serve this purpose very well.

Using a limited set of 85 clinically selected questions, we have used this questionnaire to generate a composite autonomic symptom score (COMPASS).¹ The questions comprising this instrument address eleven autonomic domains with 73 questions (orthostatic intolerance – 9 items, secretomotor – 8 items, male sexual dysfunction – 8 items, urinary – 3 items, gastrointestinal – 5 items, constipation – 4 items, diarrhea – 4 items, pupillomotor – 7 items, vasomotor – 11 items, reflex syncope – 5 items, and sleep – 8 items), and an additional 12 items to generate two validity scores (an understatement index – 6 questions and a psychosomatic index – 6 questions).¹ It has been validated and used extensively by our group and selected others.^1-7

However, over the years we have identified a number of problems with this instrument that have resulted in concerns about supporting this tool for use by other institutions in spite of broad national and international interest:

1. The scoring algorithm of COMPASS is highly complicated and requires computer analysis for score generation. The complexity and ambiguity of the extraction process has resulted in even experienced end users obtaining inconsistent scores.

2. Completion of the ASP, even if limited to the questions relevant for COMPASS, is time-consuming.

3. The questions within the domains assessed using COMPASS have not been evaluated for internal consistency.

4. A number of questions included in the original COMPASS score have over time been identified to be less meaningful or redundant for scoring autonomic function and symptom severity.

A redesign of this instrument to a simplified, more time-efficient, and statistically more robust, but still comprehensive tool to assess and grade symptoms relevant to autonomic function that can find broad application in research and clinical practice is therefore long needed.

The specific aims of this study were therefore:

1. To develop a simplified and more user friendly scoring scheme for questions that comprise COMPASS.

2. To critically assess current questions and domains and use state of the art statistical methods guided by clinical judgment to develop an updated, concise, and statistically robust stand-alone tool that provides a clinically relevant general score of autonomic symptom severity with meaningful subscores for individual autonomic domains.

Patients and Methods

Participants

Healthy control subjects (N=405), who were recruited and seen in the Autonomic Disorders Center between March 1995 and March 2010, were asked to complete the ASP after informed consent was obtained. The study was reviewed and approved by the Internal Review Board.

New Scoring Scheme

A new, simplified autonomic symptom scoring scheme was developed that follows a homogeneous pattern of scoring throughout the instrument. Simple yes/no questions were scored as “no” = 0 and “yes” = 1 point. Questions about a specific site of symptoms or symptoms under specific circumstances were scored as 0 if not present and as 1 if present for each site or circumstance. All questions regarding the frequency of symptoms were scored as “rarely” or “never” = 0, “occasionally” or “sometimes” = 1, “frequently” or “a lot of the time” = 2, and “almost always” or “constantly” = 3 points. All questions regarding the severity of symptoms were scored as “mild” = 1, “moderate” = 2, and “severe” = 3 points. When assessing the time course of a symptom, we scored “gotten somewhat better”, “gotten much better”, “completely gone”, and “I have not had any of these symptoms” = 0, “stayed about the same” = 1, “gotten somewhat worse” = 2, and “gotten much worse” = 3 points. Changes to bodily functions had to be scored dependent on the individual question asked. For example, “I get full a lot more quickly than I used to” when eating a meal was scored 2 points and “I get full a lot less quickly than I used to” was scored 0 points; while the answer “I sweat much more than I used to” was given 1 point and “I sweat much less than I used to” was scored 2 points.

For the new scoring system, we opted to eliminate scoring of questions previously comprising the “syncope” domain due to its vast overlap with the “orthostatic” domain and the questionable relevance of reflex syncope for the assessment of autonomic deficits.⁸ We also eliminated questions about male erectile dysfunction for scoring due to the low specificity of erectile dysfunction as indicator of autonomic nervous system impairment, and the difficulty of a universal scoring system with questions that relate only to one gender.^9,10 Furthermore, we opted to combine the “diarrhea” and “constipation” domain to a “lower GI” domain.

Statistical Approach to Assessing the New Scoring Scheme

Comparisons of the current scoring algorithm using the questions originally selected for COMPASS with the above described new scoring algorithm using all suitable questions from the ASP was performed by calculating Cronbach alpha coefficient as a measure of internal consistency for items comprising respective domains as they are grouped in the current version of the ASP.^11,12 A Cronbach alpha coefficient of 0.7 or higher was considered acceptable.

Factor Analysis and Approach to Content Reduction

In order to identify internally consistent question domains and to reduce dimensionality, i.e. the number of questions retained, unbiassed exploratory factor analysis of items was performed with orthogonal rotation.^11-13 The Eigenvalue rule was used to extract factors retaining only factors with Eigenvalues > 1. The resulting model was confirmed by inspecting the corresponding scree plot, which is a visual representation of where the sharp decline in factors levels off, at which point factors become less relevant (even if their Eigenvalues are > 1). Only items with factor loading ≥ 0.40 were retained and ambiguous items (i.e., items loading on more than one factor) were eliminated.

The resulting set of questions and domains was then reviewed by two autonomic experts (PAL and WS) with three goals: 1) to decide on redundant content in order to allow for further reduction in the number of questions; 2) to retain clinically important questions that were screened out based on factor analysis; and 3) to assess for clinical appropriateness of the grouping of questions into domains based on the factor analysis.

Final Assessment of Content and Design of the New Instrument

After clinical review and revision of questions and domains to be included in the final symptom score, another calculation of Cronbach alpha coefficient values took place to assess for the internal consistency of the new instrument.

To allow for the calculation of a weighted total score, the maximal raw score for each domain was determined and each domain was assigned a weight factor based on our current perception of the importance of domains for reflecting autonomic failure so that the minimal weighted score for the instrument equals 0 and the maximal weighted score 100.

All the analyses were performed using SAS version 9.2 software (SAS Inc, Cary, NC).

Results

Participants

166 of the 405 recruited controls (41%) were male and 239 (59%) were female. Age ranged from 8 to 79 years with a median age of 32 years. Race was predominately “white” (95.8%) and ethnicity was predominantly “Not Hispanic or Latino” (97.3%).

Comparison of Scoring Schemes using the previous COMPASS domains

Cronbach alpha coefficient of previous COMPASS domains ranged from -0.89 to 0.79 using the old scoring system with only two domains reaching a value above 0.70 (orthostatic intolerance and erectile dysfunction). Cronbach alpha coefficient values were markedly improved using the new scoring system, ranging from 0.40 to 0.90 with five domains reaching a Cronbach alpha coefficient greater than 0.70 (Table 1). All domains had a higher Cronbach alpha coefficient with the new compared to the old scoring system.

Table 1.

Comparison of internal consistency of the previously used domains using old and new COMPASS scoring algorithms

	COMPASS – Previous Scoring Algorithm				COMPASS – New Scoring Algorithm
		Cronbach Alpha Coefficient				Cronbach Alpha Coefficient
Domain	Number of Items	All	Females	Males	Number of Items	All	Females	Males
		(405)	(239)	(166)		(405)	(239)	(166)
Orthostatic Intolerance	9	0.79	0.79	0.78	28	0.90	0.91	0.88
Vasomotor	11	0.68	0.69	0.57	11	0.84	0.85	0.80
Secretomotor	10	0.34	0.28	0.42	13	0.52	0.55	0.44
Gastroparesis	5	0.50	0.52	0.35	7	0.66	0.69	0.56
Constipation	4	−0.32	−0.33	−0.22	10*	0.82*	0.81*	0.85*
Diarrhea	4	−0.89	−0.95	−0.82
Bladder	3	0.53	0.42	0.74	3	0.58	0.48	0.74
Pupillomotor	7	0.55	0.57	0.47	11	0.87	0.86	0.88
Sleep	8	0.36	0.31	0.39	8	0.40	0.44	0.33
Syncope	5	0.13	−0.01	0.36	NA	NA	NA	NA
Male Erectile Dysfunction	6	0.75	NA	0.75	NA	NA	NA	NA
Total	72				91

^*Constipation and Diarrhea Domains were combined.

Cronbach alpha coefficient calculations stratified by gender showed a similar trend with markedly improved values for both male and female gender using the new scoring algorithm (Table 1).

Exploratory Factor Analysis

Exploratory factor analysis based on the Eigenvalue rule and factor loading criteria described above, identified seven factors consisting of a total of 54 questions. Five of these seven factors corresponded well with the previous domains “orthostatic intolerance”, “vasomotor”, “secretomotor”, “pupillomotor”, and “bladder”; each containing only items that had previously been part of those domains. The other two factors grouped questions on gastroparesis and diarrhea as one factor (a mixed upper gastrointestinal domain) and questions regarding constipation as a separate factor. Cronbach alpha coefficient for these domains ranged from 0.71 to 0.93 and therefore exceeded the acceptable level of 0.70 for each of these seven domains (Table 2). The previous “sleep” domain was not identified as a separate factor of internal consistency and was therefore eliminated.

Table 2.

Domains identified, Cronbach alpha coefficients, and number of questions retained within each domain based solely on statistical measures (exploratory factor analysis) with factor loading ≥ 0.40

Domains	All (405)	Number of Questions
Orthostatic Intolerance	0.93	17
Vasomotor	0.91	9
Secretomotor	0.71	4
Gastrointestinal – mixed upper and diarrhea	0.87	8
Constipation	0.89	4
Bladder	0.79	2
Pupillomotor	0.90	10

Expert Revisions and Final Assessment of Content

Expert review confirmed the grouping of items into seven factors as clinically meaningful domains with content that allows for retaining previous clinical designation of domains. It was, however, felt that retaining a separation between a gastroparesis/diarrhea and a constipation domain was not clinically useful and the two gastrointestinal domains were, therefore, combined to one for a total of six autonomic domains in the new instrument.

Review for redundant content allowed for elimination of 23 more items. Both experts felt that 5 items that were originally not retained in the factor analysis should be retained due to clinical significance. Those included a question about changes in body sweating, a question about dryness of the mouth, a question about postprandial vomiting, a question about cramping/colicky abdominal pain, and a question about loss of bladder control.

The remaining 31 items and new scoring method were re-analyzed for Cronbach alpha coefficient of each domain. Compared to the statistically ideal 54 question set, Cronbach alpha coefficient values were similar for 3 of the 6 domains, slightly lower but still high for the “gastrointestinal” domain, and lower (and below 0.70) for the other two domains (0.62 for the “bladder” and 0.48 for the “secretomotor” domain, Table 3), which was expected as these domains include items with low factor loading that were retained based on clinical significance alone. Cronbach alpha coefficient for each domain was still notably higher than for the original comparable domains within COMPASS. Table 3 delineates the new six domains, number of questions per domain, and maximal weighted scores of the new instrument. The original ASP is provided in eAppendix 1, the COMPASS 31 instrument is provided in eAppendix 2, the scoring system is shown in eAppendix 3, and the weight factors for calculating weighted scores are listed in Table 3.

Table 3.

Domains and number of questions retained based on exploratory factor analysis and clinical revisions as used in the final instrument (COMPASS 31). Appropriate weighting factors for each domain result in appropriately balanced autonomic domains and a total score between 0 and 100

Domain	Number Of Questions	Max Raw Score	Weighting Factor	Max Weighted Score	Cronbach Alpha
Orthostatic Intolerance	4	10	4.0	40	0.92
Vasomotor	3	6	0.83333333	5	0.91
Secretomotor	4	7	2.1428571	15	0.48
Gastrointestinal*	12	28	0.8928571	25	0.78
Bladder	3	9	1.1111111	10	0.62
Pupillomotor	5	15	0.3333333	5	0.84
Total	31	75		100

^*Combines constipation, diarrhea, and gastroparesis domains into one domain.

Discussion

Using statistical measures and clinical autonomic expertise, we designed a new, refined, and abbreviated composite autonomic symptom score, the COMPASS 31. The need to develop this new instrument arose from problems with the old COMPASS score and the critical need for a straightforward, up-to-date, and broadly applicable self-assessment tool that can assess and quantify autonomic symptom severity across multiple autonomic domains.

Since its original description and validation, we have used the original COMPASS in many autonomic research studies and trials.^2-7 It has been an exceedingly helpful addition to our repertoire to assess autonomic nervous system function and to this point remains the only validated instrument assessing multiple domains of autonomic function. Other, more recently developed instruments are helpful but are limited to specific domains.^14,15 We regularly receive requests to support the use of COMPASS by other groups and institutions, but for a number of reasons we have been hesitant to do so.

Firstly, we have seen inconsistencies and frank errors in scoring of this instrument even among experienced users. There is little doubt that this relates to the highly complicated scoring algorithm that requires dedicated computer analysis for reliable score generation. Secondly, a number of questions included in the original COMPASS score have over time been identified as less meaningful or redundant for scoring autonomic function and symptom severity. Thirdly, completion of the ASP, even if limited to the questions relevant for COMPASS, is time-consuming. Finally, the questions within the domains assessed using COMPASS have never been evaluated for internal consistency.

When developing COMPASS 31, we sought to address each of these concerns. As a first step, we made considerable changes to the scoring algorithm applying a much simplified consistent scoring scheme. This proved not only more user-friendly, but also resulted in notably improved measures of internal consistency. Using exploratory factor analysis and critical clinical review of all questions included in the original ASP, we identified 6 internally consistent and meaningful clinical autonomic domains. With the exception of 5 questions which were retained based on clinical importance alone, we retained only questions that fulfilled both pre-identified statistical and relevance criteria. Redundancies were reduced as much as possible. As a final step, we assigned each domain a weighting factor based on the relevance of each domain for assessing autonomic function, with factors adjusted so that the minimum total score is 0 and the maximal score 100. The result was an instrument with significantly improved measures of internal consistency across all domains and with an easily interpretable score of autonomic symptom severity.

COMPASS 31 does not include assessment of erectile dysfunction. Some may see this as a shortcoming of this instrument, but we decided a priori to exclude this domain for a number of reasons. Erectile dysfunction is a common accompaniment of aging with prevalence rates as high as 52% in 40 to 70 year-old men.⁹ Many factors contribute to erectile dysfunction, including psychological, hormonal, vascular, and neurologic factors. Erectile dysfunction as the result of medication side effects is common.¹⁰ While erectile dysfunction certainly may reflect autonomic dysfunction, we feel that due to its nonspecific nature, erectile dysfunction should not be included in an autonomic severity score. These symptoms should be elicited during a comprehensive autonomic symptom assessment which can be achieved by direct patient interview and by using the ASP, which is designed for that purpose. Finally, the old version of COMPASS was associated with difficulties in statistics and reporting of group results due to the difference in maximal scores between genders related to the male-specific extra domain; these problems are now resolved with the new instrument.

Another domain eliminated for scoring was the syncope domain. It was felt that there was considerable overlap with the orthostatic intolerance domain and questions related to reflex syncope were felt to not be a meaningful measure of autonomic dysfunction.⁸ Questions previously comprising the sleep domain were found to have very low internal consistency and were not identified as an independent domain in our factor analysis. We did not feel that retaining this domain could be justified based on clinical relevance alone.

Our previous separation of the gastrointestinal domain into gastroparesis, constipation, and diarrhea domains was not maintained by factor analysis. The analysis suggested to group questions related to gastroparesis and diarrhea in one, and questions related to constipation in a second domain. We felt that such grouping would be artificial and that the future users of the instrument would be better served by providing a single gastrointestinal domain score akin to a single score for all other domains. If further discrimination is needed, this could be achieved by review of individual questions.

Conclusion

The COMPASS 31 was developed as a self-assessment instrument of autonomic symptoms and function that is up-to-date, broadly applicable, easy to administer in a short amount of time and based on a scientific approach. It was designed to provide a global autonomic severity score and domain scores that are both clinically as well as scientifically meaningful. We believe that these goals have been achieved. Further validation of this new instrument in various autonomic disorders and degrees of autonomic failure is now in progress.

COMPASS 31 is based on the well-established ASP, a comprehensive questionnaire assessing autonomic symptoms across multiple domains. All questions of the COMPASS 31 are contained in the ASP. It is, therefore, conveniently possible to derive a COMPASS 31 score either from the comprehensive ASP or from the 31 selected questions that make up the COMPASS 31 alone, depending on the goals of the clinician or investigator. The new instrument is backward-compatible with previously acquired data using the ASP.

We have included the complete instrument and tools necessary for its application and score generation in this publication. It is our hope that COMPASS 31 will be embraced by many autonomic clinicians and researchers as a concise quantitative measure of autonomic symptoms and function, and that it will find broad application in clinical autonomic research and practice.

Abbreviations

ASP

Autonomic Symptom Profile

COMPASS

Composite Autonomic Symptom Score