Validating a measure of myalgic encephalomyelitis/chronic fatigue syndrome symptomatology

Abstract

Objectives

The present study sought to validate a comprehensive self-report measure of myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS) symptomatology to aid in clinical and research assessment.

Method

Exploratory factor analysis (EFA) was used to establish the underlying factor structure of the DePaul Symptom Questionnaire (DSQ) (Jason, Evans, et al., 2010) using a well-characterized sample of individuals (92.6% met the Fukuda et al. criteria (1994) and/or the Clinical Canadian Criteria (Carruthers et al., 2003)), and this structure was then tested on a less stringently recruited sample of individuals utilizing a confirmatory factor analysis (CFA). Convergent and discriminant validity of the DSQ were also examined utilizing alternative measures of symptomatology and functioning.

Results

A 3-factor solution was found using EFA (Neuroendocrine, Autonomic & Immune Dysfunction; Neurological/Cognitive Dysfunction; Post-Exertional Malaise) and the fit of this factor structure was adequate for the second sample.

Discussion

The DSQ is a valid measure of ME/CFS symptomatology. The emergent factors were consistent with previous literature on symptom clusters, and convergent and discriminant validity were established.

Chronic fatigue syndrome (CFS) is a chronic and debilitating illness of unknown etiology affecting over 800,000 adults in the United States (Jason, Richman et al., 1999). Individuals with CFS experience marked functional disability (Tiersky et al., 2001), and face significant threats to their quality of life (Anderson, Ferrans, & Estwing, 1997). Because the cause of the condition remains unknown, individuals with CFS are often met with disbelief and are stigmatized by medical professionals, employers, friends, and family (Dickson, Knussen, & Flowers, 2007). Although atypical immune manifestations (Klimas & Koneru, 2007; Maher, Klimas, & Fletcher, 2005) and neurocognitive (Michiels & Cluydts, 2001), central nervous system (Gur & Oktayoglu, 2008; Natelson, Cohen, Brassloff, & Lee, 1993) and autonomic dysfunction (Newton et al., 2007) have been well documented in CFS samples, replication of many of these findings has been difficult, possibly due to heterogeneous patient samples (Jason, Helgerson, Torres-Harding, Carrico, & Taylor, 2003; Nacul et al., 2011). Without successful replications, our understanding of biomarkers and an organic explanation for CFS remains elusive.

The heterogeneous samples used in research may be a product of the vague and poorly operationalized diagnostic criteria that have been established for CFS (Jason, King et al., 1999). Since the illness became formally recognized as CFS in the late 1980s following reports of cluster outbreaks in Nevada (Buchwald et al., 1992) and New York (Bell, Bell, & Cheney, 1994), consensus for a singular case definition has yet to be reached by researchers, practitioners and patient advocates. The diagnosis of CFS is an exclusionary process that relies heavily on self-reported symptom profiles (Afari & Buchwald, 2003). Therefore, developing a standardized process for assessing these symptom profiles is vital (King & Jason, 2004). The issue of heterogeneous samples has plagued the research community since the Centers for Disease Control and Prevention’s (CDC) publication of the first diagnostic criteria for CFS (Holmes et al., 1988).

The Holmes et al. CFS criteria (1988) required a patient to experience persistent, unexplained fatigue at least 50% of the time with a definite onset, accompanied by eight out of 11 definitional symptoms. These criteria were criticized as vague and poorly operationalized, which led to inconsistent application by clinicians and researchers (Fukuda et al., 1994). Further, by placing the definitional symptom threshold so high (eight required symptoms), the Holmes criteria might have inadvertently selected for individuals with primary psychiatric explanations for their fatigue (Katon & Russo, 1992). Similarly, Kroenke (2003) found that with increased somatic complaints, psychiatric comorbidity rates increase in samples of individuals with chronic fatigue. If CFS diagnostic criteria select for increased numbers of individuals with primary affective disorders, the search for biomarkers is further complicated. This is evidenced by the findings of Lange and colleagues (1999) who found that individuals with CFS and no psychiatric comorbidity displayed significantly more brain abnormalities on an MRI than those individuals with CFS and a psychiatric condition, and healthy controls.

In response to the criticisms of the Holmes et al. (1988) case definition, the CDC convened an international working group to improve upon these diagnostic criteria, which resulted in the development of the Fukuda criteria (Fukuda et al., 1994). The improved criteria were also criticized as vague and clinically unhelpful (Jason, King et al., 1999; De Becker, McGregor, & De Meirleir, 2001), lacking specific guidelines or operationalizations. Further, the Fukuda et al. criteria are polythetic, meaning that individuals who meet the criteria will not necessarily have common features. To meet criteria, an individual must have at least six months of unexplained persistent fatigue of new or definite onset, experienced concurrently with just four out of eight definitional symptoms. Further, this symptom complex must cause substantial reductions in functioning. The Fukuda criteria, with updates made by Reeves et al. (2003), remain the most universally utilized criteria to date for research and clinical purposes.

In an attempt to operationalize the fatigue, symptom pattern, and substantial reductions required by Fukuda et al. (1994), the CDC developed the Empiric criteria (Reeves et al., 2005), which specifies the use of validated self-report measures and cut-off scores to aid in diagnosis. Although seemingly indicative of progress for the field, Reeves et al.’s (2005) Empiric criteria resulted in significant dissension by many researchers and patient advocates due to its broadening of the case definition (Jason, Porter, Brown, Brown, & Evans, 2010). The first community-based epidemiological study that utilized these criteria raised the CDC’s estimated prevalence rate of CFS from .24% (Reyes et al., 2003) to 2.54% of the population (Reeves et al., 2007), which was also significantly higher than previous outside estimates of .42% (Jason et al., 1999). This led many to question the validity of the criteria, and Jason, Najar, Porter, and Reh (2009) found that the Empiric criteria incorrectly identified 38% of a sample with primary Major Depressive Disorder (MDD) as having CFS due to the lack of specificity of these criteria.

In 2003 an international group working independently of the CDC developed new criteria in which the condition was explicitly labeled ME/CFS (Carruthers et al., 2003). In contrast to the polythetic CDC CFS case definitions, the Clinical Canadian Criteria (CCC) requires two of the core symptoms of ME/CFS to be present for a diagnosis, post-exertional malaise and neurocognitive impairment. Additionally, these criteria no longer emphasize fatigue as the primary symptom, as it is not required for a diagnosis. To meet the CCC, a person must experience post-exertional malaise, at least two neurocognitive symptoms, at least one symptom indicating sleep dysfunction, at least one symptom indicating significant bodily pain, and at least one symptom from two of the following categories: autonomic manifestations, neuroendocrine manifestations, and immune manifestations. Additionally, this symptom complex must result in substantial reduction of an individual’s functioning. As these criteria require specific symptoms, they may select for a more homogenous group of individuals than the polythetic approach of the Holmes et al. (1988) and Fukuda et al. (1994) case definitions. However, the CCC lack operationalization with no guidelines provided regarding frequency or severity of required symptoms (Jason, Evans, Porter et al., 2010). Therefore, although the CCC may identify a more homogenous sample with regards to what symptoms are occurring, the intensity of these symptoms could range significantly. Jason, Brown et al. (2012) compared those meeting the ME/CFS case definition to those not meeting the ME/CFS criteria but meeting the Fukuda et al. (1994) criteria only. Findings indicated that the ME/CFS case definition identified individuals with more severe symptoms and greater functional disability than those meeting only the Fukuda criteria.

Recently, an international consensus document (Carruthers et al., 2011) was published and described by the authors as an update to the ME/CFS Clinical Canadian Criteria (CCC) (Carruthers et al., 2003). The term CFS was dropped and significant changes to the CCC were made. The ME International Consensus criteria (ME-ICC) require an individual to experience post-exertional malaise, at least one symptom out of three of four distinct neurological domains, at least one symptom out of three of five distinct immune domains, and at least one energy production symptom. Additionally, an individual’s functioning must be reduced by 50% compared to their pre-illness activity level. The authors of the ME-ICC recently published a primer on this case definition (Carruthers & van de Sande, 2012), although many aspects of the criteria remain poorly operationalized.

Jason, Damrongvachiraphan et al. (2012) approached the development of a consensus ME case definition by combining descriptions of the condition from some of the key ME theorists over the past few decades (Ramsay, 1988; Dowsett et al., 1990; Hyde, 2007; Goudsmit et al., 2009). Taken together, these theorists propose a more narrow view of ME and the emergent criteria require an individual to experience post-exertional malaise, at least one neurological symptom, at least one autonomic symptom, and the onset of the condition had to have been sudden (developing over one week or less). Jason, Brown et al. (2012) found that a sample of individuals meeting these ME criteria and the Fukuda et al. criteria were more impaired and symptomatic than a sample of individuals who only met the Fukuda et al. criteria.

Given the diagnostic complexity of CFS and ME/CFS, and the multiple case definitions available for clinical and research application, the differences between which are yet to be fully understood, the need for a comprehensive self-report tool to aid in symptomatology assessment is clear. Jason, Evans, Porter et al. (2010) published the DePaul Symptom Questionnaire, comprised of questions regarding health, social and occupational history, as well as a 54-item symptom chart (included as Appendix A), designed to tap all of the major case definitions of CFS (Fukuda et al., 1994; Reeves et al., 2005), ME/CFS (Carruthers et al., 2003), and ME (Carruthers et al., 2011; Jason et al., 2012). While many questionnaires have been developed to assess CFS and ME/CFS symptomatology, few utilize a scoring system that provides a composite score for such a comprehensive list of symptoms, reflecting both the frequency and intensity of the symptom for the individual over the past 6 months. Further, Jason, Evans, Porter et al.’s criteria require an individual to endorse a symptom as occurring (at least) ‘about half the time’ and at a ‘moderate’ intensity to be counted as part of the symptom profile. A well validated measure capable of classifying individuals with ME/CFS using a variety of case definitions, relying not just on occurrence of symptoms alone, would allow researchers across settings to identify groups with more homogenous phenotypes. Consistent identification of more homogenous patient samples could possibly assist in the pursuit of biomarkers for ME/CFS (Nacul et al., 2011).

A few research groups have examined the underlying factor structure of symptom questionnaires designed to assess CFS symptomatology in order to inform an understanding of the core symptom domains of the illness. Using principal components analysis for a clinical sample of individuals with long-duration CFS, Friedberg, Dechene, McKenzie and Fontanetta (2000) found a 3-factor solution based on a 48-item questionnaire: ‘Cognitive Problems,’ ‘Flu-like Symptoms’ and ‘Neurologic Symptoms.’ Also using a well-characterized CFS sample, Jason, Corradi and Torres-Harding (2007) ran a principal components analysis on a 24-item questionnaire, which resulted in a 6-factor solution: ‘Neurocognitive,’ ‘Vascular,’ ‘Inflammation,’ ‘Muscle/Joint,’ ‘Infectious,’ and ‘Sleep/Post-exertional malaise.’ Arroll and Senior (2009) performed a factor analysis using data from two measures of symptomatology collected from individuals with CFS, resulting in a 5-factor solution: ‘Fibromyalgia Syndrome-like,’ ‘Depression/Anxiety,’ ‘Fatigue/Post-Exertional Malaise,’ ‘Cognitive/Neurological’ and ‘Irritable Bowel Syndrome-like.’

Other groups have attempted factor analytic studies aimed at understanding CFS-like groups. Nisenbaum, Reyes, Mawle and Reeves (1998) found a 3-factor solution using a community-based sample of individuals with chronic fatigue based on a 14-item questionnaire: ‘Fatigue-Mood-Cognition,’ ‘Flu-type’ and ‘Visual Impairment.’ In a later community-based study based on a 21-item questionnaire, a different 3-factor solution was reached using a chronically fatigued group: ‘Musculoskeletal,’ ‘Infection,’ and ‘Cognition-Mood-Sleep’ (Nisenbaum, Reyes, Unger, & Reeves, 2004). Hickie et al. (2009) utilized a large, international sample which combined a sample of 2,013 individuals with chronic fatigue and 1,958 with CFS into a factor analysis, resulting in a 5-factor model of symptom types: ‘Musculoskeletal pain/fatigue,’ ‘Neurocognitive Difficulties,’ ‘Inflammation,’ ‘Sleep disturbance/fatigue’ and ‘Mood disturbance.’

There is need for a structured questionnaire that has been designed to assess the varying symptoms that have been identified in prior factor analytic studies as well as case definitions. The present study examines the psychometric properties of a comprehensive self-report measure of CFS, ME and ME/CFS symptomatology. The factor structure of the DePaul Symptom Questionnaire (DSQ) is examined along with its convergent and discriminant validity.

Method

Research Participants & Procedure

DePaul sample

An international convenience sample of adults self-identifying as having CFS, ME/CFS, or ME was recruited. To be eligible an individual needed to be at least 18, capable of reading and writing English, and have a self-reported current diagnosis of ME, CFS, or ME/CFS. Following approval by DePaul University’s Institutional Review Board, participants were recruited from a variety of sources including postings on internet forums, support group visits, re-contacting of individuals who have participated in the DePaul Research Team’s studies in the past and have indicated interest in future studies, and contacting of individuals who have emailed the team’s address in the past with interest in future studies.

Participants were given three options for completion of the surveys: an electronic survey, a hard-copy survey, or a verbal survey over the telephone. All participants were given the opportunity to complete these surveys at home or in person at the DePaul. Participants were not given a timeline for survey completion, as this illness can be unpredictable and result in a rapid decline of functioning on any given day. The first 100 individuals who completed the survey received a $5.00 gift card to Amazon.com for their participation.

Of the original 217 individuals who completed the DSQ, 189 participants were included in the present study. Twenty-eight participants were excluded due to active medical conditions, active psychological conditions, and/or the endorsement of lifelong fatigue, all of which preclude a diagnosis of CFS based upon the Fukuda et al. (1994) case definition. Although there was no formal psychiatric interview, Torres-Harding, Jason, Cane, Carrico, and Taylor (2002) have demonstrated that individuals with CFS are capable of validly self-reporting psychiatric comorbidity information.

Demographically, the sample of 189 participants was 83.5% female and 16.5% male. 97.9% of the sample identified as Caucasian, 0.5% as Asian, and the remaining 1.6% identified as ‘Other.’ 55.3% of the sample stated that they were currently on disability, with only 12.8% of the sample working part or full-time. With regards to educational level, 39.9% of the sample held a professional degree, 35.6% held a standard college degree, 17.6% attended college for at least one year, and 6.9% completed high school or had a GED. The mean age was 51.6 (SD 11.2).

Solve CFS BioBank sample

Data on a separate sample of individuals were de-identified and shared with the DePaul Research Team by the CFIDS Association of America. This patient data comes from the Solve CFS BioBank, a resource with clinical information and blood samples on a sample of individuals (N=241) diagnosed by a licensed physician specializing in CFS, ME/CFS and ME care. The BioBank requirements involved stringent inclusion and exclusion criteria. The sample used in the present study included only those over 18. Participants were recruited by the CFIDS Association of America through their website and other social networking devices, internet forums, and through physician referral. All participants who met eligibility criteria completed a written informed consent process before being included into the BioBank. Participants completed the study measures electronically or by hard copy. BioBank data was de-identified and given to the DePaul ME/CFS research team following submission of a research protocol to the CFIDS Association of America which was approved by the Medical Research Advisory Committee.

The BioBank sample was 98.3% Caucasian, 0.8% Asian, and 0.8% of the sample identified as ‘Other.’ With regards to gender, 73.4% of the sample was female. Only 12.0% of the sample was working full- or part-time, with 64.7% on disability. Regarding education level, 24.4% of the sample held a graduate or professional degree; 43.2% completed college; 20.7% had completed some college; 11.2% had a high school degree or GED; and 0.4% did not answer. The average age of the sample was 49.7 (SD 12.8). The BioBank and DePaul samples did not differ significantly on any of the sociodemographic variables measured.

Measures

The DePaul Symptom Questionnaire

All participants in both samples completed the DePaul Symptom Questionnaire (DSQ) (Jason, Evans, Porter et al., 2010), a self-report measure of ME/CFS symptomatology, demographics, and medical, occupational and social history. It was developed to classify individuals on a variety of ME/CFS case definitions, but the symptom list was based upon a revised approach to the Clinical Canadian criteria (Carruthers et al., 2003). Participants are asked to rate each symptom’s frequency over the past six months on a 5-point likert scale with 0=none of the time, 1=a little of the time, 2=about half the time, 3=most of the time, and 4=all of the time. Likewise, participants rate each symptom’s severity over the past six months on a 5-point likert scale with 0=symptom not present, 1=mild, 2=moderate, 3=severe, 4=very severe. The frequency and severity scores are then multiplied to obtain a composite score for each symptom ranging from 0 to 16. Each case definition can be applied upon the completion of the questionnaire, utilizing algorithms. The current study is the first to use the DSQ for research purposes, and therefore the psychometric properties had not yet been established.

The development of the DSQ was also based upon the CFS Questionnaire, which evidences good inter-rater and test-retest reliability, and was able to sensitively distinguish between those with CFS, individuals with Major Depressive Disorder and healthy controls (Hawk, Jason, & Torres-Harding, 2006). The CFS Questionnaire also assesses for frequency and severity over the past six months, but the severity rating is on a scale from 0–100 whereas the DSQ proposes a likert scale.

The CDC Symptom Inventory

All participants in the DePaul sample completed the CDC’s Symptom Inventory (Wagner et al., 2005), a self-report measure of the frequency and severity of 9 CFS-related symptoms over the past one month. A composite score of frequency and severity is obtained for each symptom. Additionally, participants are asked the duration in years of each symptom and whether or not they currently consider the symptom to be a part of their ill health and their past ill health. The measure evidences good internal consistency (Cronbach’s alpha= .88) and good convergent validity (as indicated by significant correlations with related measures) (Wagner et al.).

The Medical Outcomes Study Short-Form 36 Survey (SF-36)

All participants in both samples completed the SF-36 (Ware, Snow, & Kosinski, 2000), a 36-item self-report measure of disability comprised of eight subscales: physical functioning, role physical, bodily pain, general health, role emotional, social functioning, vitality, and mental health. The composite score for each subscale ranges from 0–100 with higher scores indicating better functioning. This measure is frequently used in research to assess disability brought on by illness. Buchwald, Pearlman, Umali, Schmaling and Katon (1996) found that for a sample of individuals with CFS, the SF-36 had good internal reliability and convergent validity. It was also able to distinguish individuals with CFS and chronic fatigue from individuals with major depression, acute mononucleosis, and from healthy controls.

Results

Assessing Adequacy of the Correlation Matrix

Before running the exploratory factor analysis on the BioBank sample, we assessed the adequacy of the correlation matrix. Sufficient inter-item correlations were observed, but four pairs of items were highly correlated (>.80), demonstrating that multicollinearity was a potential issue. Two of the four pairs of items were determined to be theoretically important, with both items measuring distinct constructs and therefore these items were both retained (‘Sensitivity to Noise’ and ‘Sensitivity to Bright Lights’; ‘Joint Pain’ and ‘No Appetite’). One item from each of the remaining two pairs (‘Minimum Exercise Makes You Physically Tired’ and ‘Physically Drained or Sick after Mild Activity’; ‘Problems Remembering Things’ and ‘Absent-mindedness or Forgetfulness’) was dropped from the analyses. After dropping these two items, the correlation matrix was re-run. The determinant was approaching zero, but still acceptable. Bartlett’s test of sphericity indicated that the correlation matrix was not an identity matrix (X²=6085.94, p < .001). The Kaiser-Meyer-Olkin Measure of Sampling Adequacy (KMO=.87) also indicated the matrix was appropriate for EFA. Finally, measures of sampling adequacy (MSAs) for all items were > .60, also indicating that EFA was appropriate.

Factor Extraction and Determining Number of Factors to be Retained

The data were found to be non-normal, with significant left-skewedness for the majority of the scale-items endorsed by a high percentage of participants. Those scale-items which were endorsed by a low percentage of participants demonstrated right-skewedness. As maximum likelihood extraction is not robust to normality violations, principal axis factoring was utilized with an oblique rotation (promax). A combination of approaches was used to determine the appropriate number of factors to retain. Examination of the scree plot suggested a three-factor solution, as the final substantial ‘drop’ on the plot was between three and four factors. Utilizing the variance explained rule, which suggests that factors should only be retained if they explain at least 5% of the common variance, three factors were retained.

Evaluating and Refining the Factors

The factor structure matrix was examined to determine which items needed to be dropped before interpretation. Six items were dropped due to low loadings across factors (<.35): ‘Need to Nap Daily’, ‘Problems Falling Asleep’, ‘Problems Staying Asleep’, ‘Eye Pain’, ‘Sweating Hands’, and ‘Alcohol Intolerance’. Four items had multiple high loadings, and in all cases were retained and included with other theoretically similar items: ‘Flu-like Symptoms’, ‘Shortness of Breath or Trouble Catching Your Breath’, ‘Pain or Aching in Your Muscles’, ‘Mentally Tired after the Slightest Effort.’ In total, eight scale-items were removed from the original set of items (two before initial extraction due to multicollinearity and six following examination of the factor structure matrix). The EFA was re-run after the additional six items were dropped, and the rotated factor structure matrix for the final solution can be found in Appendix B. Items were dropped based upon loadings < .35 according to the factor structure matrix. When re-run, all these symptoms had loadings of at least .35 on the factor structure matrix. Some of the loadings are <.35 on the final factor pattern matrix, which suggests that less emphasis should be put on these scale-items for factor interpretation.

Factor Interpretation

Table 1 is the rotated factor pattern matrix for the final solution and both this and the factor structure matrix were used in interpretation. Thirty-one items loaded on Factor one, eight items loaded on Factor two, and seven items loaded on Factor three. Factor one was labeled ‘Neuroendocrine, Autonomic, and Immune Dysfunction’ as all scale-items that loaded to this factor belong to multiple symptom clusters. Factor one explained 31.3% of the variance with an eigenvalue of 14.4. Factor two was labeled ‘Neurological/Cognitive Dysfunction’ as all scale-items that loaded to this factor broadly fit into this symptom cluster. Factor two explained 5.8% of the variance with an eigenvalue of 2.7. Finally, Factor three was labeled ‘Post-Exertional Malaise’ as post-exertional malaise items and unrefreshing sleep loaded to this factor. Factor three explained 4.9% of the variance with an eigenvalue of 2.3. All factors were correlated as permitted by oblique rotation. Factors one and two were strongly correlated, r = .57. Factors one and three were also strongly correlated, r = .52. Finally, Factors two and three were strongly correlated, r = .46.

Table 1.

Factor pattern matrix for exploratory factor analysis with promax rotation of the DePaul Symptom Questionnaire

Scale-Item	Factor
	1 Neuroendocrine, Autonomic, and Immune Dysfunction	2 Neurological/Cognitive Dysfunction	3 Post-Exertional Malaise
58. Feeling hot or cold for no reason	.82
57. Feeling chills or shivers	.80		−.31
59. Feeling like you have a high temp.	.72
30. Abdomen/stomach pain	.63
63. Tender/sore lymph nodes	.61
50. Dizziness or fainting	.61
56. Cold limbs	.58
46. Irritable bowel problems	.57
55. Night sweats	.56
47. Nausea	.56
48. Feeling unsteady on your feet	.54
66. Chemical sensitivities	.54
62. Sore throat	.53
31. Headaches	.53
60. Feeling like you have a low temp.	.51
32. Muscle twitches	.50
25. Pain or aching in your muscles	.48		.33
64. Fever	.48
26. Multijoint pain	.48
45. Bladder problems	.47
29. Bloating	.44
65. Flu-like symptoms	.43		.30
24. Sleep all day/stay awake all night	.41
34. Sensitivity to noise	.40
35. Sensitivity to bright lights	.40
53. No appetite	.40
28. Chest pain	.39
49. Shortness of breath	.34
51. Irregular heart beats	.34
52. Losing/gaining weight w/o trying	.33
23. Waking up early in the morning	.33
39. Difficulty understanding things		.90
37. Difficulty paying attention		.89
43. Slowness of thought		.82
44. Absent-mindedness /forgetfulness		.78
41. Unable to focus vision/attention		.77
38. Difficulty expressing thoughts		.75
40. Only able to focus on one thing		.71
42. Loss of depth perception		.43
13. Fatigue/extreme tiredness			.82
15. Next day soreness or fatigue after non-strenuous activities			.81
18. Physically drained or sick after mild activity			.81
14. Dead, heavy feeling after starting to exercise			.72
33. Muscle weakness			.50
16. Mentally tired after the slightest effort		.46	.50
19. Feeling unrefreshed after waking			.50
Eigenvalue	14.4	2.7	2.3
Percentage variance explained	31.3%	5.8%	4.9%

Additionally, Cronbach’s alpha was computed for each resulting factor. Cronbach’s alpha for Factor one items = .93, for Factor two items = .92, and for Factor three items = .88, indicating that all factors possessed good internal consistency. Factor-based composite scores were constructed by taking the mean of all scale-item composite scores for each factor. Each participant thus had three distinct factor-based composite scores.

We next examined how the resulting factor solution of the DePaul Symptom Questionnaire fit a more inclusive convenience sample of individuals self-identifying as having CFS or ME/CFS. We ran a confirmatory factor analysis on the DePaul sample. MPlus statistical software was used.

Assessing Model Fit

Maximum likelihood robust estimation was used to assess the fit of the factor structure (3-factors), which was established using EFA. Figure 1 shows the path diagram for the CFA with standardized results. All path loadings were significant and indicator error terms are not included in the figure. Scale-item numbers correspond with the symptoms listed in Table 1. The model displayed some adequate fit indices (RMSEA=.06; 90% CI .06–.07; SRMR=.08) and some that were poor (X²=1760.40 (986), p <.001; TLI=.74; CFI=.75). A number of modification indices were suggested by MPlus to improve overall model fit. We made modifications that were theoretically meaningful, and a number of indicator terms were permitted to correlate. Additionally, two scale-items (‘Muscle Weakness’ and ‘Loss of Depth Perception’) were permitted to load onto Factor 1 in the alternative model. ‘Muscle Weakness’ originally loaded onto Factor 3 and ‘Loss of Depth Perception’ originally loaded onto Factor 2. This was justified as in the original EFA, both of these scale-items loaded relatively highly to more than one factor but were grouped with other theoretically similar items for the final factor structure. By allowing these scale-items to load to Factor 1 instead, model fit increased.

Figure 1.

Confirmatory factor analysis path model

Following these modifications, key model fit indices improved to the acceptable or good range. The CFI improved from .75 to .90, the RMSEA from .06 to .04, and the SRMR from .08 to .06. The TLI improved less (from .74 to .89).

Convergent and Discriminant Validity

We next ran bivariate correlations between the DSQ factor-based composite scores and the physical health SF-36 subscales (role physical, physical functioning, bodily pain, and general health) for both samples independently. Further, for the DePaul sample, the DSQ factor-based composite scores were correlated with the Symptom Inventory composite score. Symptom Inventory data was not available for the BioBank sample.

For the BioBank sample, all three factor-based composite scores were significantly and negatively correlated with the SF-36 physical health subscales, indicating that self-report of higher symptom intensity on the DSQ is associated with poorer physical functioning on the SF-36. With regards to the SF-36 mental health subscales, two of the three factor-based composite scores were significantly and negatively associated with the Mental Health subscale and all three factor-based composite scores were significantly and negatively associated with the Role Emotional subscale. This indicates that self-report of higher symptom intensity on the DSQ is associated with poorer mental health functioning as measured by the SF-36 for the BioBank sample.

For the DePaul sample, all three factor-based composite scores were significantly and negatively correlated with the SF-36 physical health subscales, indicating that self-report of higher symptom intensity on the DSQ is associated with poorer physical functioning on the SF-36. The three factor-based composite scores were not significantly associated with either of the mental health subscales for the DePaul sample. Additionally, the three factor-based composite scores were significantly and positively correlated with the Symptom Inventory composite score, indicating that higher DSQ symptom intensity are associated with higher SI symptom intensity.

The two samples were compared on all eight SF-36 subscales, as well as on factor-based composite scores. The samples are not significantly different with regards to factor-based symptom composite scores. However, the DePaul sample scored significantly worse on measures of pain, physical functioning, and social functioning. The BioBank sample endorsed worse mental health and emotional functioning. This suggests that the DePaul sample may be more physically impaired while the BioBank sample may have more mental health concerns.

Discussion

The current study utilized exploratory factor analysis to examine the underlying factor structure of the DePaul Symptom Questionnaire (DSQ) for a physician-defined sample of individuals (BioBank sample). A 3-factor solution included: ‘Neuroendocrine, Autonomic, and Immune Dysfunction’ (explaining 31.3% of the variance), ‘Neurological/Cognitive Dysfunction’ (explaining 5.8% of the variance) and ‘Post-Exertional Malaise’ (explaining 4.9% of the variance). Adequate internal consistency was found for each factor. Subsequently, a confirmation of the 3-factor solution was attempted with a more inclusive, self-defined sample of individuals with ME/CFS (DePaul sample). When a number of modifications were made to the model, the fit was satisfactory.

Validity was established for the DSQ by correlating factor-based composite scores with alternative measures of symptomatology, physical health and mental health. Good convergent validity was observed with data from both patient samples, with the alternative measures of symptomatology and physical health functioning correlating significantly with the DSQ factor-based composite scores in the appropriate direction for each alternative measure. Additionally, for the DePaul sample, no significant correlations were observed with alternative mental health scales. However, for the BioBank sample, DSQ factor-based composite scores correlated significantly with the mental health scales. The discriminant validity of the DSQ warrants future study.

Of note is that the BioBank sample reported significantly greater mental health impairment than the DePaul sample. For the Mental Health subscale of the SF-36, [t(428) = 2.49, p < .05], patients from the DePaul sample scored higher indicating better mental health functioning. For the Role Emotional subscale of the SF-36, [t(426) = 2.45, p < .05], patients from the DePaul sample also scored higher. The DePaul sample also reported significantly greater physical health impairment than the BioBank sample on the Physical Functioning and Bodily Pain subscales. Further, the DePaul sample reported significantly more impairment on the Social Functioning subscale, which is meant to tap into both physical and mental health functioning. Although the two samples were comparable sociodemographically, the DePaul and BioBank samples appear to differ significantly on several aspects of functionality. When attempting to validate a factor structure with an independent sample, it is ideal for the samples to be comparable with regards to physical and mental functioning.

A number of scale-items did not load significantly with any of the three factors. Interestingly, three of these symptoms were related to sleep dysfunction, considered a hallmark symptom cluster of CFS and ME/CFS (Fukuda et al., 1994; Carruthers et al., 2003; 2011): ‘Need to Nap Daily,’ ‘Problems Falling Asleep,’ and ‘Problems Staying Asleep.’ Two other sleep dysfunction symptoms that were included in the DSQ and were not dropped loaded to Factor 1, but weakly: ‘Waking up Early in the Morning’ and ‘Sleep All Day, Stay Awake All Night.’ The final sleep dysfunction symptom, ‘Feeling Unrefreshed after You Wake up in the Morning’ loaded to Factor 3, the Post-Exertional Malaise factor. It may be possible that sleep dysfunction does not represent its own distinct symptom cluster, which is supported by previous factor analytic studies of CFS samples (Friedberg, Dechene, McKenzie, & Fontanetta, 2000; Arroll & Senior, 2009).

Other dropped items included ‘Eye Pain,’ ‘Sweating Hands’ and ‘Alcohol Intolerance.’ With regards to the latter, many patients with CFS and ME/CFS avoid alcohol due to extreme chemical sensitivity (Ciccone & Natelson, 2003) and this might have caused participants to indicate that this symptom had not occurred over the past six months, and thus it was dropped from the list of items. ‘Eye Pain’ is considered by Carruthers et al. (2003; 2011) to belong to the pain symptom cluster but this was not supported by the current study. ‘Sweating Hands’ is considered by Carruthers et al. to belong to the neuroendocrine symptom cluster but this was also not supported by the current study.

Two of the emergent factors, Factor 2 (Neurological/Cognitive Dysfunction) and Factor 3 (Post-Exertional Malaise) fit well with previous literature indicating that these are two of the cardinal symptom clusters of ME/CFS (Jason, Corradi, & Torres-Harding, 2007; Arroll & Senior, 2009), and have both been considered mandatory for a diagnosis under multiple case definitions (Carruthers et al., 2003; 2011). Factor 1 (Neuroendocrine, Autonomic, & Immune Dysfunction) is more difficult to interpret as it incorporates many symptom clusters. This suggests that there may be core, well-defined symptom clusters such as cognitive impairment and post-exertional malaise, but also that there may be many symptoms that are experienced differently by patients.

Of note is that the current finding of three distinctive factors relates most closely to the ME case definition proposed by Jason, Damrongvachiraphan et al. (2012), based upon Ramsey (1988) and other ME theorists. This definition specifies that post-exertional malaise, neurocognitive impairments and autonomic dysfunction must be present for a diagnosis. Similarly, the present findings indicate that post-exertional malaise and neurocognitive symptoms are unique. While autonomic symptoms are included in Factor 1, they are also closely related to other pain, neuroendocrine and immune symptoms, and do not represent a distinctive factor. In contrast, the Canadian ME/CFS (Carruthers et al., 2003) and the ME-ICC (Carruthers et al., 2011) specify many more symptoms that must be present for a diagnosis.

Carruthers and colleagues’ (2011) most recent addition to the case definition literature requires an individual with ME to meet four distinct symptom categories to meet criteria. This document is referred to as an international consensus document, and the authors state that the symptom clusters were based on patient data. However, within these clusters, a total of eight symptoms are required. Multiple factor analytic studies of symptomatology have resulted in three to four symptom clusters (Nisenbaum, Reyes, Mawle, & Reeves, 1998; Friedberg, Dechene, McKenzie, & Fontanetta, 2000; Nisenbaum, Reyes, Unger, & Reeves, 2004; Arroll & Senior, 2009; Hickie et al., 2009). It is unclear how these eight required symptoms were selected within these four clusters recommended by Carruthers et al. (2011), and this issue is of some importance as a requirement of eight or more symptoms may actually be inadvertently selecting for patient samples with higher rates of psychiatric comorbidity (Kroenke, 2003).

The current study has several limitations. Most notably, the DePaul patient sample was a convenience sample, whereas the BioBank patient sample was well-characterized and each participant was evaluated by a medical specialist before being given a diagnosis. The DePaul sample was collected without physician referral, and self-report of a physician diagnosis was accepted. DePaul participants did provide a medical and psychiatric history, which was used to determine if the participants had any active exclusionary conditions. The BioBank and DePaul samples were also racially homogeneous, as both were almost entirely White. Community-based prevalence studies have found that the condition disproportionately affects ethnic minorities and individuals of low socioeconomic status (Jason, Richman et al., 1999). Thus, the symptom reporting patterns observed in the current study may not generalize to an ethnically diverse sample.

Another limitation of the current study was the sample size. Based upon recommendations by Nunnally (1978) and Everitt (1975), sample sizes should include at least ten participants per scale-item for factor analysis. However, MacCallum, Widaman, Zhang and Hong (1999) suggest that if communalities of measured variables are high, a sample as small as 100 could be adequate. In addition, although our datasets violated multivariate normality assumptions, strategies were used that are more robust to non-multivariate normality (a principal axis factoring approach was taken for the EFA and maximum likelihood robust estimation was used for the CFA).

The current study also relied upon self-reported data. Future studies might be directed towards validating this DSQ using more objective measures. For example, performance on neuropsychological measures could be related to neurocognitive factor-based scores, and immunologic testing following exercise testing could be associated with post-exertional malaise factor-based scores. Future studies might also examine subtypes, as individuals who experienced a sudden onset of the condition may be unique from those with a gradual onset (DeLuca, Johnson, Ellis, & Natelson, 1997a), and those with comorbid psychiatric issues demonstrate significant differences on fMRI scans when compared to those without psychiatric comorbidity (DeLuca, Johnson, Ellis, & Natelson, 1997b). Systematic sub-typing based upon phenotype and genotype is a necessary next endeavor for the field.

In order to achieve high levels of diagnostic reliability, it is important to specify the domains or symptoms that are to be assessed for a case definition, as well as to provide specific ways of asking the questions needed to tap these domains using standardized questionnaires (Jason & Choi, 2008). Even if the symptoms are well specified, failure to carefully operationalize how the symptoms are to be measured will compromise the diagnostic process. One of the primary problems in the CFS, ME/CFS, and ME field has been the difficulty in replicating results across data sets. In contrast to prior research, the current study found that a three factor structure in one data set was confirmed with an independent sample. It is possible that the present study successfully handled this reliability problem because of the specification of operational criteria and the use of an objective questionnaire to collect data. This study highlights the importance of using standardized questionnaires that adequately measure symptoms of case definitions. Finally, the current findings may contribute to the development of a data-driven case definition for this illness. Future work in this field should continue to utilize symptom data from well-characterized patient samples as the basis of case definition, rather than relying on clinical consensus.

Appendix A

The DePaul Symptom Questionnaire

Questions 13–66

For the following questions (13–66), we would like to know how often you have had each symptom and how much each symptom has bothered you over the last 6 months. For each symptom please circle one number for frequency and one number for severity. Please fill the chart out from left to right.

Symptoms	Frequency: Throughout the past 6 months, how often have you had this symptom? For each symptom listed below, circle a number from: 0 = none of the time 1 = a little of the time 2 = about half the time 3 = most of the time 4 = all of the time					Severity: Throughout the past 6 months, how much has this symptom bothered you? For each symptom listed below, circle a number from: 0 = symptom not present 1 = mild 2 = moderate 3 = severe 4 = very severe
13) Fatigue/extreme tiredness	0	1	2	3	4	0	1	2	3	4
14) Dead, heavy feeling after starting to exercise	0	1	2	3	4	0	1	2	3	4
15) Next day soreness or fatigue after non-strenuous everyday activities	0	1	2	3	4	0	1	2	3	4
16) Mentally tired after the slightest effort	0	1	2	3	4	0	1	2	3	4
17) Minimum exercise makes you physically tired	0	1	2	3	4	0	1	2	3	4
18) Physically drained or sick after mild activity	0	1	2	3	4	0	1	2	3	4
19) Feeling unrefreshed after you wake up in the morning	0	1	2	3	4	0	1	2	3	4
20) Need to nap daily	0	1	2	3	4	0	1	2	3	4
21) Problems falling asleep	0	1	2	3	4	0	1	2	3	4
22) Problems staying asleep	0	1	2	3	4	0	1	2	3	4
23) Waking up early in the morning (e.g. 3am)	0	1	2	3	4	0	1	2	3	4
24) Sleep all day and stay awake all night	0	1	2	3	4	0	1	2	3	4
25) Pain or aching in your muscles	0	1	2	3	4	0	1	2	3	4
26) Pain/stiffness/tenderness in more than one joint without swelling or redness	0	1	2	3	4	0	1	2	3	4
27) Eye pain	0	1	2	3	4	0	1	2	3	4
Symptoms	Frequency: Throughout the past 6 months, how often have you had this symptom? For each symptom listed below, circle a number from: 0 = none of the time 1 = a little of the time 2 = about half the time 3 = most of the time 4 = all of the time					Severity: Throughout the past 6 months, how much has this symptom bothered you? For each symptom listed below, circle a number from: 0 = symptom not present 1 = mild 2 = moderate 3= severe 4 = very severe
28) Chest pain	0	1	2	3	4	0	1	2	3	4
29) Bloating	0	1	2	3	4	0	1	2	3	4
30) Abdomen/stomach pain	0	1	2	3	4	0	1	2	3	4
31) Headaches	0	1	2	3	4	0	1	2	3	4
32) Muscle twitches	0	1	2	3	4	0	1	2	3	4
33) Muscle weakness	0	1	2	3	4	0	1	2	3	4
34) Sensitivity to noise	0	1	2	3	4	0	1	2	3	4
35) Sensitivity to bright lights	0	1	2	3	4	0	1	2	3	4
36) Problems remembering things	0	1	2	3	4	0	1	2	3	4
37) Difficulty paying attention for a long period of time	0	1	2	3	4	0	1	2	3	4
38) Difficulty finding the right word to say or expressing thoughts	0	1	2	3	4	0	1	2	3	4
39) Difficulty understanding things	0	1	2	3	4	0	1	2	3	4
40) Only able to focus on one thing at a time	0	1	2	3	4	0	1	2	3	4
41) Unable to focus vision and/or attention	0	1	2	3	4	0	1	2	3	4
42) Loss of depth perception	0	1	2	3	4	0	1	2	3	4
43) Slowness of thought	0	1	2	3	4	0	1	2	3	4
44) Absent-mindedness or forgetfulness	0	1	2	3	4	0	1	2	3	4
45) Bladder problems	0	1	2	3	4	0	1	2	3	4
46) Irritable bowel problems	0	1	2	3	4	0	1	2	3	4
Symptoms	Frequency: Throughout the past 6 months, how often have you had this symptom? For each symptom listed below, circle a number from: 0 = none of the time 1 = a little of the time 2 = about half the time 3 = most of the time 4 = all of the time					Severity: Throughout the past 6 months, how much has this symptom bothered you? For each symptom listed below, circle a number from: 0 = symptom not present 1 = mild 2 = moderate 3= severe 4 = very severe
47) Nausea	0	1	2	3	4	0	1	2	3	4
48) Feeling unsteady on your feet, like you might fall	0	1	2	3	4	0	1	2	3	4
49) Shortness of breath or trouble catching your breath	0	1	2	3	4	0	1	2	3	4
50) Dizziness or fainting	0	1	2	3	4	0	1	2	3	4
51) Irregular heart beats	0	1	2	3	4	0	1	2	3	4
52) Losing or gaining weight without trying	0	1	2	3	4	0	1	2	3	4
53) No appetite	0	1	2	3	4	0	1	2	3	4
54) Sweating hands	0	1	2	3	4	0	1	2	3	4
55) Night sweats	0	1	2	3	4	0	1	2	3	4
56) Cold limbs (e.g. arms legs, hands)	0	1	2	3	4	0	1	2	3	4
57) Feeling chills or shivers	0	1	2	3	4	0	1	2	3	4
58) Feeling hot or cold for no reason	0	1	2	3	4	0	1	2	3	4
59) Feeling like you have a high temperature	0	1	2	3	4	0	1	2	3	4
60) Feeling like you have a low temperature	0	1	2	3	4	0	1	2	3	4
61) Alcohol intolerance	0	1	2	3	4	0	1	2	3	4
62) Sore throat	0	1	2	3	4	0	1	2	3	4
63) Tender/sore lymph nodes	0	1	2	3	4	0	1	2	3	4
64) Fever	0	1	2	3	4	0	1	2	3	4
65) Flu-like symptoms	0	1	2	3	4	0	1	2	3	4
66) Some smells, foods medications, or chemicals make you feel sick	0	1	2	3	4	0	1	2	3	4

Appendix B

Factor Structure Matrix

Factor Structure Matrix for Exploratory Factor Analysis with Promax Rotation of the DePaul Symptom Questionnaire

Scale-Item	Factor
	1	2	3
58	.764	.432
57	.746	.499
59	.663	.350
48	.650	.412	.481
30	.648	.384	.353
50	.630	.326	.406
63	.603		.396
46	.596	.395
45	.585	.481	.328
651	.584	.382	.522
47	.581		.399
56	.578	.392
251	.577		.520
66	.574	.368	.311
55	.569	.326	.308
491	.565	.464	.516
32	.561	.328	.400
35	.536	.470	.316
26	.534	.326	.349
31	.529		.335
28	.525	.398	.400
62	.524		.366
29	.508	.362
34	.503	.403
52	.472	.405	.328
24	.472		.355
60	.469	.344
51	.430	.316
53	.397
64	.384
23	.377	.312
39	.463	.866	.369
43	.476	.835	.420
37	.368	.815	.362
44	.499	.814	.404
38	.493	.790	.388
41	.486	.782	.324
40	.386	.731	.459
42	.479	.545
15	.406	.390	.811
18	.408	.386	.807
13	.361	.316	.779
14	.388	.325	.722
161	.420	.628	.655
331	.506	.468	.654
19	.423	.362	.592

Factor loadings < .30 not included in table

¹Scale-items with multiple high-loadings