Abstract
Purpose
All of the major current case definitions for chronic fatigue syndrome (CFS) specify substantial reductions in previous levels of occupational, educational, social, or personal activities to meet criteria. Difficulties have been encountered in operationalizing “substantial reductions.” For example, the Medical Outcomes Study Short Form-36 Health Survey (SF-36) has been used to determine whether individuals met the CFS disability criterion. However, previous methods of using the SF-36 have been prone to including people without substantial reductions in key areas of physical functioning when diagnosing CFS. This study sought to empirically identify the most appropriate SF-36 subscales for measuring substantial reductions in patients with CFS.
Method
The SF-36 was administered to two samples of patients with CFS: one recruited from tertiary care and the other a community-based sample; as well as a non-fatigued control group. Receiver operating characteristics were used to determine optimal cutoff scores for identifying patients with CFS.
Results
The SF-36 Role-Emotional subscale had the worst sensitivity and specificity, whereas the Vitality, Role-Physical, and Social Functioning subscales had the best sensitivity and specificity.
Conclusion
Based on evidence from this study, potential criteria for defining substantial reductions in functioning and diagnosing CFS is provided.
Introduction
Chronic fatigue syndrome (CFS) is a poorly understood illness lacking a clearly defined etiology. Current case definitions of CFS are vaguely worded and not operationalized. Consequently, the reliability of the diagnostic criteria is at risk, potentially leading to differences in CFS samples across research studies. An important component of CFS diagnostic criteria that has not been well-defined is level of disability. Specifically, to diagnose CFS using the Fukuda et al. [1] criteria, it is important to assess substantial reductions in “previous levels of occupational, educational, social, or personal activities” [1, p. 956]. Objective measures of activity may be an advantageous approach to assessing this substantial reductions construct compared to self-report measures. However, for research purposes, subjective measures may be more desirable for screening this construct due to ease of administration and lower cost. An instrument that has been frequently used to assess substantial reductions is the Medical Outcomes Study Short Form-36 Health Survey (SF-36) [2]. This instrument is a 36-item, broadly-based, self report measure of functional status related to physical functioning, role physical functioning, role emotional functioning, social functioning, bodily pain, general health, vitality, and mental health. Higher scores on this scale indicate better functioning. The SF-36 has been recommended by several groups, including Reeves et al. [3] to assess substantial reductions in previous levels of occupational, educational, social, and personal activities.
Reeves et al.’s [4] empiric CFS case definition attempted to operationalize the substantial reductions construct using the SF-36. According to this empiric CFS case definition, the CFS disability criterion would be met by scoring at or below the 25th percentile on any one of the following four SF-36 subscales: Physical Functioning (less than or equal to 70), Role-Physical (less than or equal to 50), Social Functioning (less than or equal to 75), or Role-Emotional (less than or equal to 66.7). Jason et al. [5] have questioned the inclusion of the Role-Emotional subscale because a person could meet the disability CFS criterion without any reductions in key areas of physical functioning, and only have impairment in role emotional areas (e.g., problems with work or other daily activities as a result of emotional problems). Ware et al. [2] found that the mean for Role-Emotional for a clinical depression group was 38.9, indicating that almost all those with clinical depression would meet the CFS disability criterion, as they would be within the lower 25th percentile on this subscale.
Rather than arbitrarily selecting the lower 25% for four SF-36 subscales, as was recommended in Reeves et al.’s [4] empiric CFS case definition, the present study first examined the use of the SF-36 in the existing literature to determine which subscales best differentiated CFS and control groups. We next used Receiver Operating Characteristics (ROC) to determine which subscales best discriminate between patients with CFS versus controls who are not ill. Given that substantial reductions in functioning are only one criterion used to diagnose CFS and can occur in several medical and psychological illnesses, this measure alone would not suffice in differentiating CFS from other conditions. Therefore, the objective of this study was to determine the level of reductions necessary to differentiate patients with CFS from those who are not ill, as well as to examine which SF-36 subscales are optimal in making this distinction. In order to increase the generalizability of our findings, we used two CFS samples: a tertiary care sample and a community-based sample. In addition, we determined cutoff points on the most discriminating subscales with the best sensitivity and specificity. These cutoff points were subsequently used in a final ROC to determine how many subscale cutoffs needed to be met to best differentiate the CFS groups from the controls.
Methods
Literature review
Because of the importance of the SF-36 for CFS diagnostic purposes, it is useful to review the findings of CFS investigators using this scale. The authors searched for peer-reviewed articles in the following major online databases: PsycInfo, PubMed, and Google Scholar. Search terms included: chronic fatigue syndrome, Myalgic Encephalomyelitis, SF-36, MOS SF-36, Medical Outcome Study Short Form 36 Health Survey, and substantial reductions. Within this search strategy, articles included in Table 1 reported means, standard deviations, and/or confidence intervals of SF-36 subscale scores. Further, studies were included if they used one or more comparison group or subgroups of patients with CFS. Thus, this is not an exhaustive literature review but rather a representation of literature on SF-36 scores among patients with CFS. A total of 17 studies were found that included CFS samples and used the SF-36. Descriptive statistics were obtained for the SF-36 subscales. Nine of the 17 studies had a CFS and non-ill control group, and these studies were used to compare SF-36 subscale scores between patients and controls. Across these 9 studies, the SF-36 subscale scores were averaged for the CFS groups, and, separately, the control groups. Differences between the CFS and control group means were computed for each of the subscales.
Table 1.
Summary of Results from Studies Reporting SF-36 Scores for Individuals with a CFS Diagnosis
| Study | Population | n | Role-Phys. M (SD) |
Physical Func. M (SD) |
Vitality M (SD) |
Social Func. M (SD) |
Bodily Pain M (SD) |
Role-Emo. M (SD) |
Ment. Health M (SD) |
Gen. Health M (SD) |
|---|---|---|---|---|---|---|---|---|---|---|
| Boneva et al. (2007)† | CFS | 30 | 23.3 (32.1) | 55.0 (21.5) | 18.5 (12.2) | 50.8 (23.7) | 42.1 (17.1) | NR | 64.4 (20.0) | 52.9 (21.3) |
| Control | 38 | 85.5 (28.3) | 90.1 (12.0) | 70.8 (14.3) | 94.7 (10.7) | 77.7 (17.8) | NR | 86.4 (8.9) | 83.2 (15.2) | |
| Ciccone, Busichio, Vickroy, & Natelson (2003) | CFS | 24 | 17.7 (34.2) | 41.7 (18.6) | 17.7 (18.4) | 42.7 (22.7) | 36.0 (23.5) | 80.6 (35.3) | 78.8 (12.9) | 38.6 (25.0) |
| CFS w/Psy | 27 | 5.6 (14.4) | 40.6 (22.3) | 15.0 (12.9) | 37.5 (23.0) | 28.9 (18.3) | 50.6 (51.8) | 60.1 (19.0) | 32.1 (13.1) | |
| CFS w/Personality Disorder | 21 | 16.7 (28.9) | 45.0 (27.2) | 12.6 (11.7) | 45.2 (33.2) | 38.7 (22.9) | 69.8 (68.2) | 55.0 (18.7) | 27.4 (15.8) | |
| Hassan, Bannister, Akbar, Weir, & Bofill (1998)† | CFS | 44 | 10.2 (19.7) | 29.1 (16.4) | 11.9 (13.3) | 30.1 (21.6) | 32.6 (19.7) | 68.9 (42.2) | 66.6 (18.4) | 25.7 (13.4) |
| Control | 20 | 91.3 (23.3) | 94.8 (6.9) | 69.3 (20.2) | 91.1 (14.2) | 88.3 (12.7) | 85.0 (29.6) | 77.6 (16.4) | 79.4 (18.9) | |
| Jason, King, Taylor, & Kennedy (2000)† | CFS | 32 | 18.1 (23.3) | 51.6 (23.3) | 24.2 (13.6) | 44.9 (19.4) | 39.7 (24.8) | 45.2 (45.6) | 56.3 (21.3) | 47.8 (24.7) |
| Psy Exclusion | 56 | 28.3 (37.6) | 59.1 (28.0) | 33.0 (19.0) | 46.6 (23.6) | 53.7 (25.3) | 25.6 (34.7) | 47.5 (21.3) | 49.9 (21.3) | |
| Control | 47 | 77.3 (37.3) | 87.6 (24.0) | 64.2 (24.9) | 82.9 (26.5) | 80.9 (27.3) | 76.2 (39.2) | 77.4 (18.3) | 77.3 (20.3) | |
| Jason et al. (2009)*, a | CFS | 27 | 5.6 (16.0) | 37.4 (23.4) | NR | 30.1 (28.4) | NR | 69.1 (40.2) | NR | NR |
| MDD/CFS | 14 | 51.8 (41.0) | 70.4 (32.9) | NR | 42.0 (23.3) | NR | 19.1 (31.3) | NR | NR | |
| MDD | 23 | 58.7 (45.6) | 76.7 (21.3) | NR | 40.2 (25.3) | NR | 30.4 (40.1) | NR | NR | |
| King & Jason (2005)† | CFS | 15 | 13.3 | 44.0 | 14.0 | 35.0 | 38.5 | 48.9 | 39.3 | 31.6 |
| MDD | 15 | 44.3 | 70.3 | 44.0 | 60.8 | 55.6 | 37.8 | 39.1 | 49.9 | |
| Control | 15 | 90.0 | 88.0 | 66.7 | 90.8 | 76.3 | 91.1 | 79.7 | 66.7 | |
| Komaroff et al. (1996)† | CFS | 223 | 17.1 (30.7) | 54.5 (25.9) | 25.7 (21.0) | 45.8 (26.2) | 49.7 (25.4) | 62.9 (42.0) | 62.1 (20.6) | 33.8 (19.4) |
| Control | 2474 | 81.0 (34.0) | 84.2 (23.3) | 60.9 (21.0) | 83.3 (22.7) | 75.2 (23.7) | 81.3 (33.0) | 74.7 (18.0) | 72.0 (20.3) | |
| Moss-Morris & Chalder (2003) | CFS | 49 | 1.45 (.07) | 43.7 (3.7) | 20.9 (3.0) | 41.7 (3.9) | NR | NR | 65.5 (3.0) | NR |
| Rheumatoid Arthritis | 75 | 1.82 (.05) | 41.9 (2.9) | 44.4 (2.6) | 62.9 (3.1) | NR | NR | 66.9 (2.6) | NR | |
| Myers & Wilks (1999)† | CFS | 85 | 21.2 (34.4) | 52.4 (31.0) | 27.2 (22.3) | 49.3 (28.1) | 50.9 (29.5) | 60.0 (43.9) | 61.8 (21.8) | 39.8 (21.9) |
| Control | 9332 | 91.4 (23.2) | 92.5 (13.4) | 64.0 (18.2) | 91.3 (15.8) | 86.3 (17.9) | 85.6 (29.3) | 75.4 (16.3) | 78.8 (15.7) | |
| Nijs, Almond, | CFS Pre-Exerc. | 24 | 22.9 (32.9) | 52.5 (21.2) | 40.6 (17.2) | 51.0 (23.3) | 60.4 (22.0) | 70.7 (42.2) | 55.5 (17.5) | 25.0 (12.1) |
| Becker, Truijen, & Paul (2008)b | CFS Post-Exerc. | 24 | 18.8 (28.8) | 49.8 (19.6) | 36.3 (13.7) | 47.4 (20.8) | 55.6 (22.2) | 58.3 (43.1) | 56.8 (17.3) | 25.4 (12.9) |
| CFS 24 Hours-Post | 23 | 16.3 (26.8) | 50.7 (20.1) | 36.5 (12.7) | 48.9 (18.0) | 55.3 (20.3) | 63.8 (41.3) | 57.7 (13.9) | 27.8 (14.6) | |
| Nijs & Thielemans (2008)c | CFS Dutch | 100 | 14.5 (26.4) | 50.6 (22.1) | 36.1 (19.2) | 43.5 (23.2) | 41.0 (22.0) | 58.7 (45.7) | 57.6 (21.3) | 21.2 (13.7) |
| CFS French | 48 | 20.3 (31.7) | 46.0 (24.0) | 22.1 (22.6) | 38.5 (24.2) | 31.4 (22.3) | 45.8 (43.3) | 49.8 (21.4) | 20.0 (12.5) | |
| Reeves et al. (2005)† * | CFS | 43 | 18.0 (28.5) | 53.3 (21.5) | 18.6 (12.5) | 50.0 (22.7) | 41.7 (15.7) | 55.8 (42.2) | 66.4 (18.5) | 51.2 (20.4) |
| ISF | 57 | 60.7 (37.2) | 77.5 (18.9) | 37.3 (18.5) | 74.2 (19.6) | 59.7 (20.0) | 76.5 (33.5) | 74.6 (14.0) | 70.1 (13.6) | |
| Control | 64 | 88.8 (25.0) | 89.7 (13.2) | 72.3 (13.0) | 94.8 (10.6) | 77.9 (16.0) | 96.1 (13.8) | 87.5 (9.0) | 85.2 (13.3) | |
| Scheeres, Wensing, Severens, Adang, & Bleijenberg (2008)d | CFS-All | 263 | NR | 52.3 (22.6) | NR | NR | NR | NR | NR | NR |
| CFS-MHC | 114 | NR | 52.4 (23.0) | NR | NR | NR | NR | NR | NR | |
| CFS-ECCF | 149 | NR | 53.2 (22.4) | NR | NR | NR | NR | NR | NR | |
| Taillefer, Kirmayer, Robbins, & Lasry (2002) | CFS | 45 | 10.8 (22.5) | 49.6 (25.0) | 22.7 (18.7) | 38.1 (25.0) | 45.3 (24.4) | 58.3 (45.0) | 65.8 (20.4) | 38.8 (21.3) |
| Multiple Sclerosis | 40 | 33.6 (42.4) | 42.8 (33.5) | 38.8 (21.6) | 65.8 (27.1) | 65.1 (32.6) | 66.7 (37.2) | 67.7 (15.3) | 47.7 (24.8) |
| Study | Population | n | Role-Phys. M (95% CI) |
Physical Func. M (95% CI) |
Vitality M (95% CI) |
Social Func. M (95% CI) |
Bodily Pain M (95% CI) |
Role-Emo. M (95% CI) |
Ment. Health M (95% CI) |
Gen. Health M (95% CI) |
|---|---|---|---|---|---|---|---|---|---|---|
| Herrell et al. (2002)†e | CF | 119 | 21 (16–26) | 49 (46–53) | 22 (19–35) | 43 (40–47) | 40 (36–45) | 66 (59–72) | 62 (59–65) | 36 (33–40) |
| Twin Control | 119 | 89 (83–93) | 90 (86–94) | 65 (62–68) | 88 (84–91) | 74 (70–78) | 87 (80–93) | 75 (72–78) | 81 (78–84) | |
| CFS-Exclusion | 74 | 19 (14–25) | 52 (47–57) | 21 (17–25) | 44 (40–48) | 40 (36–45) | 69 (61–77) | 62 (58–63) | 35 (32–39) | |
| Twin Control | 74 | 90 (84–95) | 92 (88–97) | 70 (66–73) | 91 (87–95) | 76 (71–81) | 90 (82–98) | 77 (74–81) | 85 (81–89) | |
| CFS | 22 | 13 (4–22) | 45 (38–53) | 16 (10–22) | 44 (35–52) | 36 (26–46) | 58 (41–74) | 55 (47–62) | 32 (26–38) | |
| Twin Control | 22 | 89 (80–98) | 95 (87–100) | 72 (66–78) | 90 (82–99) | 79 (69–89) | 91 (75–100) | 80 (72–87) | 85 (85–91) | |
| Nater et al. (2006)† | CFS | 43 | 18.0 (9.3–26.8) | 53.3 (46.6–59.9) | 18.6 (14.8–22.5) | 50.0 (43.0–57.0) | 41.7 (36.9–46.5) | 55.8 (42.8–68.8) | 66.4 (60.7–72.1) | 51.2 (44.9–57.5) |
| Control | 60 | 88.8 (82.3–95.2) | 89.7 (86.2–93.1) | 72.3 (689–75.6) | 94.8 (92.0–97.5) | 77.9 (73.7–82.0) | 96.1 (92.5–99.7) | 87.5 (85.2–89.8) | 85.2 (81.8–88.7) | |
| Wells (2009)f | CFS Pet | 98 | 13.4 (8.0–18.8) | 30.8 (25.9–35.8) | 20.1 (16.7–23.6) | 31.8 (26.6–37.1) | 37.0 (30.9–43.2) | 67.9 (58.8–76.9) | 64.4 (60.0–68.9) | 30.4 (26.4–34.3) |
| CFS No Pet | 70 | 12.3 (5.3–19.3) | 32.0 (26.6–37.5) | 18.8 (15.0–22.6) | 33.5 (28.0–39.0) | 38.3 (32.2–44.4) | 66.7 (55.8–77.5) | 62.8 (57.6–68.1) | 26.4 (22.5–30.4) | |
| Combined | 168 | 12.9 (8.7–17.2) | 31.3 (27.7–35.0) | 19.6 (17.0–22.1) | 32.5 (28.8–36.3) | 37.6 (33.2–41.9) | 67.4 (60.5–74.2) | 63.8 (60.4–67.1) | 28.7 (25.9–31.6) |
Notes:
NR = Not Reported; CI = Confidence Interval; Psy = Comorbid psychiatric condition; Psy Exclusion = Does not meet criteria for CFS due to exclusionary psychiatric condition; ISF = Insufficient Symptoms or Fatigue
Study included in analysis of difference scores
Study using the Reeves et al. (2005) empiric case definition
This study evaluated the Reeves et al. (2005) empiric case definition where MDD = Major Depressive Disorder; MDD/CFS = Patients with MDD classified as CFS using this case definition.
This study explored post-exertional malaise where Pre-Exerc. = Assessment before exercise; Post-Exerc. = Assessment after exercise; 24 Hrs-Post = Assessment 24 hours after exercise.
This study compares the SF-36 scores of native Dutch speaking participants with native French speaking participants.
The MHC group was participants with CFS recruited from a community mental health center and the ECCF group was participants with CFS recruited from the Nijmegen Expert Centre for Chronic Fatigue.
This study looked at pairs of monozygotic twins where one twin was sick, meeting a diagnosis of: chronic fatigue (CF; ≥ 6 months fatigue); CFS-Excl. (≥ 6 months fatigue and atleast four minor symptoms) but exclusionary condition; or CFS (≥ 6 months fatigue and at least four minor symptoms) with no exclusionary conditions; and the control was his or her healthy twin.
This study looked at differences in people with CFS that were pet owners versus non-pet owners.
Community-based sample
The data for the community-based CFS sample and the controls in the present investigation were derived from a larger community-based study of the prevalence of CFS [6]. This larger study was carried out in three stages. Stage 1 involved administering an initial telephone screening questionnaire in order to assess for symptoms of CFS. Procedures developed by Kish [7] were used to select one adult from each household. Birth dates for each adult were gathered, and the person with the most recent birthday was selected to be interviewed. We screened a random sample of adults (aged 18 years or older) between September 1995 and May 1997. A stratified random sample of several neighborhoods in Chicago, which were 10–15 minutes from the site of the medical examinations, was utilized. We called 28,673 residential/working telephone numbers and were able to complete the interview for 18,675 adults (65.1% completion rate). If interviewees reported during the stage 1 interview that they had been suffering from severe fatigue, extreme tiredness, or exhaustion that had been present for a period of six months or longer, they were determined to have chronic fatigue and were asked additional questions which assessed more specific dimensions of their fatigue.
The stage 1 screen revealed that of the 18,675 interviewees, 780 (4.2%) had chronic fatigue. Of these, 408 reported chronic fatigue and the concurrent occurrence of four or more of the core symptoms listed in the Fukuda et al. [1] CFS case definition, and they were defined as CFS-like. The suffix ‘like’ was used to clarify that individuals in this group only met criteria by self-report, and thus did not necessarily qualify as having a final diagnosis of CFS rendered by physician evaluation.
Stage 2 involved administering a semi-structured psychiatric interview, and stage 3 involved administering a complete physical examination (see Jason et al. [6] for more details). There were 166 of the 408 CFS-like individuals who agreed to complete a structured psychiatric interview in stage 2 and also agreed to undergo a complete physical examination in stage 3. There were no significant differences on sociodemographic or fatigue scores between those 166 screened positive (CFS-like) participants and the 242 screened positive (CFS-like) non-participants.
A team of four physicians and a psychiatrist were responsible for making final diagnoses. Two physicians independently rated each file using the Fukuda et al. [1] criteria for CFS. If a disagreement occurred, a third physician rater was used. Each of the 166 CFS-like participants evaluated in stages 2 and 3 was diagnosed by the physician review panel in one of three ways: (1) individuals evaluated as meeting the Fukuda et al. criteria for CFS were given a final diagnosis of CFS (n = 32); (2) individuals not meeting full CFS criteria, but possessing unexplained chronic fatigue and no exclusionary medical conditions detected in evaluation were given a final diagnosis of idiopathic chronic fatigue (ICF; n = 45); and (3) individuals with exclusionary medical or psychiatric conditions detected in evaluation were given a final diagnosis of CF-explained (n = 89). The 32 individuals who met criteria for CFS were used in the present investigation for the community-based CFS sample.
The control group was composed of individuals selected randomly from those 18,260 screened negatives (groups included participants with no reported prolonged or chronic fatigue). Among the 199 controls randomly selected for evaluation following Stage 1, the physician review team identified 47 individuals who were medically worked up and found not to have CFS. This control group was used for comparison with both the community-based CFS sample and the tertiary care CFS sample in the present study. See Jason et al. [6] for more details regarding recruitment and procedure.
Regarding sociodemographic characteristics, of the 32 individuals with CFS, 71.9% were women. In terms of ethnicity, 46.9% were Caucasian, 28.1% were Latino, 15.6% were African American, and 9.4% were of another ethnicity. Among the CFS group, 40.6% were married, 62.5% had children, 53.1% were working either full time or part time, 84.4% had at least a high school diploma or higher education, 75.0% were at least at the occupational level of a skilled or clerical worker and higher, and the average age was 40.8.
Of the 47 controls, 48.8% were women. In terms of ethnicity 58.1% were Caucasian, 27.9% were African American, 7.0% were Latino, and 7.0% were of another ethnicity. Among the control group, 27.9% were married, 27.9% had children, 81.4% were working either full time or part time, 93.0% had at least a high school diploma or higher education, 83.7% were at least at the occupational level of a skilled or clerical worker and higher, and the average age was 41.5.
With respect to gender, significantly higher frequencies of females than males were observed in CFS than in the control group. In terms of work status, a significantly higher number of individuals in the control group reported working full-time when compared with individuals in CFS group, who were more likely to be unemployed, on disability, or working part-time. Individuals in CFS and control groups did not differ significantly with respect to racial/ethnic identification, age, and marital status.
Tertiary care sample
The data for the tertiary care CFS sample in the present investigation were baseline data derived from a larger non-pharmacological interventions study for CFS [8]. Participants were recruited from a variety of sources including physician referrals, newspaper advertisements, and CFS support groups. All participants were required to be at least 18 years of age, not pregnant, able to read and speak English, and considered to be physically capable of attending the scheduled sessions.
A semi-structured psychiatric interview (the Structured Clinical Interview for DSM-IV) [9] was administered to establish Axis I psychiatric diagnoses and rule out exclusionary psychiatric diagnoses according to the Fukuda et al. [1] criteria. Next, an in-depth medical and laboratory evaluation similar to that which was given in the community-based sample was conducted (see Jason et al. [8] for more details). One hundred and fourteen individuals were diagnosed with CFS based on the Fukuda et al. [1] criteria and enrolled in the study. See Jason et al. [8] for more details regarding recruitment and procedure.
In regards to sociodemographic characteristics, 83.3% of the 114 participants were female and the average age was 43.8 years (standard deviation = 11.6). Concerning ethnicity, 87.7% of the participants were White, 4.4% were African American, 4.4% were Latino, and 3.5% were Asian American. Regarding marital status, 49.1% of the participants were married/living with a partner, 33.3% were single, and 17.6% were either divorced or separated. As for work status, 40.4% of the participants were working or full-time students, and 59.6% were not working or were part-time students. Concerning education, 47.4% of the participants had earned a standard college degree, 21.8% had a graduate or professional degree, 21.1% had partial college, and 9.7% had a high school/GED degree or less.
Measure
CFS and control participants in the community-based study and CFS participants in the tertiary care study completed the SF-36 [2], a reliable and valid measure that discriminates between gradations of disability. This measure includes eight subscales measuring various aspects of disability: Physical Functioning, Social Functioning, Role-Physical, Role-Emotional, Vitality, Bodily Pain, General Health, and Mental Health. Scores on each subscale range from 0 to 100, with higher scores indicating better health or functioning. Reliability and validity studies for the SF-36 have shown adequate internal consistency, discriminant validity among subscales, and substantial differences between patient and nonpatient populations in the pattern of scores [10,11].
Statistical analysis
The statistical software package used for data analysis was PASW (formerly SPSS) for Windows, version 17.0. A Receiver Operating Characteristic (ROC) curve analysis [12] was used to evaluate the ability of the SF-36 subscales to discriminate between patients with CFS in the community-based sample and controls, and between those in the tertiary care sample and the same controls. The ROC curve graphically represents the probability of true positive results in diagnosis as a function of the probability of false positive results of this test. Sensitivity is defined as the probability that the test correctly classifies a CFS subject as positive. A true positive is defined as a participant who scores positive on the SF-36 subscale test for CFS and actually has the illness, whereas a false positive occurs when a participant whose SF-36 subscale tests positive for CFS but the person does not have the illness. Specificity involves a test correctly classifying a non-ill participant as negative. A true negative is defined as a participant who tests negative on the SF-36 subscale test for CFS and does not have the illness, whereas a false negative is defined as a participant who tests negative on the SF-36 subscale test for CFS and actually has the illness.
An ROC analysis is produced by plotting the sensitivity versus 1 - specificity for all cutoff points of the SF-36 subscales. The area under the curve (AUC) is an indicator of the discriminatory ability of the scale: a straight line (area = 0.5) means that the scale is doing no better than chance in classifying CFS and non-CFS, while a perfect scale would have an ROC curve with an area of 1. The area under the ROC curve is a summary measure that essentially averages diagnostic accuracy across the spectrum of test values. The informative area under the ROC curve ranges from 0.5 to 1.0, and not from 0.0 to 1.0, as would the area under a probability distribution curve. An AUC of 0.99 means that 99% of the time a randomly selected individual from the CFS group will more adequately fulfill the SF-36 subscale criterion than a randomly selected individual from the control group.
Results
Literature review
Table 1 lists the 17 studies that used the SF-36 in CFS studies with at least two comparison groups. Of these, the nine studies used to calculate difference scores are noted in table 1. Differences between the CFS and non-ill control groups were: 70.1 for Role-Physical; 48.7 for Vitality; 46.0 for Social Functioning; 41.5 for Physical Functioning; 38.5 for Bodily Pain; 38.4 for General Health; 30.9 for Role-Emotional; and 20.9 for Mental Health. These findings suggest that Role-Physical is clearly the best discriminator between CFS and controls, with the next groups being Vitality, Social Functioning, Physical Functioning, Bodily Pain, and General Health; and dimensions that showed the least discrimination between the CFS and control groups were Role-Emotional and Mental Health.
ROC analyses
Table 2 presents data on the AUC, standard error, and 95% confidence intervals. A test needs an AUC threshold of between 90–100% to have diagnostic meaning [28,29]. For the community-based sample, none of the scales had an AUC above 0.90, although Vitality, Role-Physical, and Social Functioning had 0.86 or above AUC. For the tertiary care CFS sample, three subscales had an AUC of 0.91, and they included Vitality, Role-Physical, and General Health. Physical Functioning and Social Functioning had an AUC of 0.87. Only Vitality, Social Functioning, and Role-Physical emerged as within the top 4 for each sample. For both samples, Role-Emotional and Mental Health had the lowest thresholds, similar to what had been found in the literature review from table 1. From these findings, taking into consideration both samples, it appears that Vitality, Social Functioning, and Role-Physical are among the highest AUCs for both samples. Given these findings, there is an argument for focusing on only those three subscales with the highest AUC thresholds (Vitality, Social Functioning, and Role-Physical).
Table 2.
AUC Values, Standard Errors (SE), and Confidence Intervals (CI) for Community-Based and Tertiary Care Samples
| SF-36 Subscale | Community-Based Sample | Tertiary Care Sample | ||||
|---|---|---|---|---|---|---|
| AUC | SE | 95% CI† | AUC | SE | 95% CI† | |
| Vitality | .88 | .04 | .81 – .96 | .91 | .03 | .85 – .97 |
| Social Functioning | .87 | .04 | .79 – .95 | .87 | .04 | .79 – .94 |
| Role-Physical | .86 | .04 | .77 – .94 | .91 | .03 | .84 – .97 |
| Bodily Pain | .85 | .04 | .77 – .94 | .86 | .04 | .79 – .93 |
| Physical Functioning | .84 | .05 | .75 – .93 | .87 | .04 | .80 – .94 |
| General Health | .80 | .05 | .70 – .90 | .91 | .03 | .86 – .97 |
| Mental Health | .75 | .06 | .63 – .86 | .71 | .05 | .61 – .80 |
| Role-Emotional | .67 | .07 | .54 – .80 | .63 | .05 | .53 – .73 |
Confidence intervals not including .5 are significant at the p < .05 level
We next focused on optimal cutoff points for these two subscales. For Vitality, a score of 35 or less in the community-based sample had 90% sensitivity and 81% specificity; whereas in the tertiary care sample, a score of 35 or less had 89% sensitivity and 81% specificity. For Social Functioning, a score of 62.5 or less in the community-based sample had 87% sensitivity and 72% specificity; whereas in the tertiary care sample, a score of 62.5 or less had 88% sensitivity and 72% specificity. For the Role-Physical subscale, a score of 50 or less had 97% sensitivity and 70% specificity for the community-based sample; whereas for the tertiary care sample, a score of 25 or less had 96% sensitivity and 79% specificity (there are only 4 items on this scale, so a person can only get a score of 0, 25, 50 or 100). Based on these findings, we used a cut-off score of ≤ 35 or less for Vitality, ≤ 62.5 for Social Functioning, and ≤ 50 for Role-Physical.
As it is unclear if the optimal diagnosis for CFS would require a person to meet the cutoff for one or more subscales (i.e., ≤ 35 on Vitality, ≤ 62.5 on Social Functioning, and/or ≤ 50 on Role-Physical), we conducted ROCs examining the total number of scales met by participants (range 0 to 3). For the community-based sample, the ROC resulted in an AUC of 0.89 (SE = 0.04, 95% CI 0.82 to .96, p < 0.001). If the person met the cutoff for one or more subscales, the sensitivity was 100% and the specificity was 64%. If the person met the cutoff for two or more subscales, the sensitivity was 93% and the specificity was 75%. If the person met the cutoff for three subscales, the sensitivity was 79% and the specificity was 85%.
For the tertiary care sample, the ROC resulted in an AUC of 0.90 (SE = 0.03, 95% CI 0.83 to .96, p < 0.001). If the person met the cutoff for one or more subscales, the sensitivity was 100% and the specificity was 64%. If the person met the cutoff for two or more subscales, the sensitivity was 96% and the specificity was 75%. If the person met the cutoff for three subscales, the sensitivity was 81% and the specificity was 85%.
If investigators are attempting to identify as many as possible cases of CFS from either community-based or tertiary care samples, then higher sensitivity is better. Even though specificity might be lower, when a medical and psychiatric examination is conducted, those individuals who have other fatigue-causing medical or psychiatric conditions would be excluded from the CFS samples. Therefore, we would recommend that meeting two or more of the following three subscales: Vitality, Social Functioning, or Role-Physical, with our designated cutoff scores, should be used to designate substantial reductions.
Discussion
Although the SF-36 [2] has been recommended as a good measure to achieve the goal of determining substantial reductions in functioning [3], it is important for a measure assess reductions in functioning to have high sensitivity by identifying the majority positive CFS cases, while also having high specificity to successfully identify non-ill cases. Unfortunately, few studies have used rigorous statistical methods to determine which variables and cutoff scores to use to assess substantial reductions in functioning among people with CFS. The present study found that the SF-36 Vitality, Social Functioning, and Role-Physical subscales have the best sensitivity and specificity using two different samples of individuals with CFS. The Vitality scale measures items that assess feeling full of pep and energy, as well as those that focus on feeling worn out or tired. Social functioning is assessed by items that involve interference with normal social activities with family, friends, neighbors, or groups. The Role-Physical subscale focuses on items assessing the need to cut down on or limit work or other activities, as well as accomplishing less than one might like. In other words, these three subscales capture significant limitations in the person’s energy level, involvement in social activities, and life accomplishments. Of interest, these subscales not only had the best AUC thresholds, but in Table 1, they were the subscales that had the largest difference scores between CFS and control groups. These three subscales might assess substantial reductions in “previous levels of occupational, educational, social, or personal activities” [1, p. 956].
Reeves et al.’s [4] empiric CFS case definition recommended using scores below the 25th percentile on any one of the following four SF-36 subscales to measure disability: Physical Functioning (less than or equal to 70), Role-Physical (less than or equal to 50), Social Functioning (less than or equal to 75), or Role-Emotional (less than or equal to 66.7). If a person met the criteria for Role-Emotional functioning, and did not meet any other criteria, they would meet the CFS disability criteria for the case definition. The literature review outlined in table 1 indicates that among the different subscales, the CFS versus non-ill control groups have among the least difference for the Role-Emotional subscale. In a study by Jason et al. [16] (See table 1), the mean Role-Emotional score for the chronic fatigue psychiatric exclusion group was 25.6 (SD = 34.7) versus 45.2 (SD = 45.6) for the CFS group. Therefore, using Reeves et al.’s Role-Emotional cutoff of less than or equal to 66.7 would select the majority of those with chronic fatigue explained by psychiatric reasons as meeting the CFS disability criterion. In addition, Jason et al. [5] found that almost all of those with clinical depression would meet the CFS disability criterion, and this is in part due to the inclusion of the Role-Emotional subscale. Furthermore, the AUC for the Role-Emotional subscale was the worst among the eight SF-36 subscales in table 2 for discriminating patients with CFS from controls.
It could be argued from both tables 1 and 2 that other SF-36 scales have somewhat good discrimination between the CFS and control groups including Physical Functioning, Bodily Pain, and General Health. Physical Functioning was also a subscale recommended for inclusion by Reeves et al. [1]. When we examined the cutoff score to select 90% of CFS cases for Physical Functioning, a very high score would be needed (≤ 85) and this would have poor specificity (62%). Within the data set described by Jason et al. [16], it is only with items assessing more vigorous activities that a separation occurs between those with CFS and those who are chronically fatigued due to psychiatric reasons. When examining the raw scores of the individual items comprising the Physical Functioning subscale, on relatively non-strenuous activities (e.g., bathing or walking one block), there are no significant differences between the CFS group and the control group. As the intensity of the activity increases, both fatigued groups become significantly differentiated from the control group. However, it is only for “walking more than one mile” that the CFS group demonstrates greater functional impairment than the chronic fatigue explained by psychiatric reasons group.
We considered both Bodily Pain as well as General Health beyond the scope of what was to be measured to represent substantial reductions in occupational, educational, social, or personal functioning. In addition, the cutoff for Bodily Pain (≤ 74) to detect 90% of CFS cases in the tertiary care sample had a poor specificity (66%). In addition, a major problem with the General Health subscale was that for the community-based sample, to accurately select 90% of CFS cases, an extremely high cut off score would be needed (≤ 80) and this score would have very poor specificity (53%).
The reason for using a tertiary care and community-based sample was to assess whether comparable findings would emerge with the two different samples. As shown in table 2, the community-based sample had, in general, lower AUCs than the tertiary care sample. The difference in AUCs suggests that the community-based sample was less accurately discriminated from controls than was the tertiary care sample. In CFS research, it has been found that participants recruited from the community tend to be less impaired than those recruited from tertiary care settings [30]. It is likely that our tertiary care sample was more impaired, making them more distinguishable from controls on the SF-36 subscales.
There are several limitations in this study. First, the community-based sample of participants was relatively small. In addition, there was only one control group, and it was used to contrast two different samples of individuals with CFS. In addition, our study recommends using three subscales of the SF-36, but some might argue that more subscales need to be included to provide a more comprehensive appraisal of reduced functioning. Finally, we only tested substantial reductions on self-report measures of functional disability, and there might be other more objective approaches that are worthy of consideration, including those measuring either activity, such as actigraphs [31], or more biological variables, such as Orthostatic Intolerance [32]. However, objective measures of activity can be costly, difficult to administer, and involve a greater burden on participants than completing a self-report measure. The SF-36 proved to be useful in operationalizing substantial reductions for research purposes.
In summary, our study suggests that three subscales of the SF-36 could be used to assess substantial reductions in different areas of functioning, as these scales have the best sensitivity and specificity across two CFS samples, and have demonstrated the largest discrimination between CFS and control groups in previous studies. It is hoped that future studies will assess whether these three subscales are effective in documenting substantial reductions in functioning in other CFS samples. Clearly, great care needs to be exercised when determining which scales and with what cutoff points should indicate that substantial reductions CFS criterion has been reached for CFS samples in order to identify true CFS cases.
Acknowledgments
The authors appreciate the financial assistance provided by the National Institute of Allergy and Infectious Diseases (grant number AI055735).
Footnotes
Declarations of Interest
The authors appreciate the financial assistance provided by the National Institute of Allergy and Infectious Diseases (grant numbers AI36295 and AI055735).
References
- 1.Fukuda K, Straus SE, Hickie I, Sharpe MC, Dobbins JG, Komaroff A. The chronic fatigue syndrome: a comprehensive approach to its definition and study. Ann Intern Med. 1994;121:953–959. doi: 10.7326/0003-4819-121-12-199412150-00009. [DOI] [PubMed] [Google Scholar]
- 2.Ware JE, Snow KK, Kosinski M. SF-36 Health Survey: Manual and Interpretation Guide. Lincoln, RI: Quality Metric Inc; 2000. [Google Scholar]
- 3.Reeves WC, Lloyd A, Vernon SD, Klimas N, Jason L, Bleijenberg G, Evengard B, White PD, Nisenbaum R, Unger ER. Identification of ambiguities in the 1994 chronic fatigue syndrome research case definition and recommendations for resolution. BMC Health Services Research. 2003;3:25. doi: 10.1186/1472-6963-3-25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Reeves WC, Wagner D, Nisenbaum R, Jones JF, Gurbaxani B, Solomon L, Papanicolaou D, Unger ER, Vernon SD, Heim C. Chronic fatigue syndrome – a clinical empirical approach to its definition and study. BMC Medicine. 2005;3:19. doi: 10.1186/1741–7015–3–19. (available at: http://www.biomedcentral.com/content/pdf/1741-7015-3-19.pdf) [DOI] [PMC free article] [PubMed]
- 5.Jason LA, Najar N, Porter N, Reh C. Evaluating the centres for disease control’s empirical chronic fatigue syndrome case definition. J Disabil Policy Stud. 2009;20:93–100. doi: 10.1177/1044207308325995. [DOI] [Google Scholar]
- 6.Jason LA, Richman JA, Rademaker AW, Jordan KM, Plioplys AV, Taylor RR, McCready W, Huang C, Plioplys S. A community-based study of chronic fatigue syndrome. Arch Intern Med. 1999;159:2129–2137. doi: 10.1001/archinte.159.18.2129. [DOI] [PubMed] [Google Scholar]
- 7.Kish L. Survey Sampling. NY: Wiley; 1965. [Google Scholar]
- 8.Jason LA, Torres-Harding S, Friedberg F, Corradi K, Njoku MG, Donalek J, Reynolds N, Brown M, Weitner BB, Rademaker A, Papernik M. Non-pharmacologic interventions for CFS: a randomized trial. J Clin Psychol Med Settings. 2007;14:275–296. [Google Scholar]
- 9.Spitzer RL, Williams JBW, Gibbon M, First MB. Structured Clinical Interview for the DSM-IV—non-patient edition (SCID—NP, Version 2.0) Washington, DC: American Psychiatric Press; 1995. [Google Scholar]
- 10.McHorney CA, Ware JE, Raczek AE. The MOS 36-Item Short-Form Health Survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Med Care. 1993;31:247–263. doi: 10.1097/00005650-199303000-00006. [DOI] [PubMed] [Google Scholar]
- 11.McHorney CA, Ware JE, Lu AW, Sherbourne CD. The MOS 36-item Short-Form Health Survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Med Care. 1994;32:40–66. doi: 10.1097/00005650-199401000-00004. [DOI] [PubMed] [Google Scholar]
- 12.Hanley JA, Mc Neil BJ. The meaning and use of the area under a receiver operating characteristics (ROC) curve. Radiology. 1982;143:29–36. doi: 10.1148/radiology.143.1.7063747. [DOI] [PubMed] [Google Scholar]
- 13.Boneva RS, Decker MJ, Maloney EM, Lin J, Jones JF, Helgason HG, Heim CM, Rye DB, Reeves WC. Higher heart rate and reduced heart rate variability persist during sleep in chronic fatigue syndrome: A population-based study. Autonomic Neuroscience: Basic and Clinical. 2007;137:94–101. doi: 10.1016/j.autneu.2007.08.002. [DOI] [PubMed] [Google Scholar]
- 14.Ciccone DS, Busichio K, Vickroy M, Natelson BH. Psychiatric morbidity in the chronic fatigue syndrome: are patients with personality disorder more physically impaired? J Psychosom Res. 2003;54:445–452. doi: 10.1016/s0022-3999(02)00525-1. [DOI] [PubMed] [Google Scholar]
- 15.Hassan IS, Bannister BA, Akbar A, Weir W, Bofill M. A study of the immunology of the chronic fatigue syndrome: correlation of immunologic parameters to health dysfunction. Clin Immunol and Immuopathol. 1998;87:60–67. doi: 10.1006/clin.1997.4512. [DOI] [PubMed] [Google Scholar]
- 16.Jason LA, King CP, Taylor RR, Kennedy C. Defining chronic fatigue syndrome: methodological challenges. J Chronic Fatigue Syndr. 2000;7:17–32. [Google Scholar]
- 17.King C, Jason L. Improving the diagnostic criteria and procedures for chronic fatigue syndrome. Biol Psychol. 2005;68:87–106. doi: 10.1016/j.biopsycho.2004.03.015. [DOI] [PubMed] [Google Scholar]
- 18.Komaroff AL, Fagioli LR, Doolittle TH, Gandek B, Gleit MA, Guerriero RT, Kornish J, Ware NC, Ware JE, Bates DW. Health status in patients with chronic fatigue syndrome and in general population ad disease comparison groups. Am J Med. 1996;101:281–290. doi: 10.1016/S0002-9343(96)00174-X. [DOI] [PubMed] [Google Scholar]
- 19.Moss-Morris R, Chalder T. Illness perceptions and levels of disability in patients with chronic fatigue syndrome and rheumatoid arthritis. J Psychosom Res. 2003;55:305–308. doi: 10.1016/s0022-3999(03)00013-8. [DOI] [PubMed] [Google Scholar]
- 20.Myers C, Wilks D. Comparison of Euroqol EQ-5D and SF-36 in patients with chronic fatigue syndrome. Qual Life Res. 1999;8:9–16. doi: 10.1023/a:1026459027453. [DOI] [PubMed] [Google Scholar]
- 21.Nijs J, Almond F, Becker PD, Truijen S, Paul L. Can exercise limits prevent post-exertional malaise in chronic fatigue syndrome? An uncontrolled clinical trial. Clin Rehabil. 2008;22:426–435. doi: 10.1177/0269215507084410. [DOI] [PubMed] [Google Scholar]
- 22.Nijs J, Thielemans A. Kinesiophobia and symptomatology in chronic fatigue syndrome: A psychometric study of two questionnaires. Psychol Psychother. 2008;81:273–283. doi: 10.1348/147608308X306888. [DOI] [PubMed] [Google Scholar]
- 23.Scheeres K, Wensing M, Severens H, Adang E, Bleijenberg G. Determinants of health care use in chronic fatigue syndrome patients: a cross-sectional study. J Psychosom Res. 2008;65:39–46. doi: 10.1016/j.jpsychores.2008.03.015. [DOI] [PubMed] [Google Scholar]
- 24.Taillefer SS, Kirmayer LJ, Robbins JM, Lasry J. Psychological correlates of functional status in chronic fatigue syndrome. J Psychosom Res. 2002;53:1097–1106. doi: 10.1016/s0022-3999(02)00566-4. [DOI] [PubMed] [Google Scholar]
- 25.Herrell R, Goldberg J, Hartman S, Belcourt M, Schmaling K, Buchwald D. Chronic fatigue and chronic fatigue syndrome: A co-twin control study of functional status. Quality of Life Research. 2002;11:463–471. doi: 10.1023/a:1015635113159. [DOI] [PubMed] [Google Scholar]
- 26.Nater UM, Wagner D, Solomon L, Jones JF, Unger ER, Papanicolaou DA, Reeves WC, Heim C. Coping styles in people with chronic fatigue syndrome identified from the general population of Wichita, KS. J Psychosom Res. 2006;60:567–573. doi: 10.1016/j.jpsychores.2006.04.001. [DOI] [PubMed] [Google Scholar]
- 27.Wells DL. Associations between pet ownership and self-reported health status in people suffering from chronic fatigue syndrome. J Altern Complement Med. 2009;15:407–413. doi: 10.1089/acm.2008.0496. [DOI] [PubMed] [Google Scholar]
- 28.Zou KH, O’Malley AJ, Mauri L. Receiver-operating characteristic analysis for evaluating diagnostic tests and predictive models. Circulation. 2007;115:654–657. doi: 10.1161/CIRCULATIONAHA.105.594929. [DOI] [PubMed] [Google Scholar]
- 29.Zweig MH, Campbell G. Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin Chem. 1993;39:561–577. [PubMed] [Google Scholar]
- 30.Jason LA, Plioplys AV, Torres-Harding S, Corradi K. Comparing symptoms of chronic fatigue syndrome in a community-based versus tertiary care sample. J Health Psychol. 2003;8:459–464. doi: 10.1177/13591053030084005. [DOI] [PubMed] [Google Scholar]
- 31.Tryon WW, Jason LA, Frankenberry E, Torres-Harding S. Chronic fatigue syndrome impairs circadian rhythm of activity level. Physiology & Behavior. 2004;82:849–53. doi: 10.1016/j.physbeh.2004.07.005. [DOI] [PubMed] [Google Scholar]
- 32.Streeten DH, Bell DS. The roles of orthostatic hypotension, orthostatic tachycardia, and subnormal erythrocyte volume in the pathogenesis of the chronic fatigue syndrome. Am J Med Sci. 2000;320:1–8. doi: 10.1097/00000441-200007000-00001. [DOI] [PubMed] [Google Scholar]