Operationalizing Substantial Reduction in Functioning Among Young Adults with Chronic Fatigue Syndrome

Abstract

Purpose

Chronic fatigue syndrome and myalgic encephalomyelitis are fatiguing illnesses that often result in long-term impairment in daily functioning. In reviewing case definitions, Thrope et al. (Fatigue 4(3):175–188, 2016) noted that the vast majority of case definitions used to describe these illnesses list a “substantial reduction” in activities as a required feature for diagnosis. However, there is no consensus on how to best operationalize the criterion of substantial reduction.

Method

The present study used a series of receiver operating curve (ROC) analyses to explore the use of the Medical Outcomes Study Short-Form-36 Health Survey (SF-36), designed by Ware and Shelbourne for operationalizing the substantial reduction criterion in a young adult population (18–29 years old). We compared the sensitivity and specificity of various cutoff scores for the SF-36 subscales and assessed their usefulness in discriminating between a group of young adults with a known diagnosis of chronic fatigue syndrome or myalgic encephalomyelitis (n = 98) versus those without that diagnosis (n = 272).

Results

The four top performing subscales and their associated cutoffs were determined: Physical Functioning ≤ 80, General Health ≤ 47, Role Physical ≤ 25, and Social Functioning ≤ 50. Used in combination, these four cutoff scores were shown to reliably discriminate between the patients and controls in our sample of young adults.

Conclusion

The implications of these findings for employing the substantial reduction criterion in both clinical and research settings are discussed.

Introduction

Chronic fatigue syndrome (CFS) and myalgic encephalomyelitis (ME) are illnesses characterized by fatigue, unrefreshing sleep, exacerbation of symptoms following exertion, and cognitive impairment, the combination of which often results in drastically reduced levels of daily functioning [3]. CFS has been estimated to affect about a million individuals in the USA [4]. Efforts to better understand the illness have been complicated by competing case definitions, each of which identifies differing combinations of symptoms required for the diagnosis of ME, CFS, or both. The majority of case definitions specify “substantial reduction in activity” as a key characteristic of the ME or CFS illness, making it one of the few criteria that remain consistent across case definitions. This standard requires evidence that an individual’s level of functioning has reduced significantly from pre-illness levels. However, most case definitions have not described a reliable system for characterizing this criterion [1]. For example, the case definition developed by Fukuda et al. [5] requires a decrease from pre-illness levels of occupational, educational, social, or personal activities. However, the authors do not specify methods for measuring such activity levels, or standards to define the extent of the functional decrease that should be considered substantial enough to warrant a diagnosis.

The Canadian Clinical Criteria (CCC) [6] case definition specifies different levels of substantial reduction, with a “mild reduction” defined as an approximately 50% decrease from pre-illness activity. It defines being housebound as a “moderate reduction,” being mostly bedridden as a “severe reduction,” and being completely bedridden as having a “very severe reduction” in functioning [6]. While this case definition specifies concrete diagnostic cutoffs, assessing previous activity levels continues to be a challenge as more often than not pre-illness data are not available [1].

Similarly, the Myalgic Encephalomyelitis International Consensus Criteria (ME-ICC) [7] requires that premorbid activity levels have been reduced by approximately 50%, and that the 50% standard should be evaluated through clinician dialogue with the patient over time. While this approach may prove sufficient for patients who have developed an ongoing rapport with their practitioner prior to the development of illness, one study found that 71% of patients saw four or more doctors before they were able to receive a diagnosis [8], making it unlikely that many patients will have a single physician capable of assessing a 50% reduction in activity over time.

Most recently, the Institute of Medicine (IOM) developed a case definition [9] for CFS under the illness label of “Systemic Exertion Intolerance Disease” (SEID). This IOM case definition defined a substantial reduction in activity as a decrease in pre-illness levels of personal, social, educational, or occupational activities. The authors provided several examples of scenarios which would fulfill the substantial reduction requirement, including job loss, severe limitations in one’s ability to complete personal or household chores, or being housebound or bedbound. Several tools and questionnaires to measure activity levels, including the Work and Social Adjustment Scale (WSAS) [10], the Medical Outcomes Study Short-Form-36 Health Survey (SF-36) [2], and the Lawton Instrumental Activities of Daily Living scale (IADL) [11], were suggested. Nonetheless, there continues to be ambiguity and disagreement around delineating an empirical method to operationalize the substantial reduction requirement, even if one or more of these tools are used.

Several studies have attempted to measure the substantial reduction criterion using the SF-36, a widely applied scale which assesses level of functional disability in adults [e.g., 1, 12–14]. For example, Thorpe et al. [1] assessed the association between patients’ self-reported percent reduction in the amount of hours spent on activities pre- and post-illness and the SF-36 subscales. They determined that both current hours spent on activities and percent reductions in activity hours were correlated with functioning on the Role Physical and Social Functioning subscales of the SF-36. In addition, Evans and Jason [15] found that the optimal time of recall reliability for patients with ME and CFS was 6 months. Together, these findings suggest that patient self-reports of current levels of functioning as measured by the SF-36 may be the most feasible operationalization of the substantial reduction criterion.

Reeves et al. [13] operationalized substantial reductions in activity by specifying three SF-36 subscale cutoff scores, setting the cutoffs at or below the 25th percentile of the US general population. Their operationalization required patients to meet cutoff scores on at least one of the following subscales: Physical Functioning (≤ 70), Role Physical (≤ 50), Social Functioning (≤ 75), or Role Emotional (≤ 66.7). However, Jason, Najar, Porter, and Reh [16] found that the inclusion of the Role Emotional subscale was problematic in that it would allow an individual with primarily social or emotional impairments to receive a CFS diagnosis, and that unless there was rigorous screening prior to the use of the specified SF-36 subscales, 38% of those diagnosed with major depressive disorder would meet the Reeves et al. [13] substantial reduction requirements and thus possibly be misdiagnosed with CFS. Additionally, Jason et al. [14] evaluated each SF-36 subscale using receiver operating curves (ROC) and determined the Role Emotional subscale to be the least accurate in distinguishing patients with CFS from controls.

The Jason et al. [14] ROC analysis used two adult samples of patients with CFS: a community-based sample and a tertiary care sample. They evaluated each SF-36 subscale and identified cutoff scores that optimally discriminated between patient and control groups. Based on their findings, the authors recommended that the substantial reduction criterion be operationalized by requiring patients to meet cutoff scores on at least two of the following subscales of the SF-36: the Role Physical subscale (≤ 50), the Social Functioning subscale (≤ 62.5), or the Vitality subscale (≤ 35). While these standards are helpful in that they account for physical and social impairments, they were developed using an adult population; the mean ages for the community-based sample and tertiary care patient samples were 40.8 and 43.8 years, respectively. Thus, the Jason et al. [14] operationalization may have limited applicability to younger patients, as baseline levels of functioning tend to decrease with age [17–19]. For example, Jenkinson et al. [17] found that the mean Physical Functioning score for women aged 18–24 was 90.1, whereas 45–54 and 55–64-year-old women had average scores of 84.8 and 74.8, respectively. The present study used ROC analyses to examine SF-36 subscale cutoff scores in a young adult sample (age 18–29) in order to reassess the operationalization of the substantial reduction criterion in that patient population.

Method

Sample

CFS has a prevalence rate of 0.42% in the US adult population [4]. Therefore, identifying a large sample of young adults diagnosed with CFS required the combination of information across several patient databases. By pooling de-identified participants from previous study samples, a sample of 98 patients between the ages of 18 and 29 years old was collected. Participants were drawn from the following five databases:

1. Newcastle sample. The participants in the Newcastle sample [20] were recruited through a tertiary care setting in Newcastle upon Tyne in the UK. They were at least 18 years old and were diagnosed with CFS by a specialist following a comprehensive medical examination at the Newcastle-upon-Tyne Royal Victoria Infirmary.

2. Norway samples (1 and 2). Participants in the first Norway sample [21] were recruited from towns in southern Norway, the suburbs of Oslo, and nearby communities. They were recruited by a variety of methods: patient education programs, patient organizations, the Oslo University Hospital website, and physician referrals. Diagnoses were confirmed with the patient’s physician. Participants in the second Norway sample [22] were recruited from an inpatient medical setting and an outpatient clinic. Physicians referred patients for a possible CFS diagnosis, and these individuals received an evaluation that included a comprehensive medical exam and detailed medical history to rule out exclusionary conditions. Once the suspected diagnosis was confirmed and informed consent was obtained, participants completed the study materials.

3. BioBank sample. Participants for the BioBank sample [23] were recruited through the Solve ME/CFS Initiative. They were diagnosed by a CFS specialist and were recruited through their physicians. Participants were recruited internationally but were required to read and write English and be at least 18 years of age.

4. Chronic Illness Symptom Study sample. Participants for the Chronic Illness Symptom Study sample [24] were recruited through postings on internet forums, advocacy websites, social media, and national CFS foundations. Participants were at least 18 years of age and had a current, self-reported diagnosis of CFS.

5. Prospective Health Study (control) sample. Control participants (n = 272) were drawn from the Prospective Health Study. This study is currently underway and has been recruiting healthy freshmen and sophomore college students to participate in the baseline wave of data collection, which consists of an online questionnaire and a small blood sample [25]. These students were then tracked via the campus health center to determine who developed infectious mononucleosis (IM) over the course of their college experience. Those who developed IM were then invited to participate in three additional waves of data collection: during their mono illness and at 6 and 12 months following their IM diagnosis. The goal of the study is to uncover pre-illness baseline data that may predict the development of CFS following IM. For the purposes of the present study, the control group derived from this sample included those who were fully enrolled in the baseline stage of the Prospective Health Study and then graduated from the university (and thus effectively left the study follow-up participant pool) without developing IM.

Sample Demographics

Table 1 shows the demographic breakdown for the patient and control groups. The age range for “young adults” was set at 18–29 years old. However, because the control group was limited to younger college students, there was a significant (though arguably not substantive) age difference found between patients and controls. The control group was also more diverse in terms of ethnicity, race, and gender than those with CFS (e.g., 43.4% male controls versus 12.2% male patients). In addition, although research has shown comparable scores of the SF-36 between males and females in general populations [26], Jenkinson et al. [17] described small gender differences in average subscale scores for young adults in the USA. In order to compensate for these differences, we also compared two subsamples from our total data set: (1) those 18–25 years old (controls = 268; patients = 66) and (2) only females 18–25 years old (controls = 150; patients = 58). The demographic breakdown of each of these subsamples is reported in Table 1.

Table 1.

Demographic characteristics of the sample

	Ages 18–29 (total sample)		Ages 18–25		Ages 18–25 females only
	Controls (N = 272)	Patients (N = 98)	Controls (N = 268)	Patients (N = 66)	Controls (N = 150)	Patients (N = 58)
	M (SD)	M (SD)	M (SD)	M (SD)	M (SD)	M (SD)
Age	20.2 (1.3)	23.8 (3.1)*	20.1 (0.9)	22.1 (2.1)*	20.0 (0.8)	22.1 (2.1)*
	n (%)	n (%)	n (%)	n (%)	n (%)	n (%)
Gender
Female	154 (56.6)	86 (87.8)*	150 (56.0)	58 (87.9)*	150 (100.0)	58 (100.0)
Male	118 (43.4)	12 (12.2)*	118 (44.0)	8 (12.1)*	-	-
Race
Caucasian/White	170 (64.6)	96 (98.0)*	166 (64.1)	66 (100.0)*	87 (60.8)	58 (100.0)*
Asian/Pacific Islander	47 (17.9)	1 (1.0)*	47 (18.1)	0 (0.0)*	27 (18.9)	0 (0.0)*
Other or multiracial	46 (17.5)	1 (1.0)*	46 (17.8)	0 (0.0)*	29 (20.3)	0 (0.0)*
Ethnicity
Non-Hispanic	243 (89.3)	97 (99.0)*	240 (89.6)	66 (100.0)*	134 (89.3)	58 (100.0)
Hispanic	28 (10.3)	1 (1.0)*	27 (10.1)	0 (0.0)*	16 (10.7)	0 (0.0)

^*Denotes that the values for patients are significantly different (α = 0.05) than those for controls

Measure

SF-36

Participants completed the Medical Outcomes Study Short-Form-36 Health Survey [2] online. This instrument measures physical and mental functioning and includes eight subscales: Physical Functioning, Role Physical, Bodily Pain, General Health, Social Functioning, Mental Health, Role Emotional, and Vitality. Scores for each subscale are calculated using a 100-point system, with higher scores indicating higher functioning. The SF-36 has been shown to have good discriminant validity, internal consistency, and reliability across groups [27].

Analysis

A series of receiver operating curves (ROC) was used to determine how accurately each SF-36 subscale discriminated between young adult patients and controls. The ROC analyses were run using SPSS Statistics Software (version 21) [28]. ROCs plot the probability of each score on a given test identifying true positives (i.e., patients correctly identified as having CFS) versus the probability of identifying false positives (i.e., controls falsely assigned the diagnosis of CFS). Sensitivity represents the probability that the scale correctly identifies patients as having CFS. Specificity represents the probability that the scale correctly identifies controls as NOT having CFS. In an ROC analysis, the sensitivity is plotted against 1 minus the specificity for all cutoff points. By examining the area under the ROC curve (AUC), one can determine how well the scale discriminates between patients and controls. An AUC of 1.0 represents perfect discrimination; an AUC of 0.5 represents a scale that is no better than chance. An AUC between 0.90 and 1.00 is considered an indication of a scale’s good discriminant ability [29, 30].

It is possible to determine the sensitivity and specificity of each cut-point of interest to establish the cutoff score that would most accurately discriminate between patients and controls. However, often there are practical and theoretical considerations in choosing a cutoff score because the value of selecting a sensitive cutoff (i.e., one that maximizes true positives) must be weighed against the value of selecting a specific cutoff (i.e., one that includes a minimal number of false positives). Generally, the more sensitive the cutoff, the less specific and vice versa. To optimize the balance between the two, a value known as the Youden Index (YI) can be calculated as sensitivity + specificity − 1 for each cut-point and is a general indicator of how well scores maximize both sensitivity and specificity, giving each equal weight [31]. AUC values for all SF-36 subscales, along with the sensitivity, specificity, and Youden Index scores for each cutoff within the top performing scales, were compared to determine which scale and cutoff points were best able to discriminate between young adults with CFS (patients) and young adults without a CFS diagnosis (controls).

Results

Table 2 shows the area under the curve (AUC) values and 95% confidence intervals for the ROC analyses for each subscale of the SF-36, for all three samples (the total sample and the two subsamples previously defined). The AUC values are useful in determining which subscales performed best at discriminating between the known patients and controls in each sample. In the total young adult sample, six of the eight SF-36 subscales showed good discriminant ability, as indicated by having an AUC value above 0.90 (Physical Functioning = 0.977, General Health = 0.965, Role Physical = 0.939, Social Functioning = 0.938, Bodily Pain = 0.926, and Vitality = 0.904). The two remaining subscales (Mental Health Functioning = 0.573 and Role Emotional = 0.470) had AUCs and accompanying confidence intervals that suggested these scales performed no better than chance at discriminating between young adults with CFS and those without. The two subsamples (of only 18–25 year olds and only female 18–25 year olds) had similar results, with the same rank order of highest performing subscales, but showed slightly lower AUC values.

Table 2.

The area under the curve (AUC) values for all SF-36 subscales among different subsets of the sample

	Ages 18–29 (total sample) (N = 370)		Ages 18–25 (N = 334)		Ages 18–25 females only (N = 208)
	AUC	95% CI	AUC	95% CI	AUC	95% CI
Physical Functioning	0.978	0.957–0.998	0.971	0.942–1.000	0.965	0.930–1.000
General Health	0.965	0.944–0.985	0.958	0.931–0.984	0.944	0.908–0.979
Role Physical	0.939	0.914–0.965	0.940	0.910–0.969	0.926	0.887–0.965
Social Functioning	0.938	0.909–0.968	0.927	0.888–0.966	0.905	0.857–0.954
Bodily Pain	0.926	0.897–0.955	0.921	0.885–0.956	0.902	0.856–0.948
Vitality	0.904	0.868–0.941	0.898	0.851–0.945	0.880	0.822–0.937
Mental Health Functioning	0.573	0.500–0.646	0.571	0.486–0.657	0.549	0.456–0.641
Role Emotional	0.470	0.405–1.536	0.439	0.365–0.514	0.406	0.323–0.489

To assess for the optimal cutoff values within each subscale, we compared the sensitivity, specificity, and Youden Index scores for different cut-points on each subscale. Table 3 indicates the three highest performing cutoff scores for five of the eight subscales, along with the sensitivity, specificity, and Youden Index scores for each cutoff. As multiple studies have used the Role Physical, Social Functioning, and Vitality subscales to operationalize substantial reduction in adults [14, 22, 32], these three subscales are included in Table 3, along with the two additional top performing subscales from the present study (Physical Functioning and General Health). As expected, often when the sensitivity score increased, the specificity score decreased. Therefore, the Youden Index score (sensitivity + specificity − 1) was used to determine the optimal balance between sensitivity and specificity. Among the total young adult sample, the following cutoff scores maximized sensitivity and specificity on the scales examined: Physical Functioning ≤ 80 (YI = 0.905); General Health ≤ 47 (YI = 0.828); Role Physical ≤ 25 (YI = 0.810); Social Functioning ≤ 50 (YI = 0.764); and Vitality ≤ 30 (YI = 0.663). When looking at the cutoff scores for the younger subsample (18–25), the rank order of cutoff values was similar to those for the total sample. However, for the female-only subsample, setting a lower cutoff score for both the Physical Functioning (≤ 75) and General Health (≤ 45) subscales showed slightly better performance than those indicated for the total sample, but the differences in performance were quite small.

Table 3.

Optimal cutoff values for select SF-36 subscales across different subsets of the sample

Subscale	Top 3 cutoffsa	Ages 18–29 (total sample) (N = 370)			Ages 18–25 (N = 334)			Ages 18–25 females only (N = 208)
		Sensitivity	Specificity	YI	Sensitivity	Specificity	YI	Sensitivity	Specificity	YI
Physical Functioning	80	0.949	0.956	0.905	0.939	0.966	0.906	0.931	0.940	0.871
	75	0.929	0.971	0.899	0.924	0.978	0.902	0.931	0.960	0.891
	70	0.908	0.974	0.882	0.909	0.981	0.890	0.914	0.967	0.880
General Health	47	0.939	0.890	0.828	0.924	0.895	0.820	0.914	0.867	0.780
	50	0.939	0.886	0.825	0.924	0.892	0.816	0.914	0.867	0.780
	45	0.918	0.904	0.823	0.894	0.910	0.804	0.862	0.920	0.782
Role Physicalb	25	0.939	0.871	0.810	0.924	0.877	0.801	0.931	0.860	0.791
	50	0.959	0.809	0.768	0.955	0.813	0.768	0.948	0.793	0.741
	75	0.990	0.702	0.692	0.985	0.705	0.690	0.983	0.680	0.663
Social Functioning	50	0.878	0.886	0.764	0.848	0.888	0.737	0.828	0.853	0.681
	62.5	0.949	0.765	0.714	0.939	0.765	0.704	0.931	0.727	0.658
	37.5	0.745	0.938	0.682	0.670	0.940	0.637	0.655	0.927	0.581
Vitality	30	0.806	0.857	0.663	0.788	0.858	0.646	0.793	0.820	0.613
	25	0.724	0.915	0.640	0.712	0.918	0.630	0.707	0.900	0.607
	35	0.857	0.768	0.626	0.848	0.769	0.617	0.844	0.700	0.544

^aRanked in order based on 18–29 year olds, using Youden Index (YI) values

^bThere are four items in the Role Physical subscale, resulting in the fact that the only available subscale scores are 0, 25, 50, 75, and 100

After setting optimal cutoff scores for the different subscales, we explored whether using the subscales in different combinations would result in better or worse discriminant ability. Therefore, the sensitivity, specificity, and Youden Index scores were also calculated for the use of several subscales in combination. For example, using this methodology, we examined whether it might be useful to require an individual to meet the cutoff scores for two out of three scales or three out of four, etc. Table 4 shows the sensitivity, specificity, and Youden Index values for several different combinations of subscale cutoff scores for the total 18–29-year-old sample (N = 370). We first examined the scores used by Jason et al. [14], who recommended the following cutoff points for their operationalization of substantial reduction among adults: Role Physical ≤ 50, Social Functioning ≤ 62.5, and Vitality ≤ 35.

Table 4.

Sensitivity, specificity, and Youden Index Score for different SF-36 subscale combinations

Subscale combination	Number required	Ages 18–29 (total sample) (N = 370)
		Sensitivity	Specificity	YI
Original Jason et al. (2011) substantial reduction criteria: Role Physical ≤ 50, Social Functioning ≤ 62.5, Vitality ≤ 35	1 of 3	0.980	0.798	0.777
	2 of 3	0.959	0.938	0.897
	3 of 3	0.827	1.000	0.827
Three highest performing scales and cutoffs from the present study: Physical Functioning ≤ 80, General Health ≤ 47, Role Physical ≤ 25	1 of 3	0.969	0.952	0.922
	2 of 3	0.959	0.982	0.941
	3 of 3	0.898	1.000	0.898
Four highest performing scales from the present study: Physical Functioning ≤ 80, General Health ≤ 47, Role Physical ≤ 25, Social Functioning ≤ 50	1 of 4	0.969	0.904	0.874
	2 of 4	0.969	0.967	0.936
	3 of 4	0.949	0.993	0.942
	4 of 4	0.816	1.000	0.816

Requiring one, two, or all three of the Jason et al. [14] cutoff scores yielded Youden Index scores of 0.777, 0.897, and 0.827, respectively. When the top three performing subscales and associated cutoffs from the present study (Physical Functioning ≤ 80; General Health ≤ 47; Role Physical cutoff ≤ 25) were similarly analyzed, requiring one or more of these three scales yielded a Youden Index score of 0.922, requiring two or more yielded a score of 0.941, and requiring all three yielded a score of 0.898. We also examined the Youden Index scores for requiring one (YI = 0.874), two (YI = 0.936), three (YI = 0.942), or four (YI = 0.816) of the following cutoff scores: Physical Functioning ≤ 80; General Health ≤ 47; Role Physical ≤ 25; and Social Functioning ≤ 50.

Discussion

The results of the present study show that it is important to consider age-normative functioning when evaluating whether an individual meets the criterion for substantially reduced functioning as a result of ME or CFS. Using ROC analyses, we determined that the following subscales were the best candidates for identifying young adults experiencing ME- or CFS-related substantial reduction in functioning: Physical Functioning (AUC = 0.977), General Health (AUC = 0.965), Role Physical (AUC = 0.939), and Social Functioning (AUC = 0.938). The cut-points identified in the present study for each of these subscales performed better in our sample of young adults than the Jason et al. [14] cutoff scores for older adults (Role Physical ≤ 50; Social Functioning ≤ 62.5; and Vitality ≤ 35). The present results suggest that the Physical Functioning (≤ 80) and General Health (≤ 47) subscales of the SF-36 were the best candidates for indicators of substantially reduced functioning in young adult populations. Additionally, the current study found that setting a cutoff score of ≤ 25 (YI = 0.810) rather than the Jason et al. [14] recommendation of ≤ 50 (YI = 0.768) on the Role Physical subscale and a cutoff score of ≤ 50 (YI = 0.764) rather than ≤ 62.5 (YI = 0.714) on the Social Functioning subscale would better discriminate between patients with substantially reduced functioning and healthy college students. While the optimal cutoff score for the Physical Functional subscale appears on the surface to be a fairly high score (≤ 80 out of 100), the results indicated that scores at or below this point do likely represent a meaningful reduction in normative physical functioning for young adults. In our sample of known young adult patients, this ≤ 80 cutoff score performed better than any other cut-point or scale in discriminating between patients and controls. In fact, only 5 of the 98 patients (5.1%) scored above 80 on the Physical Functioning subscale and only 12 of the 272 controls (4.4%) scored below it.

Furthermore, the ROC findings indicated that the SF-36 subscales that assessed for impairment as a result of mental health issues (Mental Health Functioning and Role Emotional) were no better than chance at discriminating between known patients and controls in our sample. This underscores the physical nature of the impairment associated with the ME and CFS and fits with previous data that questioned the accuracy the Role Emotional component of the SF-36 in measuring substantial reduction [14, 16]. In comparing different combinations of scale cutoffs, including those recommended by Jason et al. [14], it was determined that requiring cutoffs on either two out of three (YI = 0.942) or three out of four (YI = 0.941) of the top performing scales (Physical Functioning ≤ 80; General Health ≤ 47; Role Physical ≤ 25; and Social Functioning ≤ 50) resulted in equally high performing discrimination between patients and controls. The inclusion of the Social Functioning subscale as the fourth scale option has the advantage of broadening the number of functional domains being assessed, covering both physical and social domains.

While the present study employed the Youden Index to assess for the best balance between sensitivity and specificity, decisions on whether cutoff scores should place a higher priority on including true positives or on excluding false positives can be highly dependent on the intended purpose of the diagnostic grouping. In a clinical setting, one may wish to prioritize sensitivity, ensuring that the maximum number of potential patients is identified, with less concern placed on limiting over-inclusivity and false positives. When using the SF-36 to determine substantial reduction in young adults for clinical purposes, the recommendations outlined in this article may be loosened by increasing one or more cutoff values or by reducing the number of scales required to meet the substantial reduction criterion. However, in research settings, high levels of specificity may be more important to ensure that studies minimize the number of false positives. Including unacceptably high levels of non-patients in patient samples would result in skewed findings. Indeed, Jason, McManimen, Sunnquist, Newton, and Strand [32] have recommended that the field move towards developing a research case definition that would be less inclusive than most of the currently available clinical case definitions. Retaining strict cut-point standards for research purposes, thus prioritizing specificity, would ensure greater homogeneity among patient samples by excluding more false positives. Homogeneous or “pure” samples, containing a minimum of falsely identified individuals as “patients,” would reduce bias and aid in overall efforts to specify the etiology and effectiveness of treatment for this illness.

Indeed, this challenge of balancing sensitivity and specificity is applicable to research and treatment for a wide range of illnesses that are of interest to the fields of behavioral medicine and public health. ROC analyses were originally developed for use with radar in World War II, helping to reliably distinguish between radar signals related to airplanes and those related to non-combat entities (e.g., a flock of geese) [29]. Since that time, the methodology has been used in a range of clinical settings including radiology and epidemiology. For practitioners and researchers alike who seek to understand and interpret ROC analyses, it is important to remember the tradeoffs involved in the probabilistic approach to classifying individuals in this manner. In better understanding the tensions between inclusivity (sensitivity) and rigorous standards (specificity), we can better interpret the literature and apply ROC analyses in a way that recognizes this tension and gives appropriate consideration to our own specific intensions. This understanding would allow the flexible application of standards that are tailored to the intended use and setting.

This study is not without limitations. The primary limitation was that in collecting a young adult patient sample large enough for the ROC analysis, individuals were pooled from several de-identified databases. This resulted in a patient sample that was diverse in the country of residence for participants (i.e., Norway, the UK, the US), but not in gender, race, or ethnicity. On the other hand, the control group consisted of a fairly diverse group of college students from a single campus in the Midwestern United States. In order to demonstrate the potential impact of the demographic differences between these two groups, smaller, more homogeneous subsamples were also considered for the ROC analyses. The fact that the findings from the smaller subsamples were overall very similar to those of the total sample gives some indication that large degrees of bias were not present in any of our patient or control samples. However, that the patient sample consisted mainly of females of Caucasian decent is certainly a limitation in generalizing the results to other patient populations. Additionally, it is important to note that the present study used the rather straightforward probability-based receiver operator curve approach to selecting optimal subscales and cutoffs. While this methodology is not capable of indicating causal links among data, it is a widely used approach in medical and epidemiological research [29].

The present study suggests a method for operationalizing the substantial reduction criterion among young adults who may warrant a diagnosis of ME or CFS. Further research should seek to replicate the use of the recommended cutoff scores among more diverse groups of young adult patients and controls, as well as explore age-normative cutoff scores among other age groups. While the present study focused on the operationalization of but one component of the complex of diagnostic criteria for ME and CFS, we do not suggest that the substantial reduction criterion be used as the lone discriminating feature for identifying patients. Rather, we seek to add this study to the existing body of work that explores the operationalization of this one component of ME and CFS case definitions, with the hopes that future research will continue to explore the optimal operationalization of each aspect of the major case definitions related to this illness.