Defining Essential Features of Myalgic Encephalomyelitis and Chronic Fatigue Syndrome

Abstract

Considerable debate surrounds the search for the defining features of patients with Myalgic Encephalomyelitis (ME) and chronic fatigue syndrome (CFS). Current case definitions were created through clinical consensus. Failure to operationalize these case definitions has led to considerable variability in the identification of patients. In addition, some case definitions (e.g., Fukuda et al., 1994) do not require cardinal symptoms of this illness, where as other case definitions do require core symptoms of this illness (Carruthers et al., 2003, 2011), and these latter case criteria appear to identify a more impaired group of patients. Criterion variance is most likely to occur when operationally explicit criteria do not exist for diagnostic categories (Spitzer, Endicott, & Robins, 1978), or when there are varying criteria for contrasting case definitions, which is an impediment to the research in this field. To deal with this problem, it is possible to differentiate those that meet more loosely defined criteria from those that are more narrowly and defined, thus differentiating CFS from ME. In order to progress the search for biological markers and effective treatments, essential features need to be operationalized and broadly used in order to increase the probability that individuals included in samples have the same underlying illness.

Sources of diagnostic unreliability include subject, occasion, and information variance account, but criterion variance, differences in the formal inclusion and exclusion criteria used by clinicians to classify patients’ data into diagnostic categories, accounts for the largest source of diagnostic unreliability (Jason, & Choi, 2008). Criterion variance is most likely to occur when operationally explicit criteria do not exist for diagnostic categories (Spitzer, Endicott, & Robins, 1978), or when there are varying criteria for contrasting case definitions. When diagnostic categories lack reliability and accuracy, the validity (i.e., usefulness) of a diagnostic category is inherently limited by its reliability. Problems of criterion variance have plagued case definitions involving Myalgic Encephalomyelitis (ME) and chronic fatigue syndrome (CFS).

Most instances of fatigue have an identifiable cause that is either self-limiting (e.g., exercise) or treatable (e.g., flu-like illness). In addition, brief periods of fatigue (i.e., less than one month duration) are common occur in approximately 15% to 25% of the population (Lewis & Wessely, 1992). Fewer individuals experience fatigue that is either prolonged (i.e., fatigue lasting between one and five months) or chronic (i.e., fatigue lasting six months or longer). Jason et al. (1999) found that prolonged fatigue occurs in approximately 5% to 7.7% of the general population, and chronic fatigue occurs in 2.7% to 4.2% of the population. A study of fatigue in rural communities revealed somewhat higher rates of fatigue, with 18% of respondents reporting prolonged fatigue and 10.8% reporting chronic fatigue (Fukuda et al., 1997). In primary care settings, estimates of prolonged fatigue range from 10.4% (David et al. 1990) to 25% (Hickie et al., 1996). Another primary care study revealed a point prevalence of 11.3% for chronic fatigue (Wessely, Chalder, Hirsch, Wallace, &Wright, 1997). Thus, prolonged and chronic fatigue are significant concerns among patients both in the general population and in primary care settings.

Whereas most people do experience some time limiting fatigue, with the studies reviewed above, let us for the moment conservatively assume that about 4% of the population experience more long term fatigue of 6 or more months. Jason et al. (1999) found that among those with 6 or more months of fatigue, 54% had a medical or psychiatric explanation (melancholic depression, bipolar disorders, anorexia nervosa or bulimia nervosa, psychotic disorders, drug or alcohol related disorders) for this fatigue. An additional 27% did not meet the Fukuda et al. (1994) criteria for chronic fatigue syndrome (CFS). Therefore, in the community-based study by Jason et al. (1999), if about 4 percent of the general population experiences chronic fatigue, about half of these cases have medical or psychiatric reasons for the fatigue, and the other cases are divided between those who meet the Fukuda et al. (1994) criteria and those that do not. The Jason et al. (1999) study estimated that on person in every 200 individuals, or .42% of the population, had CFS.

However, when the Centers for Disease Control and Prevention (CDC) operationalized the Fukuda et al. (1994) criteria, they estimated that 2.54% had CFS (Reeves et al., 2007). This rate is considerably higher than the .42% rate found in Jason et al.’s (1999) community-based study. Clearly, this CFS Fukuda et al. (1994) case definition is being used quite differently by investigators, introducing criterion variance. In a recent systematic review, Brurberg, Fønhus, Larun, Flottorp, and Malterud (2014) identified 20 case definitions, and while the Fukuda et (1994) criteria as the most frequently applied case definition, validation studies had inconsistent results, and no studies rigorously assessed the these case definitions’ reproducibility or feasibility. It is important to review the history of the attempts to develop case definitions for individuals with CFS and ME, and propose a way of helping investigators and clinicians develop a theoretical and conceptual way of dealing with these multiple case definitions, each with different criteria and ways of operationalizing them.

Case Definitions

In the US, Holmes, Kaplan, Gantz, et al. (1988) constructed the first working case definition of CFS. To meet criteria, participants were required to report at least eight of eleven minor symptoms (fever or chills, sore throat, lymph node pain, muscle weakness, muscle pain, post-exertional malaise, headaches of a new or different type, migratory arthralgia, neuropsychiatric complaints, sleep disturbance, and a sudden onset of symptoms). However, when the Holmes, Kaplan, Gantz et al. (1988) criteria were utilized in research and practice, it became evident that there were numerous inconsistencies in interpretation and classification of cases (Holmes, Kaplan, Schonberger et al., 1988; Schluederberg et al., 1992; Straus, 1992). A major area of concern was that the requirement of eight or more minor symptoms could inadvertently select for individuals with psychiatric problems (Straus, 1992). For example, Katon and Russo (1992) noted that chronic fatigue patients with the highest numbers of unexplained physical symptoms had high rates of psychiatric disorders, while patients with the lowest numbers of unexplained symptoms displayed rates of psychiatric disorders that were similar to other clinical populations with chronic medical illnesses.

A few years later, a revised International case definition for CFS (Fukuda et al., 1994) was developed, and it required a person to experience six or more months of chronic fatigue of a new or definite onset, that is not substantially alleviated by rest, not the result of ongoing exertion, and results in substantial reductions in occupational, social, and personal activities. The Fukuda et al. (1994) CFS case definition used polythetic criteria: a set of symptoms in which all do not need to be present to make a diagnosis. Because the Fukuda et al. (1994) criteria only requires four symptoms out of a possible eight (all of which had been part of the earlier Holmes, Kaplan, Gantz, et al., 1988 criteria for CFS), critical CFS symptoms such as post-exertional malaise, and memory and concentration problems were not required for a patient to receive a diagnosis of CFS. This could have increased the heterogeneity of the population and complicated identification of comparable samples. In addition, the Holmes, Kaplan, Gantz et al. (1988) criteria (as specified by the Schluederberg et al., 1992 revision), excluded individuals with the presence of anxiety disorders, somatoform disorders and nonpsychotic or non melancholic depression prior to CFS onset, but these conditions were no longer exclusionary under the Fukuda et al. (1994) case definition. When the Fukuda et al. (1994) and Holmes, Kaplan, Gantz, et al. (1988) criteria were compared with a sample of patients, the Holmes, Kaplan, Gantz, et al. (1988) criteria selected patients with higher symptomatology and functional impairment (Jason, Torres-Harding, Taylor, & Carrico, 2001). In a primary care setting, Wessely, Chalder, Hirsch, Wallace, and Wright (1997) found rates of CFS based on the Fukuda criteria (1994) were over twice that of the Holmes et al. (1988) criteria.

Later, the Centers for Disease Control and Prevention (CDC) developed an empirical case definition for CFS (Reeves et al., 2005), which was an operationalization of the Fukuda et al. (1994) criteria. The authors of this empirical case definition felt that the specification of instruments and cut-off points would result in a more reliable and valid approach for the assessment of CFS. As noted above, using these operationalized criteria, the estimated rates of CFS increased to 2.54% (Reeves et al., 2007), rates that were considerably higher than prior community based epidemiologogical studies: .24% CDC prevalence estimate (Reyes et al., 2003) and .42% prevalence estimate of Jason et al. (1999). It is of interest that the new CFS rates were within the range of several mood disorders. Mood disorders are the most prevalent psychiatric disorders after anxiety disorders: for major depressive episode, the one-month prevalence is 2.2% (Regier, Boyd, & Burke, 1988). Jason, Najar, Porter, and Reh (2009) later found that 38% of those with a diagnosis of a Major Depressive Disorder were classified as having CFS using the more broadly based Reeves et al. (2005) CFS empiric case definition. Therefore, the broadening threshold for caseness in the Reeves et al. (2005) criteria probably led to increases in estimates of CFS prevalence by the CDC (Reeves et al., 2007).

Other efforts to describe and diagnose this illness occurred prior to the development of the Holmes, Kaplan, Gantz et al. (1988) criteria. The term Myalgic Encephalomyelitis (ME) was first described in literature of the 1930s, where an outbreak of Epidemic Neuromysthenia in L.A. County was called “atypical poliomyelitis” because of its resemblance to polio (Gilliam, 1938; Hyde, 2007). Years later, an anonymous editorial in the 1956 issue of the Lancet coined the term benign Myalgic Encephalomyelitis (Anonymous Editorial, 1956). It was called ‘benign’ because the illness did not lead to patient death. Later, Ramsay (1988) published a definition of this illness using the term Myalgic Encephalomyelitis (ME) and the term benign was dropped due to the seriousness of the disability created by the illness (Hyde, Goldstein, & Levine, 1992). Efforts to operationalize ME occurred with what are now known as the London criteria (Report from The National Task Force, 1994, pp. 96–98). These criteria recognized four cardinal features: (1) physical or mental fatigue or muscle weakness after minimal exertion which may persist long after exertion ends; (2) circulatory impairment (e.g., feeling hot when it’s cold, postural hypotension); (3) one or more symptoms indicating the involvement of the central nervous system, such as impairment of memory and concentration and disturbed sleep patterns; (4) and the marked fluctuation of symptoms (Dowsett, Ramsay, McCartney, & Bell, 1990; Goudsmit, Shepherd, Dancey, & Howes, 2009).

When Jason, Helgerson, Torres-Harding, Carrico, and Taylor (2003) attempted to operationalize the London ME criteria by selecting individuals with post-exertional malaise, memory and concentration impairment, and fluctuation of symptoms, and then compared these patients to those meeting the CFS Fukuda et al. (1994) criteria, the London ME criteria selected a more symptomatic group of patients from a community-based sample. However, the Jason, Helgerson et al.’s (2003) scoring criteria was limited by just measuring the occurrence of symptoms for the past 6 months, rather than requiring a certain degree of severity to be considered a symptom of ME. Still, in their study, of the 32 participants who were diagnosed with CFS using the Fukuda et al. (1994) criteria, 14 or 44% also would have met the criteria for London ME (an additional 3 participants from the 45 with idiopathic chronic fatigue group were classified as having ME). Therefore, the ME criteria selected a smaller group of patients than the broader CFS Fukuda et al. (1994) criteria.

Several years later, Jason, Damrongvachiraphan, et al. (2012) attempted to better operationalize the ME criteria, based on the work of a number of theorists and practitioners (Dowsett, Ramsay, McCartney, & Bell, 1990; Goudsmit, Shepherd, Dancey, & Howes, 2009; Hyde, Goldstein, & Levine, 1992; Ramsay, 1988). The major symptom categories of ME in this revised case definition included: post-exertional malaise, neurological manifestation, and autonomic dysfunction, and these investigators used more precise frequency and severity criteria for symptoms. Patients also needed to also have an acute onset to meet the ME criteria. When Jason, Brown et al. (2012) applied these revised criteria to a data set of patients in a tertiary sample diagnosed with CFS using the Fukuda et al. (1994) criteria, only 24% met these ME criteria, and they were more functionally impaired than those that just met the Fukuda et al. criteria. In addition, the patients meeting these ME criteria had higher pulse rates at resting and standing than those with CFS, as well as more self-report autonomic symptoms. In addition, on the Trailmaking test, which assesses for cognitive domains of attention, visual scanning with speed of eye-hand coordination, and information processing, the ME group had significantly poorer performance than the CFS groups. Later, Jason, Evans, Brown, Sunnquist, and Newton (in press) found that 29.6% of a CFS sample in the US, and 17.7% of a CFS sample in England met these ME criteria. Clearly, these more restrictive ME criteria selected a smaller group of patients than the Fukuda et al. (1994) criteria, and those selected with ME were more impaired.

Howes, Goudsmit, and Shepherd (2014) recently proposed revised ME London criteria that requires the following five criteria: A new onset of significantly abnormal levels of muscle fatiguability and/or muscle weakness, precipitated by relatively minor levels of activity; the presence of symptoms indicating the involvement of the brain and central nervous system; periods of impaired circulation compatible with autonomic dysfunction; fluctuation of symptoms, from hour to hour and day to day; and finally, these symptoms must have been present during the past three months. When Sunnquist et al. (2014) compared and contrasted the CFS (Fukuda et al., 1994) and this revised ME London criteria (Howes, Goudsmit, & Shepherd, 2014), only 5% of cases met the revised ME criterion. Certainly, these studies indicate that the percent of patients who met the ME criteria will vary depending the ways in which investigators operationalize the symptom criteria.

Another consensus clinical case definition has been called the Canadian Clinical Criteria, and has been referred to as ME/CFS (Carruthers et al., 2003). This ME/CFS case definition also specified core symptoms, including post-exertional malaise, impairment of memory and concentration, unrefreshing sleep, arthralgia and/or myalgia; and several autonomic, neuroendocrine, and immune manifestations. Jason, Torres-Harding, Jurgens, and Helgerson (2004) compared persons meeting the Canadian ME/CFS and Fukuda et al. (1994) criteria. The Canadian ME/CFS criteria, in contrast to the Fukuda et al. (1994) criteria, selected cases with less psychiatric comorbidity, more physical functioning impairment, more fatigue or weakness, and more neuropsychiatric and neurological symptoms. In Johnston, Brenu, Staines, and Marshall-Gradisnik’s (2013a) review of thirty-one CFS prevalence studies, eight different case definitions were used, and only one study reported prevalence according to the Canadian ME/CFS Consensus Criteria.

In an effort to better operationalize the Canadian ME/CFS criteria (Carruthers et al., 2003), Jason et al. (2010) specified explicit rules for determining ME/CFS status, and in particular, using severity and frequency criteria as opposed to just the presence of symptoms as occurred in the prior Jason, Torres-Harding et al. (2004) investigation. Using this method, Jason, Brown, et al. (2012) compared those meeting the Canadian ME/CFS case definition to those who did not meet the Canadian ME/CFS criteria but met only the Fukuda et al. (1994) criteria. Findings indicated that the Canadian ME/CFS case definition identified individuals with more severe symptoms and greater functional disability than those who met only the Fukuda et al. (1994) criteria. Finally, Jason, Brown, Evans, Sunnquist, and Newton (2013) utilized three different samples of patients, each collected through a different case ascertainment method, but used a similar assessment instrument called the DePaul Symptom Questionnaire (DSQ). Findings indicated that fewer individuals met the Canadian ME/CFS criteria (Carruthers et al., 2003) than the Fukuda et al. (1994) CFS criteria, and that those who met the ME/CFS criteria evidenced more severe symptoms and more physical impairment. For the samples, whereas from 87 to 96% of the samples met the Fukuda et al. (1994) case definition criteria, only 73 to 77% met the Canadian ME/CFS criteria. These consistent findings across the three data sets suggest that about three fourths of those within disparate samples meet the case definition for ME/CFS, whereas the Fukuda et al. (1994) criteria identify a larger group of less impaired and symptomatic patients.

A new ME case definition, the International Consensus Criteria for Myalgic Encephalomyelitis (ME-ICC), was recently published (Carruthers et al., 2011). To meet the ME-ICC criteria, a person must have symptoms from the following four domains: Post-Exertional Neuroimmune Exhaustion; Neurological Impairments; Immune, Gastro-intestinal, and Genitourinary Impairments; and Energy Production/Transportation Impairments. Brown, Jason, Evans, and Flores (2013) contrasted the ME-ICC criteria (Carruthers et al., 2011) with the Fukuda et al. (1994) CFS criteria. Findings indicated that the ME-ICC criteria identified a subset of patients with more functional impairments and physical, mental, and cognitive problems than the larger group of patients who met the Fukuda et al. criteria. However, the patients who met the ME-ICC criteria also had significantly greater rates of psychiatric comorbidity. In a larger study with samples from the US and Great Britain, Jason, Sunnquist, Brown, Evans, and Newton (2013) also compared the ME-ICC criteria to the Fukuda et al. CFS. In general, participants who met the ME-ICC criteria were more functionally impaired than those with Fukuda-defined CFS. When using the Carruthers et al. (2011) ME-ICC case definition, from 57 to 58% met the ME-ICC case definition, whereas the Fukuda et al. criteria identified a larger group of patients.

Figures 1 shows those with different categories of fatigue, with about 4% of the populations experiencing 6 or more months of fatigue. Of these 4%, about half will have medical and psychiatric exclusionary illnesses. Using the broad Reeves et al. (2005) empiric CFS criteria, about half of those with chronic fatigue (or 2.54% of the population) will be identified with CFS (See Figure 2). On the other hand, for those referred to a clinic for CFS or self-diagnosed, Figure 3 compares the various case definitions and the percentage of patients who meet these criteria from several distinct data sets. It is clear from the Figure 3 that the Fukuda et al. (1994) criteria include the most patients, with fewer being identified with the Canadian ME/CFS (Carruthers et al., 2003) and ME-ICC criteria (Carruthers et al., 2011). Still fewer meet the London ME criteria described by Jason Damrongvachiraphan, et al. (2012) which required an acute onset and three major ME symptom categories (post-exertional malaise and neurological and autonomic manifestations), and with the revised ME London criteria being the most restrictive (Sunnquist et al., 2014). These different definitions provide large problems for investigators, as if they are using case definitions with different symptoms, as well as measuring the symptoms in different ways, then criterion variance is introduced. If diagnostic categories are unreliable, then this will have severe consequences for estimating prevalence rates or in finding biological markers. Empirical approaches might help specify which symptoms and domains have both the needed sensitivity and specificity for the case definitions.

Figure 1.

Prevalence of Daily, Prolonged, and Chronic Fatigue

Figure 2.

Prevalence of Chronic Fatigue, and CFS (Reeves et al., 2005; Fukuda et al., 1994)

Figure 3.

Individuals Referred by Medical Specialists in CFS and ME/CFS

Empirical Approaches

As mentioned above, in order to provide more guidelines and specific criteria for this case definition, the CDC developed an empirical case definition for CFS that involved assessment of symptoms, disability, and fatigue using standardized scales (Reeves et al., 2005). Advanced statistical methods can be used to evaluate these types of case definition criteria. The Receiver Operating Characteristics (ROC) curve graphically represents the probability of true positive results in diagnosis as a function of the probability of false positive results of this test. The area under the ROC curve is a summary measure that essentially averages diagnostic accuracy across the spectrum of test values. For example, Jason, Evans, et al. (2011) used ROC to examine the use of the Multidimensional Fatigue Inventory (MFI) fatigue scale (Smets, Garssen, Bonke, & DeHaes, 1995), which was used to identify whether participants meet the fatigue criterion for the CFS empirical case definition (Reeves et al., 2005). Jason, Evans, et al. (2011) found that the MFI did identify all CFS cases, but these scales were not able to successfully identify those who did not have CFS. In addition, Jason, Brown, et al. (2011) examined Reeves et al.’s recommended use of selected subscales from the Medical Outcomes Study 36-Item Short-Form Health Survey (SF-36) to measure disability. Notably, Jason, Brown, et al. (2011) found that the Reeves et al. recommended cutoff of less than or equal to 66.7 on the SF-36 Role-Emotional subscale would select the majority of those with chronic fatigue explained by psychiatric reasons as meeting the CFS disability criterion. Furthermore, the area under the curve (AUC) for the Role-Emotional subscale was the worst among the eight SF-36 subscales for discriminating patients with CFS from controls (Jason, Brown, et al., 2011).

Another study compared the new diagnostic criteria for Pediatric case (Jason et al., 2006) with the Fukuda et al. (1994) CFS criteria in a sample of adolescent patients and controls. Because the Fukuda et al. adult criteria have been used to diagnose youth with CFS, an international team of investigators developed a new pediatric criteria (Jason et al., 2006) using the Canadian ME/CFS criteria (Carruthers et al., 2003) as a framework. Jason, Porter et al. (2010) found that the new pediatric ME/CFS criteria were able to correctly differentiate between the presence and absence of ME/CFS in children in more cases than the Fukuda et al. (1994) criteria. The Fukuda et al. (1994) criteria evidenced 76% sensitivity and 100% specificity, whereas the Pediatric ME/CFS criteria (Jason et al., 2006) evidenced 97% sensitivity and 100% specificity. Based on this sensitivity, diagnosticians would miss 24% of patients with ME/CFS using the Fukuda criteria; however, only 3% of patients with ME/CFS would be misdiagnosed as not having the illness with the utilization of the pediatric ME/CFS criteria.

In addition, rather than relying on consensus approaches, several studies have used statistical techniques to determine the critical domains experienced by patients with this illness. For example, Friedberg, Dechene, McKenzie and Fontanetta (2000) found the following three-factor solution: cognitive problems, flu-like symptoms, and neurologic symptoms. Jason, Corradi and Torres-Harding (2007) found a six-factor solution, consisting of: neurocognitive, vascular, inflammation, muscle/joint, infectious, and sleep/post-exertional malaise symptoms. Arroll and Senior (2009) found the following five-factor solution: fibromyalgia syndrome-like, depression/anxiety, fatigue/post-exertional malaise, cognitive/neurological, and irritable bowel syndrome-like symptoms. Hickie et al. (2009) found a five-factor model involving musculoskeletal pain/fatigue, neurocognitive difficulties, inflammation, sleep disturbance/fatigue, and mood disturbance. A factor analysis by Brown and Jason (2014) resulted in a three-factor solution with one factor comprised of post-exertional malaise items, one factor of neurocognitive items, and one larger factor that encompassed pain, immune, neuroendocrine, and autonomic items. In a more recent study with a large data set of 788 patients, Jason, Sunnquist et al. (2015) found a 4-factor solution. Three of these factors, cognitive dysfunction, post-exertional malaise, and sleep, fit well with previous literature indicating that these are cardinal symptom clusters. A combined factor involving neuroendocrine, autonomic and immune dysfunction symptoms suggests that neuroendocrine, autonomic, and immune areas that occur a lower rates than the more core domains might be better thought of as subtypes of the illness. These investigators next used an exploratory factor analysis examined solely the items within this heterogeneous factor. The results indicate that circulatory impairment, orthostatic intolerance, and gastro-intestinal distress could be thought of as subtypes. For example, an individual could present with all four core domains, plus have severe orthostatic intolerance but no circulatory impairment or gastro-intestinal distress. It is clear from these studies that the domains of post-exertional malaise, neurocognitive and sleep impairments are most common, whereas fewer studies report autonomic, immune and neuroendocrine factors.

Statistical selection techniques can help reveal which symptoms are the most useful in distinguishing between patients and healthy people, and hence which symptoms are most characteristic of the illness. Machine learning or data mining is one of these techniques which could help compare and contrast case definitions as well as determine the types of symptoms that may be most useful in accurately diagnosing CFS and ME. In particular, data mining can uncover patterns in the data that would not be evident to humans because of the size and complexity of the data. Jason, Skendrovic et al. (2012) explored the use of decision trees to implement the data mining. Decision trees attempt to predict a classification (diagnosis) for each patient based on successive binary choices: at each branch point of the tree, all the symptoms are examined with respect to their effect on the entropy of the diagnoses. Symptoms with high entropy are deemed important and used to split all the cases into two parts. Successive analysis of symptoms contributing less entropy leads to further branching of the tree, until such branchings produce groupings with homogenous labels. If the decision tree determines that a symptom is important in the classification, then this symptom can be considered an important contributor to the illness. Using this approach, Jason, Skendrovic et al. (2012) compared the Canadian ME/CFS criteria (Carruthers et al., 2003) to the empirical CFS criteria developed by Reeves et al. (2005). The Reeves et al. CFS (2005) criteria were able to correctly identify 79% of the CFS cases, whereas the Canadian ME/CFS criteria were able to discriminate 87% of the cases. In addition, the Canadian ME/CFS criteria selected items in the data mining analyses that represent core features of the illness, such as the inability to concentrate, post-exertional malaise, and unrefreshing sleep, whereas the Reeves et al. (2005) CFS criteria did not identify these fundamental items in the data mining analysis.

Data mining techniques, such as classification using decision trees, can also provide statistical analyses that identify which questionnaire items best predict class membership and are useful for indicating which symptoms should be required in the diagnostic process to ensure the most accurate classifications. In a study by Jason, Sunnquist, et al. (2014), decision trees were used to determine which symptoms were most effective at correctly classifying participants as patients or control. The data mining analysis resulted in the selection of three symptoms from the initial 54 analyzed: fatigue or extreme tiredness, inability to focus on more than one thing at a time, and experiencing a dead or heavy feeling after starting to exercise. These three symptoms, when requiring minimum frequency and severity scores of 2 (moderate severity; at least half of the time), accurately classified 95.3% of participants as CFS or control.

A more recent study by Jason, Kot et al.’s (2014) was methodologically stronger than the study by Jason, Sunnquist et al.’s (2014) in its effort to identify core symptoms by both matching sizes for patient and control groups, as well as reporting on 100 sets of data mining analyses as opposed to just one. Both studies found “fatigue or extreme tiredness” as being a core prominent symptom, but the other two symptoms identified by Jason, Sunnquist et al. (2014) were not as consistently found in the Jason, Kot et al. (2014) study. For example, Jason, Sunnquist et al. had found a core symptom being “experiencing a dead or heavy feeling after starting to exercise,” but in data mining, this symptom only emerged in 9 out of 100 trials. In contrast, the item “physically drained/sick after mild activity” appeared 70 times in data mining. In addition, the other symptom from Jason, Sunnquist et al.’s study was “inability to focus on more than one thing at a time”, which emerged for only 5 out of the 100 data mining trials, whereas “difficulty finding the right word to say or expressing thoughts,” emerged 76 times. In summary, the studies by both Jason, Sunnquist et al. (2014) and Jason, Kot et al. (2014) identified fatigue, and symptoms within the post-exertional malaise and neurocognitive areas, but the symptoms were different in the latter two areas. In addition, the Jason, Kot et al. study, unfreshing sleep was also identified as one of the best predictors in the data mining effort. Other symptoms, such as pain, autonomic, immune, and neuroendocrine symptoms were less prevalent, and symptom scores within these areas could also be specified as secondary areas of assessment.

As indicated above, criterion variance occurs when there is not a consensus among investigators for the criteria. There is a critical need to reduce unreliability in order to select those with the illness, not select those without the illness. Empirical methods such as those reviewed could be employed to help specify explicit criteria, and potentially this could be used to help develop a consensus within the scientific community on a particular case definition.

Conclusion

With high levels of exertion or during many illnesses, fatigue is commonly experienced. More long term fatigue of 6 or more months is experienced by about 4% of the population, with half explained by medical or psychiatric illness. Those without clear cut exclusionary illnesses have been provided several labels including CFS and ME. While the original US case definition of CFS required patients to have eight of eleven specified symptoms (Holmes, Kaplan, Gantz, et al., 1988), the Fukuda et al. criteria (1994) later reduced the number of symptoms required to four of eight core symptoms (Fukuda et al., 1994). The Fukuda et al. (1994) CFS case definition has been extensively used by researchers for the past two decades. Unfortunately, it is possible that some individuals who meet these criteria do not have core symptoms of the illness, such as post-exertional malaise, memory/concentration problems, or unrefreshing sleep. Also, because these criteria have been assessed with different instruments and operationalized differently, rates of CFS prevalence in community-based samples have ranged widely, from .004 to .0087% (Reyes et al., 1997), .24% (Reyes et al., 2003) and .42% (Jason et al., 1999) to 2.54% (Reeves et al., 2007) of the population.

We have argued in this article that criterion variance accounts for the largest source of diagnostic unreliability. In the ME and CFS areas, major differences have occurred in the formal inclusion and exclusion criteria to classify patients’ data into diagnostic categories. Criterion variance has occurred because operationally explicit criteria have not existed for diagnostic categories, and there has not been a consensus among investigators for the criteria. We argue that there is a need for the provision of operationally explicit criteria, to operationalize rules for determining when a symptom meets threshold for counting as a problem, and to use structured interview schedules to ensure that consistent information across settings is elicited from an interview.

Using empiric approaches, the findings of study by Jason, Sunnquist, et al. (2014) indicate that fatigue, post-exertional malaise, neurocognitive problems, and unrefreshing sleep occur in most patients. Jason, Kot et al. (2014) also found that data mining indicated that symptoms within these areas best discriminated patients from controls. Hawk, Jason and Torres-Harding (2006) also found these domains were able to successfully differentiated patients with CFS from Major Depressive Disorder. Factor analysis studies also suggest these are among the most common domains found for this illness (Jason, Sunnquist et al., 2015). Although a moderate percentage of the CFS group reports other important symptoms, within pain, autonomic, immune, and neuroendocrine domains, they not as prominent.

Up until now, more consensus based methods have been used for the case definitions. Because the Fukuda et al. (1994) criteria did not explicitly require core symptoms of the illness, the Canadian ME/CFS criteria later defined seven specific symptom requirements (Carruthers et al., 2003). The most recent case definition, the ME-ICC criteria, further increased the number of symptoms required to eight (Carruthers et al., 2011). The Canadian ME/CFS criteria (Carruthers et al., 2003) and ME-ICC criteria (Carruthers et al., 2011) do identify a smaller subset of patients with more severe symptoms and physical functioning impairment (Jason, Brown, et al., 2013). Johnston, Brenu, Hardcastle, Huth, Staines, and Marshall-Gradisnik (2014) also found that patients fulfilling the ME-ICC criteria reported significantly lower scores for physical functioning, physical role, bodily pain, social functioning as well as greater disability than those who met the Fukuda et al. (1994) criteria.

Because the ME/CFS Canadian criteria (Carruthers et al., 2003) and the more recent ME-ICC criteria (Carruthers et al., 2011) have received considerable attention by activists and increasingly researchers, it would be useful to more closely examine features of these case definitions. Regarding the Canadian ME/CFS case definition (Carruthers et al., 2003), only one symptom within a core symptom domain needs to be present for a patient to meet criteria for that domain, except for neurocognitive where two symptoms are required. Factor analytic studies do not support this seven symptom framework, nor is there any empirical justification for the basis for deciding how many symptoms are required for each domain. In addition, with both the Canadian ME/CFS (Carruthers et al., 2003) and ME-ICC criteria (Carruthers et al., 2011), requiring larger numbers of symptoms can inadvertently increase the rate of psychiatric co-morbidity within the group of individuals who meet criteria (Katon & Russo, 1992), as was noted with the Holmes et al. (1988) criteria.

Below we provide some diagnostic and conceptual challenges posed by the recent ME-ICC consensus criteria (Carruthers et al., 2011). For the ME-ICC criteria, the core symptom PEM is referred to as post-exertional neuroimmune exhaustion (PENE). For the self-report questionnaire that is included in the primer (Carruthers et al., 2012), PENE is characterized by: “1) marked, rapid physical or cognitive fatigability in response to exertion, 2) symptoms that worsen with exertion, 3) post-exertional exhaustion that may be immediate or delayed, 4) exhaustion is not relieved by rest, and 5) substantial reduction in pre-illness activity level due to low threshold physical and mental fatigability.” There are a number of problems with this level of specification. First, it is unclear whether all 5 characteristics must be present for PENE to occur. If a person meets all but 1 characteristic, it is unclear whether they would be counted as having this symptom. For example, in a separate dataset utilized by Jason, Sunnquist, Brown, and Evans (2013), it was found that out of 122 participants who met the Canadian ME/CFS criteria (Carruthers, 2011), only 77 participants would have met 5 differently worded statements that are characteristics of PEM. Therefore, whether all characteristics or just some are required for the ME-ICC case definition of PENE is of considerable importance. Furthermore, the precise operationalization of each of these characteristics is still ambiguous. For example, the requirements for the onset and duration of PENE are vague and therefore reliability problems might occur. In addition, the requirement that exhaustion is not relieved by rest might be difficult to operationalize, as many patients do report that when they rest, many of their symptoms are relieved, at least temporarily. Finally, the last characteristic involving a reduction in pre-illness activity due to fatigability does not necessarily involve exertion or the types of activity that precipitates PEM.

In addition, Carruthers et al.’s (2011) ME-ICC criteria indicates that to meet criteria for the Neurological Impairment area, a patient must have at least 1 symptom from 3 of the following 4 symptom categories “1. neurocognitive impairments (e.g,, difficulty processing information, short-term memory loss), 2. pain, 3. sleep disturbance, and 4. neurosensory, perceptual and motor disturbances (e.g. inability to focus vision, sensitivity to light, muscle weakness, feeling unsteady on feet).” Unfortunately, with these criteria, a person could have pain, sleep disturbance, and neurosensory disturbance but no neurocognitive impairments, which some believe are a fundamental feature of this illness. In addition, feeling unsteady on the feet overlaps with orthostatic intolerance, which is in another domain. Finally, there are no empirical studies that support the proposition that three of the four symptom criteria need to be present, or that these items are part of a latent factor.

Regarding the third domain of Carruthers et al.’s ME-ICC (2011) case definition, Immune, Gastro-Intestinal and Genitourinary, individuals must have at least one symptom from 3 of the following 5 symptom categories: “(1) flu-like symptoms, (2) susceptibility to viral infections with prolonged recovery periods (3) gastro-intestinal tract symptoms (e.g., nausea, abdominal pain), (4) genitourinary symptoms (e.g., urinary urgency), and (5) sensitivities to food, medications, odors, or chemicals.” Yet two of the categories have considerable overlap, flu-like symptoms that may be recurrent or chronic, and susceptibility to viral infections with prolonged recovery periods. In addition, factor analytic studies have not found these symptoms to all be part of a latent variable, nor is there any empirical justification for requiring 3 of 5 symptoms.

The last category within the Carruthers et al. (2011) ME-ICC case definition is called Energy Production/Transportation Impairments, and there needs to be at least 1 symptom from “1. cardiovascular (e.g. orthostatic intolerance), 2. respiratory (e.g. laboured breathing), 3. loss of thermostatic stability (e.g. subnormal body temperature), and 4. intolerance of extremes of temperature.” However, a person could meet criteria by having a number of symptoms within this category that are not as easily thought of as an Energy Production/Transportation Impairments (e.g., 1 of the symptoms in the respiratory category is having recurrent feelings of feverishness with or without low grade fever). There is also no evidence that these four symptoms are part of the same domain within samples of patients with ME or CFS.

Further problems are also evident with the International Consensus Primer (Carruthers et al., 2012), written following the release of the ME/ICC case definition (Carrutthers et al., 2011), and offered to medical practitioners as a guide for diagnosing ME in adults and children/adolescents. For example, this primer stated that here must be “50% reduction of premorbid activity level.” However, contradictory information on this important point was provided in the two documents. Carruthers et al. (2011) specifies that individuals must experience symptoms severe enough to cause substantial reductions in pre-illness activity. Furthermore, the Carruthers et al. (2011) criteria designates three levels of activity reduction severity (mild, moderate, and severe), with the mild level signifying an approximate 50 percent reduction in activity levels (Carruthers et al., 2011). In contrast, the Carruthers et al. (2012) primer designates four severity categories (mild, moderate, severe, and very severe): with the mild level signifies that a person “meets criteria” whereas the moderate level signifies that a person has experienced approximately a 50 percent reduction in pre-illness activity level. In other words, the mild level signifies an approximate 50 percent reduction in activity levels for the Carruthers et al. (2011) criteria, but the moderate level signifies that a person has experienced approximately a 50 percent reduction for the Carruthers et al. (2012) criteria.

Finally, the descriptions of some symptoms used in both the ME-ICC (Carruthers et al., 2011) and the ME primer (Carruthers et al., 2012) lack specific steps or assessment tools for consistently and accurately assessing substantial reductions, and severity level in individuals with the illness. Without clearly defined criteria and adequate assessment tools, the method for determining whether an individual meets the required severity level of reductions in pre-illness activity is left to clinical discretion, which may differ greatly across clinicians. In addition, the ME-ICC guidelines focus on severity assessments, but there is also a need to assess frequency in a separate rating.

Refining the case definition and bringing the varying gatekeepers (scientists, clinicians, patients, government) into this process is an important unmet need. Such a development could result in the identification of more homogenous patient samples, which could possibly assist in the pursuit of biomarkers for ME and CFS (Nacul et al., 2011). In addition, to dealing with issues of criterion variance, there is a need for the provision of operationally explicit structured interview schedules to ensure the necessary information is elicited from an interview. Some of the conflicting outcomes of studies in this area are probably due to different strategies used to assess these domains, and clearly, there is a need for investigators to use a similar comprehensive self-report tool as well as biological methods to aid in symptom assessment.

The use of structured interview schedules increase the chance that the clinical material needed to apply to the diagnostic criteria is elicited by structuring and standardizing the questions asked by each interviewer. In other words, structured interview schedules remove the unreliability introduced by differences in the way clinicians elicit clinical information. As an example, Vincent et al. (2012) reported that just 36% of Fukuda patients had PEM, whereas Maes et al. (2012) reported that 50% of Fukuda CFS patients had PEM. As one example of an instrument, Jason, Evans, et al. (2010) published the DePaul Symptom Questionnaire (DSQ), comprised of questions regarding health, social and occupational history, as well as a 54-item symptom chart, designed to tap all of the symptoms of the Fukuda et al. (1994) CFS case definition, the empiric criteria of Reeves et al. (2005), the ME/CFS Canadian criteria (Carruthers et al., 2003), and the ME (Carruthers et al., 2011; Jason, Damrongvachiraphan, et al., 2012). While questionnaires have been used to assess CFS and ME symptomatology, few have reported psychometric characteristics or a scoring system that provides decision rules reflecting both the frequency and intensity of the symptom for the individual over the past 6 months. As an example of how this can occur, Jason, Evans et al.’s (2010) scoring rules require an individual to endorse a symptom as occurring (at least) ‘about half the time’ and at a ‘moderate’ intensity to be counted as part of the symptom profile. This instrument has been shown to evidence high test-retest reliability (Jason, So, Brown, Sunnquist & Evans, 2014) as well as internal reliability (Brown & Jason, 2014). A well validated measure capable of classifying individuals with ME and CFS using a variety of case definitions, allows researchers across settings to identify groups with more homogenous phenotypes.

Future work in this area involves determining what symptom frequency and severity scores make the best threshold for discriminating those who have an illness versus those that do not. Rather than choosing an arbitrary threshold, Watson et al. (2014) dynamically adjusted the threshold for each symptom based on observed frequency and severity scores. This was achieved through a k-means clustering approach. Generally speaking, the k-means algorithm works by iteratively dividing coordinate points into a predetermined number of clusters based on which cluster center the point lies closest to. In this case, the k-means clustering algorithm was set to find two clusters, based on the underlying assumption that the data consists of patient and control groups. Frequency and severity scores for each symptom were treated as coordinate pairs for the purpose of cluster assignment, and the Euclidean distance was used to measure closeness to the cluster centers. After equilibrium was reached, the perpendicular bisector of the line between cluster centers was found. This bisecting line was used as the threshold; frequency-severity pairs above the threshold line were considered “symptom present,” whereas scores below the line were considered “symptom not present.” Using this system, the questions with the highest sensitivity and specificity involved items tapping fatigue (i.e., fatigue/extreme tiredness), post-exertional malaise (i.e., next day soreness or fatigue after non-strenuous, everyday activity; minimum exercise makes you physically tired; physically drained or sick after mild activity; dead, heavy feeling after starting to exercise), sleep (i.e., feeling unrefreshed after waking in the morning) and cognitive/memory (i.e., problems remembering things). Fortunately, the results of using the 2,2 framework that involved a frequency of “half the time” and severity of “moderate” were rather close to using this unsupervised learning strategy (Jason, Kot et al., 2014).

There are other measurement devices, such as the Patient Reported Outcomes Measurement Information System (PROMIS). This is a free database of standardized measures that assess patient reported health status. PROMIS currently includes questions related to the impact of fatigue on functioning, physical feelings of fatigue and limitations. However, PROMIS does not investigate triggers/causes of fatigue or duration of fatigue, lacks questions assessing fatigue that results from physical or mental exertion; thus lacking any questions that query post-exertional malaise.

It is also important for diagnostic criteria to have high sensitivity and specificity, particularly for disorders with low prevalence rates such as CFS (Jason et al., 1999). As an example, in a city of 1,000,000, with a true CFS rate of 4.2 per thousand (Jason et al., 1999), there would be 4,200 CFS cases. According to Bayes’ theorem (Jaynes, 2003) if a diagnostic test had a 95% rate of sensitivity, the screening test would correctly identify 3,990 of these cases. However, if the test had 95% specificity, there would be over 49,000 individuals who did not have CFS but were identified as having it using the test. Clearly, being able to identify true negatives with precision is of high importance with low prevalence illnesses.

Johnston, Brenu, Staines, and Marshall-Gradisnik (2013b) found that differences in method of assessment was possibly due to the heterogeneity in CFS prevalence, and they urged researchers to exercise caution for estimates of CFS prevalence determined by the self-reporting of symptoms alone. There is much potential in using more biological methods to confirm differences in diagnostic classifications. For example, Brenu et al. (2013) found natural killer cell activity significantly decreased for both the Fukuda et al. (1994) and the ME-ICC (Carrruthers et al., 2011) case definitions, but only those diagnosed with the ME-ICC had significant correlations between physical status and some immune parameters.

Advanced statistical procedures can also be used to better understand the biological markers that differentiate those with this illness (Hanson, Gause, & Natelson, 2001; Huang, Hsu, & Lin, 2009). Others have used machine learning algorithms to show that an inflammatory adipokine leptin could distinguish with 78.3% accuracy high from low fatigue days among a sample of women with CFS (Stringer et al., 2013). Broderick, Klimas et al. (2011) and Sorenson, Furst, Mathews, and Jason (2014) have engaged in network analyses of possible biological markers. These are all promising methods that ultimately will need to be used with the types of self-report data cited in this article.

It is possible that a broad CFS criteria could be conceptualized as having polythetic criteria as frequency and severity levels that would allow about 2% of the population to meet criteria. Such a system would only exclude from a CFS diagnosis those with clear medical and psychiatric reasons for their fatigue of 6 or more months. In a sense, this would be similar to what occurred with the Reeves et al. (2005) empiric criteria which suggested that 2.54% of the population had CFS (Reeves et al., 2007). In the first CDC community-based epidemiology study in Wichita, Kansas (Reyes et al., 2003), Nisenbaum et al. (2003) provided percentages of symptoms for 65 individuals classified as having CFS. Unusual fatigue post-exertion was found in 78.5%, and difficulty thinking/concentrating or memory problems was found in 76.9%. These classic CFS symptoms tend to be low, suggesting an identified CFS group with fewer core symptoms. In addition, 77% described their onset as gradual, which contrasts with the sudden onset found in most tertiary samples. Of the individuals who were identified as having CFS during the study that occurred over a three year period (1997 through 2000), 58 were brought back for a two day inpatient study that occurred from December 2002 to July 2003, and only 16 (28% of the original group diagnosed with CFS) had a current consistent diagnosis of CFS, using traditional methods of making this diagnosis. A review of prospective outcome tertiary care studies in CFS patients (Cairns & Hotopf, 2005; Joyce, Hotopf & Wessely, 1997) reveals that substantial recovery occurs in less than 10% of cases. It is very possible that a broad case definition identified cases that are different from that from a more restrictive case definition, which could explain why CFS status changed so dramatically over time.

Comparable findings have also occurred in a British sample, as Wessely et al. (1997) found a CFS prevalence rate of 2.6%, which is comparable to the Reeves et al. (2007) findings in the US with broad criteria. But if psychological disorders were excluded, only .5% of this sample would have CFS. Individuals diagnosed with CFS in this epidemiologic study were subsequently compared to a sample of people with CFS who had been diagnosed from a hospital unit (Euba, Chalder, Deale & Wessely, 1996). Of the community sample, 59% felt their illness might be due to psychological or psychosocial causes compared to 7% for the hospital sample. In Wessely et al.’s (1996) community based sample, 36 individuals were diagnosed as having CFS. Among this group, only 64% had sleep disturbances and 63% had post-exertional malaise. These percentages are again rather low, as both symptoms are critical features of CFS.

If the term CFS were used to designate a broad criteria, as has been used in the US and Britain, the term ME could refer to those that had required symptoms such as those proposed by the Canadian ME/CFS criteria (Carruthers et al., 2003), the ME-ICC criteria (Carruthers et al., 2011) or the more empirically based ME criteria (Jason, Kot et al., 2014). These ME criteria would identify a much smaller percent of the population than the ME criteria (see Figures 2–3), and this group would be more seriously impaired. Inspecting the findings of the CDC prevalence studies (Reeves et al., 2007) as well as research using more specific criteria (e.g., Jason, Sunnquist et al., 2014; Jason, Kot et al., 2014) provides some evidence of there being really two sets of patients, one being identified with broad criteria and the other with more narrow criteria.

In this article, we reviewed critical domains within the CFS and ME literature, and suggested ways to possibly differentiate these case definitions. Our challenge is to decide on which case definition to use, so that the core symptoms are clearly identified. In addition, decisions need to be made regarding whether or not a particular symptom is severe enough to qualify as occurrence of one of the required symptoms. Finally, we need to employ standardized procedures for assessing symptoms. Clearly, the scientific enterprise depends on reliable, valid methods of classifying patients into diagnostic categories, and this critical research activity can enable investigators to better understand etiology, pathophysiology, and treatment approaches for ME and CFS, along with other disorders. There is now a need for an open, transparent, inclusive process for developing a consensus on these issues. This might be accomplished by providing support for structural capacities that set up a mechanism for ongoing data collection and interactive feedback, one that are vetted by broad based gatekeepers representing scientists, patients and government groups.