Abstract
Background
The DePaul Symptom Questionnaire (DSQ) was developed to provide a structured approach for collecting standardized symptomatology and health history information to allow researchers and clinicians to determine whether a patient meets the diagnostic criteria for myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS), myalgic encephalomyelitis (ME), and/or chronic fatigue syndrome (CFS).
Purpose
The purpose of this study was to examine the test-retest reliability of the DSQ.
Methods
Test-retest reliability of the measure was examined with a sample of 26 adults self-identifying as having either ME/CFS, ME and/or CFS and 25 adults who did not self-identify as having these illnesses and were otherwise healthy controls.
Results
Overall, the majority of items on the DSQ exhibited good to excellent test-retest reliability, with Pearson’s or kappa correlation coefficients that were 0.70 or higher.
Conclusions
Thus, the present study suggests that the DSQ is a reliable diagnostic measure that can provide a standardized way of examining illness constructs and symptomatology among patients who identify as having ME/CFS, ME and/or CFS.
Keywords: Myalgic Encephalomyelitis, Myalgic Encephalomyelitis/chronic fatigue syndrome, chronic fatigue syndrome, reliability, DePaul Symptom Questionnaire
Introduction
In order to accurately diagnose an illness or disease, it is important for researchers and clinicians to have a reliable set of criteria to utilize. Without standardized criteria, it is possible that disagreements about diagnostic decisions may arise because of diagnostic unreliability. [1] This type of unreliability and variance may be due to subject variance, occasion variance, information variance, observation variance, and criterion variance. [2, 3] In particular, criterion variance, which refers to the classification of patients’ symptoms into diagnostic categories, can constitute the largest source of diagnostic unreliability. This typically occurs when an operationally explicit set of criteria is not being utilized in the process of diagnosing an illness. [2]
Given the number of case definitions for Myalgic Encephalomyelitis/chronic fatigue syndrome (ME/CFS), [4] Myalgic Encephalomyelitis (ME), [5] and chronic fatigue syndrome (CFS), [6] criterion variance has consistently been one of the most prominent challenges for researchers in the field. Heterogeneous samples may be selected if multiple case definitions are utilized across studies. Furthermore, additional difficulties arise if these case definitions and diagnostic criteria do not specify a diagnostic instrument that should be used or do not explicitly state how to rate the presence and severity of the symptom criteria. [7]
More importantly, because of the lack of agreed upon diagnostic biomarkers for ME, ME/CFS, and CFS, clinical interviews or self-report questionnaires are the main source of information gathered from patients about their symptoms. [6] Thus, regardless of which case definition is used, it is critical to assess symptoms in a standardized way to reduce reliability issues. By using a consistent set of items on a questionnaire or measure, as well as cut off points for defining whether a threshold has been met for symptom criteria, researchers and clinicians will be able to examine the same illness constructs among all their participants or patients. Samples generated from different sites can then be consistently compared across the same symptoms and illness constructs.
The DePaul Symptom Questionnaire (DSQ) [8] was developed to provide a structured approach to gathering standardized symptomatology and health history information to allow investigators to determine whether or not a patient meets the diagnostic criteria for ME/CFS, [4] ME, [5] and/or CFS. [6] The DSQ has been shown to have good construct, convergent and discriminant validity. [9] Additionally, a Persian version of the measure was found to have good content validity. [10] The development of the DSQ was based upon the CFS Questionnaire, [11] which has been shown to have good inter-rater and test–retest reliability. The CFS Questionnaire has also been found to sensitively distinguish among individuals with CFS, individuals with Major Depressive Disorder, and healthy controls. [12]
The DSQ is a self-report instrument that measures symptoms and relevant health history related to the dimensions of the Clinical Canadian Criteria for ME/CFS, [4] the ME International Consensus Criteria, [5] and the Fukuda et al. [6] CFS case definition. First, respondents are asked to rate the frequency of 54 symptoms over the last six months on a five-point Likert scale (0 = none of the time, 1 = a little of the time, 2 = about half the time, 3 = most of the time, and 4 = all of the time). Similarly, they are asked to rate each symptom’s severity over the last six months on a five-point Likert scale (0 = symptom not present, 1 = mild, 2 = moderate, 3 = severe, and 4 = very severe). A composite score on a scale from 0 to 100 is calculated by multiplying both the frequency and severity ratings by 25 and then dividing the sum of these two numbers by 2. In addition, the DSQ contains general demographic questions and other items that inquire about health history. The DSQ is available at REDCap’s shared library (https://redcap.is.depaul.edu/surveys/?s=tRxytSPVVw).
The DSQ utilizes a six-month time frame for assessing symptomatology, as this time frame evidenced good test-retest reliability on the CFS questionnaire. [12] Additionally, the six-month time frame was found to be the optimal time frame for reliably recalling ME, ME/CFS, and CFS symptoms when compared to the past week and past month time frames. [13] This time frame was considered optimal as individuals with these illnesses often report a fluctuating symptom pattern, which may result in lower symptomatology some days and weeks and severe symptomatology on other days and weeks. Investigators could risk losing valuable information about the true nature of patients’ symptoms when using shorter time frames (the past week or right now), as a longer recall period such as six months can capture the overall, chronic severity of the illness.
Additionally, investigators may also risk getting unreliable symptom data when using shorter timeframes. Specifically, Evans and Jason [13] found that CFS symptoms recalled at the past month and the past week time frames resulted in slope coefficients with poor predictive validity, providing further support for using a six month timeframe to assess symptoms experienced by this illness population. Broderick et al. [14] have suggested that individuals with chronic illnesses may have a good grasp of their symptom pattern over time which may provide insight into why individuals with CFS are able to reliably make a global assessment of their symptoms at the six month timeframe. At the present time, there are no published studies that have assessed the reliability and validity of the DSQ. Since the DSQ was developed using a six-month time frame, it is important to validate this measure using the same time frame.
The purpose of the present study was to determine the test-retest reliability of the DSQ in examining individuals with ME, ME/CFS, or CFS and healthy controls. It was hypothesized that this instrument would be a reliable measure for researchers and clinicians to use in order to assess ME, ME/CFS, and CFS symptoms and other illness-related constructs.
Method
Participants
An international convenience sample of 51 adults (over the age of 18) was recruited for this study from a variety of sources: internet forum posts, visits to support groups, re-contacting individuals who had participated in the DePaul Research Team’s studies in the past and had indicated interest in future studies, and contacting individuals who had emailed the team in the past with interest in future studies. The Patient group (n=26) self-identified as having ME, ME/CFS, or CFS, whereas the Control group (n=25) did not self-identify as having ME, ME/CFS, or CFS and were otherwise healthy controls without any exclusionary, self-reported medical or psychiatric illnesses. For the Patient group, individuals could not report any CFS, ME/CFS, or ME defined exclusionary medical or psychiatric illnesses.
The mean age for those in the Patient group was 49.5 years (SD = 10.0). A majority of the Patient group was female (88.5%) and identified as White (96.2%). About half were married or living with a partner (53.8%). Within the Patient group, 65.4% were on disability; 7.7% were homemakers; 3.8% were retired; 11.5% were unemployed; and 11.5% were working part-time. Regarding education, 50.0% of participants had a graduate or professional degree; 30.8% had a college degree; and 19.2% had completed at least one year of college or specialized training.
The mean age of those in the healthy Control group was 53.5 years (SD = 14.3). A majority of the Control group was female (80.0%) and identified as White (92.0%). A majority were married or living with a partner (72.0%). Within the Control group, 8.0% were students; 24.0% were retired; 8.0% were unemployed; and 60.0% were working part- or full-time. Regarding education, 48.0% of participants had a graduate or professional degree; 36.0% had a college degree; 12.0% had completed at least one year of college or specialized training; and 4.0% had a high school degree or equivalent. The Patient and Control groups had no significant socio-demographic differences, with the exception of work status, as the majority of the Patient sample reported that they were on disability.
Procedure
All participants provided informed consent to a protocol approved by the local Institutional Review Board before inclusion in the study. Participants were scheduled to complete a verbal survey twice over the telephone exactly two weeks apart. Interviews were conducted by trained Research Assistants who were supervised by a Masters level graduate student. To maintain consistency, the same interviewer conducted both interviews for each participant. Since ME, ME/CFS, and CFS can be unpredictable and result in unexpected, rapid declines in functioning, participants were given as much time as they needed to complete the survey on each interview day. All participants were sent a follow-up debriefing email explaining the purpose of the study.
Statistical Analyses
In order to evaluate test-retest reliability for each individual item on the DSQ, Pearson’s correlation coefficients were calculated for continuous items, while percent agreement and the kappa statistic were utilized for categorical items. The kappa statistic was chosen because it is more robust than percent agreement alone, as it takes agreement by chance into consideration. Test-retest reliability was analyzed for the Patient group, Control group, and the overall sample. Statistical analyses were performed using SPSS 19.0 software and significance levels were set at p < 0.05.
Results
Symptom Items
Pearson’s correlation coefficients for the frequency, severity, and composite scores of the 54 symptoms on the DSQ are presented in Table 1. Overall, the majority of DSQ symptoms demonstrated good to excellent test-retest reliability, with most correlation coefficients above 0.70. Significant test-retest correlations were found for all 54 symptoms in the Patient group and in the overall sample. Significant test-retest correlations were found for 47 of the symptoms in the Control group. In the overall sample, all symptoms classified within the Fatigue, Post-Exertional Malaise, Neurocognitive and Autonomic categories had significant test-retest correlations (0.80 or higher). Within the Control group, 7 of the 54 symptoms were not significantly correlated (Dead, heavy feeling after starting to exercise; Physically drained or sick after mild activity; Waking up early in the morning; Unable to focus vision and/or attention; Loss of depth perception; Feeling chills or shivers; and Flu-like symptoms).
Table 1.
Test-Retest Pearson’s Correlation Coefficients for the 54 DSQ Symptoms
| Patient Group (n=26) | Control Group (n = 25) | Total (N=51) | |
|---|---|---|---|
| Fatigue | |||
| Fatigue | |||
| Frequency | r = .87 *** | r = .74 *** | r = .93 *** |
| Severity | r = .84 *** | r = .61 ** | r = .90 *** |
| 100-pt Composite Score | r = .90 *** | r = .70 *** | r = .94 *** |
| Post-Exertional Malaise | |||
| Dead, heavy feeling after starting to exercise | |||
| Frequency | r = .71 *** | r = .06 | r = .85 *** |
| Severity | r = .77 *** | r = .22 | r = .88 *** |
| 100-pt Composite Score | r = .80 *** | r = .15 | r = .88 *** |
| Next day soreness/fatigue after everyday activities | |||
| Frequency | r = .73 *** | r = .82 *** | r = .90 *** |
| Severity | r = .71 *** | r = .70 *** | r = .90 *** |
| 100-pt Composite Score | r = .79 *** | r = .79 *** | r = .92 *** |
| Mentally tired after the slightest effort | |||
| Frequency | r = .74 *** | r = .78 *** | r = .90 *** |
| Severity | r = .80 *** | r = .62 ** | r = .94 *** |
| 100-pt Composite Score | r = .81 *** | r = .76 *** | r = .94 *** |
| Minimum exercise makes you physically tired | |||
| Frequency | r = .71 *** | r = .79 *** | r = .95 *** |
| Severity | r = .69 *** | r = .85 *** | r = .94 *** |
| 100-pt Composite Score | r = .72 *** | r = .83 *** | r = .96 *** |
| Physically drained or sick after mild activity | |||
| Frequency | r = .56 ** | r = .18 | r = .86 *** |
| Severity | r = .81 *** | r = .18 | r = .92 *** |
| 100-pt Composite Score | r = .77 *** | r = .18 | r = .91 *** |
| Sleep Dysfunction | |||
| Unrefreshing sleep | |||
| Frequency | r = .84 *** | r = .74 *** | r = .90 *** |
| Severity | r = .76 *** | r = .71 *** | r = .89 *** |
| 100-pt Composite Score | r = .85 *** | r = .76 *** | r = .92 *** |
| Need to nap daily | |||
| Frequency | r = .86 *** | r = .72 *** | r = .87 *** |
| Severity | r = .90 *** | r = .63 ** | r = .90 *** |
| 100-pt Composite Score | r = .89 *** | r = .75 *** | r = .90 *** |
| Problems falling asleep | |||
| Frequency | r = .70 *** | r = .76 *** | r = .79 *** |
| Severity | r = .75 *** | r = .62 ** | r = .80 *** |
| 100-pt Composite Score | r = .77 *** | r = .72 *** | r = .82 *** |
| Problems staying asleep | |||
| Frequency | r = .82 *** | r = .76 *** | r = .84 *** |
| Severity | r = .75 *** | r = .73 *** | r = .81 *** |
| 100-pt Composite Score | r = .81 *** | r = .78 *** | r = .84 *** |
| Waking up early in the morning (e.g. 3am) | |||
| Frequency | r = .72 *** | r = .32 | r = .64 *** |
| Severity | r = .64 *** | r = .34 | r = .63 *** |
| 100-pt Composite Score | r = .70 *** | r = .34 | r = .64 *** |
| Sleep all day and stay awake all night | |||
| Frequency | r = .65 *** | r = .95 *** | r = .70 *** |
| Severity | r = .77 *** | r = .42 * | r = .76 *** |
| 100-pt Composite Score | r = .76 *** | r = .53 ** | r = .78 *** |
| Pain | |||
| Muscle pain/aches | |||
| Frequency | r = .80 *** | r = .69 *** | r = .87 *** |
| Severity | r = .77 *** | r = .58 ** | r = .84 *** |
| 100-pt Composite Score | r = .82 *** | r = .72 *** | r = .89 *** |
| Joint pain | |||
| Frequency | r = .67 *** | r = .73 *** | r = .69 *** |
| Severity | r = .78 *** | r = .54 ** | r = .75 *** |
| 100-pt Composite Score | r = .74*** | r = .67 *** | r = .74 *** |
| Eye pain | |||
| Frequency | r = .96 *** | r = .81 *** | r = .95 *** |
| Severity | r = .94 *** | r = .63 ** | r = .92 *** |
| 100-pt Composite Score | r = .96 *** | r = .76 *** | r = .94 *** |
| Chest pain | |||
| Frequency | r = .62 ** | r = 1.00 *** | r = .74 *** |
| Severity | r = .76 *** | r = .96 *** | r = .79 *** |
| 100-pt Composite Score | r = .79 *** | r = .98 *** | r = .84 *** |
| Bloating | |||
| Frequency | r = .88 *** | r = .82 *** | r = .88 *** |
| Severity | r = .93 *** | r = .80 *** | r = .91 *** |
| 100-pt Composite Score | r = .95 *** | r = .82 *** | r = .93 *** |
| Stomach pain | |||
| Frequency | r = .94 *** | r = .61 ** | r = .90 *** |
| Severity | r = .92 *** | r = .52 ** | r = .90 *** |
| 100-pt Composite Score | r = .93 *** | r = .58 ** | r = .91 *** |
| Headaches | |||
| Frequency | r = .86 *** | r = .77 *** | r = .87 *** |
| Severity | r = .81 *** | r = .85 *** | r = .86 *** |
| 100-pt Composite Score | r = .87 *** | r = .82 *** | r = .89 *** |
| Neurological/Cognitive Symptoms | |||
| Muscle twitches | |||
| Frequency | r = .79 *** | r = .53 ** | r = .79 *** |
| Severity | r = .85 *** | r = .48 * | r = .83 *** |
| 100-pt Composite Score | r = .86 *** | r = .52 ** | r = .85 *** |
| Muscle weakness | |||
| Frequency | r = .55 ** | r = .83 *** | r = .77 *** |
| Severity | r = .65 *** | r = .79 *** | r = .79 *** |
| 100-pt Composite Score | r = .63 ** | r = .81 *** | r = .80 *** |
| Sensitivity to noise | |||
| Frequency | r = .74 *** | r = .53 ** | r = .83 *** |
| Severity | r = .83 *** | r = .44 * | r = .86 *** |
| 100-pt Composite Score | r = .85 *** | r = .51 ** | r = .88 *** |
| Sensitivity to bright lights | |||
| Frequency | r = .81 *** | r = .52 ** | r = .83 *** |
| Severity | r = .79 *** | r = .48 * | r = .79 *** |
| 100-pt Composite Score | r = .82 *** | r = .52 ** | r = .83 *** |
| Problems remembering things | |||
| Frequency | r = .65 *** | r = .82 *** | r = .88 *** |
| Severity | r = .75 *** | r = .72 *** | r = .87 *** |
| 100-pt Composite Score | r = .77 *** | r = .80 *** | r = .91 *** |
| Difficulty paying attention | |||
| Frequency | r = .77 *** | r = .84 *** | r = .89 *** |
| Severity | r = .80 *** | r = .73 *** | r = .88 *** |
| 100-pt Composite Score | r = .79 *** | r = .84 *** | r = .91 *** |
| Difficulty finding right words to say/express thoughts | |||
| Frequency | r = .84 *** | r = .87 *** | r = .92 *** |
| Severity | r = .85 *** | r = .73 *** | r = .90 *** |
| 100-pt Composite Score | r = .91 *** | r = .83 *** | r = .93 *** |
| Difficulty understanding things | |||
| Frequency | r = .79 *** | r = .65 *** | r = .87 *** |
| Severity | r = .80 *** | r = .65 *** | r = .87 *** |
| 100-pt Composite Score | r = .86 *** | r = .65 *** | r = .90 *** |
| Only able to focus on one thing at a time | |||
| Frequency | r = .83 *** | r = .64 ** | r = .87 *** |
| Severity | r = .85 *** | r = .69 *** | r = .90 *** |
| 100-pt Composite Score | r = .85 *** | r = .68 *** | r = .89 *** |
| Unable to focus vision and/or attention | |||
| Frequency | r = .76 *** | r = .13 | r = .84 *** |
| Severity | r = .61 ** | r = .27 | r = .80 *** |
| 100-pt Composite Score | r = .69 *** | r = .19 | r = .83 *** |
| Loss of depth perception | |||
| Frequency | r = .84 *** | r = −.04 | r = .87 *** |
| Severity | r = .77 *** | r = −.04 | r = .83 *** |
| 100-pt Composite Score | r = .82 *** | r = −.04 | r = .87 *** |
| Slowness of thought | |||
| Frequency | r = .88 *** | r = .48 * | r = .91 *** |
| Severity | r = .85 *** | r = .55 ** | r = .91 *** |
| 100-pt Composite Score | r = .91 *** | r = .53 ** | r = .92 *** |
| Absent-mindedness or forgetfulness | |||
| Frequency | r = .61 ** | r = .52 ** | r = .75 *** |
| Severity | r = .81 *** | r = .47 * | r = .85 *** |
| 100-pt Composite Score | r = .75 *** | r = .49 * | r = .81 *** |
| Autonomic Symptoms | |||
| Bladder problems | |||
| Frequency | r = .80 *** | r = .72 *** | r = .82 *** |
| Severity | r = .88 *** | r = .71 *** | r = .86 *** |
| 100-pt Composite Score | r = .85 *** | r = .74 *** | r = .86 *** |
| Irritable bowel problems | |||
| Frequency | r = .91 *** | r = .77 *** | r = .93 *** |
| Severity | r = .89 *** | r = .65 *** | r = .91 *** |
| 100-pt Composite Score | r = .94 *** | r = .73 *** | r = .94 *** |
| Nausea | |||
| Frequency | r = .77 *** | r = .62 ** | r = .84 *** |
| Severity | r = .79 *** | r = .62 ** | r = .86 *** |
| 100-pt Composite Score | r = .79 *** | r = .62 ** | r = .86 *** |
| Feeling unsteady on your feet | |||
| Frequency | r = .74 *** | r = .88 *** | r = .83 *** |
| Severity | r = .58 ** | r = .87 *** | r = .77 *** |
| 100-pt Composite Score | r = .73 *** | r = .88 *** | r = .84 *** |
| Shortness of breath | |||
| Frequency | r = .82 *** | r = .77 *** | r = .85 *** |
| Severity | r = .75 *** | r = .77 *** | r = .82 *** |
| 100-pt Composite Score | r = .82 *** | r = .77 *** | r = .86 *** |
| Dizziness or fainting | |||
| Frequency | r = .81 *** | r = .92 *** | r = .88 *** |
| Severity | r = .80 *** | r = .92 *** | r = .89 *** |
| 100-pt Composite Score | r = .83 *** | r = .92 *** | r = .91 *** |
| Irregular heart beats | |||
| Frequency | r = .89 *** | r = 1.00 *** | r = .92 *** |
| Severity | r = .80 *** | r = 1.00 *** | r = .87 *** |
| 100-pt Composite Score | r = .90 *** | r = 1.00 *** | r = .94 *** |
| Neuroendocrine Symptoms | |||
| Losing or gaining weight without trying | |||
| Frequency | r = .40 * | r = .87 *** | r = .52 *** |
| Severity | r = .41 * | r = .77 *** | r = .56 *** |
| 100-pt Composite Score | r = .41 * | r = .87 *** | r = .53 *** |
| No appetite | |||
| Frequency | r = .88 *** | r = 1.00 *** | r = .89 *** |
| Severity | r = .63 ** | r = 1.00 *** | r = .74 *** |
| 100-pt Composite Score | r = .80 *** | r = 1.00 *** | r = .85 *** |
| Sweating hands | |||
| Frequency | r = .92 *** | r = 1.00 *** | r = .93 *** |
| Severity | r = .94 *** | r = 1.00 *** | r = .96 *** |
| 100-pt Composite Score | r = .95 *** | r = 1.00 *** | r = .96 *** |
| Night sweats | |||
| Frequency | r = .65 *** | r = .93 *** | r = .78 *** |
| Severity | r = .80 *** | r = .97 *** | r = .88 *** |
| 100-pt Composite Score | r = .75 *** | r = .99 *** | r = .86 *** |
| Cold limbs | |||
| Frequency | r = .73 *** | r = .89 *** | r = .82 *** |
| Severity | r = .77 *** | r = .79 *** | r = .83 *** |
| 100-pt Composite Score | r = .76 *** | r = .84 *** | r = .84 *** |
| Feeling chills or shivers | |||
| Frequency | r = .64 *** | r = .12 | r = .63 *** |
| Severity | r = .83 *** | r = .12 | r = .80 *** |
| 100-pt Composite Score | r = .80 *** | r = .12 | r = .76 *** |
| Feeling hot/cold for no reason | |||
| Frequency | r = .60 ** | r = .78 *** | r = .74 *** |
| Severity | r = .62 ** | r = .87 *** | r = .78 *** |
| 100-pt Composite Score | r = .64 *** | r = .86 *** | r = .79 *** |
| Feeling like you have a high temperature | |||
| Frequency | r = .71 *** | r = .80 *** | r = .80 *** |
| Severity | r = .57 ** | r = .80 *** | r = .71 *** |
| 100-pt Composite Score | r = .66 *** | r = .80 *** | r = .78 *** |
| Feeling like you have a low temperature | |||
| Frequency | r = .71 *** | r = 1.00 *** | r = .79 *** |
| Severity | r = .88 *** | r = 1.00 *** | r = .92 *** |
| 100-pt Composite Score | r = .85 *** | r = 1.00 *** | r = .90 *** |
| Alcohol intolerance | |||
| Frequency | r = .92 *** | r = .61 ** | r = .91 *** |
| Severity | r = .94 *** | r = .08 | r = .86 *** |
| 100-pt Composite Score | r = .92 *** | r = .30 | r = .89 *** |
| Immune Symptoms | |||
| Sore throat | |||
| Frequency | r = .85 *** | r = .61 ** | r = .87 *** |
| Severity | r = .84 *** | r = .52 ** | r = .85 *** |
| 100-pt Composite Score | r = .76 *** | r = .57 ** | r = .82 *** |
| Tender/sore lymph nodes | |||
| Frequency | r = .90 *** | r = 1.00 *** | r = .95 *** |
| Severity | r = .87 *** | r = 1.00 *** | r = .93 *** |
| 100-pt Composite Score | r = .90 *** | r = 1.00 *** | r = .95 *** |
| Fever | |||
| Frequency | r = .70 *** | r = .55 ** | r = .69 *** |
| Severity | r = .67 *** | r = .55 ** | r = .71 *** |
| 100-pt Composite Score | r = .69 *** | r = .55 ** | r = .72 *** |
| Flu-like symptoms | |||
| Frequency | r = .84 *** | r = .22 | r = .88 *** |
| Severity | r = .80 *** | r = .23 | r = .85 *** |
| 100-pt Composite Score | r = .83 *** | r = .24 | r = .87 *** |
| Some smells, foods, medications, or chemicals make you feel sick | |||
| Frequency | r = .88 *** | r = .33 | r = .85 *** |
| Severity | r = .85 *** | r = .65 *** | r = .87 *** |
| 100-pt Composite Score | r = .91 *** | r = .49 * | r = .89 *** |
Note:
p < .05,
p < .01
p < .001
Health History Items
In addition to the 54 symptoms, test-retest reliability was analyzed for the remaining items on the DSQ that were related to each participant’s health history. Test-retest percent agreement, kappa coefficients, and Pearson’s correlation coefficients for these items are presented in Table 2. For a few items, the kappa coefficient was not calculated because at least one variable in the two-way table was a constant, or no responses were provided in the Control group. Overall, test-retest percent agreement was above 70% for a vast majority of items amongst both subgroups and the overall sample. Kappa and Pearson’s coefficients were also found to be significant and above 0.70 for most of these items.
Table 2.
Test-Retest Percent of Agreement, Kappa Coefficients, and Pearson’s Correlation Coefficients for Items 67 to 99 on the DSQ
| Patient Group (n=26) | Control Group (n = 25) | Total (N=51) | ||||
|---|---|---|---|---|---|---|
| Have you always had persistent or recurring fatigue/energy problems, even back to your earlier memories as a child? a | 100% | K = 1.0 *** | 80% | K = .66 *** | 90% | K = .82 *** |
| Since your fatigue/energy related illness began, do your headaches either happen more often, feel worse or more severe, or are they in a different place or spot? a | 96% | K = .88 *** | 84% | K = .66 *** | 90% | K = .85 *** |
| How long ago did your problem with fatigue/energy begin? a | 100% | K = 1.0 *** | 80% | K = .67 *** | 90% | K = .85 *** |
| Have you been diagnosed with CFS or ME? a | 100% | c | 100% | c | 100% | K = 1.0 *** |
| Do you currently have a diagnosis of CFS or ME? a | 100% | c | 100% | c | 100% | K = 1.0 *** |
| Who diagnosed you with CFS or ME?a | 100% | K = 1.0 *** | n/a | n/a | ||
| Have any of your family members been diagnosed with CFS or ME? a | 100% | K = 1.0 *** | 96% | K = .91 *** | 98% | K = .95 *** |
| Did you experience any of the following symptoms regularly and repeatedly in the months and years before your fatigue/energy problems began? | ||||||
| Sore throats a | 89% | K = .75 *** | 92% | c | 90% | K = .70 *** |
| Lymph nodes a | 92% | K = .76 *** | 96% | c | 94% | K = .70 *** |
| Unrefreshing sleep a | 96% | K = .89 *** | 92% | K = .70 *** | 94% | K = .81 *** |
| Impaired memory/concentration a | 100% | K = 1.0 *** | 88% | c | 94% | K = .55 *** |
| Prolonged fatigue following exertiona | 96% | K = .84 *** | 96% | c | 96% | K = .73 *** |
| Muscle pain a | 92% | K = .71 *** | 96% | c | 94% | K = .64 *** |
| Headaches a | 88% | K = .74 *** | 96% | K = .78 *** | 92% | K = .77 *** |
| Joint pain a | 89% | K = .51 ** | 96% | c | 92% | K = .46 ** |
| Not having a problem with fatigue/energy a | 96% | c | 92% | K = .82 *** | 94% | K = .87 *** |
| If you rest, does your problem with fatigue/energy go away? a | 92% | K = .84 *** | 56% | K = .33 * | 75% | K = .65 *** |
| How long do you have to rest for your problem with fatigue/energy to entirely or partially go away? a | 88% | K = .70 *** | 55% | K = .33 | 75% | K = .61 *** |
| If you were to become exhausted after actively participating in extracurricular activities, sports, or outings with friends, would you recover within an hour or two after the activity ended? a | 88% | K = .34 | 100% | K = 1.00 *** | 94% | K = .88 *** |
| Do you reduce your activity level to avoid experiencing problems with fatigue/energy? a | 100% | c | 76% | K = .56 ** | 88% | K = .80 *** |
| Do you experience a worsening of you fatigue/energy related illness after engaging in minimal physical effort? a | 92% | K = −.04 | 88% | K = .75 *** | 90% | K = .84 *** |
| Do you experience a worsening of your fatigue/energy related illness after engaging in mental effort? a | 85% | K = .25 | 90% | K = .44 * | 87% | K = .74 *** |
| If you feel worse after activities, how long does this last? a | 72% | K = .35 ** | 33% | K = −.20 | 68% | K = .38 *** |
| Are you currently engaging in any form of exercise? a | 85% | K = .68 ** | 84% | K = .25 | 84% | K = .59*** |
| If you are not, why aren’t you exercising: | ||||||
| Not interested? a | 100% | c | 92% | c | 96% | . |
| No time? a | 100% | c | 92% | K = .48 * | 96% | K = .48 ** |
| Would like to but cannot because of problems with fatigue/energy? a | 100% | K = 1.0 *** | 100% | c | 100% | K = 1.0 *** |
| Cannot because exercise makes symptoms worse?a | 92% | K = .82 *** | 100% | c | 94% | K = .79 *** |
| Over what period of time did your fatigue/energy related illness develop? a | 81% | K = .76 *** | 80% | K = .56 *** | 80% | K = .75 *** |
| How would you describe the course of your fatigue/energy related illness? a | 73% | K = .59 *** | 80% | K = .55 *** | 77% | K = .68 *** |
| Which statement best describes your fatigue/energy related illness during the last 6 months? a | 92% | K = .88 *** | 79% | K = .62 *** | 86% | K = .82 *** |
| Did you fatigue/energy related illness start after you experienced: | ||||||
| An infectious illness? a | 96% | K = .90 *** | 100% | K = 1.0 *** | 98% | K = .96 *** |
| An accident? a | 100% | c | 100% | c | 100% | c |
| A trip or vacation? a | 100% | K = 1.0 *** | 100% | K = 1.0 *** | 100% | K = 1.0 *** |
| An immunization? a | 96% | K = .84 *** | 100% | c | 98% | K = .85 *** |
| Surgery? a | 96% | K = .78 *** | 100% | K = 1.0 *** | 98% | K = .85 *** |
| Severe stress? a | 96% | K = .91 *** | 84% | K = .51 ** | 90% | K = .74 *** |
| Other? a | 85% | K = .42 * | 88% | K = .50 * | 86% | K = .46 ** |
| I am not ill a | n/a | 84% | K = .67 ** | 92% | K = .81 *** | |
| Have you ever consulted a medical doctor or health professional about your fatigue/energy problem? a | 100% | c | 96% | K = .87 *** | 98% | K = .96 *** |
| Do you currently have a medical doctor overseeing your fatigue/energy problem? a | 92% | K = .71 *** | 89% | K = .61 * | 92% | K = .81 *** |
| Do you currently have any medical illness(es) that might be causing your symptoms? a | 92% | K = .82 *** | 79% | K = .58 ** | 86% | K = .70 *** |
| Are you currently taking any medications (OTC or prescription)? a | 96% | c | 96% | K = .88 *** | 96% | K = .81 *** |
| Do you think any medication(s) is(are) causing your problem with fatigue/energy? a | 96% | K = .65 ** | 67% | K = .39 * | 82% | K = .55 *** |
| Have you ever been diagnosed and/or treated for any of the following: | ||||||
| Major depression? a | 62% | K = .76 *** | 92% | K = .78 *** | 90% | K = .77 *** |
| Major depression with melancholic or psychotic features? a | 96% | c | 96% | K = .65 ** | 96% | K = .48 ** |
| Bipolar disorder?a | 100% | c | 100% | c | 100% | c |
| Anxiety? a | 96% | K = .92 *** | 96% | K = .78 *** | 96% | K = .90 *** |
| Schizophrenia? a | 100% | c | 100% | c | 100% | c |
| Eating disorder? a | 100% | c | 100% | c | 100% | c |
| Substance abuse? a | 100% | c | 100% | c | 100% | c |
| Multiple chemical sensitivities? a | 96% | K = .89 *** | 100% | c | 98% | K = .90 *** |
| Fibromyalgia? a | 96% | K = .92 *** | 100% | c | 98% | K = .95 *** |
| Allergies? a | 81% | K = .62 ** | 100% | K = 1.00 *** | 90% | K = .80 *** |
| Other? a | 58% | K = .15 | 68% | K = .38 * | 63% | K = .26 |
| No diagnosis/treatment a | 96% | K = .78 *** | 92% | c | 94% | K = .54 *** |
| What do you think is the cause of your problem with fatigue/energy? a | 92% | K = .76 *** | 68% | K = .56 *** | 80% | K = .70 *** |
| Do you think anything specific in your personal life or environment accounts for your problem with fatigue/energy? a | 69% | K = .28 | 80% | K = .68 *** | 75% | K = .61 *** |
| In the past 4 weeks, approximately how many hours per week have you spent doing the following: | ||||||
| Household related activities? b | r = .72 *** | r = .62 ** | r = .69 *** | |||
| Social/recreational activities? b | r = .70 *** | r = .41 * | r = .61 *** | |||
| Family related activities? b | r = .75 *** | r = .76 *** | r = .76 *** | |||
| Work related activities? b | r = .93 *** | r = .94 *** | r = .97 *** | |||
| In the past 4 weeks, have you had to reduce the number of hours you previously spent (prior to your illness) on occupational, social, or family activities because of your health or problems with fatigue/energy? a | 92% | K = −.04 | 80% | K = .65 *** | 86% | K = .77 *** |
| Before your fatigue/energy related illness, approximately how many hours per week did you used to spend on the following: | ||||||
| Household related activities? b | r = .83 *** | n/a | n/a | |||
| Social/recreational activities? b | r = .80 *** | n/a | n/a | |||
| Family related activities? b | r = .82 *** | n/a | n/a | |||
| Work related activities? b | r = .61 ** | n/a | n/a | |||
| Please rate the amount of energy you had available yesterday, using a scale from 1 to 100 where 1=no energy and 100=your pre-illness energy level b | r = .52 ** | r = .55 ** | r = .80 *** | |||
| Please rate the amount of energy you expended yesterday b | r = .57 ** | r = .37 | r = .62 *** | |||
| Please rate the amount of fatigue you had yesterday, using a scale from 0–100 where 1=no fatigue and 100=severe fatigue b | r = .71 *** | r = .38 | r = .82 *** | |||
| For the past week, please rate the amount of energy you had available b | r = .68 *** | r = .46 * | r = .78 *** | |||
| For the past week, please rate the amount of energy you have expended b | r = .86 *** | r = .27 | r = .80 *** | |||
| For the past week, please rate the amount of fatigue you had b | r = .71 *** | r = .61 ** | r = .84 *** | |||
| Since the onset of your problems with fatigue/energy, have your symptoms caused a 50% or greater reduction in your activity level? a | 100% | c | 84% | K = .70 *** | 92% | K = .87 *** |
| Do you experience frequent viral infections with prolonged recovery periods? a | 92% | K = .85 *** | 100% | c | 96% | K = .89 *** |
| Are you intolerant of extremes of temperatures (when it is extremely hot or cold)?a | 88% | K = .60 ** | 92% | K = .82 *** | 90% | K = .80 *** |
Note:
Categorical data,
Continuous data,
Cannot be computed because at least one variable in the 2-way table is a constant
p < .05
p < .01
p < .001
For the overall sample, 55 of the 61 items for which test-retest percent agreement was calculated had agreements of 80% or higher. Of the 55 items for which kappa was calculated, 54 were significant for the overall sample. Correlation coefficients were calculated for ten items, all of which were significant for the overall sample.
For the Patient group, test-retest percent agreement was greater than 70% for all but three items. Additionally, the kappa or Pearson’s coefficients were significant for all but six items; however, four of these six items had high percent agreements of 88% or greater.
For the Control group, test-retest percent agreement was greater than 70% for all but six items. Additionally, the kappa or Pearson’s coefficients were significant for all but six items. These items asked about specific instances of fatigue or energy problems, such as “How long do you have to rest for your problem with fatigue/energy to entirely or partially go away?”
It should be noted that these non-significant items for the Patient and Control groups represent a small percentage of the overall items examined, as can be seen in Table 2. In general, when examining test-retest percent agreement, kappa, and Pearson’s coefficients for the overall sample, the items demonstrated good test-retest reliability.
Additionally, the kappa coefficient was calculated for the diagnosis of CFS [6] and for the diagnosis of ME/CFS. [4] Kappa was significant and greater than 0.90 for both case definitions.
Discussion
The purpose of the current study was to evaluate the test-retest reliability of the DePaul Symptom Questionnaire (DSQ). The good to excellent correlation coefficients for most items on the DSQ suggest that the overall instrument is a reliable measure for examining symptoms and illness constructs in patient and healthy control samples. When the overall sample was analyzed, all but one reliability coefficient was significant at the p < 0.01 level. This high level of reliability could be attributed to the specificity of the language used throughout the DSQ. Most items reference a specific time period, and questions about symptoms inquire separately about frequency and severity over the past six months. This level of specificity and clarity may have reduced variance in responses and contributed to the high levels of reliability observed.
In examining the symptom items, high Pearson’s correlation coefficients were consistently observed for the overall sample and the Patient group; however, a few symptoms were not significantly correlated between the test and retest time points for the Control group. Certain symptoms may not generally apply to the lives of otherwise healthy individuals; therefore, they may not be as aware of their symptoms (e.g. feeling chills or shivers) compared to those in the Patient group. Additionally, the Control group is less likely to have experienced the drastic fluctuations in functioning and symptom severity as those in the Patient group. Their daily lives may also be more variable because they are physically able to engage in more activities; thus making it more difficult to reliably recall the presence and severity of certain symptoms (e.g. physically drained or sick after mild activity or waking up early in the morning) over the last six months. Nonetheless, this measure proves to be a largely reliable measure of symptomatology amongst those in both the Patient and Control groups.
In examining items that inquire about participants’ health history, high test-retest percent agreement was also found, and kappa coefficients were generally above 0.70. In other words, participants responded to these items in a consistent manner during both interviews, so it can be reasonably concluded that these DSQ items were a reliable way of measuring illness constructs related to one’s fatigue and energy. However, there were a few exceptions. Within the Control group, two of the items that had low percent agreement and non-significant kappa coefficients were not answered by the majority of Controls (“If you feel worse after activities, how long does this last” and “How long do you have to rest for your problem with fatigue/energy to entirely or partially go away?”), as these items could be skipped if the topic was not applicable to them. Thus, the analysis for these items was less robust, and these specific questions may simply not be relevant to healthy Controls.
Several items specifically asked participants to rate their available energy, expended energy, and fatigue yesterday and during the past week on a 100-point scale. For those in the Patient group, lower Pearson’s coefficients were found for the items that referenced yesterday, whereas higher coefficients were found for the items that referenced the past week. Hawk and colleagues [12] also found that individuals with CFS evidenced higher test-retest reliability when recalling their fatigue and energy symptoms over the past week compared to the past day on the CFS Questionnaire. Thus, those in the Patient group may be able to more reliably recall their fatigue and energy over longer periods of time, since their symptoms may have more fluctuations day to day. Conversely, for those in the Control group, lower Pearson’s coefficients were found for both reference time points. Once again, those in the Control group may have more unpredictable lifestyles compared to those with a chronic illness because they are physically able to engage in more activities, so they will have less reliable responses for these types of items. Additionally, because the Control group does not suffer from fatigue or energy problems, it is also possible that they may not be as attuned to daily levels of energy and fatigue as those in the Patient group.
Furthermore, for two items on the DSQ, participants were given a list of possible response options and an additional response called other: “Did your fatigue/energy related illness start after you experienced any of the following?” and “Have you ever been diagnosed and/or treated for any of the following?” If the list provided did not apply or if there were additional comments, participants were instructed to choose other and then list any other circumstances surrounding their illness onset and any other diagnoses they may have. For both the Patient and Control groups, lower percent agreement and/or lower kappa coefficients were found for both of these items if the participant had chosen to respond other. This is likely due to the fact that other is an ambiguous category, so participants may remember different aspects of their lives during the test and retest interviews. Hence, there were slightly lower reliabilities for these two response options, but greater reliability was found for the item responses that contained concrete selections.
Overall, the DSQ was found to be a reliable measure across the test-retest phone interviews over a two-week time period. Despite fluctuations in symptoms over time, participants could consistently recall details about their illness. While individual items were assessed for test-retest reliability, the overarching purpose of this study was to determine the test-retest reliability of the DSQ as a whole and not for each individual item on the measure. Utilizing the DSQ as a standardized measure will allow for a well-defined characterization of a patient’s illness. Thus, researchers and clinicians will be able to improve diagnostic reliability when examining those with ME, ME/CFS, and/or CFS, and recruit more homogenous samples. Through the use of homogenous samples, researchers may eventually be able to establish consistent biomarkers that will help distinguish this illness from other illness groups. A recent study [15] used the DSQ as an instrument to assess whether participants met the Fukuda et al. [6] CFS criteria and the Canadian Clinical [4] ME/CFS criteria. The percentage of participants who met each case definition was consistent across three distinct samples, indicating that the DSQ can reliably classify participants.
It is important to note that a measure such as the DSQ can exhibit good reliability without necessarily possessing adequate validity. If the ultimate goal is to decrease criterion variance, a measure must be reliable and valid. It can be concluded that the DSQ possesses good face and content validity as the constructs it purports to measure (ME, ME/CFS, and CFS symptomatology and health history) appear to be comprehensively assessed within the measure, and items were constructed based upon the current case definitions used to diagnose ME, ME/CFS, and CFS. The authors have also examined construct and convergent validity of the DSQ through the use of factor analytic strategies and comparisons with theoretically-related measures of symptomatology and functioning. [9]
Results indicated that the DSQ measures three distinct factors of ME and CFS symptomatology (Post-Exertional Malaise, Neurological/Cognitive Dysfunction and Neuroendocrine, Autonomic & Immune Dysfunction). Additionally, symptomatology scores on the DSQ were significantly correlated with the Medical Outcomes Study Short-Form 36 Survey physical health subscales [16] and the Centers for Disease Control and Prevention Symptom Inventory composite score. [17] The authors also recently compared a CFS and Control group on their responses to the DSQ symptom items. [18] A data mining technique was applied to these responses, and it identified a combination of three DSQ symptom items that accurately classified over 95% of the CFS and Control groups. The results of this study provide some support for the DSQ’s validity. Further validation work should focus on the extent to which scores on the DSQ are able to distinguish between individuals with ME, ME/CFS, and CFS, healthy controls, and other illness groups. Additionally, the racial homogeneity of the sample utilized in this study is a limitation and decreases generalizability of the findings. Future reliability and validity work should aim to include representative samples of individuals with ME, ME/CFS, and CFS.
Acknowledgments
Funding: Funding was provided by National Institute of Allergy and Infectious Diseases [Grant numbers AI 49720 and AI 055735].
References
- 1.Jason LA, King CP, Richman JA, Taylor RR, Torres SR, Song S. U.S. case definition of chronic fatigue syndrome: Diagnostic and theoretical issues. J Chron Fatigue Synd. 1999;5:3–33. [Google Scholar]
- 2.Spitzer R, Endicott J, Robins E. Clinical criteria for psychiatric diagnosis and DSM-III. Am J Psyc. 1975;132:1187–1192. doi: 10.1176/ajp.132.11.1187. [DOI] [PubMed] [Google Scholar]
- 3.Jason LA, Choi M. Dimensions and assessment of fatigue. In: Yatanabe Y, Evengard B, Natelson BH, Jason LA, Kuratsune H, editors. Fatigue Science for Human Health. Tokyo: Springer; 2008. pp. 1–16. [Google Scholar]
- 4.Carruthers BM, Jain AL, De Meirleir KL, Peterson DL, Klimas NG, Lerner AM, Bested AC, Flor-Henry P, Joshi P, Powles ACP, Sherkey JA, van de Sande MI. Myalgic encephalomyelitis/chronic fatigue syndrome: Clinical working case defintion, diagnostic and treatment protocols. J Chron Fatigue Synd. 2003;11:7–116. [Google Scholar]
- 5.Carruthers BM, van de Sande MI, De Meirleir KL, Klimas NG, Broderick G, Mitchell T, Staines D, Powles CP, Speight N, Vallings R, Bateman L, Bell DS, Carlo-Stella N, Chia J, Darragh A, Jo D, Lewis D, Light AR, Marshall-Gradisbik S, Mena I, Mikovits J, Miwa K, Murovska M, Pall ML, Stevens S. Myalgic encephalomyelitis: International consensus criteria. J Int Med. 2011;270:327–338. doi: 10.1111/j.1365-2796.2011.02428.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Fukuda K, Straus SE, Hickie I, Sharpe MC, Dobbins JG, Komaroff AL. The chronic fatigue syndrome: A comprehensive approach to its definition and study. Ann Int Med. 1994;121:953–959. doi: 10.7326/0003-4819-121-12-199412150-00009. [DOI] [PubMed] [Google Scholar]
- 7.Cantwell DP. Classification of child and adolescent psychopathology. J Child Psychol Psyc. 1996;37:3–12. doi: 10.1111/j.1469-7610.1996.tb01377.x. [DOI] [PubMed] [Google Scholar]
- 8.Jason LA, Evans M, Porter N, Brown M, Brown AA, Hunnell J, Friedberg F. The development of a revised Canadian myalgic encephalomyelitis chronic fatigue syndrome case definition. Am J Biochem Biotechnol. 2010;6:120–135. [Google Scholar]
- 9.Brown A, Jason L. Validing a measure of ME/CFS symptomatology. Fatigue Biomed Health Behav. 2014;2:132–152. doi: 10.1080/21641846.2014.928014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Haddadi M, Zakerian SA, Mahmoudi M, Nasl Saraji J, Parsa Yekta Z, Aliyari A. Investigation of chronic fatigue syndrome questionnaire validity and reliability: CFS (DSQ Revised) Scientific Journal of School of Public Health and Institute of Public Health Research. 2014;12:84–93. [Google Scholar]
- 11.Jason LA, Ropacki MT, Santoro NB, Richman JA, Heatherly W, Taylor RR, Ferrari JR, Haney-Davis TM, Rademaker A, Dupuis J, Golding J, Plioplys AV, Plioplys S. A screening instrument for chronic fatigue syndrome: reliability and validity. J Chron Fatigue Synd. 1997;3:39–59. [Google Scholar]
- 12.Hawk C, Jason LA, Torres-Harding S. Reliability of a chronic fatigue questionnaire. J Chron Fatigue Synd. 2006;13:41–66. [Google Scholar]
- 13.Evans M, Jason LA. Effects of timeframe on the recall reliability of CFS symptoms. Eval Health Prof. doi: 10.1177/0163278713497014. In press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Broderick JE, Schwartz JE, Vikingstad G, Pribbernow M, Grossman S, Stone AA. The accuracy of pain and fatigue items across different reporting periods. Pain. 2008;139:146–157. doi: 10.1016/j.pain.2008.03.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Jason LA, Brown A, Evans M, Sunnquist M, Newton JL. Contrasting chronic fatigue syndrome versus myalgic encephalomyelitis/chronic fatigue syndrome. Fatigue Biomed Health Behav. 2013;1:168–183. doi: 10.1080/21641846.2013.774556. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Ware JE, Snow KK, Kosinski M. SF-36 Health Survey: Manual and Interpretation Guide. Quality Metric Incorporated; 2000. [Google Scholar]
- 17.Wagner D, Nisenbaum R, Heim C, Jones JF, Unger ER, Reeves WC. Psychometric properties of the CDC Symptom Inventory for assessment of chronic fatigue syndrome. Popul Health Metr. 2005;3 doi: 10.1186/1478-7954-3-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Jason LA, Sunnquist M, Brown A, Evans M, Vernon S, Furst J, Simonis V. Examining case definition criteria for chronic fatigue syndrome and Myalgic Encephalomyelitis. Fatigue Biomed Health Behav. 2014;2:40–56. doi: 10.1080/21641846.2013.862993. [DOI] [PMC free article] [PubMed] [Google Scholar]