Abstract
Objective:
As eating disorders (EDs) often emerge during college, managing EDs would ideally integrate prevention and treatment. To achieve this goal, an efficient tool is needed that detects clinical symptoms and level of risk. This study evaluated the performance of a screen designed to identify individuals at risk for or with an ED.
Participants:
Five hundred forty-nine college-age women.
Methods:
Participants completed a screen and diagnostic interview.
Results:
Using parsimonious thresholds for ED diagnoses, screen sensitivity ranged from 0.90 (anorexia nervosa) to 0.55 (purging disorder). Specificity ranged from 0.99 (anorexia nervosa) to 0.78 (subthreshold binge eating disorder) compared to diagnostic interview. Moderate to high area under the curve values were observed. The screen had high sensitivity for detecting high risk.
Conclusions:
The screen identifies students at risk and has acceptable sensitivity and specificity for identifying most ED diagnoses. This tool is critical for establishing stepped care models for ED intervention.
Keywords: Eating Disorders, Screening, Prevention, Treatment, Risk, Diagnosis
Eating disorders are prevalent, serious issues among college-age women1 and often emerge during college.2,3 Eating disorders can lead to medical and other psychological problems, are impairing, and can result in high service utilization and costs.4–14 Further, problematic eating behaviors (e.g., binge eating, compensatory behaviors) and concerns about one’s weight and shape are associated with increased risk for eating disorder onset.15,16 Thus, tools are needed to detect individuals experiencing symptoms associated with eating disorder risk or clinical pathology so that they can be referred to appropriate prevention or treatment.
Population health management focuses on integrating prevention and treatment to reduce the incidence and prevalence of a disease.17,18 The first step in managing the health of a defined population, such as a population of college women, is to have an efficient tool that can precisely identify individuals at high risk for an eating disorder (in need of selective prevention interventions), with early symptoms (in need of indicated preventive interventions), and with clinical diagnoses (in need of clinical treatment services). Using one tool to detect individuals at risk for or with eating disorder symptoms would have the benefit of improving clinical triage and the allocation of resources to more effectively manage the health of defined populations.
Existing screens have demonstrated predictive validity for identifying women at high-risk of developing an eating disorder19,20 or with current eating disorders.21–26 However, to our knowledge, no screen exists that sorts individuals into low-risk, high-risk, and probable subthreshold or full-syndrome eating disorders. This type of comprehensive screen would be useful for developing population health management programs for eating disorders. Thus, we have developed a comprehensive screening tool—delivered online—that identifies individuals at risk for or with eating disorder symptoms across DSM-5 eating disorder diagnostic categories. In line with national priorities for improving the precision of treatment delivery and translating evidence-based interventions into public practice,27–29 this screening tool has the potential to significantly improve the health of college students affected by or at risk for eating disorder symptoms.
The purpose of this study is to describe the development of a screening instrument and evaluate its performance, compared to a structured clinical interview, in detecting individuals with eating disorder diagnoses as well as individuals at high risk for an eating disorder among a sample of college-age women. Specifically, we assess the sensitivity and specificity of the screen for detecting DSM-5 defined eating disorder cases and individuals at high risk identified by clinical interview. We hypothesized the screen would be a valid tool for identifying college-age women at high risk for or with an eating disorder.
Methods
Participants
Participants were college-age women between the ages of 18–25 years from the St. Louis, Sacramento, and San Francisco Bay areas. Participants were individuals who self-selected to enroll in a study that broadly targeted women interested in improving body esteem, weight, or interpersonal functioning. Participants were ineligible for study inclusion if they were male, endorsed active suicidality, had a severe psychiatric disorder (e.g., psychotic disorder, bipolar disorder), or did not have Internet access.
Procedure
Participants for this study completed baseline assessments as part of recruitment procedures to enroll in one of two concurrent, multi-site randomized controlled trials evaluating Internet-based interventions for the prevention of eating disorders in college-age women.30,31 Individuals were recruited primarily from academic institutions, through flyers posted on university campuses and in the surrounding local area, advertisements through student groups, referrals from student health and counseling centers, Craigslist, Facebook, and word of mouth. Recruitment materials advertised a study for women who were interested in feeling better about their body, concerned about their weight, having interpersonal difficulties, or struggling to focus on schoolwork.
Interested individuals contacted study staff and completed a self-report screening questionnaire online or by telephone, described below. Potentially eligible individuals were then invited to complete an in-person assessment, which occurred on average two weeks later. During this in-person assessment, participants completed a semi-structured clinical interview [(i.e., the Eating Disorder Examination 14th Edition Diagnostic Version (EDE)]32,33 and self-report questionnaires, including the Weight Concerns Scale.20,34 Trained research assistants conducted the interviews, and any scoring discrepancies were resolved by study team consensus. Participants also completed height and weight measurements in-person.
Written informed consent was obtained from all participants after the study procedures had been explained. This study was approved by the Institutional Review Boards of all participating institutions.
Measures
Self-report screening instrument.
The Stanford-Washington University Eating Disorder (SWED) screen was developed to identify students at risk for or with an eating disorder. The measure is presented in Appendix A. The screen includes 11 questions, from which additional follow-up questions are administered. Items assess for demographics, height and weight, eating disorder behaviors, concerns about weight and shape, and impairment. In developing this screen, we used an approach that retained items that were intended to parsimoniously reflect current approaches to eating disorder diagnosis and classification. Items for the screen were primarily adapted from psychometrically sound, publically available measures. Items that assessed eating disorder behaviors were adapted from the Eating Disorder Examination – Questionnaire21 and the Eating Disorder Diagnostic Scale.24,25 These items used binary, “yes/no” response options to assess whether individuals engaged in specific eating disorder behaviors at all over the past four weeks, at least twice per week over the past month, and at least once per week for the past three months. For this validation study, all of these questions were presented to individuals. However, the sequential order in which the frequency of eating disorder behaviors is assessed is designed to optimize the screen’s efficiency for future online delivery: only individuals who respond “yes” to the first item (i.e., whether a specific behavior occurred at all over the past four weeks) will be prompted to complete the additional two items for that specific eating disorder behavior. Items that assessed concerns for weight and shape were from the Weight Concerns Scale (WCS),20,34 a 5-item self-report questionnaire that has been validated to identify individuals experiencing an overconcern with their weight and shape. To score the WCS, responses to each item are recoded on a 0–100 scale, and then an average is computed across the five items, such that the total WCS score ranges between 0–100.
Clinical interview.
The EDE32,33 is a semi-structured clinical interview that is used to assess for eating disorder pathology and from which eating disorder diagnoses can be derived. Items are rated on a 0–6 scale, with higher scores indicating greater psychopathology. In an effort to reduce participant burden, participants were assessed using a modified version of the EDE, in which a subset of items was administered to all participants sufficient to make a diagnosis of an eating disorder.
Anthropometrics.
Objective weight and height measurements were measured in triplicate using a calibrated scale and portable stadiometer, respectively, in order to calculate participants’ body mass index (BMI; kg/m2). Participants were weighed wearing light indoor clothing and without shoes.
Determining High Risk Status & Diagnostic Criteria
We created operational categorical screen criteria for high-risk status and DSM-5 eating disorder diagnoses based on the items in the screening instrument and EDE interview, presented in Appendix B.
High Risk Status.
For this study, “true” over-concern with weight and shape was defined as having a score ≥4 on the average of four items on the EDE: importance of weight, importance of shape, fear of weight gain, and feelings of fatness, assessed over the previous month. These four items are part of the EDE Weight Concern and Shape Concern subscales and have been calculated as an average in past research to measure weight/shape concerns.35 (We note that because we did not administer the full EDE, we were unable to calculate the EDE Weight Concern or Shape Concern subscales.) Cut-off scores of 4 are used to indicate clinically-significant symptoms on the EDE.33,36
Based on the screen, individuals were considered high risk if they had a WCS score at or above 47 and did not screen positive for having an eating disorder. Past research has shown that scores ≥47 indicate elevated weight/shape concerns and are associated with increased risk for eating disorder onset.19
Eating Disorder Diagnoses.
The criteria for eating disorder diagnoses were developed to align with DSM-5 criteria for eating disorder diagnoses.37 This study did not assess for feeding disorders (e.g., avoidant-restrictive food intake disorder) as they primarily occur in children. For this study, “true” eating disorder diagnoses were determined based on objectively-measured height and weight (which were used to calculate BMI) and items from the EDE, described in Appendix B.
Based on the screen, eating disorder diagnoses were assessed using participants’ self-reported height and weight (which were used to calculate BMI) and responses to questions about eating disorder behaviors and concerns about weight and shape. We also wanted the screen results to align with stakeholder preferences at university student health services centers. To achieve this goal, we sought input from collaborating university student health services centers to determine their priorities for referrals to their clinic based on screen results. The diagnoses identified as clinical priorities by the student health services team were anorexia nervosa, bulimia nervosa, and binge eating disorder. In addition, clinicians at the university student health services centers said they wanted to receive a clinical referral for any student who endorsed purging behaviors to control weight (i.e., self-induced vomiting or diuretic or laxative misuse) at any frequency in the previous month, given that low-frequency purging behaviors are associated with increased risk for eating disorder onset.15 Thus, based on the preferences of university stakeholders, individuals meeting the above criteria would be referred for medical assessment and indicated preventive intervention as appropriate.
Analysis Plan
We evaluated eating disorder diagnostic criteria on the screen in relation to the interview in two stages. In these analyses, diagnoses were not considered mutually exclusive (i.e., the “trumping order” of eating disorder diagnoses in the DSM-5 was ignored). First, the sensitivity and specificity of categorical diagnostic items were calculated without consideration of scores on the WCS. Second, receiver operating characteristic (ROC) analysis was used to optimize sensitivity and specificity for cases screening positive on stage 1 items (categorical symptom criteria), using WCS scores. Sensitivity was defined as the proportion of individuals with a particular eating disorder diagnosis (as diagnosed on the interview) who were correctly identified as having that eating disorder diagnosis on the screen. Specificity was defined as the proportion of individuals without a particular eating disorder diagnosis (as diagnosed on the interview) who were correctly identified as not having that eating disorder diagnosis on the screen. Total sensitivity and specificity estimates were derived by calculating the products of stage 1 and stage 2 (WCS score) sensitivity scores and the product of stage 1 and stage 2 (1 - specificity) estimates at possible cut-off points on the WCS (i.e., 47 to 69). The diagnostic performance of the screening instrument was compared to the EDE interview using ROC analysis to determine the area under the curve (AUC).38,39
We also estimated the prevalence, per 1,000 cases, of detecting each eating disorder diagnosis and detecting individuals at high risk using the screening instrument, with a mutually exclusive assignment of a diagnosis/risk status with a trumping order as follows:
Anorexia Nervosa > Bulimia Nervosa > Binge Eating Disorder > Other Specified Feeding or Eating Disorder (OSFED) (Subthreshold Bulimia Nervosa) > OSFED (Purging Disorder) > OSFED (Subthreshold Binge Eating Disorder) > High Risk
The trumping order for anorexia nervosa, bulimia nervosa, and binge eating disorder were based on DSM-5 criteria.37 Given that no trumping order is specified in the DSM-5 for the OSFED diagnoses, we elected for subthreshold bulimia nervosa to trump subthreshold binge eating disorder to mimic the trumping order for their full-syndrome counterparts, and for purging disorder to trump subthreshold binge eating disorder given university stakeholder preferences to prioritize the detection of individuals who engage in purging.
Lastly, we evaluated the performance of the screen to detect individuals at high risk for an eating disorder compared to the in-person baseline assessment. Pearson correlations were used to assess the test-retest reliability of WCS scores on the screen and during the baseline assessment among those who screened negative for an eating disorder on the screen. ROC analyses then were used to determine the sensitivity, specificity, and AUC for predicting individuals with WCS scores ≥47 on the baseline assessment by WCS scores ≥47 on the screen. We also explored the performance of the screen compared to the EDE for detecting high risk. This analysis was considered exploratory given that we used an average item score for determining “weight/shape concerns” on the EDE interview. Correlations also were used to compare scores on the WCS on the screen to the EDE “weight/shape concerns” average item score. Then, ROC analyses were used to evaluate the performance of the screen for predicting individuals identified as having elevated weight/shape concerns on the EDE (i.e., ≥4 on the “weight/shape concerns” average item score) by identified as having an over-concern with weight and shape on the screen (i.e., ≥47 on the WCS19).
Results
The sample was comprised of 549 college-age women. Of these, 56% identified as White/Caucasian. Average BMI was 24.5 kg/m2 (SD=5.02), and average age was 20.6 years (SD=1.97).
Detecting Eating Disorder Diagnoses
Results of the psychometric testing comparing the screening instrument to the EDE interview are presented in Table 1, which shows the sensitivity and specificity of identifying each eating disorder diagnosis without WCS scores (i.e., WCS = none; stage 1 results) and across a range of WCS scores (i.e., WCS = 47 to 69; stage 2 results). In choosing one WCS cut-point across the diagnoses for parsimony (selected to maximize balance in sensitivity and specificity for the full-threshold diagnoses, as preferred by the university stakeholders), categorical screening criteria coupled with a WCS cut-point of 59 offered a reasonable balance between sensitivity and specificity, as indicated in Table 1. Sensitivity ranged from 0.90 (anorexia nervosa) to 0.55 (purging disorder), and specificity ranged from 0.99 (anorexia nervosa) to 0.79 (subthreshold binge eating disorder). Using a WCS cut-point of 59, AUC values of using the screen criteria were 0.94 (95% CI = 0.84–1.00) for anorexia nervosa, 0.85 (95% CI = 0.75–0.96) for bulimia nervosa, 0.80 (95% CI = 0.72–0.89) for binge eating disorder, 0.77 (95% CI = 0.69–0.86) for subthreshold bulimia nervosa, 0.75 (95% CI = 0.65–0.85) for purging disorder, and 0.76 (95% CI = 0.69–0.82) for subthreshold binge eating disorder. All of these AUC values were in the moderate to high range.39
Table 1:
Screen sensitivity and specificity of identifying DSM-5 eating disorder diagnoses using categorical screening criteria across a range of Weight Concerns Scale scores
| WCS score ≥ cut off: | Anorexia Nervosa (AN; n = 10) |
Bulimia Nervosa (BN; n = 17) |
Binge Eating Disorder (BED; n = 32) |
Subthreshold BN (n = 41) |
Purging Disorder (n = 38) |
Subthreshold BED (n = 72) |
AN, BN, or BED (n = 40) |
Any Eating Disorder†
(n = 107) |
||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SE | SP | SE | SP | SE | SP | SE | SP | SE | SP | SE | SP | SE | SP | SE | SP | |
| none | 1.00 | 0.96 | 0.82 | 0.83 | 0.94 | 0.68 | 0.76 | 0.74 | 0.63 | 0.93 | 0.93 | 0.58 | 0.95 | 0.66 | 0.92 | 0.56 |
| 47 | 0.90 | 0.98 | 0.82 | 0.84 | 0.94 | 0.70 | 0.76 | 0.77 | 0.60 | 0.94 | 0.89 | 0.65 | 0.93 | 0.70 | 0.86 | 0.66 |
| 49 | 0.90 | 0.99 | 0.82 | 0.85 | 0.94 | 0.73 | 0.73 | 0.78 | 0.60 | 0.94 | 0.89 | 0.66 | 0.93 | 0.72 | 0.86 | 0.67 |
| 51 | 0.90 | 0.99 | 0.82 | 0.86 | 0.91 | 0.74 | 0.73 | 0.80 | 0.60 | 0.94 | 0.86 | 0.68 | 0.90 | 0.74 | 0.84 | 0.69 |
| 53 | 0.90 | 0.99 | 0.82 | 0.86 | 0.87 | 0.76 | 0.73 | 0.81 | 0.60 | 0.94 | 0.83 | 0.71 | 0.87 | 0.76 | 0.82 | 0.73 |
| 55 | 0.90 | 0.99 | 0.82 | 0.87 | 0.81 | 0.79 | 0.73 | 0.82 | 0.60 | 0.94 | 0.78 | 0.74 | 0.82 | 0.79 | 0.78 | 0.76 |
| 57 | 0.90 | 0.99 | 0.82 | 0.87 | 0.78 | 0.79 | 0.71 | 0.83 | 0.60 | 0.95 | 0.74 | 0.76 | 0.80 | 0.79 | 0.74 | 0.77 |
| 59 | 0.90 | 0.99 | 0.82 | 0.88 | 0.78 | 0.82 | 0.68 | 0.84 | 0.55 | 0.95 | 0.72 | 0.78 | 0.80 | 0.82 | 0.72 | 0.80 |
| 61 | 0.90 | 0.99 | 0.77 | 0.89 | 0.75 | 0.83 | 0.61 | 0.85 | 0.50 | 0.95 | 0.68 | 0.80 | 0.78 | 0.84 | 0.68 | 0.82 |
| 63 | 0.90 | 0.99 | 0.77 | 0.89 | 0.72 | 0.84 | 0.59 | 0.86 | 0.47 | 0.96 | 0.63 | 0.81 | 0.72 | 0.84 | 0.64 | 0.83 |
| 65 | 0.80 | 0.99 | 0.77 | 0.90 | 0.69 | 0.85 | 0.56 | 0.88 | 0.47 | 0.96 | 0.58 | 0.83 | 0.70 | 0.85 | 0.60 | 0.85 |
| 67 | 0.80 | 0.99 | 0.71 | 0.92 | 0.63 | 0.88 | 0.51 | 0.90 | 0.42 | 0.97 | 0.53 | 0.86 | 0.65 | 0.88 | 0.55 | 0.88 |
| 69 | 0.70 | 0.99 | 0.65 | 0.92 | 0.59 | 0.88 | 0.49 | 0.90 | 0.39 | 0.97 | 0.50 | 0.86 | 0.60 | 0.88 | 0.52 | 0.88 |
Notes: WCS = Weight Concerns Scale; n = number of cases diagnosed on the Eating Disorder Examination interview, without consideration of diagnostic trumping order; SE = sensitivity; SP = specificity.
The diagnostic performance of the screening instrument was compared to the Eating Disorder Examination interview using ROC analysis, and sensitivity and specificity estimates were derived by calculating the products of categorical diagnostic items and WCS scores at possible cut-off points on the WCS (i.e., none; 47 to 69).
Refers to AN, BN, BED, Subthreshold BN, Purging Disorder, or Subthreshold BED
However, use of the screen could depend on the goals of the student health service. For example, as shown in Table 1, if an organization adopting the screen was interested in identifying individuals with any diagnosis of anorexia nervosa, bulimia nervosa, or binge eating disorder (i.e., who screen positive for any of the three full-threshold diagnoses, versus identifying individuals who screen positive for each specific diagnosis), a WCS cut-point of 47 or 49 has higher sensitivity (sensitivity = 0.93) without excessively sacrificing specificity (specificity = 0.70 and 0.72, respectively) compared to a WCS cut-point of 59 (sensitivity = 0.80, specificity = 0.82).
Table 2 shows the prevalence estimates for screening positive for each eating disorder diagnosis, accounting for diagnostic trumping order, based on our sample of participants.
Table 2:
Prevalence of eating disorder diagnoses and high risk status detected by the screening instrument in a sample of women aged 18–25 who self-selected for participation
| Diagnosis | N EDE | N SWED | Prevalence from the SWED per 1,000 cases |
|---|---|---|---|
| Eating Disorder | |||
| Anorexia nervosa | 10 | 15 | 27 |
| Bulimia nervosa | 14 | 65 | 118 |
| Binge eating disorder | 16 | 42 | 77 |
| Subthreshold bulimia nervosa | 21 | 23 | 42 |
| Purging disorder | 24 | 8 | 15 |
| Subthreshold binge eating disorder | 22 | 13 | 24 |
| Anorexia nervosa, bulimia nervosa, or binge eating disorder† | 40 | 122 | 222 |
| Any eating disorder | 107 | 166 | 302 |
| High Risk (and No Eating Disorder) | |||
| High weight/shape concerns | 76 | 274 | 499 |
Note: Diagnoses account for diagnostic trumping order.
EDE = Eating Disorder Examination; SWED = Stanford Washington University Eating Disorder Screen.
Refers to full-syndrome eating disorder diagnoses
Detecting Purging Behavior based on University Preferences
Given university stakeholders’ preferences to receive a referral for individuals who engaged in purging, we evaluated the performance of the screen for detecting purging behavior. The number of women who reported purging behavior as a means to control weight or shape (i.e., vomiting, diuretic use, or laxative use) during the previous month was 60 on the screen and 46 on the EDE interview. Sensitivity and specificity of detecting purging behavior on the screen compared to the EDE interview were 0.63 and 0.94 respectively, and AUC = 0.79 (95% CI = 0.70 – 0.87). Of the 60 individuals who reported one or more purging behaviors on the screen, 22 individuals did not meet criteria for a full-syndrome eating disorder diagnosis (i.e., anorexia nervosa, bulimia nervosa, or binge eating disorder) based on the screening criteria. This finding suggests that stakeholder preferences to identify individuals who engage in purging (in addition to identifying individuals with a full-syndrome eating disorder diagnosis) does indeed help identify individuals who are not captured by the full-syndrome diagnostic criteria.
Detecting High Risk Status (Over-concern with Weight & Shape)
In this sample, there was a high correlation between scores on the WCS administered on the screen and again during the baseline assessment (r = 0.72; p < .001). Based on the screen, 274 participants (50%) had an over-concern with weight and shape and did not meet screen criteria for an eating disorder (see Table 2). In the exploratory analysis of the screen compared to the EDE for detecting high risk, among those who did not meet criteria for an eating disorder on the screen, there was a moderate correlation between WCS scores on the screen and “weight/shape concerns” average item scores on the EDE (r = 0.48; p < .001). Among those who did not meet criteria for an eating disorder on the screen, the screen had high sensitivity (0.96) and but poor specificity (0.32) for detecting individuals with scores ≥4 on the EDE “weight/shape concerns” average item (AUC = 0.64; 95% CI = 0.57–0.71).
Discussion
An optimal screen for an integrated prevention and treatment program will sort individuals into low-risk, high-risk (in need of selective preventive interventions), with early symptoms (in need of indicated preventive interventions), and with probable clinical diagnoses (in need of clinical evaluation and treatment services). The results of this study indicate the SWED screen is a useful tool through which to achieve these aims. Specifically, the screening instrument demonstrated reasonable sensitivity (0.80) and specificity (0.82) for identifying cases of combined DSM-5 anorexia nervosa, bulimia nervosa, or binge eating disorder, as well high sensitivity for detecting individuals at high risk for an eating disorder.
Other eating disorder screens exist. Compared to the Eating Disorder Diagnostic Scale screen, which identifies women with anorexia nervosa, bulimia nervosa, or binge eating disorder based on DSM-IV derived criteria, our screen results reveal lower sensitivity and specificity (0.80 and 0.82 SWED screen versus 0.88 and 0.98 Eating Disorder Diagnostic Scale, respectively).24,25,40 However, the Eating Disorder Diagnostic Scale sensitivity and specificity estimates were derived from a sample that included 34 cases of bulimia nervosa but only one case of anorexia nervosa and one case of binge eating disorder.24 The Eating Disorder Examination-Questionnaire can discriminate among eating disorder diagnoses and is widely used,21 but it is long for use as a screen and thus may be burdensome to patients and health care providers. Our screen also has the benefit of being able to administer questions adaptively (meaning that not every individual needs to complete every item) and it assesses for pathology beyond eating disorder symptoms like impairment and risk factors. The Patient Health Questionnaire–Eating Disorder Module (PHQ-ED) has demonstrated good sensitivity and acceptable specificity for identifying cases of binge eating disorder, bulimia nervosa, and sub-clinical recurrent binge eating.26 However, this screen was not used to identify cases of anorexia nervosa. Finally, the 5-item SCOFF is an easy-to-administer screener but only identifies individuals with possible anorexia nervosa or bulimia nervosa.41,42 In addition, to our knowledge, the SWED is the first screen to go beyond screening for the mere presence of an eating disorder and instead detects individuals at low-risk, high-risk, or with probable subthreshold or full-syndrome eating disorder categories.
The goal of creating this screening instrument was to develop an efficient tool that can direct individuals to appropriate interventions based on their risk and symptom profile. Towards the goal of efficiency, we selected a single WCS cut-point value for parsimony, because having one cut-point may make it easier for organizations to program and use the screen in the future. However, results showed that other cut-points offer a more reasonable balance between sensitivity and specificity for some of the other specified eating disorders. Accordingly, the use of the screen can depend on the goals of the organization (e.g., student health service) adopting it, and the ability to adapt the thresholds at which cases are identified is a strength of the tool. For example, organizations that wish to identify any and all individuals in need of services (e.g., for large-scale, population-based screening) may want to prioritize sensitivity over specificity to avoid missing cases, whereas organizations who need a screen for guiding the allocation of limited resources may prefer to limit the number of individuals incorrectly identified as symptomatic (i.e., false positives).
Including WCS scores as part of the screen criteria increased specificity for all of the diagnoses compared to the stage 1 results in which the WCS was omitted. While specificity of the screen is modest, improving specificity using current screening items would reduce sensitivity to unacceptable levels. Given the serious nature of eating disorders and their devastating consequences, prioritizing sensitivity over specificity is a desirable feature of the screen. Revision of the current SWED items may be needed before specificity can be improved. Indeed, one drawback of the current screen is that questions use binary responses, which limits the granularity of frequency data that can be collected for generating diagnoses. It is possible that re-wording survey items and response options to allow for more precise eating disorder behavior frequency data would provide an opportunity to use ROC analysis to optimize screen performance beyond the level achieved in the current study. However, when using the methods presented in this report, clinical diagnostic interview psychometrics represents a ceiling beyond which no screen can rise. Validity of the clinical interview may be compromised, for example, if the higher rate of eating disorder symptoms reported via the self-report screen is due to greater respondent discomfort with reporting potentially stigmatizing symptoms to a live person in a clinical interview. Alternative methods for assessing symptom criteria, other than the clinical interview, such as active and passive longitudinal daily symptom assessment methods (e.g., ecological momentary assessment43) may allow researchers to better gauge and compare performance of screening tools and clinical interview procedures. For now, findings from the current study show that the SWED screen provides modest but reasonable sensitivity and specificity for identifying eating disorder cases, when the gold standard comparison is a structured clinical interview.
Given that the use of the screen may lead to increased identification of students in need of intervention resources, an appropriate system of care should be in place where the screen is implemented. Using a stepped-care approach to allocate individuals to appropriate intensities of interventions has potential to reduce population morbidity and optimize distribution of limited treatment resources. Fortunately, previous efforts to establish interventions that reduce eating disorder risk and chronicity have resulted in the development of a suite of in-person and online interventions designed to reduce eating disorder risk, disordered eating behaviors, or clinical pathology among college students.44 We have been partnering with universities to implement an integrated prevention and treatment model, with early success.45–47
There are several limitations of this study. First, the psychometric properties of the screening instrument are particular to college-age women who are represented by the population of individuals who volunteered for this study (which recruited for women who were concerned about their bodies or weight). Thus, validation in additional samples is warranted to generalize across multiples ages, to men, and to college-age women in general. Another limitation is that the current version of the instrument does not screen for the full spectrum of feeding and eating disorders, such as atypical anorexia nervosa. Future work to expand the screening instrument to detect additional eating disorder diagnoses would enhance the clinical utility of the tool. Third, there was a gap between when individuals completed the screening instrument and the clinical interview, given the study procedure to schedule an in-person assessment following the screen. Symptoms may have changed in this time and therefore impacted case detection. Fourth, we used an average item score to identify individuals with “weight/shape concerns” on the EDE (based on past research35) because we did not administer the full interview. Therefore, we were unable to calculate the Weight Concerns and Shape Concerns subscales, which may have diminished our ability to identify individuals with “true” weight/shape concerns via the interview. However, we note that the WCS measure is widely used to assess for eating disorder risk among college-age women.19,30,31,45 Finally, it is important to consider that the screen may yield a large number of individuals with false positive results, who may require a subsequent clinical assessment. False positives may be preferable to false negatives when trying to detect eating disorders, given that these disorders are characterized by high chronicity if patients are left without intervention.48,49 However, the desire to capture more cases must be balanced with the capacity of university health services centers for conducting follow up assessments. Thus, organizations who are considering use of the screen might benefit from allocating resources to further evaluate those who screen positive for an eating disorder.
In conclusion, the SWED screening instrument is an easy to implement, scalable tool for identifying individuals with probable eating disorders as well as at high risk among college-age women. In this context, the screen is able to classify women by eating disorder diagnostic and risk category, which can be used to direct them to appropriate prevention or treatment intervention options. The screening instrument can be delivered online, making it low cost to implement among large populations of users. As such, the screen is ideally suited for implementation within a stepped-care model. Given that delivering preventive interventions in addition to treatment can increase costs to university health services centers, the cost-effectiveness of this approach can be enhanced through the use of online preventive interventions and online therapist guided self-help treatment programs.44,50,51 In sum, to our knowledge, this study is the first to develop and evaluate a comprehensive eating disorder screening measure to identify individuals at varied levels of risk or clinical status. Moreover, the development of the screen in partnership with university stakeholders is in line with national recommendations for dissemination and implementation research and practice.28 Results of this study are pivotal to achieving the large-scale goal of building a university stepped care model for universal and targeted prevention and treatment of eating disorders.
Supplementary Material
Acknowledgments
This work was supported by grants from the National Institutes of Health (R01 MH081125, T32 HL007456, T32 HL130357, K24 MH070446, F32 HD089586).
References
- 1.Eisenberg D, Nicklett EJ, Roeder K, Kirz NE. Eating disorder symptoms among college students: prevalence, persistence, correlates, and treatment-seeking. J Am Coll Health. 2011;59(8):700–707. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Volpe U, Tortorella A, Manchia M, Monteleone AM, Albert U, Monteleone P. Eating disorders: What age at onset? Psychiatry Res 2016;238:225–227. [DOI] [PubMed] [Google Scholar]
- 3.Stice E, Marti CN, Rohde P. Prevalence, incidence, impairment, and course of the proposed DSM-5 eating disorder diagnoses in an 8-year prospective community study of young women. J Abnorm Psychol. 2013;122(2):445–457. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.American College Health Association. American College Health Association-National College Health Assessment II: Reference Group Executive Summary Spring 2015. Hanover, MD: American College Health Association;2015. [Google Scholar]
- 5.Collaborators GBoDS. Global, regional, and national incidence, prevalence, and years lived with disability for 301 acute and chronic diseases and injuries in 188 countries, 1990–2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet. 2015;386(9995):743–800. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Keshaviah A, Edkins K, Hastings ER, et al. Re-examining premature mortality in anorexia nervosa: a meta-analysis redux. Compr Psychiatry. 2014;55(8):1773–1784. [DOI] [PubMed] [Google Scholar]
- 7.Eisenberg D, Golberstein E, Hunt JB. et al. The BE Journal of Economic Analysis & Policy. 2009;9(1):Article 40. [Google Scholar]
- 8.Agras WS. The consequences and costs of the eating disorders. Psychiatr Clin North Am. 2001;24(2):371–379. [DOI] [PubMed] [Google Scholar]
- 9.Hart LM, Granillo MT, Jorm AF, Paxton SJ. Unmet need for treatment in the eating disorders: a systematic review of eating disorder specific treatment seeking among community cases. Clin Psychol Rev 2011;31(5):727–735. [DOI] [PubMed] [Google Scholar]
- 10.Samnaliev M, Noh HL, Sonneville KR, Austin SB. The economic burden of eating disorders and related mental health comorbidities: An exploratory analysis using the U.S. Medical Expenditures Panel Survey. Prev Med Rep 2015;2:32–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Arts J, Fernandez ML, Lofgren IE. Coronary heart disease risk factors in college students. Adv Nutr. 2014;5(2):177–187. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Spring B, Moller AC, Colangelo LA, et al. Healthy lifestyle change and subclinical atherosclerosis in young adults: Coronary Artery Risk Development in Young Adults (CARDIA) study. Circulation. 2014;130(1):10–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Ashwood JS, Stein BD, Briscombe B, et al. Payoffs for California College Students and Taxpayers from Investing in Student Mental Health. Santa Monica, CA: Rand Corporation;2015. [PMC free article] [PubMed] [Google Scholar]
- 14.Striegel Weissman R, Rosselli F. Reducing the burden of suffering from eating disorders: Unmet treatment needs, cost of illness, and the quest for cost-effectiveness. Behav Res Ther. 2017;88:49–64. [DOI] [PubMed] [Google Scholar]
- 15.Jacobi C, Fittig E, Bryson SW, Wilfley D, Kraemer HC, Taylor CB. Who is really at risk? Identifying risk factors for subthreshold and full syndrome eating disorders in a high-risk sample. Psychol Med. 2011;41(9):1939–1949. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Jacobi C, Hayward C, de Zwaan M, Kraemer HC, Agras WS. Coming to terms with risk factors for eating disorders: application of risk terminology and suggestions for a general taxonomy. Psychol Bull. 2004;130(1):19–65. [DOI] [PubMed] [Google Scholar]
- 17.Austin SB. Accelerating Progress in Eating Disorders Prevention: A Call for Policy Translation Research and Training. Eat Disord. 2016;24(1):6–19. [DOI] [PubMed] [Google Scholar]
- 18.Frieden TR. A framework for public health action: the health impact pyramid. Am J Public Health. 2010;100(4):590–595. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Jacobi C, Abascal L, Taylor CB. Screening for eating disorders and high-risk behavior: caution. Int J Eat Disord. 2004;36(3):280–295. [DOI] [PubMed] [Google Scholar]
- 20.Killen JD, Taylor CB, Hayward C, et al. Weight concerns influence the development of eating disorders: a 4-year prospective study. J Consult Clin Psychol. 1996;64(5):936–940. [DOI] [PubMed] [Google Scholar]
- 21.Fairburn CG, Beglin SJ. Assessment of Eating Disorders: Interview or Self-Report Questionnaire? International Journal of Eating Disorders. 1994;16(4):363–370. [PubMed] [Google Scholar]
- 22.Mond JM, Hay PJ, Rodgers B, Owen C, Beumont PJ. Validity of the Eating Disorder Examination Questionnaire (EDE-Q) in screening for eating disorders in community samples. Behav Res Ther. 2004;42(5):551–567. [DOI] [PubMed] [Google Scholar]
- 23.Mond JM, Myers TC, Crosby RD, et al. Screening for eating disorders in primary care: EDE-Q versus SCOFF. Behav Res Ther. 2008;46(5):612–622. [DOI] [PubMed] [Google Scholar]
- 24.Stice E, Fisher M, Martinez E. Eating disorder diagnostic scale: additional evidence of reliability and validity. Psychol Assess. 2004;16(1):60–71. [DOI] [PubMed] [Google Scholar]
- 25.Stice E, Telch CF, Rizvi SL. Development and validation of the Eating Disorder Diagnostic Scale: a brief self-report measure of anorexia, bulimia, and binge-eating disorder. Psychol Assess. 2000;12(2):123–131. [DOI] [PubMed] [Google Scholar]
- 26.Striegel-Moore RH, Perrin N, DeBar L, Wilson GT, Rosselli F, Kraemer HC. Screening for binge eating disorders using the Patient Health Questionnaire in a community sample. Int J Eat Disord. 2010;43(4):337–343. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Insel TR. Translating Scientific Opportunity Into Public Health Impact: A Strategic Plan for Research on Mental Illness. Arch Gen Psychiatry. 2009;66(2):128–133. [DOI] [PubMed] [Google Scholar]
- 28.Glasgow RE, Vinson C, Chambers D, Khoury MJ, Kaplan RM, Hunter C. National Institutes of Health approaches to dissemination and implementation science: current and future directions. Am J Public Health. 2012;102(7):1274–1281. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.National Institute of Mental Health. Strategic Research Priorities Overview. 2017; https://www.nimh.nih.gov/about/strategic-planning-reports/strategic-research-priorities/index.shtml.
- 30.Kass AE, Trockel M, Safer DL, et al. Internet-based preventive intervention for reducing eating disorder risk: A randomized controlled trial comparing guided with unguided self-help. Behav Res Ther. 2014;63:90–98. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Taylor CB, Kass AE, Trockel M, et al. Reducing eating disorder onset in a very high risk sample with significant comorbid depression: A randomized controlled trial. J Consult Clin Psychol. 2016;84(5):402–414. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Cooper Z, Fairburn CG. The eating disorder examination: A semi-structured interview for the assessment of the specific psychopathology of eating disorders. International Journal of Eating Disorders. 1987;6(1):1–8. [Google Scholar]
- 33.Fairburn CG, Cooper Z. The Eating Disorder Examination (twelfth edition). In: Fairburn CG, Wilson GT, eds. Binge Eating: Nature, Assessment and Treatment. New York: Guildford Press; 1993:317–360. [Google Scholar]
- 34.Killen JD, Taylor CB, Hayward C, et al. Pursuit of thinness and onset of eating disorder symptoms in a community sample of adolescent girls: a three-year prospective analysis. Int J Eat Disord. 1994;16(3):227–238. [DOI] [PubMed] [Google Scholar]
- 35.Fitzsimmons-Craft EE, Eichen DM, Kass AE, et al. Reciprocal longitudinal relations between weight/shape concern and comorbid oathology among women at very high risk for eating disorder onset. Eating and Weight Disorders. In press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Fairburn CG, Cooper Z, O’Connor ME. Eating Disorder Examinatin (Edition 16.0D). In: Fairburn CG, ed. Cognitive Behavior Therapy and Eating Disorders. New York: Guilford Press; 2008:265–308. [Google Scholar]
- 37.American Psychiatric Association. Diagnostic and statistical manual for mental disorders. 5th ed Washington, DC: American Psychiatric Association; 2013. [Google Scholar]
- 38.McFall RM, Treat TA. Quantifying the information value of clinical assessments with signal detection theory. Annu Rev Psychol. 1999;50:215–241. [DOI] [PubMed] [Google Scholar]
- 39.Streiner DL, Cairney J. What’s under the ROC? An introduction to receiver operating characteristics curves. Can J Psychiatry. 2007;52(2):121–128. [DOI] [PubMed] [Google Scholar]
- 40.Krabbenborg MA, Danner UN, Larsen JK, et al. The Eating Disorder Diagnostic Scale: psychometric features within a clinical population and a cut-off point to differentiate clinical patients from healthy controls. Eur Eat Disord Rev. 2012;20(4):315–320. [DOI] [PubMed] [Google Scholar]
- 41.Morgan JF, Reid F, Lacey JH. The SCOFF questionnaire: assessment of a new screening tool for eating disorders. BMJ. 1999;319(7223):1467–1468. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Luck AJ, Morgan JF, Reid F, et al. The SCOFF questionnaire and clinical interview for eating disorders in general practice: comparative study. BMJ. 2002;325(7367):755–756. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Engel SG, Crosby RD, Thomas G, et al. Ecological Momentary Assessment in Eating Disorder and Obesity Research: a Review of the Recent Literature. Curr Psychiatry Rep. 2016;18(4):37. [DOI] [PubMed] [Google Scholar]
- 44.Wilfley DE, Agras WS, Taylor CB. Reducing the burden of eating disorders: a model for population-based prevention and treatment for university and college campuses. Int J Eat Disord. 2013;46(5):529–532. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Jones M, Kass AE, Trockel M, Glass AI, Wilfley DE, Taylor CB. A population-wide screening and tailored intervention platform for eating disorders on college campuses: the healthy body image program. J Am Coll Health. 2014;62(5):351–356. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Fitzsimmons-Craft EE, Balantekin KN, Eichen DM, et al. Screening and offering online programs for eating disorders: Reach, pathology, and differences across risk groups at 28 U.S. colleges. Under review. [DOI] [PMC free article] [PubMed]
- 47.Fitzsimmons-Craft EE, Firebaugh M, Kass AE, et al. Implementing an Internet-Based Platform for Eating Disorder Screening, Prevention, and Treatment on College Campuses: Preliminary Results from a State-Wide Initiative. Psychological Services. In press. [Google Scholar]
- 48.Wonderlich S, Mitchell JE, Crosby RD, et al. Minimizing and treating chronicity in the eating disorders: a clinical overview. Int J Eat Disord. 2012;45(4):467–475. [DOI] [PubMed] [Google Scholar]
- 49.Eddy KT, Tabri N, Thomas JJ, et al. Recovery From Anorexia Nervosa and Bulimia Nervosa at 22-Year Follow-Up. J Clin Psychiatry. 2017;78(2):184–189. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Kass AE, Balantekin KN, Fitzsimmons-Craft EE, Jacobi C, Wilfley DE, Taylor CB. The economic case for digital interventions for eating disorders among United States college students. Int J Eat Disord. 2017;50(3):250–258. doi: 10.1002/eat.22680. Epub 22017 Feb 22682. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Agras WS, Fitzsimmons-Craft EE, Wilfley DE. Evolution of cognitive-behavioral therapy for eating disorders. Behav Res Ther. 2017;88:26–36. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
