Four Simple Questions Can Help Screen for Eating Disorders

Abstract

Current screening instruments for eating disorders are cumbersome to administer and have not been validated in primary care populations. We compared the performance characteristics of 2 screening tools, the SCOFF clinical prediction guide, and a new set of questions, the Eating disorder Screen for Primary care (ESP), using the Questionnaire for Eating Disorders Diagnosis as the independent standard, in 104 consecutive patients from a primary care practice and 129 university students. Twelve percent of the combined population had an eating disorder. One or no abnormal responses to the ESP ruled out an eating disorder (likelihood ratio [LR] 0.0), whereas 3 or more abnormal responses ruled one in (LR 11). The SCOFF questions were less sensitive than predicted (1 or no abnormal responses, LR 0.25), but were as effective at ruling in an eating disorder (3 or more abnormal responses, LR 11).

Primary care patients are often not diagnosed with eating disorders,^1,2 possibly because they present with apparently unrelated physical³ and psychiatric complaints.⁴ A missed diagnosis can have serious implications; for example, late presentation in bulimia nervosa has been associated with a worse outcome.⁵

Some assessment tools^6–8 have been developed to help improve screening for eating disorders but have not been properly validated in primary care populations. In addition, they are cumbersome to use in practice settings because they consist of multiple questions (87 in 1 case⁸) and require careful analysis. Other potentially useful screening questions have been identified in different populations including university college students,⁹ women attending self-help groups for bulimia,¹⁰ and servicemen in the U.S. navy.¹¹ However, these studies either assessed patients already known to have an eating disorder¹⁰ or used an inappropriate nonvalidated, nonindependent reference standard,^9,11 and so have uncertain screening potential.

The aims of the present study were:

1. To validate the SCOFF clinical prediction rule questions¹² in primary care patients and a higher-risk population consisting of university students.¹³

2. To assess the screening potential of 5 potentially useful screening questions derived from other studies^9–11 (named the Eating disorder Screen for Primary care [ESP]).

3. To compare the 2 instruments.

METHODS

The local research ethics committee approved the study design and all participants gave written informed consent. Participants were selected from students at a large multifaculty college in the University of London. The students were recruited using methods similar to a student health campaign, i.e., poster campaigns and announcements during lectures. Subjects received no payment for participation. A second group of participants included patients attending a primary care clinic. Consecutive patients were approached on entering the waiting area, and the study was conducted immediately. The exclusion criteria were: 1) age under 18 or over 65 years; 2) unable to read English or give valid consent; 3) known chronic physical illness that might result in a body mass index <20.1.

Participants first completed the self-administered Questionnaire for Eating Disorder Diagnoses (Q-EDD), which was immediately placed in an opaque closed box. This was used as the reference standard, because it has been shown to be both valid and reliable in operationalizing The Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) criteria into a self-report format.¹⁴ Compared to the use of DSM-IV criteria in an interview format, the Q-EDD has been shown to have a sensitivity of 97% and a specificity of 98%.¹⁴

A psychiatrist blind to the results of the Q-EDD immediately interviewed each subject using the ESP, followed by the SCOFF questionnaire. Patients were asked the following questions:

ESP

• Are you satisfied with your eating patterns? (A “no” to this question was classified as an abnormal response).

• Do you ever eat in secret? (A “yes” to this and all other questions was classified as an abnormal response).

• Does your weight affect the way you feel about yourself?

• Have any members of your family suffered with an eating disorder?

• Do you currently suffer with or have you ever suffered in the past with an eating disorder?

SCOFF¹²

• Do you make yourself Sick because you feel uncomfortably full?

• Do you worry you have lost Control over how much you eat?

• Have you recently lost more than One stone (14 lb or 7.7 kg) in a three month period?

• Do you believe yourself to be Fat when others say you are thin?

• Would you say that Food dominates your life?

The SCOFF and ESP results and the Q-EDD were subsequently analyzed separately by 2 independent blinded clinicians.

Statistical Analysis

Each component of the SCOFF and the ESP was compared to the Q-EDD using sensitivity, specificity and likelihood ratios. A multilevel analysis was performed based on the number of abnormal responses for the ESP and SCOFF questions.

There were no conflicts of interest. Authors had full access to all the data in the study and accept full responsibility for the integrity of the data and accuracy of the data analysis.

RESULTS

One hundred twenty-nine students were recruited from the University of London between February and April 2001. None was excluded. One hundred four attendees at a large primary care practice in North London were recruited consecutively between April and May 2001. Eight were excluded: 2 patients were in remission from cancer, 2 had Crohn's disease, 3 were aged under 18 years and 1 was older than 65. The demographic information for each group and for the combined groups, and the distribution of the eating disorders identified using the Q-EDD reference standard can be found in Table 1.

Table 1.

Demographics and the Distribution of Eating Disorders in the 2 Populations Studied

	Combined (N = 233)	Student (N = 129)	Primary Care (N = 104)
Number excluded	8	0	8
Number analyzed	225	129	96
Mean age, y (range)	29 (18 to 64)	22 (18 to 44)	37 (22 to 64)
Female, %	77	77	77
Mean BMI (range)	22 (16 to 45)	22 (16 to 34)	23 (17 to 45)
With an eating disorder, % (95% CI)	12 (7.8 to 16)	12 (6.7 to 18)	12 (5.1 to 18)
Anorexia nervosa, n (%)	6 (22)	4 (25)	2 (18)
Bulimia, n (%)	11 (41)	7 (46)	4 (36)
Binge eating disorder, n (%)	9 (33)	5 (31)	4 (36)
Non-binging bulimia, n (%)	1 (4)	0 (0)	1 (9)

BMI, body mass index.

Results from both populations were combined, since there was no significant difference in the likelihood ratios, sensitivities, specificities, or prevalence of eating disorders between the 2 groups. Two students did not complete the SCOFF questions and were omitted from this analysis.

Positive likelihood ratio (LR+) and negative (LR−) likelihood ratios for the individual ESP and SCOFF questions are shown for the questions best able to diagnose or exclude an eating disorder. The best individual questions for ruling in an eating disorder were found to be:

• Do you worry that you have lost control over how much you eat? (LR+, 7.0; 95% confidence interval [CI], 4.3 to 11)

• Do you make yourself sick when you feel uncomfortably full? (LR+, 6.3; 95% CI, 2.3 to 17)

• Do you currently suffer with or have you ever suffered in the past with an eating disorder? (LR+, 6.1; 95% CI, 4.0 to 9.5)

• Do you ever eat in secret? (LR+, 6.0; 95% CI, 3.4 to 11).

The best individual questions for ruling out an eating disorder were:

• Does your weight affect the way you feel about yourself? (LR−, 0.0; 95% CI, 0.0 to 0.34)

• Are you satisfied with your eating patterns? (LR−, 0.048; 95% CI, 0.0070 to 0.33).

The question on family history made no difference to the screening potential of the ESP, so this was dropped from the subsequent analysis (see Table 2).

Table 2.

Multilevel Analysis: The Probability of an Eating Disorder Based on the Number of Abnormal Responses for ESP and SCOFF^*

Number of Abnormal Responses	Eating Disorder	No Eating Disorder	LR (95% CI)	Post-test Probability, %
ESP: 3 or 4	22	15	11 (6.4 to 18)	59
2	5	43	0.85 (0.38 to 2.0)	10
0 or 1	0	140	0.0 (0.0 to 0.15)	0
Total	27	198
SCOFF: 4 or 5	3	2	11 (1.9 to 62)	60
2 or 3	18	21	6.2 (3.8 to 10)	46
0 or 1	6	173	0.25 (0.12 to 0.51)	3
Total	27	196

^*The question about family history was omitted.

LR, likelihood ratio; CI, confidence interval.

Validation and Comparison of the Two Sets of Questions

Single Level Cutoff Analysis.

For the SCOFF questions, a cutoff of 2 or more abnormal responses gave a sensitivity of 78% (95% CI, 62% to 93%) and a specificity of 88% (95% CI, 84% to 93%). The corresponding positive LR was 6.6 (95% CI, 4.3 to 10) and the negative LR was 0.25 (95% CI, 0.12 to 0.51).

A cutoff of 2 or more abnormal responses to the ESP questions maximized sensitivity at 100% (95% CI, 90% to 100%) with a corresponding specificity of 71% (95% CI, 64% to 77%). The associated likelihood ratios were LR+, 3.4 (95% CI, 2.8 to 4.2), and LR−, 0.0 (95% CI, 0.0 to 0.15).

Multilevel Analysis

Multilevel analysis was performed by separating out the questions (Table 2). With 1 or no abnormal responses, the ESP questions provided a positive likelihood ratio of 0.0 (95% CI, 0.0 to 0.15) and the SCOFF questions gave a positive likelihood ratio of 0.25 (95% CI, 0.12 to 0.51). Three or more abnormal ESP responses gave a positive likelihood ratio of 11 (95% CI, 6.4 to 18). Likewise, 4 or more abnormal SCOFF responses gave a positive likelihood ratio of 11 (95% CI, 1.9 to 62).

DISCUSSION

This study compared the performance characteristics in a primary care population of 2 short screening instruments for eating disorders, using an independent blinded reference standard. Our results suggest that in a primary care or university student population, the SCOFF is less helpful than previously reported. The SCOFF derivation study¹² found that a cutoff of 2 or more abnormal responses maximized sensitivity at 100% (95% CI, 96.9% to 100%) with a corresponding specificity of 87.5% (95% CI, 79.2% to 93.4%), giving a positive likelihood ratio of 8.0 (95% CI, 4.7 to 15) and a negative LR of 0.0 (95% CI, 0.0 to 0.04). However, in our study, a cutoff of 2 or more abnormal responses gave a sensitivity of 78% (95% CI, 62% to 93%) and a specificity of 88% (95% CI, 84% to 93%). We concluded that since the derivation study¹² compared 2 extremely different groups and excluded patients with lost results and indeterminate diagnoses, this led to an overestimation of the test characteristics.

A cutoff of 2 or more abnormal responses to the ESP questions maximized sensitivity at 100% (95% CI, 90% to 100%) with a corresponding specificity of 71% (95% CI, 64% to 77%). The associated likelihood ratios were LR+, 3.4 (95% CI, 2.8 to 4.2), and LR−, 0.0 (95% CI, 0.0 to 0.15). At this cutoff, the ESP questions had a higher sensitivity (100%) than the SCOFF (78%), although the SCOFF questions gave a higher specificity (88%) than the ESP (71%).

This study has a number of limitations. First, the university student population was not recruited consecutively, although we attempted to mimic a health promotion campaign, which is one method of reaching students at risk of eating disorders. The likely effect of recruiting volunteers in this manner would have been to attract students with experience or concern about eating disorders, which matches the type of patient who might attend student health services with this problem. Not surprising then, the prevalence and test characteristics were similar in both student and primary care populations.

Second, there was a relatively high prevalence of eating disorders in both populations as compared with that found in other studies.¹⁵ One possible explanation is that a new category ‘eating disorders not otherwise specified’ was included. The reference standard chosen may also have had an effect on overestimating the prevalence of eating disorders. As with many psychiatric diagnostic studies, the reference standard is based on subjective criteria, increasing the possibility of bias. Only long-term follow-up and detailed interviewing could offer a better standard, but this was beyond the means of this study. Alternatively, our results may indeed have reflected a true rise in the prevalence of eating disorders. Nonetheless, this increase does not affect the validity of the results. Applying ESP or SCOFF in populations with a lower prevalence of eating disorders will enhance the screening power. For a population with a prevalence of 5%, a SCOFF score of 1 or fewer abnormal responses would indicate a post-test probability of around 1%.

Third, the study was too small to discriminate between individual question levels in the multilevel analysis, so these were combined. Even when the results from 2 questions were merged, confidence intervals were still not always narrow enough to distinguish clearly between different levels. Ideally, the ESP and the SCOFF should be validated in larger populations including other high-risk groups, such as models, dance students,^16,17 or teenage girls attending single-sex private schools.¹⁸ In addition, a multivariate analysis to adjust for confounding from related diagnostic elements could have helped to refine the 2 diagnostic tools further.

In conclusion, the SCOFF questions were found to be less sensitive than noted in the derivation study, and were unable safely to exclude a diagnosis of an eating disorder in our population. Multilevel analysis demonstrated that the ESP questions were not clearly different from the SCOFF questions at ruling in an eating disorder, but were better at ruling one out.

We propose that the ESP questions can be used to screen for eating disorders in primary care patients and university students, and to help decide whether more detailed assessment of a possible eating disorder is required.