Diagnostic accuracy of case-finding questions to identify perinatal depression

Abstract

Background:

Guidelines for perinatal mental health care recommend the use of two case-finding questions about depressed feelings and loss of interest in activities, despite the absence of validation studies in this context. We examined the diagnostic accuracy of these questions and of a third question about the need for help asked of women receiving perinatal care.

Methods:

We evaluated self-reported responses to two case-finding questions against an interviewer-assessed diagnostic standard (DSM-IV criteria for major depressive disorder) among 152 women receiving antenatal care at 26–28 weeks’ gestation and postnatal care at 5–13 weeks after delivery. Among women who answered “yes” to either question, we assessed the usefulness of asking a third question about the need for help. We calculated sensitivity, specificity and likelihood ratios for the two case-finding questions and for the added question about the need for help.

Results:

Antenatally, the two case-finding questions had a sensitivity of 100% (95% confidence interval [CI] 77%–100%), a specificity of 68% (95% CI 58%–76%), a positive likelihood ratio of 3.03 (95% CI 2.28–4.02) and a negative likelihood ratio of 0.041 (95% CI 0.003–0.63) in identifying perinatal depression. Postnatal results were similar. Among the women who screened positive antenatally, the additional question about the need for help had a sensitivity of 58% (95% CI 38%–76%), a specificity of 91% (95% CI 78%–97%), a positive likelihood ratio of 6.86 (95% CI 2.16–21.7) and a negative likelihood ratio of 0.45 (95% CI 0.25–0.80), with lower sensitivity and higher specificity postnatally.

Interpretation:

Negative responses to both of the case-finding questions showed acceptable accuracy for ruling out perinatal depression. For positive responses, the use of a third question about the need for help improved specificity and the ability to rule in depression.

The occurrence of depressive symptoms during the perinatal period is well-recognized. The estimated prevalence is 7.4%–20% antenatally^1,2 and up to 19.2% in the first three postnatal months.³ Antenatal depression is associated with malnutrition, substance and alcohol abuse, poor self-reported health, poor use of antenatal care services and adverse neonatal outcomes.⁴ Postnatal depression has a substantial impact on the mother and her partner, the family, mother–baby interaction and on the longer-term emotional and cognitive development of the baby.⁵

Screening strategies to identify perinatal depression have been advocated, and specific questionnaires for use in the perinatal period, such as the Edinburgh Postnatal Depression Scale,⁶ were developed. However, in their current recommendations, the UK National Screening Committee⁷ and the US Committee on Obstetric Practice⁸ state that there is insufficient evidence to support the implementation of universal perinatal screening programs. The initial decision in 2001 by the National Screening Committee to not support universal perinatal screening⁹ attracted particular controversy in the United Kingdom; some service providers subsequently withdrew resources for treatment of postnatal depression, and subsequent pressure by perinatal community practitioners led to modification of the screening guidance in order to clarify the role of screening questionnaires in the assessment of perinatal depression.¹⁰

In 2007, the National Institute for Health and Clinical Excellence issued clinical guidelines for perinatal mental health care in the UK, which included guidance on the use of questionnaires to identify antenatal and postnatal depression.¹¹ In this guidance, a case-finding approach to identify perinatal depression was strongly recommended; it involved the use of two case-finding questions (sometimes referred to as the Whooley questions), and an additional question about the need for help asked of women who answered “yes” to either of the initial questions (Box 1).

Box 1: Case-finding questions recommended for the identification of perinatal depression¹⁰.

• “During the past month, have you often been bothered by feeling down, depressed or hopeless?”

• “During the past month, have you often been bothered by having little interest or pleasure in doing things?”

• A third question should be considered if the woman answers “yes” to either of the initial screening questions: “Is this something you feel you need or want help with?”

Useful case-finding questions should be both sensitive and specific so they accurately identify those with and without the condition. The two case-finding questions have been validated in primary care samples^12,13 and examined in other clinical populations^14–16 and are endorsed in recommendations by US and Canadian bodies for screening depression in adults.^17,18 However, at the time the guidance from the National Institute for Health and Clinical Excellence was issued, there were no validation studies conducted in perinatal populations. A recent systematic review¹⁹ identified one study conducted in the United States that validated the two questions against established diagnostic criteria in 506 women attending well-child visits postnatally;²⁰ sensitivity and specificity of the questions were 100% and 44% respectively at four weeks. The review failed to identify studies that validated the two questions and the additional question about the need for help against a gold-standard measure.

We conducted a validation study to assess the diagnostic accuracy of this brief case-finding approach against gold-standard psychiatric diagnostic criteria for depression in a population of women receiving perinatal care.

Methods

Participants and setting

We recruited participants during a seven-week period from September 2010 to November 2010 in a maternity unit in a UK National Health Service general hospital, where more than 90% of the women in the local area receive their antenatal care. We approached women attending the clinic at about 26–28 weeks’ gestation for a routine appointment who were also recruited to a large population cohort study (the Born in Bradford study).²¹ Women were excluded from our validation study if they were not participating in the Born in Bradford study, did not speak English, were not literate, were planning to move from the Bradford area and were less than 18 years old at delivery.

Screening measures

Index test

The two brief case-finding questions recommended in the UK perinatal clinical guidelines¹¹ were included in a self-administered questionnaire in written format. Participants were required to indicate “yes” or “no” in response to each question. A “yes” response to either question was considered a positive screen.

To assess the usefulness of the addition of a third question about the need for help, we considered a “yes” response to either case-finding question plus a “yes” response to the third question to be a positive screen.

Diagnostic gold standard

To confirm the presence or absence of a current depressive episode, the DSM-IV (Diagnostic and Statistical Manual of Mental Disorders, fourth edition) diagnostic criteria for major depressive disorder were administered in an interview by telephone.²² Guidance for the administration and interpretation of the criteria was taken from the Structured Clinical Interview for DSM-IV-Clinical Version.²³ Semi-structured questions to identify depressive symptoms were asked in a format suitable for verbal interview²⁴ and have previously been shown to be valid when used over the telephone or face to face.^25,26 The interview questions and DSM-IV criteria used in our study are available in Appendix 1 (www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.111213/-/DC1).

Participants who did not meet the criteria for major depressive disorder but who had either depressed mood or anhedonia and met one other criterion for major depressive disorder were considered to have minor depression.

Procedure

The study was conducted in two phases in order to validate the utility of the questions in the antenatal and postnatal periods, as recommended by National Institute for Health and Clinical Excellence.¹¹ The study is reported according to the STARD (Standards for the Reporting of Diagnostic accuracy studies) statement.²⁷

During the antenatal phase, participants were recruited sequentially while attending a routine antenatal appointment at about 26–28 weeks’ gestation (mean 26.7 weeks, standard deviation 1.7). Participants completed the self-administered questionnaire in private; a researcher (R.M.) was available to answer questions if necessary.

During the postnatal phase, a copy of the questionnaire with a covering letter was mailed to the participants at about five to six weeks postnatally. The women were asked to return the completed questionnaire within seven days as well as a reply form to indicate a time and date when they would be available for the diagnostic interview by telephone. If necessary, nonrespondents were mailed another copy of the questionnaire and covering letter, sent up to two reminder letters and contacted once by telephone.

Within 14 days after completion of each of the case-finding questionnaires, a diagnostic interview was conducted by telephone by one of us (R.M.). The researcher had previous clinical and research experience with the administration of diagnostic interviews. To prevent review bias, she was unaware of the participant’s responses to the case-finding questionnaire before the interview.

Statistical analysis

The calculation of the sample size was based on the method developed by Flahault and colleagues.²⁸ For an expected sensitivity of 95% with 0.95 probability that the minimum acceptable lower 95% confidence interval (CI) limit would not fall below 70%, and where the prevalence of antenatal and postnatal depression was estimated to be up to 20%, we determined that a sample size of 120 was required.

We calculated the sensitivity, specificity, and positive and negative likelihood ratios, and the associated 95% CIs, using the online calculator available on the University of Toronto’s website for the Centre for Evidence-Based Medicine.²⁹ Sensitivity, specificity and likelihood ratios were interpreted according to established recommendations.³⁰ If a 2 × 2 cell contained zero, the value of 0.5 was added in order to calculate likelihood ratios and associated 95% CIs.³¹ Statistical significance was interpreted at the 5% level. Additional analyses were conducted with the use of PASW Statistics 18 (formerly SPSS Statistics).

Results

Figure 1 displays the selection and flow of participants through the study. The 152 eligible participants and the 106 women who were eligible but who declined to participate did not differ significantly in terms of age (p = 0.2) or ethnic background (p > 0.9).

Figure 1:

Flow of patients through the study.

Antenatal phase

A total of 152 women completed the case-finding questionnaire during their antenatal visit. The sample was diverse, with a varied range of ethnic backgrounds and levels of education (Table 1). Of these women, 126 (82.9%) completed the telephone interview to confirm the presence or absence of depression.²⁷ These participants and the 26 women who could not be contacted for the interview did not differ significantly in terms of age (p = 0.1) or ethnic background (p = 0.46).

Table 1:

Characteristics of study participants

Characteristic	No. (%) of patients* n = 152
Age, yr, mean (SD)	27.4 (5.8)
Ethnic background
White British	81 (53.3)
White other	5 (3.3)
Mixed (white and black)	4 (2.6)
Mixed (white and South Asian)	3 (2.0)
Black	6 (3.9)
Indian	5 (3.3)
Pakistani	38 (25.0)
Bangladeshi	5 (3.3)
Other	5 (3.3)
Primiparous	73 (48.0)
Marital status
Married, first marriage	79 (52.0)
Remarried	10 (6.6)
Single, never married	60 (39.9)
Divorced	3 (2.0)
Cohabitation
Lives with baby’s father	122 (80.3)
Lives with another partner	1 (0.7)
In a relationship, not living with partner	20 (13.2)
Not in a relationship	9 (5.9)
Housing
Owns house, paying mortgage/loan	68 (44.7)
Owns house outright	13 (8.6)
Rents	54 (35.5)
Lives rent free	17 (11.2)
Highest level of education
None	22 (14.5)
High school	35 (23.0)
College or university	31 (20.4)
Postgraduate school	49 (32.2)
Other	12 (7.9)
No response	3 (2.0)
Employment status
Currently employed	94 (61.8)
Previously employed	34 (22.4)
Never employed	24 (15.8)
Smoking status
Current smoker	22 (14.5)
Former smoker	42 (27.6)
Never smoked	88 (57.9)
Self-reported history of diagnosed depression
No prior diagnosis by general practitioner	128 (84.2)
≥ 1 diagnosed episode of depression	24 (15.8)

Note: SD = standard deviation.

^*Unless stated otherwise.

The proportion of women who met the criteria for depression (minor and major) during the antenatal phase was 13.5% (95% CI 8.3%–21%). The sensitivity, specificity and likelihood ratios of the two case-finding questions and the additional question about the need for help are reported in Table 2. A positive response to either of the case-finding questions (positive screen) showed acceptable sensitivity (100%); all of the women with depression were classified correctly, with no false-negative results. The negative likelihood ratio of 0.041 indicated acceptable accuracy to rule out the presence of depression given a negative response to both questions.

Table 2:

Performance of two case-finding questions (index test) and an additional question about the need for help in identifying perinatal depression

Measure	Antenatal phase		Postnatal phase
	Case-finding questions* n = 126†	Question about need for help‡ n = 52	Case-finding questions* n = 94†	Question about need for help‡ n = 45
True positive, no.	17	10	18	7
False negative, no.	0	7	0	11
True negative, no.	74	32	49	27
False positive, no.	35	3	27	0
Sensitivity, % (95% CI)	100 (77–100)	58 (38–76)	100 (78–100)	39 (18–64)
Specificity, % (95% CI)	68 (58–76)	91 (78–97)	65 (53–75)	100 (85–100)
Positive likelihood ratio (95% CI)	3.03 (2.28–4.02)	6.86 (2.16–21.7)	2.73 (2.0–3.74)	21.4 (1.3–354.2)
Negative likelihood ratio (95% CI)	0.041 (0.003–0.63)	0.45 (0.25–0.80)	0.042 (0.003–0.65)	0.62 (0.43–0.90)

Note: CI = confidence interval.

^*There were no missing responses to the case-finding questions. A positive response to either of the two questions was considered a positive screen.

^†This is the number of women who also completed the telephone interview. A total of 152 women (antenatal phase) and 97 women (postnatal phase) answered the case-finding questions.

^‡Responses to an additional question about the need for help were elicited only from participants with a positive response to either of the case-finding questions.

Among the women with a positive screen, use of the additional question about the need for help improved specificity to 91%. The positive likelihood ratio of 6.86 showed that the additional question was moderately good at ruling in antenatal depression.

Postnatal phase

Of the 152 women eligible for postnatal follow-up contact, 4 were excluded because of a breach of study protocol (Figure 1). Of the remaining 148 women to whom the questionnaire was mailed, 2 withdrew consent and 97 returned a completed questionnaire within 13 weeks of delivery (response rate 66%). A total of 94 (97%) completed the telephone interview.

The mean number of weeks from delivery to participation in the postnatal diagnostic interview was seven weeks and four days (standard deviation [SD] 1.7 weeks). Compared with the 94 women who participated in the postnatal phase of the study, the 52 women who did not respond to postnatal contact were younger, less educated and less likely to be employed (Table 3).

Table 3:

Characteristics of the 94 women who participated in the postnatal phase of the study and the 52 women who did not

Characteristic	Participation in postnatal phase; no. (%) of women*		p value§
	Participated† n = 94	Did not participate‡ n = 52
Age, yr, mean (SD)	28.5 (6.2)	25.4 (4.4)	0.002
White	58 (62)	25 (48)	0.1
Primiparous	48 (51)	21 (40)	0.2
Married	60 (64)	25 (48)	0.06
Living with baby’s father or another partner	81 (86)	38 (73)	0.05
House owner	47 (50)	18 (35)	0.07
Attended school (any level)	76 (81)	33 (64)	0.02
Currently employed	65 (69)	25 (48)	0.01
Nonsmoker during pregnancy¶	84 (89)	43 (83)	0.3
Self-reported history of ≥ 1 diagnosed episodes of depression	16 (17)	6 (12)	0.4

Note: SD = standard deviation.

^*Unless stated otherwise.

^†Completed the questionnaire in both the antenatal and postnatal phases and completed the telephone interview.

^‡Completed the questionnaire in the antenatal phase only (49 did not respond to any postnatal contact and 3 did not complete the postnatal telephone interview).

^§Values were compared by means of the t test and χ² test.

^¶Includes former smokers and those who never smoked.

The proportion of participants who met the criteria for depression (minor and major) during the postnatal phase was 19.2% (95% CI 12%–28.9%). The sensitivity, specificity and likelihood ratios of the two case-finding questions and the additional question about the need for help are reported in Table 2. The sensitivity of a positive response to either case-finding question was 100%; all of the women with postnatal depression were classified correctly. The negative likelihood ratio of 0.042 showed acceptable accuracy to rule out the presence of postnatal depression given a negative response to both questions.

Among the participants with a positive screen, use of the additional question about the need for help improved specificity to 100%. The positive likelihood ratio of 21.4 showed that a positive response to the additional question was acceptable at ruling in postnatal depression.

Interpretation

We found that the two case-finding questions endorsed by the National Institute for Health and Clinical Excellence offered a brief, simple and precise approach for identifying perinatal depression. Negative responses to both questions showed acceptable accuracy for ruling out perinatal depression. For positive responses, the use of an additional question about the need for help improved specificity and the ability to rule in perinatal depression.

This approach has important implications for clinical practice. Use of the case-finding questions should be considered in the context of a triage test, rather than as a replacement test to existing methods of assessment.³² Triage tests are simple and noninvasive, have no wait time and do not aim to improve the diagnostic pathway; instead, they reduce the number of patients who need further assessment. The benefit of using the brief case-finding approach in clinical settings where routine perinatal care takes place is not necessarily to diagnose perinatal depression per se. It would, however, reduce the number of women who need extensive clinical assessment or evaluation with much longer questionnaires, such as the Edinburgh Postnatal Depression Scale⁶ or the Patient Health Questionnaire-9,³³ by more than 50%. The use of good psychometric instruments with acceptable discriminative properties is a necessary but not sufficient step for use in routine practice in larger population-based screening strategies, and the adoption of screening for disorders such as postnatal depression should be based on evidence of benefit derived from randomized trials. In other areas, this has not been shown to be an effective strategy, and screening should therefore not be considered without a wider consideration of the policy implications, costs and benefits.³⁴

The sensitivity of the two case-finding questions in our antenatal and postnatal validation study (100%) was the same as that in a previous postnatal validation study,²⁰ providing further evidence of a simple approach to rule out perinatal depression. The number of false-positive responses to the two questions was substantial in the previous postnatal study and in our study.

Our results should be considered alongside the results of a similar validation study in a primary care population.¹³ The increased specificity of an additional question about the need for help effectively discriminated between positive screens. In our study, the added question resulted in the number of false-positive responses in the postnatal phase dropping to zero.

Increased specificity compromises sensitivity, however, and increases the risk of depression being missed (false negative). Arroll and colleagues¹³ reported increased specificity and unchanged sensitivity (96%) when respondents answered either “yes” or “yes, but not today” or “no” to the additional question about the need for help. They identified 12 false-negative cases but did not refer to this issue in their discussion. In our study, the answer to the additional question was dichotomized as “yes” or “no”; sensitivity was reduced, and the number of false-negative cases was 7 in the antenatal phase and 11 in the postnatal phase. Poor discrimination between true-negative and false-negative cases in practice may mean that patients with depression are effectively lost to follow-up. The benefit of the additional question about the need for help was therefore not conclusive in our perinatal sample.

Limitations

Twenty-six women in the antenatal phase did not complete the diagnostic telephone interview. However, these women did not differ significantly from the 126 who were interviewed in terms of age and ethnic background.

In the postnatal phase, the 52 women who did not respond to postnatal contact were significantly younger, less educated and less likely to be employed than those who returned the questionnaire. This is an important limitation in our sample. In terms of age, it may reflect difficulties surrounding the competing demands associated with the transition to motherhood for younger mothers. In addition, some women may not have responded because of postnatal depressive symptoms.

Implications for future research

Further studies are warranted because our study was limited to the third trimester and first three postnatal months. Studies that involve other perinatal populations, include other trimesters and have longer postnatal follow-up are required. Specific strategies to retain nonrespondents, especially those who are younger and less educated, might be considered. Validation of the case-finding approach among pregnant women who do not speak English and those who are less than 18 years old is needed. Finally, the effect of the questions on outcomes of perinatal care warrants evaluation.

Conclusion

The brevity of the case-finding questions has substantial appeal for the identification of perinatal depression in frontline health care services. In our study, the use of specific case-finding questions had acceptable validity in the perinatal setting. The ability to rule out depression would help to substantially reduce the number of women needing more extensive evaluation of their antenatal and postnatal mental health issues. Identification of perinatal depression is important but represents only the first step. It must be followed by confirmation of the diagnosis and appropriate treatment or referral. Ultimately, the findings of our study may assist the utility of clinical guidelines that advocate the brief case-finding approach for the identification of perinatal depression.