Abstract
There is increased emphasis on routine assessment of depression in primary care. This report is the first UK validation of two self-completed measures: the Patient Health Questionnaire (PHQ–9) and the Clinical Outcomes in Routine Evaluation – Outcome Measure (CORE–OM). Optimum cut-off points were established against a diagnostic gold standard in 93 patients. PHQ–9 sensitivity = 91.7% (95% confidence interval [CI] = 77.5 to 98.3%) and specificity 78.3% (95% CI = 65.8 to 87.9%). CORE–OM sensitivity = 91.7% (95% CI = 77.5 to 98.2%) and specificity = 76.7% (95% CI = 64.0 to 86.6%). Brief self-rated questionnaires are as good as clinician-administered instruments in detecting depression in UK primary care.
Keywords: depression, diagnosis, primary care, screening
INTRODUCTION
Recent guidelines, such as those issued by the National Institute for Health and Clinical Excellence,1 recommend standardised instruments to improve the recognition and management of depression. The role of standardised assessment of depression is emphasised in the Quality and Outcomes Framework,2 where routine assessment in established depression, and depression case-finding in diabetes and ischaemic heart disease, are rewarded. Two specific instruments have been proposed:3,4 the Patient Health Questionnaire (PHQ–9)5 and the Clinical Outcomes in Routine Evaluation — Outcome Measure (CORE–OM).6 The PHQ–9 is a self-administered nine-item depression-specific questionnaire developed in the US.5 The CORE–OM is a longer 34-item generic instrument developed in the UK, which measures common mental health problems (including four items tapping depression), functional capacity, and risk. Both of these have clear advantages over other instruments, in that they are self-completed and are freely available to end-users. However, there is no published UK primary care validation study of these instruments assessed against a diagnostic gold standard of depression.
METHOD
A randomised trial was conducted of collaborative care for depression in a UK primary care setting where both the PHQ–95 and CORE–OM6 were used at 3 months' follow-up. At 3 months' follow-up a trained interviewer carried out a diagnostic interview using the Structured Clinical Interview for DSM (SCID),7 without foreknowledge of PHQ–9 or CORE–OM scores. One hundred and fourteen patients were recruited, of whom 96 were followed up and received a diagnostic interview, PHQ–9, and CORE–OM (22 males, 74 females; mean age 42.5 years, standard deviation 13.6 years).
Sensitivity, specificity, and likelihood ratios for various PHQ–9 scores were calculated (including ≥10, as recommended in US primary care5) and for the CORE–OM-clinical and depression scores.6 Receiver operator characteristic (ROC) analysis was also conducted.
RESULTS
Thirty-six of 96 patients were diagnosed as having major depressive disorder on the SCID. Forty-seven out of the 60 SCID non-depressed patients scored below the recommended PHQ–9 cut off of ≥10, giving PHQ–9 sensitivity of 91.7% (95% CI = 77.5 to 98.3%), specificity 78.3% (95% CI = 65.8 to 87.9%), positive likelihood ratio 4.2 (95% CI = 2.6 to 6.9), and negative likelihood ratio of 0.11 (95% CI = 0.04 to 0.32). Increasing the cut-off point of the PHQ–9 to ≥12 improved the specificity slightly, without compromising sensitivity (Table 1). For the CORE–OM-clinical score the optimum cut-off point was ≥13, giving the same sensitivity of 91.7% (95% CI = 77.5 to 98.2%), specificity 76.7% (95% CI = 64.0 to 86.6%), positive likelihood ratio 3.9 (95% CI = 2.5 to 6.3), and negative likelihood ratio of 0.10 (95% CI = 0.04 to 0.32). Using the depression subscale (CORE-OM-D) improved the psychometric properties marginally (optimum cut off ≥17 sensitivity = 94.4%, 95% CI = 81.3 to 99.3%; specificity = 78.3%, 95% CI = 65.8 to 87.9%).
Table 1.
Table 1. Sensitivity, specificity, and likelihood ratios at various cut-off points of the PHQ–9 and CORE–OM.
| Sensitivity Specificity | ||||||
|---|---|---|---|---|---|---|
| Cut off | % | % | LR+ | LR− | AUC (95% CI) | |
| PHQ–9 | 0.94 (0.89 to 0.98) | |||||
| ≥9 | 94.4 | 73.3 | 3.542 | 0.076 | ||
| ≥10 | 91.7 | 78.3 | 4.231 | 0.106 | ||
| ≥11 | 91.7 | 81.7 | 5.000 | 0.102 | ||
| ≥12 | 91.7 | 85.0 | 6.111 | 0.098 | ||
| ≥13 | 88.9 | 86.7 | 6.667 | 0.128 | ||
| CORE-OM-clinical | 0.92 (0.87 to 0.97) | |||||
| ≥12 | 91.7 | 71.7 | 3.235 | 0.116 | ||
| ≥13 | 91.7 | 76.7 | 3.929 | 0.109 | ||
| ≥14 | 88.9 | 80.0 | 4.444 | 0.139 | ||
| ≥15 | 83.3 | 83.3 | 4.545 | 0.204 | ||
| ≥16 | 75.0 | 85.0 | 5.000 | 0.294 | ||
| CORE-OM-depression | 0.92 (0.86 to 0.97) | |||||
| ≥17 | 94.4 | 78.3 | 4.359 | 0.071 | ||
| ≥18 | 88.9 | 81.7 | 4.848 | 0.136 | ||
| ≥19 | 88.9 | 81.7 | 4.848 | 0.136 | ||
| ≥20 | 88.9 | 81.7 | 4.848 | 0.136 | ||
| ≥21 | 83.3 | 85.0 | 5.556 | 0.196 | ||
AUC = area under the curve. CORE-OM = Clinical Outcomes in Routine Evaluation – Outcome Measure. LR+ = positive likelihood ratio. LR− = negative likelihood ratio. PHQ–9 = Patient Health Questionnaire.
ROC curve analysis indicated that the PHQ–9 and CORE–OM-clinical performed well: PHQ–9 area under the curve (AUC) = 0.94 (95% CI = 0.89 to 0.98); CORE–OM-clinical AUC = 0.92 (95% CI = 0.87 to 0.97); CORE–OM-D AUC = 0.92 (95% CI = 0.86 to 0.97) (Figure 1 and Table 1; see Supplementary Table 1 for detailed psychometric properties, including CIs). At cut offs of ≥10 and ≥13 respectively, PHQ–9 and CORE–OM-clinical delivered almost identical sensitivity and specificity.
Figure 1.
Receiver Operator Characteristic (ROC) curve of CORE–OM (Clinical Outcomes in Routine Evaluation − Outcome Measure) depression scores; CORE–OM clinical scores; and Patient Health Questionnaire (PHQ–9) scores in the presence or absence of major depression diagnosed by the Structured Clinical Interview for DSM.
By applying Bayes' theorem8 to these performance data, at a commonly encountered baseline prevalence of 10% for depressive disorders, a positive screen of ≥10 on the PHQ–9 will increase the post-test probability of depression from 10 to 32% (95% CI = 22 to 43%). At a 20% prevalence of depression, typical of that encountered among those with chronic diseases such as diabetes, a positive screen on the PHQ–9 will increase the post-test probability from 20 to 51 % (95% CI = 39% to 63%).
DISCUSSION
This is the first UK validation of the PHQ–9 and CORE–OM against a diagnostic gold standard for depression in a UK primary care population. There are relative merits to both instruments: the PHQ–9 can be self-completed in less than 2 minutes, while the CORE–OM measures a greater range of mental health problems in addition to depression, and assesses functional capacity and risk. The cut-off points of ≥10 for the PHQ–9 and ≥13 for the CORE–OM achieve good performance, if not better than other longer and clinician-competed instruments.9 As this is a relatively small cross-sectional study, replication of these findings would be helpful.
How this fits in
There is an increased emphasis on the recognition and management of depression in the UK under the Quality and Outcomes Framework. PHQ–9 and CORE–OM are brief self-completed instruments, which have been recommended as screening and assessment tools. There is no published report from UK primary care on the performance of the PHQ–9 or CORE–OM as assessment tools for depression. The PHQ–9 and CORE–OM perform as well in UK primary care as clinician-rated instruments. GPs may consider using either the PHQ–9 or CORE–OM alongside enhancements of care for depression.
While these performance characteristics are impressive according to accepted criteria,10 it is unlikely that case-finding instruments, by themselves, will improve the quality and outcome of primary care for depression.1,11
Supplementary Material
Acknowledgments
We are grateful to the Medical Research Council who funded the initial trial from which these data are drawn, and to our co-investigators of the MRC Collaborative care platform trial group.
Supplementary information
Additional information accompanies this article at http://www.rcgp.org.uk/bjgp-suppinfo
Ethics committee
Not applicable
Competing interests
Michael Barkman was involved in the development of the CORE–OM instrument, but does not gain financially from its use. The CORE–OM instrument is free within the NHS.
REFERENCES
- 1.National Institute for Clinical Excellence. Depression: core interventions in the management of depression in primary and secondary care. London: The Stationery Office; 2004. [Google Scholar]
- 2.BMA and NHS Employers. Revisions to the GMS contract, 2006/7. Delivering investment in General Practice. London: British Medical Association; 2006. [Google Scholar]
- 3.Fletcher J, Bower P, Richards D, Saunders T. Enhanced services specification for depression under the new GP contract. Manchester: NIMHE Northwest; 2004. [Google Scholar]
- 4.Fletcher J, Bower P, Gask L, et al. (2006) Primary care service for depression — a guide to best practice. Manchester: Care Services Improvement Partnership (CSIP)/NPCRDC; 2006. [Google Scholar]
- 5.Kroenke K, Spitzer RL, Williams JB. The PHQ–9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606–613. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Barkham M, Margison F, Leach C, et al. Service profiling and outcomes benchmarking using the CORE–OM: towards practice-based evidence in the psychological therapies. J Consult Clin Psychol. 2001;69(2):184–196. doi: 10.1037//0022-006x.69.2.184. [DOI] [PubMed] [Google Scholar]
- 7.Spitzer RL, Williams JB, Gibbon M, First MB. The Structured Clinical Interview for DSM-III-R (SCID). I: History, rationale, and description. Arch Gen Psychiatry. 1992;49(8):624–629. doi: 10.1001/archpsyc.1992.01820080032005. [DOI] [PubMed] [Google Scholar]
- 8.Sackett DL, Haynes RB. Evidence base of clinical diagnosis: the architecture of diagnostic research. BMJ. 2002;324(7336):539–541. doi: 10.1136/bmj.324.7336.539. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Williams JW, Pignone M, Ramirez G, Stellato CP. Identifying depression in primary care: a literature synthesis of case-finding instruments. Gen Hosp Psychiatry. 2002;24(4):225–237. doi: 10.1016/s0163-8343(02)00195-0. [DOI] [PubMed] [Google Scholar]
- 10.Streiner D, Norman G. Health measurement scales: a practical guide to their development and use. 3rd edn. Oxford, UK: Oxford University Press; 2003. [Google Scholar]
- 11.Gilbody S, Whitty P, Grimshaw JG, Thomas R. Improving the recognition and management of depression in primary care. Effective Health Care Bulletin. 2002;7(Number 5):1–12. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.