Skip to main content
PMC home page
International Journal of Methods in Psychiatric Research logoLink to International Journal of Methods in Psychiatric Research
. 2006 Mar 24;13(3):165–175. doi: 10.1002/mpr.172

Self‐reported use of mental health services versus administrative records: care to recall?

Anne E Rhodes 1,✉, Kinwah Fung 2
PMCID: PMC6878470  PMID: 15297900

Abstract

Estimates of the level of unmet need for mental health treatment often rely on self‐reported use of mental health services. However, depressed persons may over‐report their use in relation to administrative records if they are highly distressed. This study seeks to replicate and explicate the finding that persons at a high level of distress report more mental health service use than recorded in their healthcare records. The study sample, N=36,892, 12 years and older, was drawn from the 1996/97 Ontario portion of the Canadian National Population Health Survey. Respondents were individually linked to their administrative mental healthcare records 12 months backward in time. Of these, 96.5% agreed to the link and 23,063 (62.5%) were linked. Almost two‐thirds of those who were depressed in the past year were currently at a high level of distress. Differential reporting of use for highly distressed persons in excess of 100% remained in the use of different types of physician providers after adjustments for other potential determinants of use. Telescoping was also not an explanation. The patterns of differential reporting between groups expected to diverge and converge in their recall ability were consistent with a recall bias. As this study was not able to rule out a recall bias, it further accentuates concerns about the impact of bias in the measurement of mental health‐service use and inferences made concerning the determinants of use. Copyright © 2004 Whurr Publishers Ltd.

Keywords: self‐reports, mental health services, health services research, depression, mental recall

Full Text

The Full Text of this article is available as a PDF (173.2 KB).

References

  1. Alegria M, Bijl R, Lin E, Walters E, Kessler R. Income differences in persons seeking outpatient treatment for mental disorders. A comparison of the United States with Ontario and the Netherlands. Archives of General Psychiatry 2000; 57, 383–91. [DOI] [PubMed] [Google Scholar]
  2. Allison P. Multinomial logit analysis In Logistic Regression Using the SAS System: Theory and Application. 2 edn. Cary NC: SAS Institute, 2000, pp. 111–31. [Google Scholar]
  3. Andrews G, Slade T. Interpreting scores on the Kessler Psychological Distress Scale (K10). Australian and New Zealand Journal of Psychiatry 25: 494–7. [DOI] [PubMed] [Google Scholar]
  4. Aneshensel C, Estrada A, Hansell M, Clark V. Social psychological aspects of reporting behavior: lifetime depressive episode reports. J Health Soc Behav 1987; 28: 232–46. [PubMed] [Google Scholar]
  5. Austin P, Hux J (2002). A brief note on overlapping confidence intervals. Journal of Vascular Surgery 36: 194–5. [DOI] [PubMed] [Google Scholar]
  6. Beaudet M. Depression. Health Reports 1996; 7: 11–24. [PubMed] [Google Scholar]
  7. Bijl R, De Graff R, Hiripi E, Kessler R, Kohn R, Offord D, Ustun T, Vicente B, Volleburgh W, Walters E, Wittchen H‐U. The prevalence of treated and untreated mental disorders in five countries. Health Affairs 2003; 22, 122–33. [DOI] [PubMed] [Google Scholar]
  8. Bradburn N. Response Effects In Rossi P, Wright J, Anderson A. (eds) Handbook of Survey Research. New York: Academic Press, 1983, pp. 289–328. [Google Scholar]
  9. Cairney J, Thorpe C, Rietschlin J, Avison W. 12‐month prevalence of depression among single and married mothers in the 1994 National Population Health Survey. Canadian Journal of Public Health 1999; 90: 320–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. CANMAT . Canadian Network for Mood and Anxiety Treatments: Guidelines for Diagnosis and Pharmacologic Treatment of Depression. Toronto, Ontario, 1999. [Google Scholar]
  11. Chan B. Supply of Physicians' Services in Ontario (Rep. No. 1). Toronto, Ontario: Institute for Clinical Evaluative Sciences, 1999. [Google Scholar]
  12. Chan BTB, Austin P. Patient, physician and community factors affecting referrals to specialists in Ontario, Canada. A Population‐based, multi‐level modelling approach. Medical Care 2003; 41: 500–11. [DOI] [PubMed] [Google Scholar]
  13. Ciok R. The Non‐Telephone Population Social Survey Methods Division, Statistics Canada; Ottawa, Canada: Statistics Canada, 1993. [Google Scholar]
  14. Clark R, Ricketts S, McHugo G. Measuring hospital use without claims: a comparison of patient and provider reports. Health Services Research 1996; 31: 153–69. [PMC free article] [PubMed] [Google Scholar]
  15. Diverty B, Beaudet M. Depression. An undertreated disorder? Health Reports 1997; 8: 9–18. [Google Scholar]
  16. Evans C, Crawford B. Patient self‐reports in pharmacoeconomic studies. Their use and impact on study validity. Pharmacoeconomics 1999; 15: 241–56. [DOI] [PubMed] [Google Scholar]
  17. Furukawa T, Kessler R, Slade T, Andrews G. The performance of the K6 and K10 screening scales for psychological distress in the Australian National Survey of Mental Health and Well‐Being. Psychological Medicine 2003; 33: 357–62. [DOI] [PubMed] [Google Scholar]
  18. Greenland S, Finkle WD. A critical look at methods for handling missing covariates in epidemiologic regression analyses. American Journal of Epidemiology 1995; 142: 1255–64. [DOI] [PubMed] [Google Scholar]
  19. Hennesy K, Reed S. Validating self‐reports of mental health service use in a chronic population. Journal of Nervous and Mental Disorders 1992; 180: 399–400. [DOI] [PubMed] [Google Scholar]
  20. Horwath E, Johnson J, Klerman G, Weissman M. Depressive symptoms as relative and attributable risk factors for first‐onset major depression. Arch Gen Psychiatry 1992; 49: 817–23. [DOI] [PubMed] [Google Scholar]
  21. Hosmer D, Lemeshow S. Applied Logistic Regression. New York: Wiley; 1989. [Google Scholar]
  22. Judd L, Paulus M, Wells K, Rapaport M. Socieconomic burden of subsyndromal depressive symptoms and major depression in a sample of the general population. Am J Psychiatry 1996; 153: 1411–17. [DOI] [PubMed] [Google Scholar]
  23. Katz SJ, Kessler RC, Frank RG, Leaf P, Lin E. Mental health care use, morbidity, and socioeconomic status in the United States and Ontario. Inquiry 1997; 34: 38–49. [PubMed] [Google Scholar]
  24. Kessler R, Andrews G, Colpe L, Hiripi E, Mroczek D, Normand S, Walters E, Zaslavsky A. Short screening scales to monitor population prevalences and trends in non‐specific psychological distress. Psychological Medicine 2002; 32, 959–76. [DOI] [PubMed] [Google Scholar]
  25. Kessler R, Andrews G, Mroczek D, Ustun B, Wittchen H‐U. The World Health Organization Composite International Diagnostic Interview – Short Form (CIDI–SF). International Journal of Methods in Psychiatric Research 2001; 7: 171–85. [Google Scholar]
  26. Kessler R, Frank RG, Edlund M, Katz SJ, Lin E, Leaf P. Differences in the use of psychiatric outpatient services between the United States and Ontario. New England Journal of Medicine 1997; 336: 551–7. [DOI] [PubMed] [Google Scholar]
  27. Kessler R, Wittchen H‐U, Abelson J, McGonagle K, Schwarz N, Kendler K, Knäuper B, Zhao S. Methodological studies of the Composite International Diagnostic Interview (CIDI) in the US National Comorbidity Survey (NCS). International Journal of Methods in Psychiatric Research 2001; 7: 33–55. [Google Scholar]
  28. Koriat A, Goldsmith M, Plansky A. Towards a psychology of memory accuracy. Annu Rev Psychol 2000; 51; 481–537. [DOI] [PubMed] [Google Scholar]
  29. Leaf PJ, Bruce ML, Tischler GL, Freeman DHJ, Weissman MM, Myers JK. Factors affecting the utilization of specialty and general medical mental health services. Medical Care 1988; 26: 9–26. [DOI] [PubMed] [Google Scholar]
  30. Pignone M, Gaynes B, Rushton J, Burchell C, Orleans C, Multrow C, Lohr K. Screening for depression in adults: a summary of the evidence for the U.S. Preventive Services Task Force. Annals of Internal Medicine 2002; 136: 765–76. [DOI] [PubMed] [Google Scholar]
  31. Pincus H, Davis W, McQueen L. ‘Subthreshold’ mental disorders. A review and synthesis of studies on minor depression and other ‘brand names’. Br J Psychiatry 1999; 288–96. [DOI] [PubMed] [Google Scholar]
  32. Rao J, Wu C, Yue K. Some recent work on resampling methods for complex surveys. Survey Methodology, Statistics Canada, Catalogue 1992; 18: 209–17. [Google Scholar]
  33. Rhodes A, Lin E, Mustard C. Self‐reported use of mental health services versus administrative records: should we care? International Journal of Methods in Psychiatric Research 2002; 11: 125–33. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Rothman K, Greenland S. Modern Epidemiology. 2 edn. Philadelphia: Lippincott‐Raven; 1998. [Google Scholar]
  35. Rust K, Rao J. Variance estimation for complex surveys using replication techniques. Stat Methods Med Res 1996; 5: 283–310. [DOI] [PubMed] [Google Scholar]
  36. Schwartz A, Perlman B, Paris M, Schmidt B, Thornton J. Psychiatric diagnosis as reported to Medicaid and as recorded in patient charts. American Journal of Public Health 1980; 70: 406–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Simon GE, VonKorff M. Recall of psychiatric history in cross‐sectional surveys: implications for epidemiologic research. Epidemiol Rev 1995; 17: 221–7. [DOI] [PubMed] [Google Scholar]
  38. Statistics Canada . 1996/97 National Population Health Survey. Public Use Microdata File Documentation. Ottawa, Ontario: Minister of Supply and Services; 1998. [Google Scholar]
  39. Towery OB, Sharfstein SS, Goldberg ID. The Mental and Nervous Disorder Utilization and Cost Survey: an analysis of insurance for mental disorders. American Journal of Psychiatry 1980; 137: 1065–70. [DOI] [PubMed] [Google Scholar]
  40. Vach W, Blettner M (1991). Biased estimation of the odds ratio in case‐control studies due to the use of ad hoc methods of correcting for missing values for confounding variables. American Journal of Epidemiology 1991; 134: 895–907. [DOI] [PubMed] [Google Scholar]; Published erratum appears in Am J Epidemiol 1994; 140(1): 79. [Google Scholar]
  41. Wells KB, Manning Jr, W , Duan N, Newhouse J, Ware Jr, J . Cost‐sharing and the use of general medical physicians for outpatient mental health care. Health Services Research 1987; 22: 1–17. [PMC free article] [PubMed] [Google Scholar]

Articles from International Journal of Methods in Psychiatric Research are provided here courtesy of Wiley

RESOURCES