Skip to main content
PMC home page
International Journal of Methods in Psychiatric Research logoLink to International Journal of Methods in Psychiatric Research
. 2006 Mar 24;11(3):125–133. doi: 10.1002/mpr.130

Self‐reported use of mental health services versus administrative records: should we care?

Anne E Rhodes 1,✉, Elizabeth Lin 2, Cameron A Mustard 3
PMCID: PMC6878364  PMID: 12459825

Abstract

Studies of mental health services have emphasized that people in need are not receiving treatment. However, these studies, based on self‐reported use, may not be consistent with administrative records. This study compared self‐reports of mental health service use with administrative records in a large representative sample. Respondent reports within the Ontario portion of the 1994/95 Household Component of the National Population Health Survey (NPHS) were individually linked to the provincial mental‐health physician reimbursement claims. A total of 5,187 Ontarians, aged 12 years or more, reported on their use of mental healthcare within the NPHS and 4,621 (89%) consented and were successfully linked to administrative records.

Comparisons between the two sources identified that the agreement for any use and volume of use was moderate to low and varied according to select respondent characteristics. These differences affected estimates of the associations with use and volume of use. People who reported high levels of distress reported more visits than those who did not and this effect was stronger in the self‐reported data.

These results suggest that recall bias may be present. Regardless of the definition of care, access for those in need remains a concern despite universal medical insurance coverage. Copyright © 2002 Whurr Publishers Ltd.

Keywords: self‐reports, mental health services, administrative records

Full Text

The Full Text of this article is available as a PDF (75.9 KB).

References

  1. Aneshensel CS, Estrada AL, Hansell MJ, Clark VA. Social psychological aspects of reporting behavior: lifetime depressive episode reports. J Health Soc Behav 1987; 28(3): 232–46. [PubMed] [Google Scholar]
  2. Bland RC, Newman SC, Orn H. Health care utilization for emotional problems: results from a community survey. Can J Psychiatry 1990; 35(5): 397–400.] [DOI] [PubMed] [Google Scholar]
  3. Bland RC, Newman SC, Orn H. Help‐seeking for psychiatric disorders. Can J Psychiatry 1997; 42(9): 935–42. [DOI] [PubMed] [Google Scholar]
  4. Beaudet MP. Depression. Health Reports 1996; 7(4): 11–24. [PubMed] [Google Scholar]
  5. Cairney J, Thorpe C, Rietschlin J, Avison WR. 12‐month prevalence of depression among single and married mothers in the 1994 National Population Health Survey. Can J Public Health 1999; 90(5): 320–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Cannel CF, Fisher G, Bakker T. Reporting of Hospitalization in the Health Interview Survey Department of Health, Education and Welfare, National Center for Health Statistics, Washington DC: US Government Printing Office; Series 2(6), 1965. [Google Scholar]
  7. D'Arcy C. Patterns in the delivery of psychiatric care in Saskatchewan 1971–72: an overview of service sectors and patient volumes. Can Psychiatr Assoc J 1976; 21(2): 91–100. [DOI] [PubMed] [Google Scholar]
  8. D'Arcy C, Bold G, Schmitz JA. Psychiatric healthcare and costs under comprehensive public health insurance: experience in a Canadian province. Med Care 1981; 19(9): 881–94. [DOI] [PubMed] [Google Scholar]
  9. Diverty B, Beaudet, MP . Depression. An undertreated disorder? Health Rep 1997; 8(4): 9–18. [Google Scholar]
  10. Fleiss JL. Statistical Methods for Rates and Proportions. New York: Wiley, 1981. [Google Scholar]
  11. Golding JM, Gongla P, Brownell A. Feasibility of validating survey self‐reports of mental health service use. Am J Community Psychol 1988; 16(1): 39–51. [DOI] [PubMed] [Google Scholar]
  12. Kessler RC, Andrews G, Mroczek D, Ustun B, Wittchen H‐U. The World Health Organization Composite International Diagnostic Interview Short Form (CIDISF). Int J Psychiatr Res. 1998; 7: 171–85. [Google Scholar]
  13. Kessler RC, McGonagle K, Zhao S, Nelson CB, Hughes M, Eshelman S, Wittchen HU, Kendler KS. Lifetime and 12‐month prevalence of DSM‐III‐R psychiatric disorders in the United States. Results from the National Comorbidity Survey. Arch Gen Psychiatry 1994; 51: 8–19. [DOI] [PubMed] [Google Scholar]
  14. Kessler RC, Zhao S, Katz SJ, Kousiz AC, Frank RG, Edlund M, Leaf P. Past‐year use of outpatient services for psychiatric problems in the National Comorbidity Survey. Am J Psychiatry. 1999; 156(1): 115–23. [DOI] [PubMed] [Google Scholar]
  15. Koriat A, Goldsmith M, Plansky A. Towards a psychology of memory accuracy. Annu Rev Psychol 2000; 51: 481–537. [DOI] [PubMed] [Google Scholar]
  16. Kramer MS, Feinstein AR. Clinical biostatistics. LIV. The biostatistics of concordance. Clin Pharmacol Ther 1981; 29(1): 111–23. [DOI] [PubMed] [Google Scholar]
  17. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977; 33: 159–74. [PubMed] [Google Scholar]
  18. Lin E, Goering P. The Utilization of Physician Services for Mental Health in Ontario. Technical Report. Toronto: Institute for Clinical Evaluative Sciences, 1999. [Google Scholar]
  19. Lin E, Goering P, Offord DR, Campbell D, Boyle, MH . The use of mental health services in Ontario: epidemiologic findings. Can J Psychiatry 1996; 41(9): 572–7. [DOI] [PubMed] [Google Scholar]
  20. Manderschied RW, Rae DS, Narrow WE, Locke BZ, Regier DA. Congruence of service utilization estimates from the Epidemiologic Catchment Area Project and other sources. Arch Gen Psychiatry 1993; 50: 108–14. [DOI] [PubMed] [Google Scholar]
  21. Mechanic D. Sociological dimensions of illness behavior. Soc Sci Med 1995; 41(9): 1207–16. [DOI] [PubMed] [Google Scholar]
  22. Murray CJL, Lopez AD. Evidence‐based health policy–lessons from the Global Burden of Disease Study. Science 1996; 274: 740–3. [DOI] [PubMed] [Google Scholar]
  23. Mustard CA, Goel V, Barer M, Muharjarine N. Reliability and Validity of Self‐Reported Utilization of Insured Health Care Services. Institute for Work and Health Working Paper. No 108, 2000.
  24. National Population Health Survey . Public Use Microdata File User Documentation, Ottawa, 1996.
  25. Parikh SV, Lin E, Lesage AD. Mental health treatment in Ontario: Selected comparisons between the primary care and specialty sectors. Can J Psychiatry 1997; 42: 929–34. [DOI] [PubMed] [Google Scholar]
  26. Rao JNK, Wu CFJ, Yue K. Some recent work on resampling methods for complex surveys. Survey Methodology Statistics Canada, Catalogue 12‐001 1992; 18(2): 209–17. [Google Scholar]
  27. Redelmeier DA, Schull MJ, Hux JE, Tu JV, Ferris LE. Problems for clinical judgment: 1. Eliciting an insightful history of present illness. CMAJ 2001; 164(5): 647–51. [PMC free article] [PubMed] [Google Scholar]
  28. Regier DA, Kaelber CT, Rae DS, Farmer, ME , Knauper B, Kessler RC, Norquist GS. Limitations of diagnostic criteria and assessment instruments for mental disorders. Implications for research and policy. Arch Gen Psychiatry 1998; 55(2): 109–15. [DOI] [PubMed] [Google Scholar]
  29. Regier DA, Narrow WE, Rae DS, Manderschied, RW , Locke BZ, Goodwin FK. The de facto US mental and addictive disorders service system. Epidemiologic Catchment Area prospective 1‐year prevalence rates of disorders and services. Arch Gen Psychiatry 1993; 50: 85–94. [DOI] [PubMed] [Google Scholar]
  30. Robinson, JR. , Tataryn, DJ . Reliability of the Manitoba Mental Health Management Information System for Research. Can J Psych 1997; 42(7): 744–9. [DOI] [PubMed] [Google Scholar]
  31. Rust KF, Rao JNK. Variance estimation for complex surveys using replication techniques. Stat Methods Med Res 1996; 5: 283–310. [DOI] [PubMed] [Google Scholar]
  32. SAS Institute Inc . SAS Release 8.2 for Unix Environments. SAS Campus Drive, Cary, North Carolina 27513: SAS Institute Inc., 2000.
  33. Schwartz AH, Perlman BB, Paris M, Schmidt BS, Thornton JC. Psychiatric diagnoses as reported to Medicaid and recorded in patient charts. Am J Public Health 1980; 70(4): 406–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Simon GE, VonKorff M. Recall of psychiatric history in cross‐sectional surveys: implications for epidemiologic research. Epidemiol Rev 1995; 17(1): 221–7. [DOI] [PubMed] [Google Scholar]
  35. Spector PE, Bedell JR. Measuring program effectiveness: self‐report versus objective indicators of recidivism. Am J Community Psychol 1982; 10: 613–16. [DOI] [PubMed] [Google Scholar]
  36. Swain L, Catlin G, Beaudet MP. The National Population Health Survey–its longitudinal nature. Health Rep 1999; 10(4): 69–82. [PubMed] [Google Scholar]
  37. Tataryn DJ, Mustard C, Derksen S. The utilization of medical services for mental health disorders Manitoba: 1991–1992. Department of Community Health Sciences, Faculty of Medicine, University of Manitoba: Manitoba Centre for Health Policy and Evaluation, 1994.
  38. Taube CA, Schlenger WE, Rupp A, Whitmore RW. Validity of Medicaid household respondent reporting of ambulatory visits for mental disorders. J Econ Soc Meas 1986; 14: 243–56. [PubMed] [Google Scholar]
  39. Towery OB, Sharfstein SS, Goldberg ID. The mental and nervous disorder utilization and cost survey: an analysis of insurance for mental disorders. Am J Psychiatry 1980; 137(9): 1065–70. [DOI] [PubMed] [Google Scholar]
  40. Wittchen HU. Reliability and validity studies of the WHO–Composite International Diagnostic Interview (CIDI): a critical review. J Psychiatr Res 1994; 28(1): 57–84. [DOI] [PubMed] [Google Scholar]
  41. World Health Organization . Composite International Diagnostic Interview (CIDI): a) CIDI Interview b) CIDI‐User Manual c) CIDI Training Manual d) CIDI Computer Programs. Geneva: World Health Organization, 1990. [Google Scholar]

Articles from International Journal of Methods in Psychiatric Research are provided here courtesy of Wiley

RESOURCES