In July 2018, the only fiscal commitment retained by the newly elected conservative government in Ontario from the previous liberal government was to mental health and addictions.1 This commitment acknowledges Ontario’s concerns about the challenges surrounding mental health in the population—concerns raised by the Auditor General of Ontario in 20162 and Provincial Advocate for Children and Youth in 20123 that draw attention to deficiencies associated with children’s mental health services. At the heart of these deficiencies is the lack of information on: 1) the prevalence of children’s mental health problems in Ontario, and 2) the characteristics and outcomes of children receiving mental health services. In combination, the 1983 and 2014 Ontario Child Health Studies (OCHS) addressed this lack of information by identifying: 1) changes in the prevalence and determinants of child and youth mental disorder over the past 30 years, and 2) the continuing challenges with access and targeting of children’s mental health services.4–10
The overall goal of health policies and programs in Canada to improve population health should be guided by 2 basic principles: accountability11 (being answerable for meeting defined objectives) and equity (reduction in poor health among disadvantaged groups).12 Ontario can stay true to these principles and constructively address the weaknesses that continue to undermine the effective provision of children’s mental health services in 2 ways: 1) by developing an information system that measures children’s mental health in the general population every 5 years, and 2) by incorporating identical measurement into intake and follow-up assessments of all children accessing provincially funded, community-based mental health agencies. These measures could be used in tandem to monitor the success of our provincial response to children’s mental health needs and identify specific changes needed, ensuring that services are responsive to the configuration of needs in the general population. This would be achieved by testing for parallel changes in the epidemiology of childhood mental disorders among children in the general population, and those accessing services at children’s mental health agencies.
This commentary: 1) summarizes findings from the 2014 OCHS to argue that core measures of children’s mental health be collected in the general population at regular intervals; 2) points out the limited information recorded on the mental health of children accessing community-based mental health agencies, to argue that core measures be administered to all children at baseline and follow-up; 3) explains the value of collecting identical information in the general population and community-based children’s mental health agencies; 4) describes the content, requirements (practical and scientific), and operational features for the core measures; 5) demonstrates how the core measures can be used to improve decision making aligned with the basic health principles underlying policies and programs; and 6) concludes with a brief summary. Although Ontario is home to the 1983 and 2014 OCHS, we believe that this commentary has broad relevance to other provinces and territories in Canada.
2014 OCHS—Changes in Children’s Mental Health
“Change between 1983 and 2014” is a powerful, recurring theme in the OCHS papers.8,9 The proportion of males aged 4 to 11 years with attention-deficit/hyperactivity disorder has jumped dramatically. In adolescence, there has been a steep increase in anxiety and depression among males and females and a substantial decrease in the prevalence of conduct disorder among males. The prevalence of disorder among children in immigrant v. non-immigrant families dropped by almost 50% from 1983,9 and there is strong evidence that children in poor households are at elevated risk for disorder when this occurs in combination with contextual factors like neighbourhood antisocial behaviour.13 There appears to have been a geographic shift in the prevalence of disorder from large urban areas to small-medium urban and rural areas.8,9 Finally, concerns persist about access to mental health agencies among children identified with disorder: although the proportion of children with mental disorder having service contact increased from 1983 to 2014, most remained without contact.5,8 The dramatic changes in children’s mental health documented by the 2014 OCHS indicate that core information on children’s mental health in the general population is needed at more frequent intervals than 30 years.
Limited Information Recorded on Children’s Mental Health
In Ontario, children with mental health challenges access community-based mental health agencies by way of referral (general practitioners, centralized intake) or direct requests for help. Within these agencies, psychiatrists, psychologists, and social workers collect information on children’s mental health relevant to their practice. However, there is no provision across agencies to collect the same measures of children’s mental health that could be used system-wide to examine the characteristics of those accessing services or the benefits accrued by doing so. The administrative information most relevant to understanding children’s mental health in Ontario is limited to diagnoses recorded by physicians at each visit and related billing codes for services provided. This information limits the identification of children with mental health challenges to the minority accessing walk-in clinics, urgent care, emergency rooms, hospitals, and physician practices. In addition, the failure to implement standard assessment methods has led to varying degrees of reliability and validity for the information collected. Although health-information abstracted from administrative records has important, specific uses for health services research,14 it is not a viable option for assessing or ensuring public accountability.
Alignment of Core Children’s Mental Health Measures
In the study of children’s mental health, there is a deep divide between the evidence and inferences on health system performance obtained in general population studies, such as the 2014 OCHS, and administrative data cumulated by service providers on patients. Differences in the approaches to defining children’s mental health and in the methods of assessment and sampling of respondents have made it impossible to assess the policies, programs, and services developed by governments to address children’s mental health needs among the general population and for those accessing community-based mental health agencies. These 2 approaches could be aligned by collecting a core set of children’s mental health measures in the general population and that in the subset of children accessing community-based children’s mental health agencies. Such alignment would provide the information needed to understand the scope of children’s mental health needs in the general population and the adequacy of provincial responses to meet these needs.
Content, Prerequisites and Organization of a Children’s Mental Health Information System
We believe that the core measures of children’s mental health should include 3 indicators: 1) emotional and behavioural problems, measured as both dimensional and categorical phenomena; 2) a perceived need for professional help with emotional or behavioural problems; and 3) the level of functioning represented as academic achievement and social competence. Although there are various ways to conceptualize children’s mental disorder,15,16 there is a general consensus among child psychiatrists, psychologists, and service providers that assessments of child mental health need to revolve around problems of emotion and behaviour.17,18 The underlying continua of these problem behaviours (number, intensity, frequency) provide direct insight into the severity of the mental disorder. In contrast, perceived need may be linked more closely to help seeking and the potential to benefit from services provided.19 It speaks to child, youth, and parent subjective recognition of mental health problems. Indicators of functioning focus on elements of human capability that are essential for engaged, productive and fulfilled lives. However, if compromised by mental disorder, they must be addressed in our service response.
The core indicators of the information system should: 1) be operationalized in a single instrument that demonstrates reliable and valid measurement; 2) be inexpensive and practical to implement in general population surveys and as part of intake and follow-up assessments completed by service agencies; 3) pose minimal burden to respondents and service practitioners; and 4) represent the perspectives of youth and families on important mental health outcomes. We believe that these requirements can be met by a questionnaire that is self-completed by parents of children aged 4 to 17 and youth aged 12 to 17 in less than 7 or 8 min—a time threshold at which survey completion rates start to drop-off.20 This type of instrument is inexpensive to implement; poses little time burden to respondents; can be completed in almost any setting and adapted to various modes of administration (e.g., in person via tablet/computer, internet); can be computerized to eliminate data entry costs; can be implemented in mental health agencies with little involvement of service practitioners; and can be incorporated into general population surveys at modest additional cost. We see this instrument being implemented in the general population at 5-year cycles—a time interval suitable for identifying constancy or change in mental health need—and sampling children with enough precision in census boundaries associated with service catchment areas to provide reliable estimates of population need. This cycle could either capitalize on existing data collection opportunities provided by Statistics Canada (Canadian Health Survey of Children and Youth,21 Canadian Community Health Survey22); or identify simple, unique data collection mechanisms through the internet or school-based anonymous assessments. We see this instrument being implemented in service agencies at intake and at the 3- or 6-month follow-up, depending on service length. To implement these assessments consistently, agencies would require investments in change management methods to overcome resistance among those opposed to modifying their data collection processes, as well as investments into adequate, longer-term funding to support data analysis to address these system evaluation questions.
In work associated with the 2014 OCHS, we have shown that self-completed questionnaires can meet all of the requirements discussed above and that brief problem checklists, developed to measure children’s mental disorders as dimensional phenomena, can be converted to binary measures of mental disorder (categories) that achieve levels of reliability and validity comparable to standardized diagnostic interviews implemented by lay interviewers in general-population studies.23,24 An example questionnaire that includes measures of mental health disorder symptoms and perceived need for help appears in the Appendix.
Improved Decision Making
How might this information system be used to better align policy decisions with the underlying health principles adopted by government? In general, this system would provide the evidence needed to evaluate the extent to which these principles are operating in practice. For example, at the population level, the system would track changes over time in child mental health, its geographic distribution, and its socioeconomic determinants. On its own, population-level information would provide core data for evaluating the effects of major government policy and program initiatives associated with prevention and treatment (improved health outcomes) on children’s mental health, and inform government about changes in socioeconomic gradients for child mental health (health equity). Coupled with identical assessments obtained by service providers, the system would quantify the responsiveness of community-based children’s mental health agencies to population shifts in children’s mental health need (accountability). At the individual level, routine intake and follow-up assessments would provide evidence to assess change among children accessing community-based mental health services (improved health outcomes).
Aggregating population-level information on the service catchment area needs of children’s mental health would provide estimates of the needs of independent catchment areas. Bringing together these area estimates of children’s mental health need with identical information collected by children’s mental health service agencies could be used to assess the principle of accountability by evaluating service access among children in the general population with mental health needs. This would provide estimates of coverage (the percentage of the children in catchment areas receiving services) and service targeting (the percentage of children receiving services meeting criteria for need). Bringing together catchment area estimates of children’s mental health need with service expenditure and resource allocations could be used to assess the extent to which geographic expenditures and human resource allocation match general population needs (equity).
Summary
There are high levels of children’s mental health need in the general population, temporal shifts in the configurations of mental disorders experienced by children and youth, and important information gaps about who receives community-based mental health services from agencies in Ontario and the types of services that are obtained by children. An information system that uses an identical instrument to measure children’s mental health in the general population and in children receiving mental health services would provide government policy makers with the evidence to assess the extent to which the principles of accountability and equity apply to the provision of children’s mental health services. Although many different factors influence policy development, evidence can only contribute to the process if the appropriate information is available.
Ontario has provided needed leadership in the past—the Ontario Ministry of Community and Social Services commissioned the 1983 OCHS. Along with the Canadian Institutes for Health Research, 3 Ontario ministries (Health, Education, Children and Youth Services) contributed funds to the 2014 OCHS. The Ontario Ministry of Children and Youth Services introduced the idea of core measures25 by requesting children’s mental health centres to implement the Brief Child and Family Phone Interview26 and Child and Adolescent Functional Assessment Scale27 between 1999 and 2015. The information system proposed here constitutes little time burden on service practitioners and no impediment to the collection of clinical information most relevant to their practice and objectives. There would certainly be start-up challenges to overcome associated with informed consent; the protection of privacy; the institution of consistent methods for collecting, processing and transmitting data for system-wide evaluation; and the establishment of ways for individual agencies to track outcomes if practitioners and administrators wished to do so.28 These challenges are well worth addressing in view of the potential benefits to planning associated with the proposed information system. Surely it is time be strategic in our planning for children’s mental health, linking what we do (process) to what we achieve (outcomes) with the goal of better addressing children’s mental health needs.
Supplemental Material
Supplemental Material, ExampleQuestionnaire for Tracking Children’s Mental Health in the 21st Century: Lessons from the 2014 OCHS by Michael H. Boyle, Laura Duncan, Katholiki Georgiades, Jinette Comeau, Graham J. Reid, Warren O’Briain, Robert Lampard, Charlotte Waddell and 2014 Ontario Child Health Study Team in The Canadian Journal of Psychiatry
Acknowledgements
The authors would like to acknowledge Nancy Pyette for technical assistance with editing and proofreading the manuscript.
Footnotes
Declaration of Conflicting Interests: The primary authors (MB, LD, KG, JC, GR, WB, RL and CW) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: During the preparation of this manuscript, Dr. Waddell was supported by the Canada Research Chairs program; Dr. Georgiades was supported by the David R. (Dan) Offord Chair in Child Studies and Dr. Reid was supported by the Children’s Health Research Institute, London, ON.
ORCID iD: Laura Duncan, MA
https://orcid.org/0000-0001-7120-6629
Supplemental Material: Supplemental material for this article is available online.
References
- 1. Rapid policy update: Ontario PC party platform. 2018. Available from: https://occ.ca/rapidpolicy/ontario-pc-party-platform-2018/ (Cited 2018 Nov 17).
- 2. Queens Printer for Ontario. Office of the Auditor General of Ontario. 2016. Annual Report Ottawa (ON): Queens Printer for Ontario; 2016:111–147. [Google Scholar]
- 3. Provincial Advocate for Children and Youth. 2011/2012 Report to the Legislature. Toronto, ON: Provincial Advocate for Children and Youth; 2012. [Google Scholar]
- 4. Boyle MH, Offord DR, Hofmann HF, et al. Ontario child health study: I. methodology. Arch Gen Psychiatry. 1987;44(9):826–831. [DOI] [PubMed] [Google Scholar]
- 5. Offord DR, Boyle MH, Szatmari P, et al. Ontario Child Health Study: II. Six-month prevalence of disorder and rates of service utilization. Arch Gen Psychiatry. 1987;44(9):832–836. [DOI] [PubMed] [Google Scholar]
- 6. Boyle MH, Offord DR. Prevalence of childhood disorder, perceived need for help, family dysfunction and resource allocation for child welfare and children’s mental health services in Ontario. Can J of Behav Sci. 1988;20:374–388. [Google Scholar]
- 7. Boyle MH, Georgiades K, Duncan L, et al. The 2014 Ontario Child Health Study—Methodology. Can J Psychiatry. Forthcoming. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Georgiades K, Duncan L, Wang L, et al. Six-month prevalence of mental disorders and service contacts among children and youth in Ontario: evidence from the 2014 Ontario Child Health Study. Can J Psychiatry. Forthcoming. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Comeau J, Georgiades K, Wang L, et al. Changes in the prevalence of child mental disorders and perceived need for professional help between 1983 and 2014: evidence from the Ontario Child Health Study. Can J Psychiatry. Forthcoming. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Duncan L, Georgiades K, Birch S, et al. Children’s mental health need and expenditures in Ontario: Findings from the 2014 Ontario Child Health Study. Can J Psychiatry. Forthcoming. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Deber R. Thinking about accountability. Healthc Policy. 2014;10(SP):12–24. [PMC free article] [PubMed] [Google Scholar]
- 12. Lane H, Sarkies M, Martin J, et al. Equity in healthcare resource allocation decision making: a systematic review. Soc Sci Med. 2017;175:11–27. [DOI] [PubMed] [Google Scholar]
- 13. Boyle MH, Georgiades K, Duncan L, et al. Poverty, neighbourhood antisocial behaviour and child mental health problems: findings from the 2014 Ontario Child Health Study. Can J Psychiatry. Forthcoming. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. MHASEF Research Team. The Mental Health of Children and Youth in Ontario: 2017 Scorecard. Toronto (ON): Institute for Clinical Evaluative Sciences; 2017. [Google Scholar]
- 15. Kendler KS. Classification of psychopathology: conceptual and historical background. World J Psychiatry. 2018;17(3):241–242. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Krueger RF, Kotov R, Watson D, et al. Progress in achieving quantitative classification of psychopathology. World J Psychiatry. 2018;17(8):282–293. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Lewis M, Rudolph KD, eds. Handbook of Developmental Psychopathology. 3 rd ed New York (NY): Springer; 2015. [Google Scholar]
- 18. Rutter M, Bishop DVM, Pine DS, et al., eds. Rutter’s Child and Adolescent Psychiatry. 5th ed London (UK): Blackwell Publishing; 2008. [Google Scholar]
- 19. Wichstrøm L, Belsky J, Jozefiak T, et al. Predicting service use for mental health problems among young children. Pediatrics. 2014;133(6):2013–3184. [DOI] [PubMed] [Google Scholar]
- 20. Chudoba B. How much time are respondents willing to spend on your survey?; 2018. Available from: https://www.surveymonkey.com/curiosity/survey_completion_times/ (Cited 2018 Nov 17).
- 21. Statistics Canada. Canadian Health Survey on Children and Youth (CHSCY). Ottawa (ON): Statistics Canada; 2018. [Google Scholar]
- 22. Statistics Canada. Canadian Community Health Survey: Annual component (CCHS), Ottawa (ON): Statistics Canada; 2018. [Google Scholar]
- 23. Duncan L, Georgiades K, Wang L, et al. The 2014 Ontario Child Health Study Emotional Behavioural Scales (OCHS-EBS) Part I: a checklist for dimensional measurement of selected DSM-5 disorders. Can J Psychiatry. 2018:706743718808250 doi:org/10.1177/0706743718808250. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Boyle MH, Duncan L, Georgiades K, et al. The 2014 Ontario Child Health Study Emotional Behavioural Scales (OCHS-EBS) Part II: psychometric adequacy for categorical measurement of selected DSM-5 disorders. Can J Psychiatry. 2018:706743718808251 doi:org/10.1177/0706743718808251. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Barwick M, Boydell KM, Cunningham CE, et al. Overview of Ontario’s screening and outcome measurement initiative in children’s mental health. J Can Acad Child Adolesc Psychiatry. 2004;13(4):105–109. [PMC free article] [PubMed] [Google Scholar]
- 26. Cunningham CE, Boyle MH, Hong S, et al. The Brief Child and Family Phone Interview (BCFPI): 1. Rationale, development, and description of a computerized children’s mental health intake and outcome assessment tool. Journal of Child Psychology and Psychiatry. 2009;50(4):416–423. [DOI] [PubMed] [Google Scholar]
- 27. Hodges K, Doucette-Gates A, Liao Q. The relationship between the Child and Adolescent Functional Assessment Scale (CAFAS) and indicators of functioning. Journal of Child and Family Studies. 1999;8(1):109–122. [Google Scholar]
- 28. Canadian Psychological Association. Outcomes and Progress Monitoring in Psychotherapy: A report of the Canadian Psychological Association; 2018. Available from: https://www.cpa.ca/docs/File/Task_Forces/Treatment%20Progress%20and%20Outcome%20Monitoring%20Task%20Force%20Report_Final.pdf (Cited 2018 Nov 17).
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental Material, ExampleQuestionnaire for Tracking Children’s Mental Health in the 21st Century: Lessons from the 2014 OCHS by Michael H. Boyle, Laura Duncan, Katholiki Georgiades, Jinette Comeau, Graham J. Reid, Warren O’Briain, Robert Lampard, Charlotte Waddell and 2014 Ontario Child Health Study Team in The Canadian Journal of Psychiatry