Abstract
Objective
This report describes the impact of Project TEACH (PT), a statewide training and consultation program for pediatric Primary Care Providers (PCPs), on the identification and treatment of mental health conditions.
Methods
In this observational study, the intervention group had 176 voluntary PT-trained pediatric PCPs found in New York State Medicaid files (2009-2011). Comparisons were a stratified random sample of 200 PCPs. Prescription practices; diagnoses and follow-up care for youth were compared between the groups.
Results
The percentage of children prescribed psychotropic medication increased in the PT-trained group (9% to 12%, p<.001), a larger increase than in the untrained group (p<.001). Fewer differences were seen in diagnoses, and in medication use and follow-up care among those with depression, but the data trended in the direction of a positive effect.
Conclusions
This educational/consultation intervention may be able to change providers’ behaviors but further research is needed to clarify its effectiveness.
INTRODUCTION
Between 13% and 37% of school age children and adolescents have mental health problems,1-3 but up to 80% do not receive appropriate mental health services.4,5 Many studies have documented pediatric primary care providers’ (PCPs’) discomfort with making mental health diagnoses, resulting from providers’ inadequate training and confidence.6-8
In New York State, Project TEACH (PT - Training and Education for the Advancement of Children's Health) was developed to enhance PCPs’ knowledge and skills in the identification and treatment of the most common pediatric mental health issues. In this paper, we examine the effectiveness of PT in changing providers’ diagnosis, prescription and follow-up practices.
METHODS
PT is comprised of two programs; this paper focuses on Child and Adolescent Psychiatry for Primary Care (CAP-PC), a collaboration between the REACH Institute (Resource for Advancing Children's Health9) and five Departments of Psychiatry.
The CAP-PC curriculum consists of 15 hours of in-person training, a toolkit, and web-based learning tools followed by a six-month distance learning program including 12 one-hour long consultation calls with child psychiatrists. The program goals are for PCPs to correctly identify pediatric behavioral health problems, effectively manage psychopharmacology, and create and implement treatment plans by linking to existing resources. This evaluation includes training from July 16, 2010 to February 3, 2012.
PCPs who treated children age 0 to 21 in New York State (n~7,000) and were members of the American Academy of Pediatrics or the American Academy of Family Practitioners (about 70%) were recruited through outreach from those organizations. Of the 178 voluntary providers who were trained in CAP PC during our timeframe, 176 were found in the Medicaid files and included in these analyses. PCPs’ practice geographic regions were categorized as New York City/Long Island (downstate, 33%) vs. the rest of New York State.
A comparison sample was created by randomly selecting 200 pediatric PCPs who were active in Medicaid during the study timeframe and were exposed to the same recruitment as the trained PCPs, but did not participate in any PT training. They were matched to PT providers by region with 28.5% of the comparison PCPs practicing downstate.
Children who saw the trained and comparison providers were included if they were continuously enrolled in Medicaid during the pre- or post-training period. There were a total of 21,784 unique children served by the PT-trained PCPs and 46,607 served by the comparison PCPs.
All data were extracted from the NYS Medicaid system. We focused on prescription of psychotropic medication, diagnoses, and antidepressant use, and follow-up care among those with a depression diagnosis because of the high public health relevance of these issues for children and adolescents 3, 10-15
Prescription of psychotropic medications included ADHD medications, antidepressants, antipsychotics, anxiolytics, alpha agonists and mood stabilizers. Antipsychotic polypharmacy was defined as use of 2 or more antipsychotic medications during the pre- or post-training period.
Diagnosis of depression or bipolar disorder was determined following DSM IV definitions. Antidepressant use and follow-up services were assessed among those with a depression diagnosis within 90 days of the PCP visit. Follow-up services included psychiatric inpatient hospitalization, psychiatric emergency room visits and mental health outpatient service use.
The need for Institutional Review Board (IRB) Review was waived by the New York State Office of Mental Health (OMH) IRB. Medicaid data files from the OMH database were analyzed to compare the variables of interest in the pre-training period to the post-training period, and between the trained and comparison providers. PT-trained providers were identified using their national provider identifiers, prescribing license number and provider specialty. Children were de-duplicated with their Medicaid identification numbers. The pre-training period was defined as all initial service claims between April 1, 2009 and March 31, 2010, with an additional 90 days for follow-up services (June 30, 2010); the post-training period was defined as March 1, 2012 to February 28, 2013, with an additional 90 days for follow-up services (June 1, 2013).
Chi-square analyses were conducted for within practitioner group comparisons. Cochran-Mantel Hanzel statistics were used to examine outcomes over time, controlling for practitioner group. Differences were considered to be statistically significant at p<.05. All analyses used SAS software, Version 9.2 (Cary, North Carolina).
RESULTS
Pre-training, PT-trained providers prescribed psychotropic medication more frequently than comparison providers (9% vs. 4%, p<.001). However, PT-trained providers prescribed multiple antipsychotic medications (polypharmacy) about 5 times less often than the comparisons (1% vs. 4%, p<.001). PT-trained providers were also slightly more likely to diagnose depression and bipolar disorder. (Table 1)
Table 1.
PT-trained PCPs | Comparison PCPs | Comparison of changes | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Pre-training | Post-training | P | Pre-training | Post-training | P | P | |||||
N | % | N | % | N | % | N | % | ||||
Children prescribed psychotropic medication | 1,886 | 9 | 3,335 | 12 | <.001 | 1,861 | 4 | 2,676 | 5 | <.001 | <.001 |
Children prescribed more than 1 antipsychotic medication (antipsychotic polypharmacy) | 17 | 1 | 21 | 1 | .184 | 67 | 4 | 45 | 2 | <.001 | <.001 |
Children diagnosed with depression | 105 | 1 | 246 | 1 | <.001 | 138 | .3 | 180 | .4 | .060 | <.001 |
Children diagnosed with bipolar disease | 47 | .2 | 41 | .2 | .080 | 52 | .1 | 38 | .1 | .083 | <.001 |
The percentage of children prescribed a psychotropic medication increased among the PT-trained group from 9% pre-training to 12% post-training (p<.001). Prescribing also went up in the comparisons, from 4% to 5% (p<.001), but a comparison between the two changes showed a larger increase among the PT-trained PCPs (p<.001). Antipsychotic polypharmacy did not significantly decrease among the PT-trained group, although it did decrease significantly among the comparisons (4% to 2%, p<.001).
Although statistical significance was found in changes in depression and bipolar diagnoses, they were too small to be of clinical significance (Table 1). When we looked at changes among children who were diagnosed with depression, we found no differences in receipt of antidepressants, psychiatric inpatient follow-up services, mental health-related emergency room use or outpatient mental health services, although the data trended in a direction suggesting a potential intervention effect. For example, outpatient mental health service use increased among children who saw the PT-trained providers (85% to 94%, p=.009) and no such change was observed among the comparison group, but there was no statistical difference between the changes in each group.
DISCUSSION
Our study suggests potential benefits of training PCPs to identify and treat child mental health conditions. After training and consultation, we saw an increase in psychotropic medication use among PT-trained providers, compared to non-trained providers. Fewer changes were seen in depression and bipolar diagnoses, and medication use and follow-up care among those with a depression diagnosis, but the data trended in a direction suggesting an intervention effect. Provider training and consultation may be a meaningful way to help reduce the number of children who go untreated for mental health conditions, but further research is necessary to determine if this type of model will be useful as the responsibility for mental health care and outcomes shifts under health care reform.
A large part of the PT training was focused on the proper prescription of psychotropic medications for children, and our data suggest that the training may have helped providers become more comfortable prescribers: we saw a greater increase in psychotropic prescriptions among the trained, compared to non-trained, PCPs. Unfortunately, our analyses could not clarify whether the increase in prescription of psychotropic medications reflects more appropriate care. There was a large decline in polypharmacy among the comparison PCPs, probably due to the considerable attention to polypharmacy in the lay and professional literature in recent years. There was no such change among the PT-trained PCPs, whose baseline rate of polypharmacy was already five times lower than the comparison PCPs.
Our data indicate that there were differences between providers who volunteered for this intervention and those who did not. Participating PCPs had smaller practice volumes and served more youth with psychiatric diagnoses. Recruitment from large practices was challenging, and new methods, such as offering web-based versions of the in-person training, should be tried to improve the participation rates from busy offices. Further, it seems that those who participated and those who did not had different diagnostic and prescribing practices. Unfortunately, we were not able to control for any other provider differences in these analyses, and it may be that the providers who volunteered were already attuned to mental health diagnosing and prescribing issues (e.g., polypharmacy), and the training provided them the extra confidence they needed. How well this same training would perform on a different population of PCPs is unknown and should be tested.
This study is not without limitations. First, our sample was too small to enable robust analyses among subgroups, such as those diagnosed with depression. This may be why we did not see statistical changes in some of our analyses that trended in a positive direction. Second, the providers participated in PT voluntarily and those who participated were a small portion of all providers in the state, which biased our sample and limits its generalizability; the impact of PT on providers’ knowledge and behaviors should be tested using a randomized trial to avoid the biases inherent when using a small sample of volunteers. Further, there was a limited amount of data available on the providers for this analysis. If future studies are not randomized, investigators should consider collecting additional data, such as provider- and practice-specific characteristics, on both intervention- and comparison-providers. This would allow for the creation of a more similar comparison group. Third, we were limited by design issues and the outcomes data available to us in the Medicaid database. Future research should collect and analyze data on specific outpatient services used by patients, including the types of non- pharmacological treatments utilized, the distribution of medications used among children with specific diagnoses and whether diagnoses were newly determined after training. This would allow researchers to identify the appropriateness of care, and specify hypotheses a priori. Finally, our analyses only included providers who were reimbursed by Medicaid and thus cannot be generalized to providers who are paid by private sources.
CONCLUSIONS
Primary care is an important setting for detection and treatment of pediatric psychiatric conditions. Further research is needed to ascertain the impact of different training and consultation models. This study suggests that the PT model may be helpful in changing providers’ behaviors, particularly around prescribing medication. As a result of the small changes shown here, the New York State OMH will be conducting a more formal PT evaluation to address some of the limitations described above. More extensive testing is needed to have confidence in its value for broader improvements of primary care practice.
Acknowledgments
CAP PC Work Group: David Kaye and Beth Smith, University at Buffalo; Rachel Zuckerbrot, Columbia University; Carmel Foley and Victor Fornari, Hofstra North Shore/LIJ; Michael Scharf and Jim Wallace, University of Rochester; Wanda Fremont and Terri Hargrave, Upstate Medical University/Syracuse
Financial support: National Institute of Mental Health P30MH090322
Footnotes
Disclosures: None for any author
Previous presentation: The data presented in this manuscript were presented at the National Institute of Mental Health's Mental Health Services Research meeting in Bethesda, Maryland on April 25, 2014.
Contributor Information
Bonnie D Kerker, New York University School of Medicine Child and Adolescent Psychiatry New York; The Nathan S. Kline Institute for Psychiatric Research Orangeburg New York Bonnie.kerker@nyumc.orgBonniekerker@yahoo.com.
Ka Ho Brian Chor, New York University School of Medicine - Department of Child and Adolescent Psychiatry New York, New York.
Kimberly Hoagwood, New York University School of Medicine - Department of Child and Adolescent Psychiatry New York, New York New York State Office of Mental Health Albany, New York.
Marleen Radigan, New York State Office of Mental Health - Office of Performance Measurement and Evaluation Albany, New York.
Matthew Perkins, New York State Office of Mental Health - Division of Children and Family Services Albany, New York.
Jade Setais, New York State Office of Mental Health - Office of Performance Measurement and Evaluation Albany, New York.
Rui Wang, New York State Office of Mental Health - Office of Performance Measurement and Evaluation Albany, New York.
S. Serene Olin, New York University School of Medicine - Department of Child and Adolescent Psychiatry New York, New York.
Sarah McCue Horwitz, New York University School of Medicine - Department of Child and Adolescent Psychiatry New York, New York.
REFERENCES
- 1.Costello EJ, Mustillo S, Erkanli A, et al. Prevalence and development of psychiatric disorders in childhood and adolescence. Archives of General Psychiatry. 2003;60:837–844. doi: 10.1001/archpsyc.60.8.837. [DOI] [PubMed] [Google Scholar]
- 2.Briggs-Gowan MJ, Horwitz SM, Schwab-Stone ME, et al. Mental health in pediatric settings: Distribution of disorders and factors related to service use. Journal of the American Academy of Child and Adolescent Psychiatry. 2000;39:841–849. doi: 10.1097/00004583-200007000-00012. [DOI] [PubMed] [Google Scholar]
- 3.Merikangas KR, He J, Burstein M, et al. Lifetime prevalence of mental disorders in U.S. adolescents: Results from the national comorbidity survey replication – adolescent supplement (NCS-A). Journal of the American Academy of Child and Adolescent Psychiatry. 2010;49:980–989. doi: 10.1016/j.jaac.2010.05.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Kataoka SH, Zhang L, Wells KB. Unmet need for mental health care among U.S. children: Variation by ethnicity and insurance standing. The American Journal of Psychiatry. 2002;159:1548–1555. doi: 10.1176/appi.ajp.159.9.1548. [DOI] [PubMed] [Google Scholar]
- 5.Jensen PS, Goldman E, Offord D, et al. Overlooked and underserved: “Action signs” for identifying children with unmet mental health needs. Pediatrics. 2011;128:970–979. doi: 10.1542/peds.2009-0367. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Williams J, Klinepeter K, Palmes G, et al. Diagnosis and treatment of behavioral health disorders in pediatric practice. Pediatrics. 2004;114:601–606. doi: 10.1542/peds.2004-0090. [DOI] [PubMed] [Google Scholar]
- 7.Horwitz SM, Kelleher KJ, Stein REK, et al. Barriers to the identification and management of psychosocial issues in children and maternal depression. Pediatrics. 2007;119:e208–e218. doi: 10.1542/peds.2005-1997. [DOI] [PubMed] [Google Scholar]
- 8.Horwitz SM, Caspary G, Storfer-Isser A, et al. Is developmental and behavioral pediatrics training related to perceived responsibility for treating mental health problems? Academic Pediatrics. 2010;10:252–259. doi: 10.1016/j.acap.2010.03.003. [DOI] [PubMed] [Google Scholar]
- 9.The Resources for Advancing Children's Health 2012 Retrieved from www.thereachinstitute.net.
- 10.Kessler RC, Walters EE. Epidemiology of DSM-III-R major depression and minor depression among adolescents and young adults in the national comorbidity survey. Depression and Anxiety. 1998;7:3–14. doi: 10.1002/(sici)1520-6394(1998)7:1<3::aid-da2>3.0.co;2-f. [DOI] [PubMed] [Google Scholar]
- 11.Brent DA, Perper JA, Moritz G, et al. Psychiatric risk factors for adolescent suicide: A case-control study. Journal of the American Academy of Child and Adolescent Psychiatry. 1993;32:521–529. doi: 10.1097/00004583-199305000-00006. [DOI] [PubMed] [Google Scholar]
- 12.Rapoport JL. Pediatric psychopharmacology: Too much or too little? World Psychiatry. 2013;12:118–123. doi: 10.1002/wps.20028. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Koelch M, Schnoor K, Fegert JM. Ethical issues in psychopharmacology of children and adolescents. Current Opinion in Psychiatry. 2008;21:598–605. doi: 10.1097/YCO.0b013e328314b776. [DOI] [PubMed] [Google Scholar]
- 14.Gould MS, Greenberg T, Velting DM, et al. Youth suicide risk and preventive interventions: A review of the past 10 years. Journal of the American Academy of Child and Adolescent Psychiatry. 2003;42:386–405. doi: 10.1097/01.CHI.0000046821.95464.CF. [DOI] [PubMed] [Google Scholar]
- 15.Shaffer D, Gould MS, Fisher P, et al. Psychiatric diagnosis in child and adolescent suicide. Archives of General Psychiatry. 1996;53:339–348. doi: 10.1001/archpsyc.1996.01830040075012. [DOI] [PubMed] [Google Scholar]