Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Nov 5.
Published in final edited form as: Psychiatr Serv. 2015 Jul 1;66(11):1141–1148. doi: 10.1176/appi.ps.201400315

Effect of Behavioral Health Screening and Co-located Services on Ambulatory and Inpatient Uitilization

Karen Hacker 1, Robert Penfold 2, Lisa Arsenault 3, Fang Zhang 4,5, Stephen B Soumerai 6,7, Lawrence S Wissow 8
PMCID: PMC4633707  NIHMSID: NIHMS693637  PMID: 26129994

Abstract

Objective

The study sought to determine the impact of a pediatric behavioral health (BH) screening and co-location model on BH care utilization

Methods

In 2003, Cambridge Health Alliance, a Massachusetts public health system introduced BH screening and co-location of social workers within its pediatric practices in a sequential manner. An interrupted time series study of the change in trends of ambulatory, emergency and inpatient BH utilization in the 30 months following model implementation compared to the 18 months prior was conducted to determine the impact of this model on BH care utilization, Utilization data on 11,223 children ≥4 years 9 months to < 18 years 3 months seen from 2003 to 2008, contributed to the study.

Results

In the 30 months following implementation of pediatric BH screening and co-location there was a 20.4% cumulative increase in specialty BH visit rates (trend = 0.013% per month; p=0.049), and 67.7% cumulative increase in BH primary care visit rates (trend = 0.019% per month; p=0.002) compared to the expected rate predicted by the 18 month pre- intervention trend. In addition, BH emergency department visit rates increased 245% compared to the expected rate (trend = 0.01% per month; p<.001).

Conclusions

Following the implementation of a BH screening/co-location model, more children received BH treatment. Contrary to expectations, BH emergency department visits also increased. Further study is needed to determine if this is an effect of how care was organized for children newly engaged in BH care or a reflection of secular trends in BH utilization or both.

Background

Child behavioral health (BH) issues present frequently in primary care settings.(1) Screening for BH issues is promoted in national guidelines as a strategy for early identification and treatment of BH conditions.(1, 2) Simultaneously, these recommendations include a variety of possible mechanisms for increasing the capacity of primary care to respond to these issues, including task-shifting, BH screening, collaborative care, and co-location of BH and physical health services in the same location. Trials of various integrated models (screening, co-location and collaboration) demonstrate improved mental health outcomes (3-5), provider satisfaction, and identification rates.

In studies of screening alone, findings suggest that identification rates increase (6) as do mental health referrals.(7, 8) Unfortunately, referral completion rates remain low with studies reporting rates ranging from 17% to 45%.(7-10) In a recent study by the authors, using Medicaid claims data, only 30% of newly identified children utilized BH services.(11) However, studies note higher BH initiation rates (>80%) with adult co-located/collaborative models.(12-14) Co-location reduces the stigma associated with seeking BH care and logistical barriers for patients and specialist-primary care collaboration.(15). While studies have examined the impact of co-location on BH initiation and clinical outcomes, few have examined the impact on primary care related BH visits, inpatient or emergency department mental health services. These are important effects to explore because models for financing integrated care frequently call for “offsets” or “shared savings” attributable to either reductions in inpatient, urgent care, and “masked” mental health presentations or shifting of specialty services to lower-costs settings such as primary care.

The goal of the current study is to understand how a child BH screening and co-location program impacts health care utilization; including primary care, specialty BH care, and BH-related emergency and inpatient utilization.

Methods

Conceptual Framework

Our underlying conceptual framework assumes that as screening increases, patients are identified and enter either specialty BH services or BH services in primary care settings. This results in increased ambulatory BH services. If early mental health treatment proves efficacious, inappropriate emergency department and inpatient BH admissions should decrease over time. To test our assumptions, we used data from the Cambridge Health Alliance's pediatric clinics to capitalize on a natural experiment. CHA clinics phased in the use of a validated screening tool during well-child visits from 2004-2007. Using data from the CHA data warehouse (16, 17) we conducted an interrupted time series analysis of utilization rates in the months pre and post the BH screening/co-location program implementation among a rolling cohort of primary care pediatric patients receiving care. The CHA Institutional Review Board approved the study in 2011.

Context

Cambridge Health Alliance is a public hospital and clinic network in Cambridge Massachusetts. At the time of this study, CHA operated three acute care hospitals with three emergency departments, two child mental health units, an inpatient pediatric medical unit, and multiple ambulatory health clinics as well as a large division of child and adolescent psychiatry. All ambulatory clinics used the same inpatient and outpatient psychiatric resources located <1 mile away. In 2004, CHA began screening children ≥4 years 9 months (5th year well-child visit) to < 18 years using the Pediatric Symptom Checklist (PSC) (18) (for those <14) and the Youth-PSC (Y-PSC) (18) (for those ≥14) in three of its largest pediatric sites. Screening occurred at the annual well-child visit for children in the age range. These sites treated almost 16,000 children (0-18 years) a year during 2004-2008 (Cambridge Pediatics 6,672, Somerville Pediatrics 7,267, and Union Square 1,452). There were 24 providers at these sites (Cambridge 7, Somerville 7, and Union Square 10) and none had used BH screens or co-located social workers prior to project implementation. Sixty-four percent of children had a minority race/ethnicity (Cambridge 61%, Somerville 55%, and Union Square 70%); and 43% spoke a language other than English.(19) As noted elsewhere, the majority of mental health referrals for primary care patients were made to CHA providers.(20)

During the study timeframe, many changes occurred nationally in the delivery of child and adolescent mental health including the black box warning on the use of antidepressants (21), rising rates of specific disorders (i.e. bipolar disorder) (22), and higher use of antipsychotic medications (23). There was also heightened interest in identifying and treating BH issues in pediatric offices.(24)

Screening and Co-location Model

The BH screening and co-location program (still in existence today) was initially implemented at CHA in a phased in manner; Cambridge began screening in December 2003, followed by Somerville in July 2005, and Union Square in April 2007. The PSC/YPSC are used as screening tools since they are well validated in diverse urban populations (6, 25) and no cost to providers. PSC results have compared favorably to the Child Behavior Checklist and the Children's Global Assessment Scale.(26) These 35 item instruments are completed in the waiting room by either parent or teen prior to the well-child visit. The physician scores the screen during the visit and later the medical assistant inputs the information into the electronic medical record (EMR). This also automates provider reporting of screening rates. This process is described indepth elsewhere.(10) Screening was not reimbursable until January 2008 as part of the Rosie D vs. Patrick remedy.(27)

The co-location of BH providers in the clinics happened simultaneously with screening implementation. At each site, a part-time licensed clinical social worker (4 total), supervised by CHA Division of Child and Adolescent Psychiatry, conducted on-site BH services including.initial assessments, therapy and consultation to pediatricians..This collaborative care model is similar to Kolko and Perrin's description of on-site interventions with aspects of care coordination (24). Providers made BH referrals to child psychiatry and based on patient preference, services were conducted by on-site social workers or at the child psychiatry main office.

Prior to program implementation primary care providers and social workers were trained at each site in the use of the PSC/YPSC tools and relevant BH diagnosis codes which were already being reimbursed in MA. Regular reports on screening were generated and shared with the clinical staff monthly. Within 6 months of starting the program, each of the sites was able to increase the share of well-child visits with a screen for children in the age range to over 50%/month

Study Population

We extracted clinical and demographic data for youth aged ≥4 years 9 months to < 18 years 3 months seen between 2003 and 2008. These ages include “screenable” youth and allow for sufficient time post-screening to capture any subsequent utilization. The CHA data warehouse which contains all ambulatory encounters within CHA, emergency department and inpatient hospitalizations at a CHA hospital; and includes demographic data such as gender, race and language.(17) We extracted all encounter and inpatient data including location of service, provider department (pediatrics, family medicine), date of service, CPT4 procedure codes and ICD9 diagnosis codes for each visit. To calculate monthly utilization rates we identified patients for inclusion in the denominator (patient panel) based upon the location of their well-child and ambulatory care visits (defined by location and CPT code) and eligible screening age. Any child without a primary care visit at the site in the prior year was excluded from the denominator for the given month. Children who had well-child care at multiple CHA sites were likely to have experienced the impact of the intervention prior to other sites beginning to screen. Since this would have caused contamination, we excluded them (n=1900; 14.7%) from both the pre and post index date panels. This group was significantly different on all demographic variables from the rest of the sample (P<.001). They were more likely to be female (55% vs. 49%), to speak other languages (2% vs. 5%), to be Black (16% vs. 12%), to be under 7 years of age (19% vs. 15%) between 16-18 years (18% vs. 12%) and less likely to be Hispanic (18% vs. 21%). Demographic data on children included in the denominator of any study month was generated and aggregated by pre-policy and post-policy periods for reporting purposes.

The period of patient panel identification and the calendar date of implementation (the index date) was different for each of the 3 sites. The denominator for the time series was the patient panel counts in each month. The pre-period time frame (18 months) was limited by data availability but allowed for a stable baseline and control for seasonality. For the post-period, the longest time frame possible (30 months) was used. We initially constructed stratified time series analysis of monthly utilization using the pre and post time frames for each site. We also conducted a sensitivity analysis with and without Union Square given it spanned the Rosie D remedy (January 2008). No significant differences in individual site analyses or in our sensitivity analysis were found and thus data for the three sites was merged and centered on the index date.

Variables

We collected utilization data for each month of the observation period. The primary outcomes of interest were defined as follows: Specialty BH Services – ambulatory psychiatric services delivered by BH clinicians either co-located in the sites or in the department of child and adolescent psychiatry; BH-related Primary Care Visits - well-child visits or other ambulatory visits with an associated MH related ICD9 code delivered by primary care providers; BH-related Hospitalizations- inpatient hospitalizations occurring in the CHA inpatient psychiatry unit or under the specialty of inpatient psychiatry; BH-related Emergency Department Visits - visits occurring in the CHA psych emergency department or in any CHA emergency department with an associated MH related ICD9 code.

Demographic variables included sex, age, race/ethnicity (white, black, Hispanic, Asian, other, or unknown), and language of care (English, Portuguese, Spanish, Haitian, other, or unknown). Race and language is self-reported and entered into the medical record by CHA clerical staff. Age and race/ethnicity were dichotomized (under 13 years vs. 13 years or older and white, non-Hispanic vs. all other races) in order to standardize utilization rates in preparation for Interrupted Time Series.

Interrupted Time Series Analysis (ITS)

We used segmented regression models (28, 29) to evaluate the effect of the simultaneous screening and co-location model on population utilization rates. ITS is the strongest quasi experimental design for evaluating the effects of natural experiments.(30) One threat to the validity of the ITS design is changes in the composition of the population. To account for some change in the clinic demographics (i.e. age, sex and race/ethnicity) over time, we standardized (31, 32) the distribution of demographics in each month to that in month 1 at each site. After standardization, we adjusted for seasonality.(33) The segmented models included terms for the intercept, pre-screening slope, change in level, change in slope and time as well as autoregressive terms for up to 6 month lags in rates. Because clinics took approximately 6 months to ramp-up screening, we censored the 6 months immediately following the BH intervention as has been done elsewhere.(34) All analyses were conducted using SAS version 9.3 (SAS Institute, Cary NC).(35)

Results

The total unique number of children included in the study was 11,223 and the number of children appearing in the denominator for any given month ranged from 6,833 to 7,281. Forty percent of children appeared in the dataset at least once in each of the 4 years of the study period and 59% appeared in both the pre and post policy periods. In the post-policy period, the population was 51% male, 56% minority largely English speakers, and well distributed across age groups (Table 1).

Table 1.

Demographics

CHA Pediatric Primary Care Patients (3 Sites) N=11,223
Pre Policy Post Policy
N % N %
Sex
    Female 4,032 49.0 4,641 48.6
    Male 4,203 51.0 4,911 51.4
Language
    English 6,475 78.6 7,392 77.4
    Portuguese 758 9.2 1,069 11.2
    Spanish 342 4.2 487 5.1
    Haitian 177 2.2 254 2.7
    Other 409 5.0 323 3.4
    Unknown 74 0.9 27 0.3
Race/ethnicity
    White, NH 3,795 46.1 4,175 43.7
    Black, NH 1,734 21.1 2,075 21.7
    Hispanic 1,014 12.3 1,257 13.2
    Asian 384 4.7 537 5.6
    Other, NH 1,095 13.3 1,315 13.8
    Unknown 213 2.6 193 2.0
Age, at start of the time perioda
    under 4.8 years 603 7.3 1,243 13.0
    4.8 to 6.9 years 1,219 14.8 1,330 13.9
    7 to 9.9 years 1,756 21.3 1,771 18.5
    10 to 12.9 years 1,794 21.8 1,862 19.5
    13 to 15.9 years 1,717 20.9 2,083 21.8
    16 to 18.0 years 1,008 12.2 1,080 11.3
    over 18.0 years 138 1.7 183 1.9
Insurance, at policy implementationb
    Medicaid 2,451 29.8 3,587 37.6
    Free Care/Public 1,218 14.8 1,736 18.2
    Private 3,035 36.9 3,579 37.5
    Other 41 0.5 50 0.5
    Not Indicated 1,490 18.1 600 6.3
First behavioral health diagnosis at specialty BH visits (n=3,382 pre/n=6,123 post)c
    Episodic Mood Disorder (bipolar) 849 25.12 1249 20.6
    Hyperkinetic Syndrome 833 24.6 1471 24.1
    Adjustment Reaction 631 18.7 1691 27.7
    Anxiety, dissociative and somatoform disorder 579 17.1 782 12.8
Open in a new tab
a

Pre-policy age calculated as of beginning of pre-policy period, post-policy age calculated as of beginning of post-policy period

b

Insurance category was based on the type indicated at the most proximal encounter to policy implementation

c

These are the top four diagnoses based on the fist diagnostic code at the first specialty BH visit occurring during the time period.

Total N: 8,235 pre-policy; 9,552 post-policy. Patients appearing in both pre and post policy periods; N=6,630 (59.1%)

Figures 1-3 show the baseline and post-screening/co-location rates of our various outcomes. Table 2 shows the results of the segmented regression analysis. There was a significant increase in rates of specialty BH visits, BH-related primary care visits, and BH-related emergency department visits during the 30 months post-screening/co-location period . The trend in specialty BH visits increased .013% per month (slowly initially and accelerating over time) from approximately 1.7% per month to almost 2.5% per month (p=.049) a cumulative increase of 20.4% compared to expected. In other words, by the end of the post implementation observation period, the rate of BH visits was 20% higher than expected based on pre-program trends. Similarly, the trend in BH-related primary care visits increased by .019% per month from approximately 1% to 1.5% per month (p=.002), a cumulative increase of 67.7% compared to expected. Finally, the trend in BH-related emergency department visits increased .01% per month from 1.5/1,000 to almost 3/1,000 (p< .001) a cumulative increase of 245% compared to expected. BH-related inpatient admissions at CHA remained stable (data not shown)

Figure 1.

Figure 1

Open in a new tab

Specialty Behavioral Health Visits

Figure 3.

Figure 3

Open in a new tab

Behavioral Health-Related Emergency Department Visits

Table 2.

Results of Segmented Regression Analysis

Estimate (Visits per 1,000 Patients) Standard Error 95% Confidence Interval P-Value
Specialty Behavioral Health Servicesa
    Intercept 14.99 .51 13.96 to 16.02 <.001
    Trend Change .13 .06 .00 to .25 .049
    Trend .07 .04 −.01 to .16 .099
Behavioral Health Related Primary Care Visitsb
    Intercept 8.48 .32 7.84 to 9.12 <.001
    Trend Change .19 .02 .15 to .23 <.001
Behavioral Health Related Emergency Department Visitsc
    Intercept 1.42 .26 .89 to 1.95 <.001
    Trend Change .10 .03 .04 to .16 .002
    Trend −.03 .02 −.08 to .01 .144
Behavioral Health Related Inpatient Hospitalizationsd
    Intercept .32 .12 .08 to .57 .012
    Level Change −.47 .32 −1.12 to .18 .154
    Trend Change .04 .02 .01 to .07 .019
Behavioral Health Related Inpatient Hospitalizations or Emergency Department Visitsc,d
    Intercept 1.47 .29 .88 to 2.05 <.001
    Trend Change .11 .03 .04 to .18 .003
    Trend −0.03 .02 −.08 to .01 .157
Any Behavioral Health Related Utilizationa,b,c,d
    Intercept 23.93 .32 23.29 to 24.57 <.001
    Level Change 1.50 .90 −.30 to 3.31 .102
    Trend Change .36 .04 .28 to .45 <.001
Non-Behavioral Health Related Utilizatione
    Intercept 83.70 1.00 81.68 to 85.71 <.001
    Trend .06 .03 −.01 to .13 .072
Open in a new tab
a

Psychiatric services such as diagnostic interviews, psychopharmacology management and psychotherapy (CPT codes 90801-90899), health behavioral assessment and intervention services (CPT codes 96100-96103, 96105,96111, 96115-96120, 96125, 96150-96155), Healthcare Common Procedural Codes (HCPC) codes for other mental health professionals (All H-codes) and the Massachusetts Behavioral Health Partnership's (MBHP) additional HCPC codes introduced in Massachusetts to track CBHI remedy services including crisis intervention (S9484 and S9485) and family counseling and case management (T1027, T1017 and T2022).

b

Well-child visits (CPT Code 99383--99383, 99393--99395) or ambulatory visits (CPT Code 99201--99205, 99211--99215, 99401--99404, 99241--99245) with an associated behavioral health related ICD9 code (290 to 319).

c

Inpatient hospitalizations defined by specialty (psychiatry inpatient) and/or location (inpatient psychiatry unit)

d

Emergency department visits defined by specialty (psych emergency or ED) with an associated behavioral health related ICD9 code (290-319)

e

Inpatient hospitalizations, emergency department visits, well-child visits or ambulatory visits not meeting the above criteria

Discussion

In this local study of a pediatric BH screening/co-location program, rates of BH utilization, including both specialty and primary care, increased significantly after implementation of the program. While BH-related inpatient admissions were stable, there was also a significant increase in rates of emergency admissions with BH diagnoses. To our knowledge, this is the first study to report increases in emergency department use associated with screening and co-location for a pediatric population.

As noted in the literature, BH screening in primary care leads to increased rates of referral by primary care providers (8) particularly when there are mental health services co-located in the same practice.(14) In this study, utilization of specialty BH services increased significantly suggesting that referrals were completed and more children with BH needs were getting into treatment. However, our data does not include clinical outcome measures, thus, we cannot determine whether more care is actually better care. It is also important to note that, the pre-policy BH care utilzation was higher than reported elsewhere for Massachusetts statewide. (15% versus 12%).(36) This may reflect access to available child psychiatry. In addition, nationally, outpatient pediatric visits with mental health diagnoses were increasing from 1999-2010 but there were no increases identified simultaneous to CHA program implementation.(37)

The greater increase in CHA's BH-related primary care visits in comparison to the more limited increase in specialty BH services mimics the national trends seen over the last decade. (37) This is encouraging given pediatrician's limited knowledge of BH codes, discomfort with BH treatment and concerns about reimbursement (38, 39) It suggests that screening/co-location may influence task-shifing of BH treatment from specialty to primary care. Unfortunately, our data does not reveal whether primary care providers were actually delivering BH treatment or simply more comfortable with coding.

Perhaps the most surprising study finding is that emergency department visit rates increased following the intervention (the rate doubled but the number remained low). We suspect that many of these visits were avoidable.(40) There are many possible reasons for this increase. For example, increased screening may identify issues that cannot be adequately triaged or managed on site requiring emergency visits. Also, providers increasing awareness of emergent mental health issues may increase emergency referrals. In the recent study of Oregon Medicaid expansion, both primary care and emergency department utilization increased initially.(41) Further, children and families entering treatment may experience BH crises. Therapists themselves could increase utilization by telling patients to go to the emergency department if crises occur in their absense. Unfortunately, we do not know whether crises increased following the intervention. We also recognize that the upwards trend in BH-related emergency visits (42-44) nationally might have contributed to our findings, but this is unlikely to have happened simultaneously to the intervention at each of our sites. To understand the validity of these explanations additional provider level data and a longer time frame are required to determine whether this phenomenon is merely a short-lived consequence of improving BH identification.

Despite increased emergency utilization, we did not find a concomitant increase in BH-related inpatient admissions which is surprising since most would emanate from emergency department visits. But, we recognize that CHA patients are not necessarily sent to CHA facilities for inpatient BH admissions.Children are sent to open beds throughout the region. Thus it is possible that inpatient utilization increased without our knowledge.

Limitations

The current study has a number of limitations. First, the study lacked information on BH need and clinical outcomes required to assess clinical improvement. Also, since the study occured at one delivery system and excluded children who used primary care at multiple CHA sites, it may not be representative of all CHA children or other non-urban populations. However, while this may limit generalizability, it would not explain the sudden slope changes in outcomes. Second, because data were collected from the CHA data warehouse, visits that took place outside CHA were incomplete and the accuracy of diagnostic codes could not be validated. However, because our data shows increases in utilization it is likely that we are underestimating the impact of the program to the degree that outside utilization is not captured. Third, we were unable to identify any change in BH hospitalization due to a lack of statistical power. A claims-based study that captures inpatient utilization more completely would help to clarify these findings. Fourth, the study relied on one validated screening tool (PSC) and it is possible that results might have differed with other instruments. Fifth, since the study examined the impact of a combined screening and co-location model, we are unable to estimate the independent impact of screening and co-location. We also lacked a concurrent comparison group. Finally, other factors such as staffing changes, EMR adoption and coding awareness may have influenced our findings. However, none took place concurrently with the implementation of screening and co-location at each CHA site and thus would not explain our findings. Further, our ITS design explicitly controls for national, secular trends impacting child and adolescent mental health delivery (24, 37, 44, 45).

Conclusion

This is the first study of its kind to examine health care utilization rates after the initiation of a BH screening and co-location model using an ITS design. There is strong evidence that the intervention led to increased use of BH services and provider identification of BH issues - both of which are positive developments. However, the sharp two-fold increase in admissions rates to the emergency department is concerning given the assumption that early identification of mental health issues and appropriate outpatient care should decrease emergency department utilization. More work is required to understand this unanticipated outcome.

Figure 2.

Figure 2

Open in a new tab

Behavioral Health-Related Primary Care Visits

Acknowledgments

Grant Support: All phases of this study were supported by National Instittute of Mental Health (NIIMH) grant R21MH094942- Exploring the Impact of Pediatric Behavioral Health Screening on Health Care Use and grant U19MH092201 – Mental Health Research Network: A Population-Based Approach to Transform Research.

Footnotes

Disclosures: None of the authors have any conflicts of interest or financial support to disclose.

Contributor Information

Karen Hacker, Allegheny County Health Department, Pittsburgh, Pennsylvania, KHacker@achd.net.

Robert Penfold, Group Health Research Institute - Department of Health Services Research, Seattle, Washington.

Lisa Arsenault, Institute for CommunityHealth, Cambridge, Massachusetts.

Fang Zhang, Harvard Pilgrim HealthCare Institute, Boston, Massachusetts; Harvard Medical School - Department of Population Medicine, 133 Brookline Ave 6th Floor 4, Boston, Massachusetts 02215.

Stephen B Soumerai, Harvard Medical School - Department of Population Medicine, Boston, Massachusetts; Harvard Pilgrim HealthCare Institute, Boston, Massachusetts.

Lawrence S Wissow, Johns Hopkins Bloomberg School of Public Health - Department of Health Policy and Management.

References

  • 1.Foy JM. Enhancing pediatric mental health care: report from the American Academy of Pediatrics Task Force on Mental Health. Introduction. Pediatrics. 2010;125(Suppl 3):S69–74. doi: 10.1542/peds.2010-0788C. [DOI] [PubMed] [Google Scholar]
  • 2.Appendix S4: The Case for Routine Mental Health Screening. Pediatrics. 2010;125:S133–S9. [Google Scholar]
  • 3.Kolko DJ, Campo J, Kilbourne AM, et al. Collaborative care outcomes for pediatric behavioral health problems: a cluster randomized trial. Pediatrics. 2014;133:e981–92. doi: 10.1542/peds.2013-2516. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Wissow LS, Gadomski A, Roter D, et al. Improving child and parent mental health in primary care: a cluster-randomized trial of communication skills training. Pediatrics. 2008;121:266–75. doi: 10.1542/peds.2007-0418. [DOI] [PubMed] [Google Scholar]
  • 5.Kolko DJ, Campo JV, Kilbourne AM, et al. Doctor-office collaborative care for pediatric behavioral problems: a preliminary clinical trial. Arch Pediatr Adolesc Med. 2012;166:224–31. doi: 10.1001/archpediatrics.2011.201. [DOI] [PubMed] [Google Scholar]
  • 6.Jellinek MS, Murphy JM, Little M, et al. Use of the Pediatric Symptom Checklist to screen for psychosocial problems in pediatric primary care: a national feasibility study. Arch Pediatr Adolesc Med. 1999;153:254–60. doi: 10.1001/archpedi.153.3.254. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Chisolm DJ, Klima J, Gardner W, et al. Adolescent behavioral risk screening and use of health services. Adm Policy Ment Health. 2009;36:374–80. doi: 10.1007/s10488-009-0245-8. [DOI] [PubMed] [Google Scholar]
  • 8.Wissow LS, Brown J, Fothergill KE, et al. Universal mental health screening in pediatric primary care: a systematic review. J Am Acad Child Adolesc Psychiatry. 2013;52:1134–47. doi: 10.1016/j.jaac.2013.08.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Hacker K, Arsenault L, Franco I, et al. Referral and follow-up after mental health screening in commercially insured adolescents. J Adolesc Health. 2014;55:17–23. doi: 10.1016/j.jadohealth.2013.12.012. [DOI] [PubMed] [Google Scholar]
  • 10.Hacker KA, Myagmarjav E, Harris V, et al. Mental health screening in pediatric practice: factors related to positive screens and the contribution of parental/personal concern. Pediatrics. 2006;118:1896–906. doi: 10.1542/peds.2006-0026. [DOI] [PubMed] [Google Scholar]
  • 11.Hacker KA, Penfold RB, Arsenault LN, et al. Behavioral health services following implementation of screening in massachusetts medicaid children. Pediatrics. 2014;134:737–46. doi: 10.1542/peds.2014-0453. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Auxier A, Runyan C, Mullin D, et al. Behavioral health referrals and treatment initiation rates in integrated primary care: a Collaborative Care Research Network study. Transl Behav Med. 2012;2:337–44. doi: 10.1007/s13142-012-0141-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Kessler R. Mental health care treatment initiation when mental health services are incorporated into primary care practice. J Am Board Fam Med. 2012;25:255–9. doi: 10.3122/jabfm.2012.02.100125. [DOI] [PubMed] [Google Scholar]
  • 14.Calkins LE, Michelson IR, Corso AS. Provider proximity as a predictor of referral rate and success. Psychol Serv. 2013;10:395–400. doi: 10.1037/a0029988. [DOI] [PubMed] [Google Scholar]
  • 15.Eapen V, Jairam R. Integration of child mental health services to primary care: challenges and opportunities. Ment Health Fam Med. 2009;6:43–8. [PMC free article] [PubMed] [Google Scholar]
  • 16.Hacker KA, Williams S, Myagmarjav E, et al. Persistence and change in pediatric symptom checklist scores over 10 to 18 months. Acad Pediatr. 2009;9:270–7. doi: 10.1016/j.acap.2009.03.004. [DOI] [PubMed] [Google Scholar]
  • 17.Hacker KA, Arsenault LN, Williams S, et al. Mental and behavioral health screening at preventive visits: opportunities for follow-up of patients who are nonadherent with the next preventive visit. J Pediatr. 2011;158:666–71. e2. doi: 10.1016/j.jpeds.2010.09.059. [DOI] [PubMed] [Google Scholar]
  • 18.Massachusetts General Hospital Psychiatry. Pediatric Symptom Checklist. 2012 Accessed at http://www.massgeneral.org/psychiatry/services/psc_home.aspx.
  • 19.Cambridge Health Alliance: Patient demographic distribution, 2012-2013. Cambridge, MA: 2014. [Google Scholar]
  • 20.Hacker K, Arsenault L, Franco I, et al. Referral and Follow-Up After Mental Health Screening in Commercially Insured Adolescents. J Adolesc Health. 2014 doi: 10.1016/j.jadohealth.2013.12.012. [DOI] [PubMed] [Google Scholar]
  • 21.Lu CY, Zhang F, Lakoma MD, et al. Changes in antidepressant use by young people and suicidal behavior after FDA warnings and media coverage: quasi-experimental study. BMJ. 2014;348:g3596. doi: 10.1136/bmj.g3596. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Moreno C, Laje G, Blanco C, et al. National trends in the outpatient diagnosis and treatment of bipolar disorder in youth. Arch Gen Psychiatry. 2007;64:1032–9. doi: 10.1001/archpsyc.64.9.1032. [DOI] [PubMed] [Google Scholar]
  • 23.Olfson M, Blanco C, Liu L, et al. National trends in the outpatient treatment of children and adolescents with antipsychotic drugs. Arch Gen Psychiatry. 2006;63:679–85. doi: 10.1001/archpsyc.63.6.679. [DOI] [PubMed] [Google Scholar]
  • 24.Kolko DJ, Perrin E. The integration of behavioral health interventions in children's health care: services, science, and suggestions. Journal of clinical child and adolescent psychology : the official journal for the Society of Clinical Child and Adolescent Psychology, American Psychological Association, Division. 2014;53(43):216–28. doi: 10.1080/15374416.2013.862804. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Murphy JM, Arnett HL, Bishop SJ, et al. Screening for psychosocial dysfunction in pediatric practice. A naturalistic study of the Pediatric Symptom Checklist. Clin Pediatr (Phila) 1992;31:660–7. doi: 10.1177/000992289203101104. [DOI] [PubMed] [Google Scholar]
  • 26.Massachusetts General Hospital Psychiatry: The Pediatric Symptom Checklist, Research. 2012 Accessed at http://www.massgeneral.org/psychiatry/services/psc_research.aspx.
  • 27.Center for Public Representation. Rosie D. The Remedy: The Pathway to Home-Based Services Edited by Center for Public Representation. Northampton, MA: 2007. [January. 2015]. Reforming the Health Care System in Massachusetts. http://www.rosied.org/page-84564? [Google Scholar]
  • 28.Gillings D, Makuc D, Siegel E. Analysis of interrupted time series mortality trends: an example to evaluate regionalized perinatal care. Am J Public Health. 1981;71:38–46. doi: 10.2105/ajph.71.1.38. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Cook TD, Campbell DT. Quasi-experimentation: design and analysis issues for field settings. Houghton Miffliin Company; Boston, MA: 1979. [Google Scholar]
  • 30.Shadish W, Cook T, Campbell D. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin Company; Boston: 2002. [Google Scholar]
  • 31.Brownson RC, Petitti DB. Applied Epidemiology: Theory to Practice. Oxford University Press; New York: 1998. [Google Scholar]
  • 32.Kleinbaum DG, Kupper LL, Morgenstern H. Epidemiologic Research: Principles and Quantitive Methods. Van Nostrand Reinhold; New York, NY: 1982. [Google Scholar]
  • 33.Bobbit LG, Otto MC. Effects of Forecasts on the Revisions of Seasonally Adjusted Data Using the X-11 Adjustment Procedure. Business and Economic Statistcs Section of the American Statistical Association. 1990 [Google Scholar]
  • 34.Schneeweiss S, Maclure M, Soumerai SB, et al. Quasi-experimental longitudinal designs to evaluate drug benefit policy changes with low policy compliance. J Clin Epidemiol. 2002;55:833–41. doi: 10.1016/s0895-4356(02)00437-7. [DOI] [PubMed] [Google Scholar]
  • 35.SAS Institute Inc. SAS OnlineDoc, Version 9.3. SAS Institute Inc.; Cary, NC: 2002-2006. [Google Scholar]
  • 36.Sturm R, Ringel JS, Andreyeva T. Geographic disparities in children's mental health care. Pediatrics. 2003;112:e308. doi: 10.1542/peds.112.4.e308. [DOI] [PubMed] [Google Scholar]
  • 37.Olfson M, Blanco C, Wang S, et al. National trends in the mental health care of children, adolescents, and adults by office-based physicians. JAMA psychiatry. 2014;71:81–90. doi: 10.1001/jamapsychiatry.2013.3074. [DOI] [PubMed] [Google Scholar]
  • 38.Kelleher KJ, Campo JV, Gardner WP. Management of pediatric mental disorders in primary care: where are we now and where are we going? Curr Opin Pediatr. 2006;18:649–53. doi: 10.1097/MOP.0b013e3280106a76. [DOI] [PubMed] [Google Scholar]
  • 39.Kelleher KJ, Stevens J. Evolution of child mental health services in primary care. Acad Pediatr. 2009;9:7–14. doi: 10.1016/j.acap.2008.11.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Billings J, Parikh N, Mijanovich T. Emergency Department Use: The New York Story; in Issue Brief. The Commonwealth Fund; New York, NY: 2000. [PubMed] [Google Scholar]
  • 41.Taubman SL, Allen HL, Wright BJ, et al. Medicaid increases emergeny-department use: Evidence from Oregon's health insurance experiment. Science. 2014;343:263–8. doi: 10.1126/science.1246183. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Dolan MA, Fein JA. Pediatric and adolescent mental health emergencies in the emergency medical services system. Pediatrics. 2011;127:e1356–66. doi: 10.1542/peds.2011-0522. [DOI] [PubMed] [Google Scholar]
  • 43.Grupp-Phelan J, Harman JS, Kelleher KJ. Trends in mental health and chronic condition visits by children presenting for care at U.S. emergency departments. Public Health Rep. 2007;122:55–61. doi: 10.1177/003335490712200108. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Pittsenbarger ZE, Mannix R. Trends in pediatric visits to the emergency department for psychiatric illnesses. Acad Emerg Med. 2014;21:25–30. doi: 10.1111/acem.12282. [DOI] [PubMed] [Google Scholar]
  • 45.Olfson M, Kroenke K, Wang S, et al. Trends in office-based mental health care provided by psychiatrists and primary care physicians. J Clin Psychiatry. 2014;75:247–53. doi: 10.4088/JCP.13m08834. [DOI] [PubMed] [Google Scholar]

RESOURCES