Abstract
Objective
Interorganizational relationships (IORs) between child welfare agencies and mental health service providers may facilitate mental health treatment access for vulnerable children. This study investigates whether IORs are associated with greater use of mental health services and improvement in mental health status for children served by the child welfare system.
Methods
This was a longitudinal analysis of data from a 36 month period in the National Survey of Child and Adolescent Well-Being (NSCAW). The sample consisted of 1,613 children within 75 child welfare agencies who were 2 years or older and had mental health problems at baseline. IOR intensity was measured as the number of coordination approaches between each child welfare agency and mental health service providers. Separate weighted multilevel logistic regression models tested associations between IORs and service use and outcomes, respectively.
Results
Agency level factors accounted for 9% of the variance in the probability of service use and 12% of mental health improvement. Greater intensity of IORs was associated with higher likelihood of both service use and mental health improvement.
Conclusions
Having greater numbers of ties with mental health providers may help child welfare agencies improve children's mental health service access and outcomes.
Practice Implications
Policymakers should develop policies and initiatives to encourage a combination of different types of organizational ties between child welfare and mental health systems. For instance, information sharing at the agency level in addition to coordination at the case level may improve the coordination necessary to serve these vulnerable children.
The insufficient provision of mental health services to emotionally disturbed children in the child welfare system has been well documented (Burns et al., 2004; Glisson, 1996; Hurlburt et al., 2004; Trupin, Tarico, Benson, Jemelka, & McClellan, 1993). Within the US child welfare system, for instance, one recent study found 42.4% of children to have mental health problems; however, only 28.3% received outpatient specialty mental health services (Hurlburt et al., 2004). Although many child welfare agencies provide mental health services, other agencies often provide at least some of the treatment children need. Inadequate interorganizational relationships (IORs) between child welfare and mental health providers thus may partly account for the substantial gap between needs and use of mental health services (Halfon et al., 2002; Halfon, Zepeda, & Inkelas, 2002).
Relationships between child welfare and mental health service providers may occur purely at the case level (i.e., relative to individual child or parent treatment), or may extend to additional information sharing, cross-training of staff, collective development of service delivery policies, and even joint budgeting. Such interagency cooperation may improve services in a variety of ways, including enhancing access and providing caseworkers with more timely and complete information about child and family progress.
The current study applied multilevel modeling to trace interagency ties to both children's mental health specialty service use and mental health improvement over time. Our research questions were: first, do child welfare agency IORs increase the use of mental health services for the children they serve? Second, do IORs improve children's mental health status?
Literature Review
Two previous studies suggest that interagency coordination can improve children's mental health service use. Most recently, Hurlburt et al. (2004) found that the intensity of coordination, defined as number of ties, between local child welfare and mental health service providers was related to stronger associations between Child Behavior Checklist (CBCL) scores and service use. In addition, coordination intensity was associated with decreased differences in service use between white and African American children. These findings implied that more intense interagency coordination could improve the equity of access across racial/ethnic groups. In a previous quasi-experimental study (Glisson 1994), children at intervention sites characterized by coordination teams were more likely to receive mental health services than those in control areas.
There is less evidence that IORs among child-serving agencies improve children's psychosocial functioning. On average, Glisson (1994) found that children with greater levels of initial disturbance improved more in mental health in the pilot area with interagency coordination teams, but children with lower levels of initial disturbance actually fared worse. Another study did not find any effects of IORs on changes in mental health status over 3 years (Glisson & Hemmelgarn, 1998). Empirical findings beyond child welfare have also suggested that IORs could increase children's access to health care but have not found effects on clinical outcomes (Bickman, 1996; Bickman, Lambert, Andrade, & Penaloza 2000).
One possibility for the ambiguous results of improved coordination is that the mental health treatments were on average not effective, suggesting that the next logical step in this line of inquiry would be testing how to implement evidence-based interventions in community-based practices (Bickman, Noser, & Summerfelt, 1999). Another possibility is that some groups of particularly vulnerable children, such as those in child welfare, might benefit clinically from better coordination between child welfare agencies and mental health service providers, while other children would not. Whether such potential benefits accrue because of psychotherapeutic impact or other reasons, such as neglected children's need to have people pay more careful attention to them, finding such benefits would be noteworthy.
Research to date has left some key questions unanswered. First, there is still limited information about how IORs between child welfare agencies and mental health service providers influence children's mental health specialty service use. Second, there are mixed results concerning whether coordination between child welfare and mental health providers improves children's psychosocial functioning (Glisson, 1994; Glisson & Hemmelgarn, 1998). Hurlburt et al (2004) could not investigate the role of coordination in improvement of children's mental health status because longitudinal data at other waves were unavailable at that time. Third, Glisson (1994) and Glisson and Hemmelgarn (1998) measured IORs only in terms of case coordination. However, this interagency activity is only one of several types of mutually reinforcing interagency ties, which may also include joint planning and resource allocation for service delivery (Rogers & Whetten, 1982). Thus, examining case coordination only may lead to underestimation of IOR impact. Finally, previous work has been unable to demonstrate how much variance in child outcomes has been attributable to agency factors, and thus how much management actions at the agency level could potentially improve individual children's outcomes.
Conceptual Framework
According to the health services utilization model, the probability of a child's entry into the mental health system is influenced by the structure of the delivery system itself (e.g., the availability of mental health services, and interaction between mental health providers and child welfare agencies), the child's predisposing characteristics (e.g., age, gender, race/ethnicity, maltreatment type, family risks, placement settings, receipt of child welfare services, and nature of maltreatment report), individual and community enabling resources (e.g., health insurance and geographic location), and the needs s/he may bring to the care-seeking process (as manifested by a clinical CBCL score) (Figure 1) (Aday & Andersen, 1975; Aday, Gretchen, & Andersen, 1984; Andersen, 1995). In turn, mental health services use may affect the child's psychological outcomes.
Figure 1. Applying the health services utilization model to mental health utilization and psychological outcomes for children in child welfare.
As in the most relevant previous study (Hurlburt et al., 2004), the focus in the current investigation is on the intensity of interagency connections, defined as the number of different types of linkages connecting the focal actor (in this case, the child welfare agency), and any given type of partner agency. Based on previous theory and evidence, we speculated that more intense interagency ties, involving a greater number of different types of cooperation, would yield more benefits for the children served. For instance, the presence of ongoing interagency information sharing and policy development should reinforce the benefits of coordination concerning individual cases by improving the channels of communication between the two agencies. Intensity of interagency relationships may be particularly important in child welfare because staff from different agencies often have conflicting views of children's needs (Darlington, Feeney, & Rixon 2004). Better communication therefore may be necessary to achieve adequate interagency service coordination. Both the general management literature (Kogut 1989; Uzzi 1997, where this is referred to as “multiplexity”) and research in mental health services (Provan & Sebastian, 1998; Hurlburt et al., 2004) have linked IOR intensity to better performance. Thus, we predicted that IOR intensity would be positively associated with the likelihood of mental health service use.
The ultimate goal of the IORs examined here is to facilitate children's mental health improvement through timely and appropriate mental health services. More complete and continuous information exchanges should lead to prompt identification of children's mental health problems and swifter subsequent recalibration of treatment plans. In turn, the relationship tactics children and parents learn through therapy may further improve child psychological functioning by reducing family conflict (Hukkanen, Sourander, Bergroth, & Piha, 1999). Caseworkers may also apply the skills they learn from mental health providers through cross-training to help children and parents more effectively.
More intense IORs should facilitate placement permanence, which is a major contributor to children's mental health status (Newton, Litrownik, & Landsverk, 2000). Caseworkers and mental health providers are actively involved in placement recommendations/decisions (Butler, Atkinson, Magnatta, & Hood, 1995), and these decisions may further affect child mental health. Previous research has found significant differences in mental health status after children were placed in different settings (Holtan, Ronning, Handegard, & Sourander, 2005; Keller et al., 2001). Overall, the authors expected that IOR intensity would positively affect the likelihood of improvement in psychological functioning.
Methods
Data source
Data were drawn from the Child Protective Services (CPS) cohort of the National Survey of Child and Adolescent Well-Being (NSCAW), the only national study of children in the US child welfare system (Dowd et al., 2004). The Administration for Children and Families within the US Department of Health and Human Services funded NSCAW, contracting with Research Triangle Institute (RTI) International to conduct data collection. A two-stage stratified design was used to sample children in 92 primary sampling units within 48 states throughout the US. Assessments of child context and well-being, using a variety of standardized instruments, were conducted through interviews with each child, the current caregiver, and the child welfare worker at 2-6 months (Wave 1, beginning in 1999), at 12 months (Wave 2), at 18 months (Wave 3), and at 36 months (Wave 4) after the close of the investigation or assessment. A subsequent fifth wave was limited to subsets of the initial sample, and thus was omitted from the current analyses.
Weighting adjusted the estimates for the differential probabilities of inclusion in the sample: the probability of selecting the county of residence for the child and the probability of selecting the child given that the child's county of residence was sampled (Dowd et al., 2004). Two-level probability weights adjusted for stratification by state and receipt of child welfare services, with person-level weights at Level 1 and agency weights at Level 2. When incorporated in analyses, these weights yield approximately design-unbiased and consistent estimates for the corresponding population quantities (Pfefferman, 1996). In this case, analysts can thus derive an almost nationally representative sample of children who have had encounters with the child welfare system (excluding 2 states whose human subjects' requirements would have consumed enormously disproportionate resources).
Child welfare agency directors were interviewed at Wave 1 about agency cooperation with mental health service providers, among a number of other topics. Some county-level information in the Area Resource File of 2000, including population characteristics and medical resources, was also combined with NSCAW data using county identifiers.
Analytic samples
We were interested in the subsample of children in NSCAW who were in need of mental health services, whom we identified using the CBCL clinical cut point at Wave 1 (total, internalizing, or externalizing CBCL score ≥64). That is, children with scores above the cut point were considered to have mental health problems that indicated a need for mental health services. Criteria for inclusion in addition to need for mental health services were age of at least two years at baseline and the presence of a CBCL assessment at Wave 4. The reason for the lower age limit was availability of the CBCL only for children aged 2 or over (Achenbach & Edelbrock, 1991). Among the 5,501 children in the overall NSCAW CPS sample, the final sample size was 1,613, nested within 75 child welfare agencies.
Measures
Use of mental health services
Use of mental health services was used as a dependent variable to answer the first research question. Mental health services included clinic-based specialty mental health services (i.e., community mental health clinics), day treatment, therapeutic nursery, and private professionals (i.e., psychiatrists, psychologists, social workers, and psychiatric nurses). At each wave, caregivers were asked whether their children had received any mental health services between the prior and current interviews. This service use measure reflects the presence of at least one mental health visit from Wave 1 to Wave 4, a time span of 36 months.
Mental health improvement
This measure was used as a dependent variable to answer the second research question. The evaluation of children's mental health problems was based on caregiver responses to the Child Behavior Checklist (CBCL). The CBCL reflects a detailed picture of the child's behavioral problems within the past 2 months and has high reliability and validity (Achenbach & Edelbrock, 1991; Heflinger, Simpkins, & Combs-Orme, 2000). The total problem score identifies the overall level of problem behaviors present. In addition, the CBCL includes two broad-band syndromes, internalizing problems (e.g., withdrawal, somatic complaints, anxiety/depression, and sleep problems) and externalizing problems (e.g., aggressive and destructive behavior). Recalling that all children in this sample had clinical CBCL scores at baseline (i.e., ≥64), if all three CBCL scores (total, externalizing, and internalizing) at Wave 4 were less than 64, that child was considered to have improved mental health status.
Intensity of IORs
Intensity of IORs was used as an independent variable, measured as the total number of different types of linkages connecting each child welfare agency with mental health providers. Types of ties were: joint budgeting or resource allocation, cross-training of staff, working with the agency on child welfare cases, development of interagency agreement and memoranda of understanding, joint planning/policy formulation for service delivery, discussion and information sharing, and other approaches.
Other variables
Covariates in this study included child level measures such as demographic characteristics (age, gender, and race/ethnicity), maltreatment type, health insurance coverage, placement settings, receipt of child welfare services, the family's overall risk level, and whether or not there had been a substantiated maltreatment report. Agency level information (such as whether the child welfare agency was located in an urban or rural county) was also included in the models as controls. The availability of mental health services in the agency's county was measured by the total number of mental health professionals per 100,000 children (i.e., general psychiatrists, child psychiatrists, and psychologists) and the number of community mental health centers per 100,000 children.
Analysis plan
First, the characteristics of children, child welfare agencies, and agency service areas were described. Then, an unconditional multilevel model was used to examine the degree to which child welfare agency level factors contributed to the total variance observed in each dependent variable. A very small percentage of variance in child outcomes due to agency level factors would indicate that the agency effect was ignorable. If agency effects were non-ignorable, it would be appropriate to include agency attributes as a separate level in multiple regression models assessing the associations between agency IORs and individual child outcomes. In these models, child-level factors would be included as “Level 1” predictors. In addition, agency level factors (including county attributes such as rurality and numbers of providers) would be included in a “Level 2” equation predicting each child's initial service use or mental health improvement.
Multilevel models were considered for this study because of the hierarchical data structure, with children nested within child welfare agencies. Multilevel analysis accommodates nested data structures by separating within-agency and within-person variance from between-agency and between-person variance (Bryk & Raudenbush, 1992). This type of model corrects for disaggregation bias by estimating parameters at different levels of analysis, thus explicitly accounting for the non- independence of individuals within the same group (Hofmann, 1997).
We expected that greater IOR intensity between child welfare agencies and mental health service providers would be related to greater probability of both mental health services use and mental health improvement over time. In operational terms, we expected that the odds ratios for the associations between intensity of IORs and each dependent variable would be significantly greater than 1, at alpha = 0.05.
The software package we used was GLLAMM supported by Stata 9.2. This package estimates model coefficients at each level and predicts the random effects associated with each sampling unit at each level, allowing for probability weights (Rabe-Heskech & Skrondal, 2005). This secondary data analysis was approved by the Institutional Review Board at the Pennsylvania State University. The original data collection was approved by an institutional review board at the RTI International.
Results
Descriptive analysis
Table 1 displays the characteristics of children in the analytic sample, all of whom had one or more CBCL score of 64 or higher at baseline. Only weighted percentages are reported here. CBCL scores for more than half of these children fell below 64 by Wave 4 (54.4%). Just over half (55.6%) of the sample of children with mental health problems at wave 1 received mental health services during the study period. The ages of the children at baseline ranged from 2 to 14 years old: 23.8% were from 2 to 5 years old, 39.6% were from 6 to 10 years old, and 36.6% were above 10 years of age. There were slightly fewer girls (48%) than boys in the sample. The sample was 50.1% non-Hispanic White, 26.3% non-Hispanic Black, 17.0% Hispanic, and 6.6% of other race/ethnicity.
Table 1.
Characteristics of children with clinical Child Behavior Checklist scores (>=64) at baseline
| Unweighted frequency (n=1,613) | Weighted percent | |
|---|---|---|
| Mental health status | ||
| Improved | 794 | 54.4 |
| Not improved | 819 | 45.6 |
| Use of mental health services | ||
| Yes | 793 | 55.6 |
| No | 820 | 44.4 |
| Age | ||
| 2-5 | 353 | 23.8 |
| 6-10 | 659 | 39.6 |
| >=11 | 601 | 36.6 |
| Gender | ||
| Male | 817 | 52.0 |
| Female | 796 | 48.0 |
| Race | ||
| Non-Hispanic White | 791 | 50.1 |
| Non-Hispanic Black | 462 | 26.3 |
| Hispanic | 240 | 17.0 |
| Other race | 120 | 6.6 |
| Maltreatment types | ||
| Physical maltreatment | 411 | 27.9 |
| Sexual abuse | 289 | 12.9 |
| Neglect | 570 | 40.6 |
| Other maltreatment | 343 | 18.6 |
| Health insurance status | ||
| Medicaid | 1,140 | 65.3 |
| Private | 461 | 34.0 |
| Self pay | 12 | 0.7 |
| Placements at baseline | ||
| In-home | 1,174 | 87.6 |
| Out-of-home | 439 | 12.4 |
| Composite measure of family risks a | ||
| Low | 439 | 42.9 |
| Medium | 494 | 33.9 |
| High | 578 | 23.2 |
| Receipt of child welfare services | ||
| Yes | 1,213 | 30.2 |
| No | 400 | 69.8 |
| Substantiated maltreatment case | ||
| Yes | 831 | 74.7 |
| No | 782 | 25.3 |
102 observations have missing values.
Agency and county characteristics are reported in Table 2. Seventy percent of child welfare agencies were located in urban areas. The number of IORs between child welfare agencies and mental health providers ranged from 0 to 7 with an average of 3.8 (SD=2.0). The average child welfare agency was in a county with one (1.1) community mental health center and 876 mental health professionals per 100,000 children.
Table 2.
Child welfare agency and service area characteristics
| N | Mean | S.D. | |
|---|---|---|---|
| Child welfare agency location (1=urban, 0=rural) | 75 | 0.7 | 0.5 |
| Intensity of IORs with local mental health service providers | 75 | 3.8 | 2.0 |
| Total number of mental health care professionals | 75 | 875.9 | 644.9 |
| Number of mental health centers | 75 | 1.1 | 1.9 |
Unconditional multilevel model
Given binary dependent variables, two-level weighted logistic regression models were used to evaluate whether the random effect at the agency level equaled zero, and if not, then how much of the variation was attributable to factors at the agency level (recalling that the 1,613 children were clustered in a total of 75 agencies). Level-1 variance in multilevel logistic models is π2/3, the variance for a standard logistic distribution (Browne, Subramanian, Jones, & Goldstein, 2005; Guo & Zhao, 2000; Hedeker, 2003). The Level-2 variance was 0.33 with a standard error of 0.12 in an unconditional model for use of mental health services. Comparing the Level-2 variance with its standard error, this random effect in the model was statistically significant at the 5% level (z=0.33/0.12=2.75). The proportion of variance explained at Level 2 was 9.1% [0.091=0.33/(0.33+ π2/3)].
In the unconditional model for improvement of mental health status, the Level-2 variance was 0.48 with a standard error of 0.11. Comparing this agency-level variance with its standard error, the random effect in the model was also statistically significant at the 5% level (z=0.48/0.11=4.36). The proportion of variance explained at agency level was 12.7% [0.127=0.48/(0.48+ π2/3)]. Given Level-2 variance in each model significantly different from zero, conditional multilevel models including agency level factors at Level 2 were used to test agency effects.
Multilevel logistic regression model on use of mental health services
The number of IORs was positively related to mental health services use [OR=1.04, 95% CI (1.01, 1.07)] (Table 3, Model 2). For every additional type of tie with mental health service providers the odds of mental health service use for individual children increased by 4%. Use of mental health services was also significantly positively related to many other factors, such as severity of mental health status at baseline (i.e., higher initial CBCL scores), being older, white, neglect as initial report, not having health insurance, out-of-home placement (i.e., foster or kinship care), higher family risk, receiving child welfare services, and child welfare agency location in an urban area. In addition, local medical resource availability had mixed effects on use of mental health services. A greater number of mental health professionals per 100,000 children was positively related to the use of mental health services, however the number of community mental health clinics per 100,000 children was inversely associated with services use.
Table 3.
Weighted multilevel logistic model of use of mental health services a
| Use of mental health services | Model 1 (child-level variables) | Model 2 (child and agency level variables) | ||||||
|---|---|---|---|---|---|---|---|---|
| OR | 95% CI | P | OR | 95% CI | P | |||
| Child level | ||||||||
| Severity (centered CBCL) | 1.13 | (1.09,1.17) | 0.000 | 1.12 | (1.08,1.17) | 0.000 | ||
| Male | 1.39 | (0.55,3.52) | 0.492 | 1.36 | (0.53,3.49) | 0.527 | ||
| Age 6-10 | 1.85 | (1.44,2.39) | 0.000 | 1.82 | (1.36,2.43) | 0.000 | ||
| Age 11+ | 4.75 | (3.08,7.31) | 0.000 | 4.55 | (2.84,7.49) | 0.000 | ||
| Non-Hispanic Black | 0.23 | (0.19,0.28) | 0.000 | 0.22 | (0.18,0.27) | 0.000 | ||
| Hispanic | 0.57 | (0.38,0.84) | 0.005 | 0.48 | (0.33,0.70) | 0.000 | ||
| Other race | 1.23 | (0.99,1.51) | 0.053 | 1.05 | (0.85,1.28) | 0.668 | ||
| Sexual abuse | 1.87 | (1.07,3.25) | 0.027 | 1.71 | (0.99,2.94) | 0.052 | ||
| Neglect | 0.64 | (0.46,0.91) | 0.011 | 0.63 | (0.44,0.89) | 0.009 | ||
| Other maltreatment | 0.36 | (0.14,0.91) | 0.031 | 0.36 | (0.14,0.89) | 0.027 | ||
| Private insurance | 0.78 | (0.60,1.00) | 0.057 | 0.76 | (0.60,0.97) | 0.026 | ||
| Self pay | 5.66 | (3.97,8.01) | 0.000 | 5.35 | (3.63,7.89) | 0.000 | ||
| In-home setting | 0.40 | (0.30,0.54) | 0.000 | 0.44 | (0.32,0.59) | 0.000 | ||
| Risk level | 1.33 | (1.10,1.61) | 0.003 | 1.35 | (1.11,1.62) | 0.002 | ||
| Receipt of child welfare service | 1.42 | (1.25,1.61) | 0.000 | 1.30 | (1.11,1.53) | 0.001 | ||
| Substantiated case | 0.72 | (0.48,1.08) | 0.115 | 0.71 | (0.45,1.13) | 0.149 | ||
| Agency/county level | ||||||||
| Urban | 1.47 | (1.02,2.31) | 0.041 | |||||
| Number of mental health care professionals in county per 100,000 children | 1.00 | (1.00,1.00) | 0.015 | |||||
| Number of mental health centers in county per 100,000 children | 0.87 | (0.84,0.94) | 0.000 | |||||
| Intensity of IORs with local mental health service providers | 1.04 | (1.01,1.07) | 0.009 | |||||
| Variance components (S.E.) b | ||||||||
| Level-2 | 0.56 (0.14) | 0.38 (0.13) | ||||||
Reference groups are female, age 2-5, non-Hispanic white, physical maltreatment, Medicaid, in-home setting, not receiving child welfare service, not a substantiated case, and rural county.
Level-1 variance of multilevel logistic model is π2/3.
Multilevel logistic regression modeling improvement of mental health status
The intensity of child welfare agency ties with mental health service providers was positively related to improved mental health status by the end of the study period [OR=1.07, 95% CI (1.03, 1.12)] (Table 4, Model 2). For every additional type of IOR, the odd of mental health status improvement increased by 7%. Mental health improvement was also significantly positively related to many other factors, such as severity of mental health status at baseline (i.e., lower initial CBCL score), being male, non-White, having Medicaid coverage, out-of-home placement, low-level family risk, and unsubstantiated case. Mental health services use and mental health care resource availability were negatively associated with improvement in psychological functioning.
Table 4.
Weighted multilevel logistic model of mental health status improvement a
| Improvement of mental health status | Model 1 (child-level variables) | Model 2 (child and agency level variables) | ||||
|---|---|---|---|---|---|---|
| OR | 95% CI | P | OR | 95% CI | P | |
| Child level | ||||||
| Receipt of mental health specialty services | 0.36 | (0.27,0.49) | 0.000 | 0.36 | (0.27,0.47) | 0.000 |
| Severity (centered CBCL) | 0.82 | (0.77,0.86) | 0.000 | 0.82 | (0.78,0.87) | 0.000 |
| Male | 1.32 | (1.13,1.53) | 0.000 | 1.30 | (1.12,1.51) | 0.001 |
| Age 6-10 | 2.75 | (1.88,4.00) | 0.000 | 2.69 | (1.88,3.86) | 0.000 |
| Age 11+ | 2.20 | (1.61,3.02) | 0.000 | 2.16 | (1.60,2.91) | 0.000 |
| Non-Hispanic Black | 1.04 | (0.74,1.45) | 0.842 | 0.99 | (0.71,1.38) | 0.963 |
| Hispanic | 1.79 | (1.19,2.70) | 0.005 | 1.72 | (1.20,2.47) | 0.003 |
| Other races | 2.83 | (1.93,4.15) | 0.000 | 2.66 | (1.82,3.87) | 0.000 |
| Sexual abuse | 1.37 | (0.99,1.87) | 0.051 | 1.32 | (0.96,1.83) | 0.090 |
| Neglect | 0.78 | (0.57,1.09) | 0.150 | 0.77 | (0.57,1.05) | 0.096 |
| Other maltreatment | 1.05 | (0.65,1.70) | 0.835 | 1.04 | (0.64,1.70) | 0.868 |
| Private insurance | 0.92 | (0.66,1.27) | 0.600 | 0.91 | (0.66,1.26) | 0.579 |
| Self pay | 0.24 | (0.16,0.36) | 0.000 | 0.26 | (0.18,0.37) | 0.000 |
| In-home setting | 0.56 | (0.38,0.83) | 0.004 | 0.57 | (0.39,0.84) | 0.005 |
| Risk level | 0.68 | (0.58,0.79) | 0.000 | 0.68 | (0.57,0.81) | 0.000 |
| Receipt of child welfare service | 0.73 | (0.51,1.04) | 0.081 | 0.68 | (0.45,1.02) | 0.064 |
| Substantiated case | 0.55 | (0.43,0.70) | 0.000 | 0.52 | (0.42,0.64) | 0.000 |
| Agency Level | ||||||
| Urban | 1.22 | (0.66,2.29) | 0.520 | |||
| Number of mental health care professionals in county per 100,000 children | 1.00 | (0.99,1.00) | 0.931 | |||
| Number of mental health centers in county per 100,000 children | 0.93 | (0.89,0.96) | 0.000 | |||
| Intensity of IORs with local mental health service providers | 1.07 | (1.03,1.12) | 0.002 | |||
| Variance components b | ||||||
| Level-2 | 0.47(0.22) | 0.40 (0.18) | ||||
Reference groups are not receiving mental health services, girl, age 2-5, non-Hispanic white, physical maltreatment, Medicaid, in-home setting, not receiving child welfare service, not a substantiated case, and rural areas.
Level-1 variance of multilevel logistic model is π2/3.
Discussion
Children involved in child welfare services frequently have health problems which require evaluation and treatment services from multiple providers. Timely and appropriate mental health services are especially important for maltreated children, given the potential for abuse to affect development over time (English, 1998). Child welfare agencies play an important role in children's experiences in child welfare (Wells, 2006). Moreover, the Adoption and Safe Families Act of 1997 (ASFA) requires child welfare agencies to extend their responsibilities by focusing on child well-being. But a recent literature review found that the effects of child welfare agency management were mixed on the quality of service maltreated children received as well as on their well-being (Wells, 2006).
The results of the current study showed that agency level factors accounted for significant variance in the probability of service use and mental health improvement. In addition, greater intensity of IORs was associated with higher likelihood of both service use and mental health improvement, controlling for a variety of predisposing, enabling, and need factors.
The health services utilization model posits that system-level factors affect individual service use in addition to factors that predispose individuals toward utilization and underlying need (Aday & Andersen, 1975; Aday, Gretchen, & Andersen, 1984; Andersen, 1995). The current study focused on IORs as a pivotal aspect of the child welfare system structure that might affect mental health service utilization and outcomes. The premise was that more child welfare agency cooperation with mental health service providers would lead to better outcomes for the children involved. Information exchanges among agencies could enhance assessments, planning, and coordination (Rivard, Johnsen, Morrissey, & Starrett, 1999). Such information could inform agency-level agreements (Morrissey, Johnsen, & Calloway, 1997), which could in turn support coordination for individual children. Similar mutually supportive effects could occur between agency level initiatives, such as joint planning and policy formulation and staff training, and day-to-day case management. Overall, we proposed, the more ways child welfare agencies coordinated with mental health service providers, the better.
The association found here between the intensity of interagency cooperation and children's clinical status contrasts with the null results among general pediatric populations in previous large scale studies of systems of care (Bickman, 1996). One possible reason for this contrast may be the difference in the populations. Abused and neglected children may benefit more from enhanced interagency coordination because they lack the family resources that other children have. A related possibility is that certain types of interagency ties matter more than others. Specifically, perhaps relationships between the child welfare agency, with its overall case management responsibilities, and mental health service providers, with their potential for encouraging behavioral change, are particularly important.
Contributions
This study makes several research contributions. First, coordination between child welfare and mental health providers is often necessary, yet little is known about how these types of agencies work together and what outcomes are thereby achieved. The current study uses nationally representative data to investigate the effects of IORs between child welfare agencies and mental health providers on both service use and psychological functioning for children over time. Second, this study evaluates the magnitude of agency-level effects on psychological change among child welfare children, finding that a small but significant percentage of the variation in both children's service access and mental health improvement over time was attributable to factors at the agency level. Third, the IORs examined in this study were not limited in terms of case coordination because this interagency activity is only one of several types of potentially mutually reinforcing interagency ties (Rogers & Whetten, 1982). Fourth, some methodological limitations in previous studies were addressed in this study. For instance, models have not always controlled for demographic characteristics, family risks, and maltreatment types in individual level analyses (Glisson, 1994; Glisson & Hemmelgarn, 1998). Those factors may influence health care access and utilization (Aday & Andersen, 1975; Burns et al., 2004; Hurlburt et al., 2004; Shin, 2005). Hurlburt et al. (2004) used CBCL at Wave 1 as a measure of mental health needs to predict services use in the past 12 months. However, the CBCL only reflects behavior problems over the past 2 months (Achenbach & Edelbrock, 1991). Thus, with cross sectional data, Hurlburt et al. (2004) had to use “current” mental health status to predict “previous” service use. The current study contributes to this evolving literature by controlling for child and family covariates and by examining how service use affects subsequent mental health status.
Limitations and future directions
This study has several limitations. First, the authors were unable to test the stability of child welfare-mental health agency IORs over time because NSCAW only includes this information at baseline. Second, this study does not fully explore the causal chain between agency IORs and changes in children's mental health status. Future studies should clarify how IORs affect children and their families over time. Third, a mediational model was not conducted in this study. NSCAW data are extremely complex, deriving from a stratified, clustered sample. Although in theory the model in Figure 1 would best be tested through a single mediational model, the complexities of the data made this intractable. We therefore broke the analyses out into two separate models.
Other more proximate factors explain substantially more variance in children's mental health service use and clinical outcomes than do interagency ties. These include factors that public policies might affect, such as children's insurance status. The current study has not addressed the relative cost efficiency of improving interagency ties relative to affecting other forms of systems change. However, these findings do lend credence to the view that more interagency cooperation may improve both access and outcomes for children in the child welfare system, at least slightly. The authors hope future studies will build on this investigation by examining conditions that may lead to greater benefits associated with IORs.
Practice implications
The current study provides evidence that more intense interagency coordination can lead to both more service access and better child outcomes. Based on these findings, policymakers should develop policies and initiatives to strengthen the coordination between child welfare and mental health systems. Coordination should be not limited to the individual case level. Rather, multiple ties across hierarchical levels should be developed among agencies. Elements that can be coordinated include information, resources, clients, and programs (Klonglan, Warren, Winkelpleck, & Paulson, 1976). Thus, the strategies for the interagency activities between child welfare and mental health service providers include exchange of children's maltreatment, medical reports, and other relevant information, resource exchanges of funds and materials (i.e., joint budgeting and resource allocation), cross-training of staff, working with agency on individual cases, written agreements to share activities between organizations, and joint programs to plan and implement service delivery.
Acknowledgments
This research was supported by NIDA R03 DA020748-01A1.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Contributor Information
Yu Bai, Department of Health Policy and Administration, Pennsylvania State University, University Park, Pennsylvania, United States.
Rebecca Wells, Department of Health Policy and Administration, School of Public Health, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, United States.
Marianne M. Hillemeier, Department of Health Policy and Administration, Pennsylvania State University, University Park, Pennsylvania, United States.
References
- Achenbach TM, Edelbrock C. Manual for the child behavior checklist and 1991 profile. Burlington, VT: University of Vermont Department of Psychiatry; 1991. [Google Scholar]
- Aday LA, Andersen RM. Development of indices of access to medical care. Ann Arbor, MI: Health Administration Press; 1975. [Google Scholar]
- Aday LA, Gretchen VF, Andersen RM. Access to medical care in the U.S.: Who has it, who doesn't. Chicago, IL: Center for Health Administration Studies, University of Chicago; 1984. [Google Scholar]
- Andersen RM. Revisiting the behavioral model and access to medical care: Does it matter? Journal of Health and Social Behavior. 1995;36(1):1–10. [PubMed] [Google Scholar]
- Bickman L. Implications of a children's mental health managed care demonstration evaluation. Journal of Mental Health Administration. 1996;23(1):107–117. doi: 10.1007/BF02518647. [DOI] [PubMed] [Google Scholar]
- Bickman L, Lambert EW, Andrade AR, Penaloza RV. The Fort Bragg continuum of care for children and adolescents: Mental health outcomes over 5 years. Journal Of Consulting And Clinical Psychology. 2000;68(4):710–716. [PubMed] [Google Scholar]
- Bickman L, Noser K, Summerfelt WT. Long-term effects of a system of care on children and adolescents. Journal of Behavioral Health Services & Research. 1999;26(2):185–202. doi: 10.1007/BF02287490. [DOI] [PubMed] [Google Scholar]
- Browne WJ, Subramanian SV, Jones K, Goldstein H. Variance partitioning in multilevel logistic models that exhibit overdispersion. Royal Statistical Society. 2005;168(3):599–613. [Google Scholar]
- Bryk AS, Raudenbush SW. Hierarchical linear models. Newbury Park, CA: Sage; 1992. [Google Scholar]
- Burns BJ, Phillips SD, Wagner HR, Barth RP, Kolko DJ, Campbell Y, Landserk J. Mental health need and access to mental health services by youths involved with child welfare: A national survey. Journal of the American Academy of Child and Adolescent Psychiatry. 2004;43(8):960–970. doi: 10.1097/01.chi.0000127590.95585.65. [DOI] [PubMed] [Google Scholar]
- Butler S, Atkinson L, Magnatta P, Hood E. Child maltreatment - The collaboration of child-welfare, mental-health, and judicial systems. Child Abuse & Neglect. 1995;19(3):355–362. doi: 10.1016/s0145-2134(94)00136-7. [DOI] [PubMed] [Google Scholar]
- Darlington Y, Feeney J, Rixon K. Complexity, conflict, and uncertainty: Issue in collaboration between child protection and mental health services. Children and Youth Services Review. 2004;26(12):1175–1192. [Google Scholar]
- Dowd K, Kinsey S, Wheeless S, Thissen R, Richardson J, Suresh R, Mierzwa F, Biemer P, Johnson I, Lytle T. National Survey of Child and Adolescent Well-Being (NSCAW): Combined Waves 1-4 Data File User's Manual Restricted Release Version. Ithaca, NY: Cornell University; 2004. [Google Scholar]
- English DJ. The extent and consequences of child maltreatment. The Future of Children. 1998;8(1):39–53. [PubMed] [Google Scholar]
- Glisson C. The effect of services coordination teams on outcomes for children in state custody. Administration in Social Work. 1994;18(4):1–23. doi: 10.1300/J147v18n04_01. [DOI] [PubMed] [Google Scholar]
- Glisson C. Judicial and service decisions for children entering state custody: The limited role of mental health. Social Service Review. 1996;70(2):257–281. [Google Scholar]
- Glisson C, Hemmelgarn A. The effects of organizational climate and interorganizational coordination on the quality and outcomes of children's service systems. Child Abuse & Neglect. 1998;22(5):401–421. doi: 10.1016/s0145-2134(98)00005-2. [DOI] [PubMed] [Google Scholar]
- Guo G, Zhao H. Multilevel modeling for binary data. Annual Review of Sociology. 2000;26:441–462. [Google Scholar]
- Halfon N, Inkelas M, Flint R, Shoaf K, Zepeda A, Franke T. Assessment of factors influencing the adequacy of health care services to children in foster care. 2002 Retrieved October, 2005, from http://www.healthychild.ucla.edu/Publications/ChildrenFosterCare/Documents/EXECUTIVE%20SUMMARY%20standalone.pdf.
- Halfon N, Zepeda A, Inkelas M. Mental Health Services for Children in Foster Care. 2002 Retrieved October, 2005, from http://www.healthychild.ucla.edu/Publications/ChildrenFosterCare/AssessmentFactors.asp.
- Hedeker D. A mixed-effects multinomial logistic regression model. Statistics in Medicine. 2003;22:1433–1446. doi: 10.1002/sim.1522. [DOI] [PubMed] [Google Scholar]
- Heflinger CA, Simpkins CG, Combs-Orme T. Using the CBCL to determine the clinical status of children in state custody. Children and Youth Services Review. 2000;22(1):55–73. [Google Scholar]
- Hofmann D. An overview of the logic and rationale of hierarchical linear models. Journal of Management. 1997;23:723–744. [Google Scholar]
- Holtan A, Ronning JA, Handegard BH, Sourander A. A comparison of mental health problems in kinship and nonkinship foster care. European Child & Adolescent Psychiatry. 2005;14(4):200–207. doi: 10.1007/s00787-005-0445-z. [DOI] [PubMed] [Google Scholar]
- Hukkanen R, Sourander A, Bergroth L, Piha J. Psychosocial factors and adequacy of services for children in children's homes. European Child & Adolescent Psychiatry. 1999;8(4):268–275. doi: 10.1007/s007870050101. [DOI] [PubMed] [Google Scholar]
- Hurlburt MS, Leslie LK, Landsverk J, Barth RP, Burns BJ, Gibbons RD, Slymen DJ, Zhang JJ. Contextual predictors of mental health service use among children open to child welfare. Archives of General Psychiatry. 2004;61(12):1217–1224. doi: 10.1001/archpsyc.61.12.1217. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Keller TE, Wetherbee K, Le Prohn NS, Payne V, Sim K, Lamont ER. Competencies and problem behaviors of children in family foster care: Variations by kinship placement status and race. Children and Youth Services Review. 2001;23(12):915–940. [Google Scholar]
- Klonglagn E, Warrenr D, Winkelplecjk M, Paulsons K. Interorganizational measures in the social services sector: Differences by hierarchical level. Administrative Science Quarterly. 1976;21:675–687. [Google Scholar]
- Kogut B. The stability of joint ventures: Reciprocity and competitive rivalry. Journal of Industrial Economics. 1989;38:183–198. [Google Scholar]
- Morrissey JP, Johnsen MC, Calloway MO. Evaluating performance and change in mental health systems serving children and youth: An interorganizational network approach. Journal of Mental Health Administration. 1997;24(1):4–22. doi: 10.1007/BF02790476. [DOI] [PubMed] [Google Scholar]
- Newton RR, Litrownik AJ, Landsverk JA. Children and youth in foster care: Disentangling the relationship between problem behaviors and number of placements. Child Abuse & Neglect. 2000;24(10):1363–1374. doi: 10.1016/s0145-2134(00)00189-7. [DOI] [PubMed] [Google Scholar]
- Pfefferman D. The use of sampling weights for survey data analysis. Statistical Methods in Medical Research. 1996;5:239–261. doi: 10.1177/096228029600500303. [DOI] [PubMed] [Google Scholar]
- Provan KG, Sebastian JG. Networks within networks: Service link overlap, organizational cliques, and network effectiveness. Academy of Management Journal. 1998;41(4):453–463. [Google Scholar]
- Rabe-Heskech S, Skrondal A. Multilevel and longitudinal modeling using Stata. College Station, TX: Stata Press; 2005. [Google Scholar]
- Rivard JC, Johnsen MC, Morrissey JP, Starrett BE. The dynamics of interagency collaboration: How linkages develop for child welfare and juvenile justice sectors in a system of care demonstration. Journal of Social Service Research. 1999;25(3):61–82. [Google Scholar]
- Rogers DL, Whetten DA. Interorganizational coordination: theory, research, and implementation. IOWA State University Press; Ames: 1982. [Google Scholar]
- Shin SH. Need for and actual use of mental health service by adolescents in the child welfare system. Children and Youth Services Review. 2005;27(10):1071–1083. [Google Scholar]
- Trupin E, Tarico VS, Benson PL, Jemelka R, McClellan J. Children on child protective service caseloads: Prevalence and nature of serious emotional disturbance. Child Abuse & Neglect. 1993;17:345–355. doi: 10.1016/0145-2134(93)90057-c. [DOI] [PubMed] [Google Scholar]
- Uzzi B. Social structure and competition in interfirm networks: The paradox of embeddedness. Administrative Science Quarterly. 1997;42(1):35–67. [Google Scholar]
- Wells R. Managing child welfare agencies: What do we know about what works? Children and Youth Services Review. 2006;28:1181–1194. [Google Scholar]