For Better Or Worse? Change In Service Use By Children Investigated By Child Welfare Over A Decade

Abstract

Background

Children referred to child welfare (CW) due to suspected maltreatment are vulnerable and need many services, particularly minority children.

Objective

To assess whether service use has improved over the past decade and whether racial-ethnic disparities in service use have decreased.

Methods

We used two national data sets (the National Survey of Child and Adolescent Well-Being I and II) collected a decade apart to assess changes over time in health, education, mental health (MH), dental services and overall service use.

Results

In NSCAW II, more children were young, had lower child behavior checklist (CBCL) scores, and were Hispanic. We found significant increases in dental services, a decrease in special education services and a decrease in MH services on the bivariate level (all p<.01). A large proportion of the change in MH services occurred in school settings, but the pattern continued when examining only those services delivered outside of school. The greatest decrease occurred for children with CBCL scores <64. However, in multivariate analyses, older children, white non-Hispanic children and children placed out of the home were significantly more likely to receive MH services. Rates of MH services controlling for CBCL scores showed no improvement over the decade, nor was there a decrease in racial and ethnic disparities

Conclusion

These data show no change in MH services over time for children referred for CW evaluation, but improvement in dental services. Racial and ethnic disparities persist. MH services decrease occurred predominantly among children whose MH symptoms are below the clinical range.

INTRODUCTION

Children in foster care have significant physical and mental health (MH) needs that impair their functioning and require services across multiple sectors.^1,2 This well-established need for multiple services prompted calls for comprehensive evaluations to identify needs of individual children at, or shortly after, the time of the child welfare CW) investigation.^3,4

While it is well recognized that children placed in out-of-home settings have extensive needs, most children referred to CW remain at home. Of the approximately 6 million children who are reported to CW agencies in the United States annually, only about 700,000 have substantiated abuse and neglect.⁵ A small subset are placed in foster care and this is the subset that has most often been the center of research and discussion. However, we recently documented that children who are placed with kin⁶ and those who remain in their own homes also have health and MH concerns nearly commensurate with those placed out-of-home,^7,8 a need also documented by Burns et al.⁹ Awareness of the heightened needs of this group of children prompted increased mandates for comprehensive assessments and services to all children who are investigated by CW.^10,11,12 Class action lawsuits against CW systems in >32 states have resulted in settlements requiring states to provide a range of services, including services to address MH needs.¹³ Professional groups and experts also advocated for quality standards for screening, assessment, and treatment to increase access to services for all children who come into contact with CW services.¹⁴

Beyond closing the gap between need for and use of services for children investigated by CW, attention has also addressed consistent disparities in service use among minority children.¹⁵ For example, even after controlling for need, African-American children are less likely to receive services, including developmental, school, and health-related services.^16,17 However, system-level coordination between CW and MH has been associated with reduction of disparities.¹⁸ The emphasis on closing disparities in services across race and ethnicity has been a priority for a number of professional groups, including the American Academy of Pediatrics.¹⁹ System-wide emphasis on increasing access has been paired with attention to understanding and addressing disparities in service utilization.²⁰ Improved awareness of the need to treat physical and MH conditions should increase provision of services to children involved with CW services and, hopefully, place greater emphasis on consistent screening for all youth. A direct focus on disparities should also result in reduction of disparities in service use.

However, important countervailing events occurred that might reduce availability of services to these children. The first was passage of Temporary Assistance for Needy Families,²¹ which limited the time that families could depend on public support. In many states parents were required to be employed at least part-time by 1999, potentially limiting their availability to attend their children’s appointments.²² Additionally, the number of individuals on Medicaid decreased as a consequence of this legislation, reducing families’ ability to pay for services.²³ These changes undoubtedly affected some families referred to CW.

Second, the deep economic recession decreased publicly funded services. Since 2008, >45 states have reduced services, including Medicaid (31 states) and other health care programs and services to the disabled (29 states).²⁴ Many MH agencies and school systems also experienced significant cut-backs.²⁵ Both these factors would be expected to reduce the services provided to vulnerable children.

So, although research and policy provide evidence to support an increase in services and additional attention to persistent disparities in service use, it is unclear whether changes in patterns of service use have occurred. Two national surveys of children and adolescents investigated by CW agencies, conducted a decade apart, present an opportunity to examine changes in services delivered to these high need children. We hypothesized that: 1) more youth would be identified with MH problems, but 2) services would remain static or decrease, and 3) racial disparities would have decreased. We based these hypotheses on the increased focus on earlier identification of needs and on reduction of racial disparities in services, coupled with concern that overall services were likely to be adversely affected by economic pressures during the period of data collection.

METHODS

Design and Sample

Data come from the first and second National Surveys of Child and Adolescent Wellbeing (NSCAW I and NSCAW II, respectively), two nationally representative, longitudinal surveys tracking experiences of children whose families were investigated by CW. The surveys involved interviews with caregivers, caseworkers, and youths, and contained similar measurement tools to facilitate cross-time comparisons.

NSCAW I and II employed the same general procedures. A national probability sampling strategy was used to select primary sampling units (PSUs), typically counties, from which a sample of families was drawn. Enrollment for NSCAW I occurred from October, 1999 to December, 2000, and for NSCAW II from March, 2008 to September, 2009. In NSCAW I, cases were selected using stratified random sampling within 92 PSUs. Whenever possible, the same PSUs were included in NSCAW II: Of the 92 original PSUs, 71 were eligible and agreed to participate. Ten additional PSUs were added to replace non-participating PSUs. Both surveys excluded agencies in 8 states in which laws required first contact of a caregiver by agency, rather than study staff.²⁶

Current analyses utilize caseworker and caregiver interviews to assess children’s health, MH, educational and dental service use from just prior to the index CW investigation through 18 months thereafter. Initial NSCAW interviews typically occurred approximately 4–5 months after the index CW investigation. Follow-up interviews occurred at 12 and 18 months in NSCAW I and at 18 months in NSCAW II, but both surveys included questions asking about service use across the same cumulative time period.

Data on children ages 2–14 years old were analyzed, since this is the age-span common to both surveys for which MH needs assessments were available. All procedures were approved by the Research Triangle Institute’s IRB. All analytic work on de-identified data was approved by Rady Children’s Hospital’s IRB.

Measures

Background Characteristics

Questions about demographics, health, behavior, and child placement were asked in initial parent and caregiver interviews. Age is the age of the child at the time of the investigation. Caregivers were asked to report on the child’s overall health using a standardized question and to complete the Child Behavior Checklist (CBCL). There were some minor wording differences in the versions used in the two surveys.^27,28

Services

Caregivers answered questions about children’s use of services. Time frames differed slightly for different service types, but we included only variables that were comparable between NSCAW I and II.

Acute Health Services

Comparable health services questions across the surveys were limited to children’s use of an emergency room or urgent care for illness or injury in the period from 12 months before the Wave I interview through the 18 month interview in both NSCAW surveys (~2 years total).

MH Services

MH service use included individual questions about types of services for emotional, behavioral, learning, attention, or substance abuse problems during the same two-year period as for health services. Individual questions addressed individual service types including use of: a) therapeutic nursery; b) inpatient service (psychiatric hospital or unit in a medical hospital, detoxification unit or inpatient drug/alcohol unit, hospital medical unit); c) day treatment; d) outpatient treatment (MH or community health center, outpatient drug or alcohol clinic, or private professional help from a psychiatrist, psychologist, social worker, or psychiatric nurse); e) primary care outpatient services; f) in-home counseling or in-home crisis services; g) school counseling (from a school guidance counselor, school psychologist, or school social worker); and h) children’s use of an emergency room for emotional, behavioral, learning, attention, or substance abuse problems. We used only items with comparable timeframes between the surveys. Composite MH service use variables were created to reflect use of any of these service types.

Educational Services

Questions about educational services focused on special education services. Caregivers were asked whether the index child ever had an Individualized Educational Plan (IEP) or Individualized Family Service Plan (IFSP), or was currently receiving special education services. Follow-up questions ask about whether children had or currently were receiving special education services or had received such services by the time of the 18-month interviews.

Dental Services

Baseline questions differed slightly in time frame (since contact date in NSCAW I and since start date of current living arrangement in NSCAW II), but both asked about whether a child had gone to a dentist or dental hygienist for a cleaning or a checkup. Identical questions were asked for the18 months follow-up period.

Analyses

Descriptive statistics summarize children’s service use in NSCAW I and NSCAW II. There were very few missing data (<10%): these cases were excluded from the analyses for which their data were missing. Multiple regressions were used to test for differences in rates of service use between NSCAW I and NSCAW II after controlling for confounders. We selected covariates to include all the variables that were significant at the p <.05 level and also tested for possible significant interactions. All variables that met this criterion were included in the models. Beyond examining differences across the decade, we also tested whether patterns of racial/ethnic differences in service use were smaller at NSCAW II than NSCAW I, whether service delivery was more targeted to children with higher levels of need in NSCAW II than NSCAW I, and whether similar trends were evident for children remaining at home as for children removed from their homes.

All analyses utilize recommended weighting. Analysis weights were constructed in stages corresponding to the stages of the sample design, accounting for the probability of selection for each PSU and each child within a PSU. Weights were adjusted for population estimates, small sampling deviations from the original plan, non-response patterns and replacement PSUs. The specific set of weights was designed to support direct comparisons of findings between NSCAW I and NSCAW II, across the span of nearly a decade. All statistics (i.e., means, percentages, etc.) generated using weights can be generalized to the US population of children referred to CW for whom an investigation was initiated. Analyses were conducted using SAS-Callable SUDAAN, version 11.²⁹

RESULTS

Consistent with national demographic trends, there were more Hispanic children in NSCAW II compared to a decade earlier (Table 1). The proportion of very young children was significantly higher in NSCAW II, and contrary to our hypothesis, fewer children were in the clinical range on the CBCL at NSCAW II compared with NSCAW I (22.3% versus 32.1%, p<.0001). No other significant differences in overall characteristics were present in the two samples.

Table 1.

Characteristics of NSCAW I and NSCAW II Samples Age 2–14 years

Sample Characteristics	NSCAW I (n=3331)	NSCAW II (n=2854)
Demographic information
Child age**
2–5	30.6(1.7)	39.2(1.3)
6–10	43.5(1.9)	36.3(1.4)
11–14	25.9(1.9)	24.5(1.4)
Child gender
Male	50.3(2.1)	51.9(1.4)
Female	49.7(2.1)	48.1(1.4)
Child Race**
Black	27.1(2.9)	21.7(2.5)
White	46.6(3.9)	42.4(3.9)
Hispanic	19.1(2.6)	28.6(3.8)
Other	7.2(0.9)	7.2(1.1)
Child Overall health
Good/Fair/Poor	25.2(1.3)	22.7(1.5)
Excellent/Very Good	74.8(1.3)	77.3(1.5)
Child Behavior Checklist: Total***
>=64	32.1(1.7)	22.3(1.2)
<64	67.9(1.7)	77.7(1.2)
Child placement
In Home	90.3(1.1)	88.7(1.1)
Out of Home	9.7(1.1)	11.3(1.1)

^*P<0.05,

^**P<0.01,

^***P<0.001

When we examined children’s reported service use over a two-year period (including 6 months prior to contact with CW and 18 months following) there were few differences in health services, but striking differences in dental, MH and special educational services (Table 2). A significant increase in dental service use occurred over time (from 73.5% to 82%; p<.001). In contrast, there were significant declines in both MH (from 42.8% to 35.1%; p< .01) and special education services (from 30.4% to 23.7%; p<.05). For MH services, the decrease was across all subcategories of services except for Emergency Department use. Because of the large decrease in school counseling (33.6% to 20.0%; p<.001), we assessed whether MH service use differed when excluding school counseling services. When these services were removed the trend remained (34.2% to 29.5%; p=0.07), suggesting that the preponderance of the decrease was in the in-school MH sector. NSCAW II children were also significantly less likely to have an IEP/IFSP and less likely to receive all kinds of special education services (30.4% to 23.7%; p<.05). When total educational, health and MH services were examined with and without school MH services, there was an overall decline between NSCAW I and NSCAW II (from 71.2% to 65.3% with and 67.7% to 63.0% without; both p<.05).

Table 2.

Comparison of Overall Service Use During Two Years in NSCAW I and II (Ages 2–14 Years)

	Total
	NSCAW I Wave 123	NSCAW II Wave 123
Any Health (Child went to ER/Urgent care for illness/injury)	45.9(2.2)	43.9(1.6)
Any Dental	73.5(2.1)***	82.0(1.5)
Any Mental Health	42.8(2.1)**	35.1(1.7)
In School Mental Health	33.6(1.9)***	20.0(1.3)
Any Mental Health if drop School counseling	34.2(2.1)	29.5(1.7)
Any Special Education	30.4(2.1)*	23.7(1.5)
Any Health, Mental Health or Education services	71.2(2.1)*	65.3(1.6)
Any Health, Mental Health or Education services if drop School counseling	67.7(2.5)*	63.0(1.7)

^*P<0.05,

^**P<0.01,

^***P<0.001

To insure that these differences were not due to inconsistencies in caregivers during the 2-year recall period, we also restricted the analyses to those children who remained at home (with consistent caregivers over the 2-years). The resulting pattern and difference in use of services were the same (data available upon request), suggesting that patterns in Table 2 are not due to reporting inconsistency in reporters.

To test whether the differences in MH services were related to the lower rates of clinical need between NSCAW I and NSCAW II, we ran multivariate analyses (Table 3) using in the model all variables that were significant on the bivariate level. These are all listed on the lefthand side of the Table. The first step, assessing MH needs (Step 1), controls for baseline CBCL scores, age, race, and placement. After controlling for these factors, overall differences in MH service use across the decade were no longer significant. However, the decade regression estimate of .89 reflects a slight trend toward reduction in services. We found that MH service use was greater for older children, children with high CBCL scores, and children placed out-of-home, and was consistently lower for non-white than for white children.

Table 3.

Multivariate Logistic Regression Models Predicting Mental Health Service Use Across One Decade for Children Investigated by Child Welfare

	Step 1 (No Interactions)			Step 2 (Decade/Race Interaction Added)			Step 3 (Decade/CBCL Interaction Added)
	OR	95%CI		OR	95%CI		OR	95%CI
Child’s Age	***			***			***
0–5	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00
6–10	2.72***	2.17	3.41	2.72***	2.17	3.41	2.71***	2.16	3.39
11–14	3.47***	2.63	4.57	3.49***	2.66	4.58	3.46***	2.63	4.56
Child Race	***						***
Black/Non-Hispanic	0.53***	0.39	0.71	0.61*	0.40	0.95	0.52***	0.39	0.71
White/Non-Hispanic	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00
Hispanic	0.53***	0.39	0.72	0.76	0.46	1.24	0.53***	0.39	0.72
Other	0.56*	0.35	0.90	0.60	0.32	1.13	0.56*	0.35	0.91
CBCLa
>=64	4.75***	3.69	6.13	4.77***	3.69	6.15	3.83***	2.64	5.57
<64	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00
Placement
In Home	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00
Out of Home	2.05***	1.52	2.78	2.10***	1.55	2.84	2.07***	1.53	2.80
Decade
NSCAW I	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00
NSCAW II	0.89	0.69	1.15	1.12	0.83	1.53	0.77	0.58	1.04
Decade × Black				0.71	0.39	1.30
Decade × Hispanic				0.51*	0.27	0.99
Decade × Other				0.85	0.39	1.86
Decade × CBCL							1.64*	1.02	2.63

^*P<0.05,

^**P<0.01,

^***P<0.001

^aCBCL= Child Behavior CheckList

In a follow-up multivariate model, we explored whether reductions in race/ethnic disparities occurred across the decade (Step 2, Table 3). The interaction between decade and race was not significant overall, although there was a trend for increased disparity across the decade between white and Hispanic children. These results confirm that there was no reduction in disparities in service use between minority and non-minority children over time.

Finally, we examined whether service delivery has increasingly emphasized children with high levels of need. (Step 3, Table 3). The significant interaction between decade and CBCL shows that, while CBCL is a major predictor of service use at both time points, the difference in rates of service use among children with low and high CBCL scores was considerably higher at the end of the decade than at the beginning. This does not appear to have affected race/ethnic disparities in service use.

DISCUSSION

These analyses of two surveys of 2–14 year olds investigated by CW agencies a decade apart suggest that, despite class action lawsuits to increase services and increased awareness of service needs, services for vulnerable children served by CW agencies have not increased except for dental services. We suspect that the increase in dental services reflects national attention to the implications of oral health for overall health and its role in the onset of chronic diseases and poor pregnancy-related outcomes. This may result from the Surgeon General’s report on Oral Health³⁰ and the requirements of Early and Periodic Screening, Diagnostic and Treatment³¹ which include the provision of dental care under Medicaid. Additionally, the passage of the Children’s Health Insurance Program Reauthorization Act of 2009 led to an increased requirement for dental services provided to beneficiaries.^32,33 We cannot, however, rule out the possibility that slight wording changes in the baseline interview may have partially accounted for this increase.

We were surprised to find that fewer children in NSCAW II had CBCL scores in the clinical range. Although this may be partly explained by the higher preponderance of young children in NSCAW II, the decrease was quite substantial and persisted in the weighted data that should correct for sampling differences and despite controlling for age. To rule out a methodological basis for this decrease, we explored minor differences in the survey versions of the CBCL and repeated the analyses looking at rates for identical items. We found a persistent reduction in the rates of individual and total CBCL symptoms in that analysis as well—a reduction that is quite puzzling, as it differs considerably from national trends in children’s MH problems.^34,35 The reason for this decrease in MH symptoms of the children in NSCAW II remains a mystery as we are unaware of any programmatic changes that would have led to this finding. In the absence of any data to inform us about the reason for the reduction of MH symptoms we sought to rule out methodological reasons, but none of them were explanatory. Therefore we are left without a good explanation for this finding.

In multivariable analyses, this decrease in the number of children in the clinical range accounted for the decline in service use over the decade. Nevertheless, the lack of an increase and the substantial reduction of MH and special education services in the bivariate analyses and the failure to increase MH services over time should be matters of concern. The proportion of children in need of MH services has been documented to far exceed the proportion receiving them, so we would hope to see an increase in the absolute proportions who are receiving services over time. Additionally, while an extremely high proportion of children with CBCL scores >64 have serious MH conditions, it is well known that those below this level may also have significant symptoms and that it is among those in the subclinical range that preventive interventions can be most effective.³⁶ Further, there is growing evidence for the persistence of problems that do not receive intervention.⁸ Moreover, this is only a count of the proportion of children who received any MH service and it is well recognized that only a minority of treated receive evidence-based interventions of appropriate dose and duration to have an impact on their developmental trajectory.³⁷ In relationship to need, the data here likely substantially over-represent the provision of MH services of adequate quality, dose, or duration to improve the MH of children with MH impairments. It is also notable that there were no differences across the surveys in the proportion of children who received emergency room MH services, suggesting that the intensity of MH needs is not being fully addressed. Given the recognition of existing deficits in MH services and the barriers to getting children to community MH providers, it is also concerning that the decrease was mainly in the school setting. The school is a site where children can have easier access to care in a less stigmatizing way without requiring stressed families to keep outside appointments on a regular basis. Additionally, these vulnerable children are unlikely to have high levels of receipt of care in a comprehensive continuous medical home, and few primary care providers are providing comprehensive MH services directly or through co-management.³⁸

The decrease in the number of IEPs and IFSPs is also a concern, especially in face of evidence that a large proportion of children involved with CW have special health care needs,⁷ many of which affect education and therefore would be expected to require individualized plans. This decrease may have been the result of states’ responses to the economic downturn experienced when NSCAW II was fielded.²⁵ However, only a third NSCAW survey could determine whether this is the explanation for the decrease in school-based services and, unlike the MH service variables, we had no measure of the need for educational services, so we could not examine this in a similar multivariate analysis.

Finally, it is discouraging that there is no decrease in racial/ethnic disparities in receipt of MH services over the decade. Such disparities are well documented,^39,40 and the improved targeting of children with high CBCL scores would be expected to result in fewer disparities if services were available based entirely on a high-need basis. Nevertheless, we did not see the expected reduction in disparities and those between Hispanics and white non-Hispanics actually increased over the decade. The persistence of these disparities should be a major national concern, especially as a growing proportion of the nation’s children are non-white and Hispanic. Without proper assistance many of these children and the nation will continue to pay the long-term price for their MH problems.^{31, 32}

All secondary analyses have limitations in the data that are available. For example, the data on overall health care use was limited to acute care because of the lack of other types of health services questions asked in a consistent way across surveys. Further, there is very little specificity to the information on the content, frequency, appropriateness or duration of services. Thus, we cannot comment on their quality. This is particularly a problem for the treatment of chronic conditions, including MH problems, which are unlikely to be satisfactorily addressed on single or occasional visits, which is all we are capturing here. We are therefore left with a somewhat partial picture of overall service use and of service needs except for MH. In addition, despite attempts to account for issues that might explain our findings, it is possible that unmeasured variables or minor differences in data collection account in part for them.

Despite these limitations, these analyses document significant improvement in the percentage of children receiving dental services and decreases in the percentage of children who are investigated by CW agencies who are receiving educational and MH services in school over the decade. While the decrease in MH services is targeted to children whose CBCL scores are below the clinical range and does not persist in multivariate analyses, there is no evidence of the needed increase in the proportion of children receiving MH services. Moreover, there is evidence that disparities across racial and ethnic groups are not reduced and may, in fact, be increasing. These analyses using two nationally representative surveys suggest that, despite recognition of the need for services, public funding cutbacks have left a group of high need vulnerable children without services that potentially could significantly improve their well-being and health and impact their long-term futures. As a nation we should be concerned that failure to address their very real needs affects them individually and all of us as members of society.

What’s New.

Children in contact with child welfare have high service needs, and often fail to get needed services. Dental services increased, but mental health services decreased, especially in schools. Controlling for need, racial and ethnic disparities have not decreased.

Abbreviations

MH

Mental Health

CBCL

Child Behavior CheckList

PSU

primary sampling units

NSCAW

National Surveys of Child and Adolescent Well-being

CW

Child Welfare

IEP

Individualized Educational Plan

IFSP

Individualized Family Service Plan