Cost calculator methods for estimating casework time in child welfare services: A promising approach for use in implementation of evidence-based practices and other service innovations

Abstract

Estimating costs in child welfare services is critical as new service models are incorporated into routine practice. This paper describes a unit costing estimation system developed in England (cost calculator) together with a pilot test of its utility in the United States where unit costs are routinely available for health services but not for child welfare services. The cost calculator approach uses a unified conceptual model that focuses on eight core child welfare processes. Comparison of these core processes in England and in four counties in the United States suggests that the underlying child welfare processes generated from England were perceived as very similar by child welfare staff in California county systems with some exceptions in the review and legal processes. Overall, the adaptation of the cost calculator for use in the United States child welfare systems appears promising. The paper also compares the cost calculator approach to the workload approach widely used in the United States and concludes that there are distinct differences between the two approaches with some possible advantages to the use of the cost calculator approach, especially in the use of this method for estimating child welfare costs in relation to the incorporation of evidence-based interventions into routine practice.

1. Introduction

In child welfare services in the United States and England, there is an increasing emphasis on adopting and implementing new practice models and evidence-based interventions which promise to increase benefit for children and families involved in child welfare processes. A critical element in decision-making for fitting innovative research-based interventions on child welfare platforms is the consideration of the economic costs involved in the delivery, dissemination and implementation processes. In fact, both perception and estimation of these costs may play a decisive role in the entire process, from the early exploration of what new services to consider, through the adoption and preparation stage, to actually implementing, and finally sustaining the service as it becomes part of usual care.

There are two major barriers to a systematic understanding of these costs and the use of cost information for data-informed decision-making in the implementation process. First, as Goldhaber-Fiebert, Snowden, Wulczyn, Landsverk, and Horwitz (2011) have shown, economic evaluation research has been under-utilized in the area of child welfare services, and especially in the decision-making process of importing innovative and evidence-based interventions into the child welfare context. Second, unlike the robust development and use of unit cost structures in health services, child welfare traditionally has not developed a unit cost structure, especially for the calculation of time and costs related to casework practice. This has made it difficult to estimate child welfare costs accurately when considering new services to be brought on line. As Goldhaber-Fiebert, Bailey et al. (2011) recently demonstrated in a proof-of-concept paper about a potential new child welfare service being considered for scale-up, the use of modern decision-analytic microsimulation models to assist the consideration of new practices relies ultimately on an accurate consideration of economic costs.

Measurement of time associated with child welfare service delivery activities is essential to the development of unit costs because time units are linked to salaries (including benefits) within budgetary calculations. Since child welfare service activities are typically carried out by case workers, estimation of the time they take to carry out these activities is required. In England a precise method using a cost calculator tool has been developed and used extensively for estimating case work time in child welfare activities (Holmes & McDermid, 2012; Holmes, McDermid, Padley, & Soper, 2012; Ward, Holmes, & Soper, 2008) and for explicitly linking time estimates for what are termed child welfare processes to costs that may vary from context to context. The cost calculator tool is an agency-specific interactive computer system that generates reports and analyses that link the time estimates and unit costs with the outcomes achieved for children and young people. In addition, the cost calculator tool and underpinning methodology (referred to as the cost calculator approach throughout this paper) has been used in assessing costs of implementing new practices in the child welfare system and comparing them with usual services (Holmes, Ward, & McDermid, 2012).

This paper first presents the basic elements of the cost calculator approach. Second, the paper presents preliminary findings from pilot studies examining the potential use of the cost calculator for estimating time and unit costs in child welfare services in Oregon and California in the United States. Third, the elements and processes of the cost calculator approach are juxtaposed with a second methodology for estimating time and generating costs for child welfare services that has been widely used in the United States, namely, the workload study approach. The comparison between the two approaches also is used to examine the utility of the cost calculator for estimating time and costs for implementing and sustaining new evidence-based practices in the context of child welfare services, especially within the framework of the four stages of exploring, adopting, implementing, and sustaining these practices in child welfare settings (Aarons, Hurlburt, & Horwitz, 2011). Finally, we speculate about a vision for the future in when unit costs can be easily estimated and used to improve child welfare services for the benefit of children and their families who receive these services.

2. The cost calculator approach

In England, the cost calculator tool has been utilized in many local child welfare services to calculate unit costs of core casework processes and associated administrative costs. The tool was initially developed to cost services provided to children in out of home care (Ward & Holmes, 2008;Ward et al., 2008) and has since been extended to cover a range of other services including the implementation of Multidimensional Treatment Foster Care (MTFC), an evidence-based intervention developed in the US and implemented in English child welfare systems (Chamberlain et al., 2011; Holmes, Westlake, &Ward, 2008). The methodology allows for a very flexible application and has been extended from focusing exclusively on out of home care to include cost calculations for all ‘children in need’¹ (comparable to children supported in ‘in home’ care in the US) and vulnerable families requiring additional support other than child welfare services (Holmes & McDermid, 2012; Holmes, McDermid, & Sempik, 2010; Holmes, Ward, & McDermid, 2012). The methodology also has been extended to include activities carried out by education departments to support children with additional educational needs (Holmes, Ward, & Lam, in press). These extensions provide evidence to explore costs and outcomes longitudinally, following cases over time as they move into the child welfare system and between in home and out of home care. They also are critical for work on all child welfare services and associated costs and for providing away to assist in the dissemination and implementation of evidence-based treatment and preventive interventions.

2.1. Development of a conceptual framework

Child welfare services are provided by 152 local authorities in England. All of these local authorities are mandated by legislation from central government to provide similar processes and functions. The conceptual framework for the cost calculator approach was originally developed using nationally applicable documentation, primarily the Core Information Requirements Process Model (Department of Health, 2001), which specifies the core activities that underpin the delivery of placements and services to children in out of home care. These activities were then broken down and organized into eight social care processes (detailed in Box 1) that are carried out for children placed in the care of local authorities. This conceptual framework then was piloted across a number of local authorities. This pilot research indicated that this is a universally applicable conceptual framework that is transferable across all local authorities in England.

Box 1.

Child welfare processes for children placed in out of home care in England.

Process number	Process description
Process 1:	Deciding a child needs to be looked after and finding a first placement
Process 2:	Care planning
Process 3:	Maintaining the placement
Process 4:	Leaving care/accommodation
Process 5:	Finding a subsequent placement
Process 6:	Review
Process 7:	Legal interventions
Process 8:	Transition to leaving care services

All children placed in out of home care go through the first four processes: in every case, a decision has to be made as to whether the child needs to be looked after by the local authority and a placement found (Process One), care planning is mandatory in England (Process Two) and all children need to be supported once in placement (Process Three). Process Four is carried out at the end of the care episode, whether the child moves on to adoption, returns home or becomes independent. Processes 5 to 8 are undertaken for some children: in England those placed in out of home care for more than 28 days are subject to the review process, many will move onto subsequent placements and some will require legal interventions, such as Care Orders. Young people who come under the provision of the Children (Leaving Care) Act 2000 (Department of Health, 2000) also will be entitled to leaving care services (Process 8).

The development of a standardized, universally applicable conceptual framework facilitates an exploration of activity over time. Fig. 1 illustrates how the conceptual framework can be depicted for a period in out of home care over the course of one year time period.

Fig. 1.

Timeline showing the English processes for out of home care.

All of the processes, except Process 3, are discrete events that happen on a specific date, and as Fig. 1 shows, may occur on several occasions during a care episode. Process 3 constitutes ongoing support that is provided for the duration of the care episode, and the activities are reported on a per day, per week or per month basis. Once the unit costs for each of these processes have been developed (see Section 4 below), the costs accrued by a child over a specific time period can be calculated by multiplying the Process Three per diem costs by the number of days covered and then adding up the number of times each of the other processes occurred and multiplying these by their specific unit costs.

2.2. Calculation of unit costs within the longitudinal framework

To estimate the unit costs of child welfare services, the cost calculator approach uses a ‘bottom up’ methodology, in which costs are built up from an individual child level, based on all the support and services that they receive during the time that they are placed in out of home care (Beecham, 2000; Holmes, Lawson, & Stone, 2005; Ward et al., 2008). The amounts of time spent on each of the eight processes outlined above are costed using appropriate hourly rates. The method then links time spent to data concerning salaries, administrative and management overheads and other expenditure, including any fees or allowances paid for the child's placement.

In order to carry out the ‘bottom up’ estimations that form the basis of the cost calculator approach, a number of different data items are required. These can be separated into financial information, for example, salaries and organizational overheads, service information and the time use activity data for the eight processes outlined above. The cost calculator tool also makes use of child level data, concerning needs, placements and outcomes. Bringing together these data items facilitates the longitudinal analysis of costs and outcomes for children placed in out of home care. Fig. 2 outlines the different types of data that are used by the cost calculator tool and the outputs that are produced.

Fig. 2.

The cost calculator approach.

3. Cost calculator inputs for the calculation of unit costs

3.1. Input: time use activity data

To ascertain the time spent on each of the eight processes, all of the activities carried out by a range of child welfare practitioners have been identified. The activities have been divided into two distinct categories: ‘direct work’ with children and families, including both face-to-face meetings and telephone calls and also ‘indirect work’ including attendance at meetings, liaising with other professionals and case recording. Non-client related activity also is handled with the activity being allocated at an agency level. Supervision also is handled with the time spent on supervision being included as an administrative activity. Research in England has identified a standardized allocation of non-client related activity that has been adopted for all local authorities to account for activities such as training and supervision (Curtis, 2011).

3.1.1. Collection of activity data

Unlike some other approaches to collection of activity data, the cost calculator methodology does not adopt a 100% stance but instead focuses on the level of activity required to support specific types of cases, or for children with differing needs.

A range of methods have been used to collect the time use activity data. These included retrospective methods utilizing focus groups and online surveys as well as diary methods. In the latter instance, event records (diaries) were developed to collect prospective data, and child welfare case workers were required to record all the activity that was undertaken for a specific case, for a three month time period. The data collected from all three methods has been triangulated, to test out its reliability and validity (Becker & Bryman, 2012).

For event records used in England the focus of the cost calculator time use data collection is placed on the child rather than the caseworker. Across all of the methods, participants were initially asked to report the time spent on all of the activities for each of the eight processes for a standard case. Preliminary discussions with child welfare workers led to the definition of a standard case as a child or young person with no identified additional needs, placed in local authority (child welfare agency) provided foster care, within the area of the placing authority. Further discussions with child welfare workers led to the identification of a number of variations.

3.2. Variations in activity

Although the cost calculator approach is based on a universally applicable framework that is standardized across the eight processes, research in England identified variations in activity according to children's needs and circumstances, according to the placement type or service provided, and as a result of differences in local area policies and procedures. These variations in activity and placement costs can produce dramatic differences between care populations and service providers.

3.2.1. Variation by children's needs

The research identified four types of additional needs that impact the level of activity and the cost of service provision. The various different combinations of these needs produce 11 groups of children: five simple groups which display none or one of the additional, cost related needs, and six complex groups of children which display two or more. These groups are detailed in Box 2.

Box 2.

Child needs that impact on costs.

Simple groups
	Children with no evidence of additional support needs
	Children with emotional or behavioral difficulties (EBD)
	Young offenders (Offend)
	Unaccompanied asylum-seeking children (UASC)
	Children with disabilities (CWD)
Complex groups
	CWD + EBD
	EBD + Offend
	UASC + EBD
	CWD + Offend
	CWD + EBD
	UASC + CWD + EBD

Children with the most complex needs were identified as requiring higher levels of ongoing support. Child welfare workers consistently reported that placement finding activities take significantly longer for children classified as difficult to place (those with emotional or behavioral difficulties who have experienced more than three placements in the last year). The research also identified examples of how children with particularly challenging needs can require disproportionate levels of activity; these children are very few – about 3% of the care population – but their cost-related needs are so extensive that they can skew the whole budget within a local authority. Furthermore, the research highlighted that these children were some of the least likely to return home and so were likely to remain in placements until the age of eighteen — a major factor to take into account in long-term budgetary planning.

3.2.2. Variations by placement type

Children's needs should be reflected in the type of placement considered most appropriate. Placements in residential care in England cost eight times those of placements in local authority foster care and are generally reserved for those with the most challenging needs. Exploring whether children are placed appropriately may lead to more efficient use of resources.

Variations in placement finding activities also were identified for different types of provision with it taking up to eight times longer to find a placement in a residential setting than to find one with local authority foster carers. Child welfare workers also reported that placements out of the area of the authority were more time consuming to support as a result of the additional travel time to visit the children in their placements.

3.2.3. Variations according to local authority practices and procedures

The vast majority of the activities carried out by child welfare workers were comparable across all local authorities; however, there were some instances when local authorities had introduced specific policies or procedures that resulted in different levels of activity, or additional activities. For example, some authorities had introduced placement disruption meetings following an unplanned change of placement; other authorities had introduced panel meetings attended by a range of professionals to make the initial decision as to whether to place a child away from home.

3.3. Input: overheads (i.e. the cost of running the organization)

In addition to the identification of variations in levels of activity to support children in out of home care, research in England has identified that differences in costs also are attributable to the range of elements included in the calculation of overheads (child welfare running costs) as well as the cost of these items. The salaries of the staff employed by the child welfare agency also impact unit costs. A study carried out by Selwyn, Sempik, Thurston, and Wijedasa (2009) demonstrates that the way in which overheads are calculated leads to dramatic differences in costs between agencies performing the same function. This study has produced an overheads framework to provide a standardized approach to include these costs (Selwyn et al., 2009); subsequently this overheads framework has been piloted and utilized across a number of research studies (Holmes & McDermid, 2012; Holmes et al., 2010). This has produced a standardized formula: the studies have shown that the mean value for overheads is 45% of employee costs (see Curtis, 2011), this is now commonly used in England. Notably, the standardized overhead framework has made it possible to deconstruct the percentage allocated for overheads in English studies. This facilitates the identification and comparison of the cost of the various elements that go into this calculation, thereby making it possible to identify common reasons for discrepancies in costs incurred by different agencies.

4. Cost calculator outputs

As Fig. 2 illustrates, the cost calculator tool makes use of the different data elements and produces a range of outputs making it possible to link children's needs and experiences with the cost of providing services and outcomes. Furthermore, the cost calculator tool stores each of the different types of data separately. This structure makes the model very flexible, because partial new data can be substituted and new cost estimations can immediately be carried out to produce revised outputs.

By adopting a longitudinal approach, the cost calculator research in England has highlighted how the needs and circumstances of cases change over time, for example, there may be improvements or deteriorations in emotional or behavioral well-being. Key life events for children in out of home care, such as a change of school, also impact on the level of support that is provided. Furthermore, children's age has been identified as a factor that impacts on activity and costs (Holmes &McDermid, 2012). As such, the level of activity carried out by child welfare workers to support children also changes as they grow older (Holmes & McDermid, 2012;Ward et al., 2008).

The longitudinal approach also enables cost modeling and future predictions. For example, identification of a cohort of younger children placed away from home that are likely to remain in placement can facilitate a prediction of the future level of activity and also the projected costs associated with their anticipated care episodes. The cost calculator produces ‘what if’ analyses to investigate differences in costs. For example, it is possible to explore how costs differ if children with a specific type of need are moved to alternative placement type. In England, this function has been used to inform decision making when commissioning placements from a new provider.

5. Utility and adaptation of the cost calculator for the US

As part of the adaptation of the cost calculator tool to Child Welfare systems in the United States, discussions were held with a small group of child welfare administrators in Oregon (a state-administered child welfare system) and California (a county-administered child welfare system) to determine whether the 8 core processes developed during the UK work were applicable to the US. Although the terminology is different between the two countries, the underlying child welfare processes were perceived as very similar, with the exception of Process 6: Reviews, which in both Oregon and California are closely connected with Process 7: Legal. Case workers typically conduct a review during the development of a court report and there often is not a review process that is separate from the court hearings. Table 1 provides a comparison of the core processes between England and California.²

Table 1.

Description of the eight child welfare processes for children placed in out of home care in England and California.

Process	Description	Comparison between England and California
Process 1: decide child needs to be placed	Includes all the activities directly related to placing a child in out of home care and finding the initial placement.	Although the type of assessments and some of the activities carried out differ between England and California, this process is comparable.
Process 2: care/case planning	Includes all the activities associated with the completion of the Care Plan (England) or Case Plan (California).	Although the Care/Case plans are comparable, in California the activities would usually be carried out within the first 30 days of being placed. In England the Care Plan will be updated at key intervals during the time the child is placed in care and therefore this process is repeated. Any on-going Case Planning activity in California has been included in Process 3. In England this process also includes the activity for a statutory annual health assessment and education plan.
Process 3: maintaining the placement	Includes all the on-going activity to support the child in their placement. This process includes visits to the child and family and indirect case work, such as recording and liaising with other agencies.	Directly comparable.
Process 4: child ceased to be in care	This process begins with the decision for the child to transition from being in care. This may be as a result of a return home, adoption, transition to adulthood or the case being closed.	Directly comparable.
Process 5: find subsequent placement	This process is concerned with placement change and includes all the activity to find any subsequent placement that may be required. This process does not include the activity to find a first placement following the initial decision to place a child (covered in Process One).	Directly comparable.
Process 6: review	In England this process includes all the activities associated with statutory reviews required for all children in care.	Comparable reviews are not carried out in California. Reviews of cases are typically carried out as part of the legal process and as such have been classified as ‘Legal Reviews’ and therefore included in Process 7.
Process 7: legal	This process captures activity related to court involvement, including preparation of court reports and attending hearings.	Although the legal orders and involvement of the courts differs between England and California, this process encompasses comparable activities.
Process 8: transition to leaving care/ addition of independent living services	This process begins when a young person reaches the age when he/she becomes eligible for leaving care (England)/independent living (California) services. It excludes activity when a child returns home (covered in Process 4).	This process is largely comparable in that it includes all the activities carried out to support a young person as they leave care and make the transition into independent living.

After verifying the applicability of the 8 core processes to the US system, focus groups and web surveys were conducted with agency staff in several counties. Staffing in several of the US child welfare systems was more specialized than in the UK, resulting in a larger number of staff types participating in data collection. As mentioned above, the differences in the legal system between the UK and US resulted in the merging of Processes 6 and 7 for the US data collection.

Other differences between the UK and US were related to the variances (described in Section 4). The UK has a significant number of unaccompanied asylum-seeking children who have different levels of need and corresponding associated case worker time. This issue is not commonly seen in California; however, a similar variance is seen for children of Native American heritage due to the requirements of the Indian Child Welfare Act (ICWA). A frequently cited variance in California was the need for translators, particularly for children and families speaking less common languages.

6. Cost calculator in the context of the US workload approach

In the United States, the work of the authors is the first examination of the cost calculator for potential use in the United States. However, to date the workload approach has been used widely in the US (Arnold-Williams & Graham, 2008; deVaron Reynolds, Costello, & Edwards, 2008; Perry & Murphy, 2008; Tooman & Fluke, 2002; Wagner, Johnson, & Healy, 2008) to estimate the time required to carry out child welfare case work activities for the eventual task of calculating costs and carrying out budgetary processes. Tooman and Fluke (2002) provide an excellent review of this approach. They state that “in measuring workload, the first step is to determine the nature of work (services and the tasks performed to deliver services)” (Tooman & Fluke, 2002, p. 1) and further indicate that while services represent the “broad categories of work such as ‘CPS investigation’ or the provision of out-of home services for children”, tasks describe “specific, shorter-term activities, such as telephone calls, home visits, filing, and preparing forms”. Examination of a wide range of workload studies confirms that this step is consistently used for all workload studies (http://www.americanhumane.org/children/professional-resources/research-evaluation/workload.html).

This underscores that a “conceptual” framework is developed de novo each time a child welfare system workload study is accomplished, and infers that each child welfare system is unique. One possible downside of this element is it constrains any comparison across workload studies since each one serves as an individual case study. In other words, workload studies start and end as context specific, and do not allow for real comparisons between contexts since the framing of work and time is idiosyncratic rather than comparable. This severely constrains research that might empirically build findings from across the studies into either common themes (to use a qualitative perspective) or statements that characterize consistent and robust relationships between variables leading to models that can explain and predict (to employ a quantitative perspective).

The multiplicity of ways to conceptually frame case work services and tasks from context to context also is associated with different methods for measurement of case work time to carry out the services and tasks. Complete time logs for all cases worked on during a specified period of time, time sampling, and other methods for capturing time are seen across multiple workload studies. However, the lack of standardization in time estimation methods coupled with the absence of a common conceptual framework regarding the units of work involved make it virtually impossible to compare and contrast findings from what might be characterized as case studies. This makes the development of a standardized application for estimating time and costs that could then be employed in the process of modeling and eventually conducting simulations or ‘what-if?’ scenarios virtually impossible.

While there is variation across the workload studies in methodology for estimating time for casework activities, most of the studies use a 100% census approach as Tooman and Fluke (2002) have put forth, namely, that “data were collected over a period of weeks using self-reporting by social workers. This common use of a ‘100% workload study’ approach has two important facets. First, every worker in the agency who met the criteria for inclusion was asked to participate, making the study a virtual census of all employees rather than a sampling of a handful. Second, all participants recorded all of their work time in a day, whether they worked on a weekday or weekend, during regular working hours or late evening (p. 1)”.

This 100% census approach as the dominant method observed across most of the workload studies leads to our observation that the approach places considerable burden for participating agencies and their personnel. While all collection of time for caseworkers delivering services is an estimation because of the inevitable “holes” in the time diaries, the workload approach is an estimation from a census of 100% attempt. This is likely the reason that workload studies are static or cross-sectional in nature and one-time estimations. No workload studies that we have read report on time and costs associated with longitudinal trajectories. In this sense, the workload approach is a static rather than dynamic picture of time and associated costs in the delivery of child welfare services. This aspect suggests that workload studies have also led to single reports of case work time and associated costs rather than to a management tool or set of procedures that could be used to estimate potential changes in time requirements and costs that might be expected with the introduction of new services to replace or augment existing provision.

A third element in the workload approach is the handling of “noncase-related time” (e.g., supervision and training). Tooman and Fluke (2002) see this as terminology used instead of “administrative” and further suggest these are really activities done in the service of the agency rather than of the clients it serves. They write that “non-case-related time therefore could be viewed as time a worker provides services to an agency, as opposed to time he or she provides services to clients” (pp. 1–2).Two comments could be made in response to their statement. First, and importantly, if administrative time/costs are not related to specific cases, then these costs cannot be linked to client level outcomes, making it difficult to consider these costs within the context of a new practice that may improve such outcomes. Second, if these costs are in service to the agency, then calculation of them may be considerably non-standardized because of marked differences in agency processes and overall organizational agenda. In either case, conducting research to model time and costs would again require standardization of administrative costs and time that was independent of agency and context. In short, standardization counts.

The fourth and possibly most critical element of workload studies is that they follow caseworkers' time as they fulfill the requirements for service delivery and tasks associated with these elements of services. The unit of analysis is the time spent by caseworkers fulfilling job requirements associated with cases. Cases are further differentiated by “special characteristics”; Tooman and Fluke (2002) give examples of “in-home services and placements cases [which will be] fundamentally different, given that a different type of service is provided (p. 5).” They also discuss types of cases such as rural or urban settings, and number of children in a household. Furthermore, the amount of time spent on a case is influenced by caseload size, again making comparisons between agencies problematic. These aspects of the workload studies contrasts with following individual children who may differ remarkably in the types of needs they may have for services, the strategy taken by cost calculator approach which has a sharp focus on the needs of individual children. In essence, the workload studies do not use the unit of individual children but the unit of caseworkers fulfilling practice and policy requirements. This is a case-based approach that is cross-sectional in nature rather than a child based approach that is longitudinal in nature. The workload framework is time associated with fulfilling requirements at the case level rather than an “over-time” or longitudinal approach linked to different levels of needs, that allows children's needs to be linked with costs and subsequently outcomes.

To understand the cost calculator approach within the US with its current dominant use of the workload approach, we summarize the major similarities and differences between the two approaches.

There are a few fundamental similarities between workload studies and the cost calculator approach. Both focus on the activities carried out by child welfare case workers, initially identifying the nature of their work and the specific tasks that are carried out to deliver services to vulnerable children and their families. In both approaches, these activities are then linked with salaries and non-case-related administrative running, or overhead costs, with the purpose of using this information to calculate costs and plan future child welfare budgets. Across workload studies carried out in the US, Tooman and Fluke (2002) report an estimate of around a third for administrative (overhead) costs and further suggest that there is ‘a good deal of convergence around a general one third rule for human services’ (p. 2). This compares to current estimates of 45% of employee costs for administrative (overhead) costs in the UK.

However, we also believe there are three fundamental differences between the two approaches. Firstly, the cost calculator approach has been developed as a standardized framework that is transferable across all child welfare agencies in England, thereby introducing transparency into the services provided for children placed in out of home care and facilitating comparisons between different local agencies (called local authorities in England) and service providers. Secondly, the approach identifies different levels of activities, and costs, associated with supporting children and young people with different levels of needs, and different approaches to service delivery between providers (both local authorities and independent and voluntary agencies), thus further facilitating comparisons. Thirdly, the approach provides a longitudinal perspective rather than a snap shot so it is possible to explore how costs accrue over time, link these to outcomes, model likely future costs, and introduce ‘what if’ analyses.

7. Utility of cost calculator approach for evaluating the use of evidence-based practices in child welfare settings

The introduction noted the current emphasis on considering and implementing evidence-based interventions for child welfare populations and in child welfare settings. This section discusses the potential utility of the cost calculator approach with some comparisons made into usual care in child welfare. From the outset, we note that the workload and cost calculator approaches have both demonstrated general utility in their respective United States and English contexts by repeated and continuing use and that this is likely due to their effectiveness in addressing the central purpose for which they were developed. However, we also note that each approach was developed to fulfill somewhat different purposes, with the workload approach developed to estimate time and costs at the casework level necessary for caseworkers to carry out all activities mandated by federal and state regulations and the cost calculator approach developed to estimate time and costs for carrying out social care activities at the individual child level within the regulatory context. In the United States, the usual outcome has been to demonstrate that the work force necessary to carry out all of the service delivery mandates is much larger than provided by current budgets for the specific child welfare systems. This serves a budgetary and ultimately political allocation purpose.

In England, the cost calculator approach with the focus on time and costs associated with individual children was developed as a tool to link time and costs with outcomes, especially as they relate to child trajectories that had been observed to emanate from greater need for services and result in poor outcomes. The more standardized approach employed by the cost calculator also allows for examining differences in time, costs and outcomes between differing types of children, especially based on need, and different types of child welfare service contexts. This difference in approach may explain why the cost calculator has been used in many research studies in England while the workload approach has generated no research studies of children and outcomes but remains a basic tool for budgetary and administrative purposes.

Although there are a number of similarities between the two approaches to estimating time and costs, our comparative examination suggests that specific differences between the approaches make the two quite different in their ability to be used in understanding and assisting the dissemination and implementation processes for bringing innovative practices into child welfare services. We believe that the cost calculator approach has at least four principal advantages.

The first advantage is that the cost calculator, in contrast to the workload approach, assesses time and costs using care trajectories for individual children over time. This longitudinal characteristic allows for considering time and cost shifts associated with the potential exploration, preparation, implementation and sustainment of new innovative practices. A cross-sectional approach, such as that used by the workload approach, was not designed to consider change in service activity over time.

While it is possible to consider multiple workload studies to estimate change in time and costs associated with change in practice related to implementation of new services, the 100% method makes the workload approach unrealistic because of the high burden.

The multiple step estimation methodology used in the cost calculator approach is the second advantage because its estimation process lowers the burden of repeated estimations in typical pre/post evaluations, thereby leading to a method for calculating the cost–benefit ratios needed in understanding cost as an implementation outcome (Proctor et al., 2011). Furthermore, the costs associated with diverse longitudinal trajectories at the child level are ideally suited for connecting child level outcomes to caseworker effort and costs. Ultimately, the only way to estimate both the well-being outcomes and associated costs is through following individual children over time. The workload approach simply was not designed for that purpose.

This emphasis on the child as the unit of analysis, in contrast to the case worker, is the third advantage because it allows the cost calculator the ability to carry out time and cost studies within implementation research. This advantage is multiplied by the ability of the cost calculator to link time and costs to outcomes, an essential part of understanding and predicting the change in outcomes that are linked to the mechanisms that drive the specific evidence based interventions. Implementation of innovative practices is essentially about the promise of improved outcomes at the level of individual child and families and the sustainability of these potentially improved outcomes over time.

Finally, the fourth advantage of the cost calculator approach is the utility of its standardized conceptual framework for comparing outcomes and associated time and costs across service delivery settings and systems. The universal or standardized methods allow for confidence that differences in aggregated outcomes, time and costs are not due to method or framework differences but to real differences in contextual features while holding framework and methodology constant.

The advantages of the cost calculator approach in addressing the four stages of implementation is well illustrated by its application to the implementation of Multidimensional Treatment Foster care (MTFC) in England to examine the cost of setting up and maintain the MTFC program, compared with the costs for other treatment as usual programs to serve children with similar needs. In addition, costs for individual children receiving MTFC services were examined and compared to their costs in the year prior to MTFC involvement, as well as the costs of the placement they would have been in if MTFC were not available. The average monthly costs of an MTFC placement were higher than those for regular foster care, but less than the costs for treatment foster care or residential care. Overall, the study showed a reduction in child welfare costs when children were placed in MTFC (Holmes et al., 2008).

As described above, the cost calculator is being adapted for use in California Child Welfare agencies, with the goal of generating a standardized method that can be used in decision-making. This effort in being conducted in collaboration with the California Partners for Permanency (CAPP, www.reducefostercarenow.org), one of six federally funded projects in the US to reduce long term foster care. The cost calculator is documenting the costs of implementing the CAPP intervention in four counties through a two stage process. Baseline time and cost data were collected prior to implementation of the CAPP practice model in each county, and following the full roll-out of the model, data collection will be repeated. This will allow for the examination of the cost of delivering the services in the intervention's practice model, as well as examining the new practices' impact on overall child welfare costs, such as caseworker time and expenditures for out of home placements. We will also be able to examine differences in how case workers spend their time before and after implementation. Given the goal of CAPP to reduce out of home through increased support and partnership with children and families, it would be expected to alter the amount of time workers spend on each process; for example, increasing the amount of time spent in direct and indirect contact with families and children during the case planning process, while decreasing the amount spent on maintaining out of home placements.

8. Conclusion — a vision for the future

This paper has compared two different methodologies for costing child welfare services, the workload approach commonly used in the United States, and the cost calculator approach developed in England. However, analyses of costs for child welfare services provide only part of the picture. Reducing child welfare costs may increase youth justice costs, for instance, as children and young people who are refused entry to care may be more likely to commit offenses and enter the criminal justice system. The longitudinal approach modeled by the cost calculator for children's services is sufficiently flexible to allow for calculations of costs for all services accessed over a specific time period, so that costs to the public purse can be calculated, rather than those incurred by any specific agency. In England, some of this work has already been undertaken through a program to add education costs to those of child welfare in the cost calculator model (Holmes, Ward, & Lam, in press).

However, calculating the costs of services can do little more than help monitor expenditures and identify where efficiencies might be made. The costs themselves tell us little about whether or not children benefit from services in the short or the long term. The Holy Grail is to link costs to well-being outcomes so that the effectiveness of services in promoting children's satisfactory development and enabling them to achieve their potential can be monitored over time for individuals and groups, and comparisons made.

The cost calculator approach has the potential to link costs to outcomes. Data concerning child wellbeing outcomes, such as educational attainment, emotional and behavioral development and criminal convictions, can be included amongst the child level data that is inputted into the model (see Fig. 2), and the demonstration version of the model includes data on educational qualifications that can be linked to the cost of care episodes. However, although the model can accommodate such data, they are rarely accessible or routinely collected. In order to explore the cost effectiveness of services in terms of child wellbeing outcomes, we would need baseline data on child welfare outcomes at entry to a service, and routine collection at regular intervals thereafter. In the UK, there are regulations in place to require some of these data collections. For instance, local authorities are now required to collect and publish annually scores from the Strengths and Difficulties Questionnaire on all children who are placed away from home for a year or more, thus providing evidence of changes in the emotional and behavioral problems of the care population. However, these data are not always presented in such a way that they can be linked to individual children and related to costs, and indeed collection is often incomplete or inadequate. So while the potential to link costs to outcome is there, it cannot yet be fully realized.

In the US, it may be possible to utilize data collected for the federal Child and Family Service Reviews to compare costs and outcomes; however, it is not clear at this time whether the data is sufficiently detailed to be able to link to cost data. However, more promising are the efforts at the federal level to focus on measurement of child well-being that are exemplified in recent information memoranda from the Administration for Children and Families such as the two issued this year, namely, “Promoting the Safe, Appropriate, and Effective Use of Psychotropic Medication for Children in Foster Care” and “Promoting Social and Emotional Well-Being for Children and Youth Receiving Child Welfare Services.” We also note the robust use of standardized measures of child well-being used in the landmark study, National Survey of Child and Adolescent Well-Being and the large number of papers published from this study (http://www.acf.hhs.gov/programs/opre/research/project/national-survey-of-child-and-adolescent-well-being-nscaw-1), which are being used by the leadership of the Administration for Children and Families in their policy briefs.

These promising developments for a sharper focus on child well-being outcomes and their measurement in child welfare service systems in both England and the United States provide hope that the impact of innovative and more evidence-based interventions may demonstrate strong improvement in child well-being. If the recording of well-being and outcomes indicators is improved at the individual child level then the advantages of the cost calculator to link outcomes to costs could be utilized. This in turn would then bring far stronger evidence for the benefits and costs for providing improved services in both countries.