Measuring Time Costs in Interventions Designed to Reduce Behavior Problems Among Children and Youth

Abstract

The economic evaluation of psychosocial interventions is a growing area of research. Though time costs are central to the economist’s understanding of social costs, these costs generally have been ignored by prevention scientists. This article highlights the need to measure such costs and then reviews the principles economists use in valuing time. It then considers the specific time costs that often arise in interventions designed to reduce behavior problems among children and youth. These include classroom time devoted to program activities, the time of parents or other caregivers, the time of teachers (outside of the classroom), and the time of volunteers. We consider the economic principles that govern how economists value these inputs and then apply these principles to data from an evaluation of a prominent intervention in the field, the Incredible Years Program. We find that the time costs are potentially rather large and consider the implications for public policy of ignoring them.

Interest in the economic analysis of interventions and services to prevent or treat behavioral health problems among children and youth is growing rapidly (Aos, Lieb, Mayfield, Miller, & Pennucci, 2004; Foster, Dodge, & Jones, 2003). These analyses take various forms, such as cost‐effectiveness analyses (Jensen et al., in press) or various forms of benefit‐cost analyses, including analyses of the effect of improved services on spending on public services (Foster & Connor, in press‐a, in press‐b). These different forms of economic analyses all build on a common foundation, an estimate of program or service cost.

Much of the information needed for estimating these costs can be found in program budgets¹, and the methodology for estimating those costs has been developed and refined by health economists (Drummond & Mcguire, 2001; Drummond, O'Brien, Stoddart, & Torrance, 1997; Gold, Russell, Siegel, & Weinstein, 1996; Haddix, Teutsch, & Corso, 2003). (Note that by budgetary costs we mean expenditures actually realized rather than those planned.) One area that remains unresolved, however, involves the value of time that study participants spend in intervention activities and treatment, especially when they are not compensated.

Participants’ time costs can be important in understanding the true costs of an intervention, especially from a societal perspective. Although budget expenditures for two interventions may be comparable, one intervention may require a great deal less of participants. In that case, the less intensive intervention is less costly in terms of society’s resources. The time costs generally will not appear on program budgets, and for that reason, an agency or other payor may view them as irrelevant. However, participation may be lower and dropout higher in a time‐intensive program. Time costs also may influence parent involvement in other ways, such as the quality of participation. Given the stresses of modern family life, a program that requires less of parents‐all else equal‐likely will be more successful in engaging parents. For that reason, the time costs still may be of interest to community agencies and policy makers.

The issue of participants’ time costs seems important especially in interventions targeted to preventing or reducing emotional and behavioral problems in children and youth. These interventions are often labor intensive. In addition to clinical staff, teachers, and volunteer personnel (such as mentors), children and/or their caregivers spend time in intervention activities. This time includes that spent in group or individual sessions as well as time spent preparing for those sessions. Such activities could involve homework for upcoming sessions or practicing skills learned previously. These time costs may be rather large as a proportion of total costs, especially in mental health interventions where medical equipment or devices play a smaller role (relative to, for example, treatment for cardiovascular disease or cancer).

Economists disagree on how to value such unpaid time (Drummond & Mcguire, 2001; Drummond et al., 1997; Hargreaves, Shumway, Hu, & Cuffel, 1998; Posnett & Jan, 1996). For example, a recent controversy involves whether such costs should be treated as “frictional”‐i.e., as vanishing after an adjustment period (Brouwer, Koopmanschap, & Rutten, 1997; Koopmanschap, Rutten, van Ineveld, & van Roijen, 1999). Such disagreement, however, should not obscure the fact that economists share a core set of beliefs about how markets work and how individuals make choices. These beliefs imply that time should be valued in terms of its “opportunity costs”‐that is, the value of the time that would have been spent in activities that were foregone as a result of treatment and/or illness. The disagreement often starts with identifying, measuring, and valuing those foregone activities. This article reviews the possible alternatives for different types of time costs below.

These issues are not only relevant but also more complicated in the case of programs targeted at behavior problems in children and youth. Unlike treatment for cancer or heart disease, participation in parenting groups or other prevention programs is not inherently unpleasant. As discussed below, however, this enjoyment need not imply that participation is costless.

This article considers the issue of participants’ time costs in preventive interventions. It begins by highlighting the need to measure such costs and then reviews the principles economists use in valuing time. The article then considers the time costs that often arise in interventions designed to reduce behavior problems among children and youth. Such interventions may involve working with the child directly, parent training, and/or teacher training. The intervention also may involve a classroom program delivered by the teacher during the school day or by a program provider after school. These components raise the possibility of four types of unpaid time costs—classroom time devoted to program activities; the time of parents or other caregivers; the time of teachers (outside of the classroom); and the time of volunteers. The article considers the economic principles that govern how economists value these inputs and then applies these principles to data from an evaluation of a prominent intervention in the field, the Incredible Years Program.

BACKGROUND

Why time costs count

Why should one assign costs to participants’ time spent in intervention activities? After all, individuals generally are free to participate in a program (or not), so the intervention could not be that unpleasant. The motivation for counting these costs differs somewhat depending on the perspective from which costs (and benefits) are assessed.

Every economic analysis has to establish a perspective from which costs (and benefits) are valued (Gold et al., 1996; Haddix et al., 2003). Such a perspective is essential: The magnitude of many costs depends on the perspective from which they are assessed. For example, one effect of nurse home visitation with new mothers may be to reduce welfare dependence among participating women (Olds, Henderson, Phelps, Kitzman, & Hanks, 1993). This loss of income is a cost to mothers. In contrast, these savings are a benefit to taxpayers.

In thinking about prevention programs, three perspectives seem especially important—that of participants, that of an agency (or other potential funding source), and society as a whole. The last includes all members of society and is the perspective that most interests economists (Gold et al., 1996). Those costs capture the use of society's resources in a program or intervention and are essential to assessing whether a given program is a good use of those resources. Like any scarce resource, time costs are clearly a component of societal costs. The time children, parents, teachers, mental health professionals, or volunteers spend in a program is time that could have been spent on other uses.

Time costs also are important from the participant's perspective. Such costs may have a large (negative) effect on participation‐families may be more likely to drop out of programs that require a lot of their time and during periods when their time is particularly valuable (e.g., during the work day). Understanding these time costs also may illumine whether and how participation varies across families—families for whom the participation costs are lower likely will participate more.

Similarly, as discussed below, teachers participating in training courses in classroom management, for example, may experience time costs as well. These courses may be held on the weekends or in the evenings in order to encourage participation. Such activities reduce leisure time, however. Even if held during the work day, participants may spend their evenings or weekends attending to tasks they otherwise would have accomplished during the workday.

The designers of parenting programs likely (or should) prefer an intervention that requires less of a participant’s time and that produces the same benefit over a more time‐intensive alternative. Furthermore, program implementers may plan intervention activities such that time costs are lower as a means of boosting participation. On the other hand, time costs borne by participants may hold little interest for funding agencies. As noted above, such participant time costs do not appear on program budgets.

How economists value resources

If one is convinced that participant time costs are essential to economic analyses, then how does one proceed? In measuring the amount of time spent on an activity, one can rely on self‐reports. Such reports can be retrospective or can involve time diaries (Hargreaves et al., 1998). The latter presumably would be more accurate, but as with other aspects of study design, one must balance accuracy and reliability of measures with the costs of collecting the information. These costs depend on several other aspects of the study design, such as whether an evaluation is already collecting information from parents or teachers (e.g., reports on children’s behavior).

Having collected information on the amount of participant time, one must attach dollar values to that time in some way. Before considering the specifics of time costs, a brief review of how economists measure the value of resources in general is useful.

Two principles guide how economists conceptualize and measure costs of any type. First, they are most interested in the marginal opportunity costs of a program. Second, in trying to measure the costs of resources, they look to markets to provide a measure of the value of the resources involved. In some cases, the input of interest is not bought or sold; in those instances, economists still try to infer the value of the input from markets whenever possible. In the case of unpaid time, economists look to labor markets to infer the value of this time.

Marginal opportunity costs are the costs that matter

In assessing the application of resources to one use over another, economists are most interested in the marginal costs—the incremental or additional costs of the activity or intervention of interest relative to the activity that would have occurred in the absence of the program. These costs correspond to changes in resource use actually caused by the decision of interest. Depending on the scale of the intervention, the marginal cost is often less than the average cost per participant of a program. For example, consider the costs of educating an additional student in elementary school. Those costs may be quite a bit less than the average cost per child. Increasing enrollment by a single student most likely will not require the hiring of an additional teacher. In many cases, all that will be required will be a desk as well as additional materials.²

While seemingly straightforward, the definition of marginal costs is a bit slippery. One reason is that the marginal costs depend on how “the margin” (or unit) of interest is defined and on the time frame of interest. Regarding the former, if we are interested in adding a classroom of students, then marginal costs include the costs of hiring and paying a new teacher. For that margin, the average and marginal costs (per student) are likely similar. As noted, however, the additional cost of a single student at the margin is substantially less.

A related issue involves the time frame of interest. In the short term, the funds used to construct the classroom building are “sunk” in the sense that they have been expended and cannot be used elsewhere. As a result, their opportunity costs are zero. A long‐run perspective, however, recognizes that educational facilities depreciate and must eventually be replaced. For long‐term planning, the marginal costs of educating an (additional) student would include those facilities. As the time frame for a study lengthens and as the scale of the program grows, marginal and average costs converge.

Economists look to markets to define the value of resources

For many program inputs, such as materials, information on costs can be obtained from project budgets. Payments in those budgets reflect market prices for the inputs involved, and from the payor perspective, these payments represent costs. In many cases, those prices reflect the societal or opportunity costs of the inputs involved as well, especially if those inputs are bought and sold in competitive markets. Such markets are characterized by many buyers and sellers, none of whom is large enough to control the price of the good involved (Browning & Browning, 1992). (Competitive markets have other characteristics, such as free entry and exit of firms. For details, see Browning and Browning, 1992.) In such a market, supply and demand drive prices to a level equal to the marginal cost of producing the good involved. If prices are above that level, firms enter the market and drive down prices. If prices are below marginal costs, firms lose money and exit the market over time, reducing supply and raising the price.

The link between market and social prices is broken in two situations. The first situation involves “imperfect markets,” such as markets into which firms cannot enter. Unfortunately, these types of markets are quite common in health care. A good example is the market for non‐generic pharmaceuticals. These drugs can be produced only by patent holders; new firms cannot enter. In that case, the economic theory of monopoly would apply, and the price of a medication will be higher than the marginal cost of producing it. The degree to which price exceeds marginal costs depends on several factors, including the elasticity of demand (i.e., how sensitive quantity demanded is to the price of the good).

The second situation involves other kinds of key inputs that are bought and sold outside of a competitive market. For example, prevention, educational, or mental health program materials may be purchased from program developers, who may hold copyrights. However, estimating whether prices exceed marginal costs for these inputs is difficult. The developers of these programs likely do not maximize profits in the same way that drug companies do. The prices paid for these materials represent the costs to the payors, but whether and how these payments correspond to social costs is very difficult to say.

Economists often are interested in the value of goods that are not traded in markets. For example, environmental economists may want to place a value on a large, beautiful lake (or rather, estimate the value consumers place on the lake). There is generally no market for such lakes, but economists estimate their “price” by examining other, related markets. For example, economists may examine the costs of traveling to and from the lake for individuals living varying distances from it and how travel costs relate to their use of the lake. The link between use and travel costs reveals the consumers’ willingness to pay (Zerbe & Dively, 1994).

In the case of prevention, educational, or mental health interventions, non‐market inputs might include donated materials or volunteer time. For the former, economists attempt to estimate what the items would have sold for if purchased. For the latter, economists rely on their understanding of markets in which time is bought and sold—i.e., the labor market. In the next subsection, we briefly review how economists view the functioning of labor markets.

How labor markets work and what their workings reveal about the value of time

In general, economists treat the labor market as they would any other market, where the “good” is a worker’s time (Phelps, 1978). The normal role of buyer and seller is reversed in that the firm buys the “product” of consumers (i.e., workers), who are the sellers. The demand for labor is based on the productivity of the worker and the value of the good or service produced. Workers who work harder or who produce goods that are in greater demand—all else equal—earn higher wages.

On the supply side, workers decide how much to work as part of their allocation of household resources. In particular, they divide their time among competing uses—leisure activities, household production (e.g., housework, yardwork, or caring for their children) and employment. Workers determine this allocation based on the value of their time in alternative activities. Workers with higher wages will work more; those with lower wages are more likely to stay home and care for their children. This allocation also depends on the price of substitutes, such as child care. Individuals will devote more of their time to household production in instances where substitutes are more costly. For example, mothers living in areas where child care is more expensive may be less likely to enter the labor force. The decisions of individuals in the same family are made interdependently.

Economists view workers as selecting the amount to work up to the point where the wage rate (marginal benefit) equals the value of another hour of leisure or household work lost (marginal cost). This practice assumes a well‐functioning market in which workers can choose how much to work. There are a variety of implications of this framework. First, in equilibrium, workers work the amount they desire given the wage which employers are willing to pay them, which is based on their skills. In that sense, unemployment is “voluntary” in that individuals who are not working value their alternate time use above the wage that they can earn. Second, in this framework, leisure or household production time is actually more valuable than work time. Leisure or time in household production is time individuals chose not to work because the value of that time exceeded the wage rate.

As stated above, this framework assumes a well‐functioning labor market, but the equilibrium it describes occurs over time. “Shocks” occur (such as a recession), causing a disjuncture between supply and demand, which cause the market to adjust. That adjustment occurs with some lag. During this period of unemployment, workers may be unable to find work, and the resulting unemployment is “involuntary.” In that case, a worker might have “excess” leisure time, and the market wage he or she might earn is greater than the value of leisure time.

Finally, note that taxes drive a wedge between the price paid and the price received both in labor and product markets. In the case of goods and services, consumers make their choices based on prices including sales taxes. Producers, on the other hand, make decisions based on prices net of taxes because those prices represent the firm’s revenues. In labor markets, two additional taxes are relevant: payroll taxes paid by employers and income taxes paid by workers. Firms make hiring decisions based on wages or salaries including any taxes. On the other hand, workers make choices based on wages after taxes because net wages represent the money they actually receive in exchange for their labor.³

Note that this entire framework is not without controversy

Wages may reflect a variety of market imperfections, such as discrimination (Princeton University Conference on Discrimination in Labor Markets (1971), 1973; Kain, 1969). In that case, persons of color may be paid less than other, equally productive individuals. As a result, there would be less value placed on their time. In that case, delivering the intervention would be less costly for those families.

One alternative frequently recommended in the literature is to use the same wage for all individuals, and we use this option in our empirical example below (Hargreaves et al., 1998). This practice is simply a way of avoiding the value judgment that some individual’s time is worth more than that of others. Obviously, this arbitrary practice is less than ideal and does not reflect the broader theoretical considerations considered above. However, it is a commonly used alternative to actual wages and does provide some comparability across studies (e.g., Chappell, Dlitt, Hollander, Miller, & McWilliam, 2004; Feldman et al., 2004; Shireman, Tsevat, & Goldie, 2001).

Using this framework in prevention research

There are four broad categories of time costs that one might encounter in a prevention or treatment program targeted to behavior problems among children and youth. These time costs involve classroom time devoted to program activities, the time of parents or other caregivers, the time of teachers (outside of the classroom), and the time of volunteers. We consider each of these in turn and examine the resulting costs from the perspective of the participant, the funding source (such as the school district) and society as a whole. (This information is summarized in the Appendix.)

Appendix: Conceptual Basis for Time Costs.

Participant	Treatment tradeoffs	Circumstances	Perspective Societal perspective = sum of all three
			Participant	Employer	Others (such as taxpayers)
Child	Child learning is reduced		Lost (net) future earnings	NA	Lost tax revenues + other social benefits
	Child learning is not reduced		None	NA	None
Parent	Parent loses leisure time		Value of foregone leisure	None	Same
	Parent misses hourly paid work	Parent is unable to make up time	Net wage lost	Friction costs related to replacing worker; or lost production	Difference between pay and value of leisure for replacement worker
		Parent is able to make up time	Value of foregone leisure	Related friction costs	None
	Parent misses work and is salaried	Parent is unable to make up time	None	Value of lost output	Reduced tax revenues
		Parent is able to make up time	Value of foregone leisure	Related friction costs	None
Teachers	Teacher loses leisure time		Value of foregone leisure	None	None
Volunteers	Volunteers reduce time donated to other activities		None	NA	Value of foregone activities
	Volunteers reduce their leisure time		Value of foregone leisure	NA	None

Classroom time

Time spent in classroom activities represents a potential cost of an intervention to both participating children (and their families) and society as a whole. These costs, however, are relevant only if the intervention displaces other productive classroom activities and in doing so reduces learning. If so, one needs a measure of the value of learning. For participating children, this value reflects the impact on future (net) earnings (and fringes) and includes some nonmonetary benefits, such as the benefits of education in raising one’s own children (Haveman & Wolfe, 1984).

The benefits of education to society are even larger for two reasons. First, the costs involve gross earnings (as a measure of productivity) rather than net earnings. And second, the non‐monetary benefits of education spill over onto other members of society. Such benefits might include improved citizenship (Haveman & Wolfe, 1984). Haveman and colleagues have estimated that these non‐monetary benefits can be quite large. While these costs would not appear on program budgets, it would be hard to argue that school districts would not be quite concerned about them, especially in an era when schools are increasingly accountable for student performance.

How might we measure these costs? The typical prevention study will not be able to follow study participants into adulthood to examine the effect on future earnings. Furthermore, teasing out the effect of lost learning time from that of the intervention is likely to be difficult. (Rather, what we will observe is the net effect of the intervention.) As an alternative, one might measure society’s (or taxpayers’) willingness to pay for improved learning. The best measure of this concept is the salary teachers earn for the period during which the intervention occurs (Caulkins, Rydell, Everingham, Chiesa, & Bushway, 1999).

This measure may represent an overestimate of the time costs for participants and payors. As we discuss below, improved classroom behavior may increase the efficiency of learning in the classroom, partially offsetting the reduction in classroom time devoted to the normal academic subjects. If those effects are fully offsetting, there are no opportunity costs at all. (Alternatively, this estimate may be an underestimate. Taxpayers may systematically underestimate the benefits to society of educational quality, and as a result, their willingness to pay teachers may be lower than the social benefits of education. Given the poor funding of many American schools, this is rather plausible.)

Parental time

The time that parents spend in intervention‐related activities is a second time cost. These costs depend on several factors such as (1) whether time is missed from work, or leisure time is reduced; (2) whether the parent is paid by the hour or salaried; and (3) whether lost work time has to be made up. The different combinations of these factors imply different values for time. The reader should remember, however, that the key underlying principle is that in all cases, the lost time is valued by the opportunity cost pertaining to the foregone activities.

Many prevention programs schedule activities when the parent is most likely available. As a result, these activities are scheduled outside of work hours on nights or weekends. In that case, as discussed above, the opportunity costs of participating in a prevention program involve foregone time spent in leisure or household production. If workers are employed, then from the participant perspective, the value of this time is at least as great as their wage (plus fringes minus taxes).⁴ The societal costs are larger—they include the value of the taxes as well. (The gross wage is a measure of the worker’s productivity.) These costs are irrelevant for the payor or agency involved.

Many parenting programs are targeted to lower‐income families, and to describe their unemployment as voluntary is questionable. As a result, the value of their leisure time may be less than their (potential) earnings, which is the reason they would like to work more hours. As a result, Hargreaves and colleagues (1998) recommend using half the available net wage.⁵ Given the economic status of the families in our study (lower middle class), this rate is comparable to the minimum wage figure we use below.

In some instances, a parent will miss work to attend intervention activities. If the worker is paid by the hour and loses earnings, the costs to the participant are easy to measure. The costs to the employer (and as a result, society) are harder to measure. The nature and magnitude of these costs depend on whether and how the employer responds. If the output of the worker is lost, then the cost to the employer is the value of that output, which should equal the gross wage (plus fringes). If the firm simply calls in another employee to work, then there are no costs to the employer and society, other than small, frictional (or temporary) administrative costs.⁶

In other cases, salaried individuals miss work time. Presumably, those individuals make up the lost time using their leisure time. In that case, the value of the lost work time is the value of their leisure time, which we value as discussed above. If the worker does not make up the lost time, then there are no costs to the participant. The costs to society include the value of the lost production (i.e., an appropriate fraction of the worker’s gross salary).

Teacher time

The opportunity cost for teachers involves preparation for intervention activities delivered during the school day. If preparation for the intervention reduces other preparation time, then there are no costs to the teachers. (There are no costs to society, other than the lost learning, which we discussed above.) If added preparation is necessary, then we value the lost leisure time using the methods described above.

Volunteer time

At first blush, it may seem that the costs of volunteer time would be zero. After all, the volunteer is “volunteering” to participate, and so the benefits must outweigh the costs of participation. Such may be the case, but this assumption implies only that there are no costs from the volunteer’s perspective. However, participation in one intervention generally will reduce the ability to volunteer for other activities. In that case, the value of the foregone volunteer activities represents costs to society in general (and to the individuals who would have benefited in particular). Such activities are difficult to value. In some cases, the agency pays some individuals to perform similar functions, and those payments could be used to value the time of volunteers.

Another possibility is that the volunteers reduce their leisure time to participate in an intervention. In that case, foregone leisure is a cost to the volunteer and to society in general; the value of that time would be determined as discussed above. Such an estimate likely would represent an upper bound on the related time costs because they would not include any enjoyment the volunteer experienced.

Other wrinkles

Two additional considerations complicate this framework. The first involves incentives paid to parents or teachers. If such payments are made, then these payments turn the nonbudgetary time costs into explicit budgetary costs. Those payments should be subtracted from the estimates described above. Similarly, budgetary costs for child care services during intervention activities should be subtracted as well. Presumably, one of the things the parent would be doing in the absence of the intervention—and one of the reasons leisure (nonwork) time has value—is that it can be used to care for one’s children (a form of household production). These payments shift the costs of participation from parents to the payor. There is no net effect on society (other than the costs of administering the payments).⁷

A second complication was mentioned earlier—the enjoyment directly provided by participation. Conceptually, it seems straightforward that the costs of participation should be reduced by the value of this participation. However, that value should not include the benefits of the intervention, which presumably are measured on the outcome side of the cost‐effectiveness or cost‐benefit analysis. Measuring this enjoyment and its value would be rather difficult. A first step might be to ask parents whether they would participate in the intervention even if it had no effect. If respondents answered “yes”, then it would be fair to count the time costs of the intervention for that period as zero. If they answered “no”, then the time costs must outweigh the direct enjoyment provided by the program. In that case, valuing the time costs as described above would be considered conservative, unless one developed some means of measuring the value of that enjoyment.

RESULTS: ANALYSES OF AN ACTUAL INTERVENTION

To illustrate these methods, we examine data from a large intervention study in Oregon. We use data from the study to estimate relevant time costs and then—to provide a sense of their relative magnitude—we compare these figures to costs found in project budgets.

We examine these figures from two perspectives—that of an agency or payor, for whom the costs that matter are those that appear on project budgets, and that of society. A key difference involves the time costs we identify below.⁸

Success Through the Incredible Years

The Success Through the Incredible Years Project (“Success”) is a five‐year intervention study designed to evaluate relatively low‐cost behavioral interventions for young children at risk of long‐term conduct problems (Webster‐Stratton & Reid, 2003; Webster‐Stratton, 2000; Webster‐Stratton & Taylor, 2001). The project included three carefully integrated interventions, the Incredible Years Parents, Teachers, and Children Training Series, designed to maximize consistency and follow‐through in two environments in a young child’s life—home and school.

An extensive body of evidence supports the efficacy and effectiveness of the parenting component alone (Spaccarelli et al., 1992; Scott et al., 2001; Taylor et al., 1998; Webster‐Stratton, 1984, 1990; Webster‐Stratton et al., 1988) and in combination with the teacher training (Webster‐Stratton, 1998; Webster‐Stratton et al., 2001; Gross et al., 2003) or child training programs (Webster‐Stratton & Hammond, 1997). The Success study was the first study designed to evaluate the efficacy and cost effectiveness of all 3 three program components together under real‐world implementation conditions in the classroom.

The Success Project was comprised of (a) group‐based behavioral parenting training for parents of at‐risk children (11–13 weekly 2‐hour sessions), (b) classroom‐wide social problem‐solving training for first‐grade children (64 lessons, to be implemented for at least 30 minutes per week for most of the school year), and (c) teacher training in classroom management and in classroom implementation of the social problem‐solving curriculum (6 full‐day workshops, spread out over August to January). Thus, the teacher training and classroom intervention components were directed at the entire classroom of children, while the parent training was delivered only to parents of at‐risk children (i.e., those whose Kindergarten teachers had rated them as relatively more disruptive). This project represents the first effort to test all three intervention components in a real‐world setting.

The programs used trained parent group facilitators for the family component of the intervention and trained classroom management interventionists for the teacher training component; the teachers themselves delivered the social problem‐solving curriculum to their own classrooms of children. Across all components of the intervention, videotapes were used to illustrate curriculum content.

Recruitment and Description

For each of three cohorts, children were selected for possible recruitment to the study based on elevated levels of teacher‐rated disruptive behavior. Within participating schools, kindergarten teachers completed a brief questionnaire (CADBI screener; Burns, Taylor & Rusby, 2001) assessing the disruptive behavior of all children in their classroom in November. Children whose parents had explicitly requested not to have their child rated and children who were autistic or who exhibited moderate or profound mental retardation were considered ineligible for the study and were excluded. Of the total possible sample, 7% of parents declined, and 2% were determined ineligible, leaving 91% of children in the classrooms to be rated (a total of 1438 children). Families of children who were rated above the 65^th percentile on disruptive behavior were considered for participation in the study. Recruitment began with children who were rated the highest. Recruitment stopped when parents of enough children agreed to participate (typically 3 boys and 2 girls per classroom). Almost all of the children who were invited to participate had ratings above the 75^th percentile. For families with two eligible children, we randomly selected one for participation. A total of 321 eligible families were invited to participate.

Parental consent for participation was obtained by a three‐step process. First, a secretarial or other support person from the school called the parents at home. During the call, the school employee explained to the parents that they had been selected to participate in a research study that was partnered with their child’s elementary school and asked permission to provide their phone number to the research team (85% agreed). Next, a trained research assistant would call the family to schedule a visit to the family’s home, answer questions, and go over the packet of information about the study in person (86% scheduled a home visit). Finally, a home visit took place, lasting an average of 90 minutes, during which time the aims of the study, the time commitment involved, and the nature of the intervention were described in detail. To participate, families had to be willing to participate in a parenting group, if offered, and to allow their child to be randomly assigned to a first‐grade classroom. Of the eligible families who agreed to a home visit, 87% consented to participate in the study (a total of 203 families).

Children were retained in the study only if they continued at the same school at least one day in first grade, or if they entered first grade in another school participating in the Success project. An additional 18 families moved prior to first grade, leaving 187 fully eligible for the study.

Of 187 eligible students 59% were boys and 11% were of Hispanic ethnicity. Most of the non‐Hispanic students were Caucasian, 6% were American Indian, 2% were African‐American, and 3% were of “other” or did not report their race. Seven families reported having an annual income of less than $4,069 and 11 families reported an annual income greater than $70,000. The median of families fell into the $30,000 to $34,069 income range. During the spring of first grade a total of 160 families participated in this classroom observation study.¹

Additional detail on the recruitment and implementation procedures are available (Taylor et al., 2004). Johnson‐Shelton and colleagues (Johnson‐Shelton, Foster, & Taylor, 2004) provide a full description of methods used to estimate budgetary costs.

This cost analysis focuses on the third and final cohort of teachers and families recruited into the project. Study participants included 16 first‐grade intervention teachers from 8 project schools, and 62 parent participants from 38 families of at‐risk children.⁹ Among these families, 33 attended 6 or more parent training sessions, which we defined as a minimal level of participation required for families to benefit. (In other words, aggregate costs included all expenditures related to serving 38 families, but we calculated per‐family costs using the smaller figure.)

This study also focused on the real‐world maintenance costs for the respective interventions. Our parent group leaders had been trained in the program as part of their regular employment elsewhere, and thus the project did not incur this cost. This is similar to having an experienced staff at an institution. Similarly by this point in the project our Ph.D. level teacher trainers were well experienced with the training, much as a teacher trainer working in a school district might be, and no training costs were incurred Therefore, the costs of training the individuals who conducted the teacher training workshops and who facilitated the parenting groups were not included.¹⁰ We recognize that during an initial or start‐up period for an intervention, a service agency might incur some of these additional costs for training in the intervention protocol.

Furthermore, the costs presented here may not generalize to situations where the intervention is implemented differently (e.g., with two parent group leaders rather than one). Nonetheless, the exercise presented below illustrates the calculations needed for any intervention, even one that was implemented quite differently.

Data Sources

Costs were calculated using several sources of data: attendance data, project budgets, and eight separate cost tracking instruments. These instruments were created to measure direct costs (such as curricula and rent for facilities) as well as the time expended by all participants. (For a full description of instruments, see Johnson‐Shelton, Foster, & Taylor 2004). These forms had several advantages. First, they were completed prospectively and thus are likely more accurate than retrospective reports. Parallel forms existed for parents and teachers, which allowed us to compare time costs across different participants.

In the following sub‐sections, we discuss how this information was used to calculate the main types of costs—those related to recruitment, teacher training, child training, and parent training. Throughout, we focus our discussion on the calculation of participant time costs. Detailed information on time costs per hour is provided in Table I. The way in which that information was used is described below.

Table I.

Computation of Individual Hourly Values for Staff and Participants in Success

	Compensation
Participants	Unit	Rate	Fringe	Rate + fringes	Unit
Project staff
Child care providersa	Hourly	$8.76	13%	$9.90	Per hour
Parent group leadersb	Hourly	$18.93	17%	$22.17	Per hour
Project administrator	Hourly	$24.96	33%	$33.20	Per hour
Interventionist supervisor	Hourly	$35.37	33%	$47.04	Per hour
Interventionistsc	Hourly	$34.51	33%	$45.90	Per hour
Support staff	Hourly	$12.39	33%	$16.48	Per hour
Teachers
Training‐substitutes	Per day	$200.00		$572.00	Per teacher(2.86 days)
Training‐stipend	Per day	$150.00		$429.00	Per teacher(2.86 days)
Classroom social skillsd	Hourly(gross)	$34.12	39.9%	$47.73	Per hour
Recruitment, homework, preparation, and consultatione	Hourly(net)	$21.59	39.9%	$26.32	Per hour
Principals
Recruitment	Hourly	$45.05	33.0%	$59.92	Per hour
Parents
Training, consultation, homework	Hourly	$7.05	33.0%	$9.38	Per hour
Stipends (33 out of 38 families)	Per session	$60.00		$60.00	Per family(6 sessions)

^aAveraged across 11 providers.

^bAveraged across 4 leaders.

^cAveraged across 3 interventionists.

^dCalculated based on classroom time of teacher classroom instruction.

^eCalculated based on annual work hours of 2,080 hours. This figure also reflects withholding at 18%.

Table IIthrough Table IV present the key cost estimates. One should note that each table has three columns of costs—budgetary costs, “other” costs and total costs. (Additional information on the budgetary costs is provided in Johnson‐Shelton, Foster and Taylor (2004). That report describes the calculation of direct intervention costs from project budgets. For example, since training was provided in idle school space, the only space costs involved two sessions that required the rental of community space when a school was not available.)

Table II.

The Costs of the Classroom Intervention

	Time and costs
Classroom intervention	Hours	BudgetaryCosts	Other Costs	Totals
Recruitment of schools and teachers
Teachers	32		$842	$842
Principals	32		$1,917	$1,917
Project staff	80	$3,672		$3,672
Schools (stipend)		$12,000		$12,000
Sub‐total	144	$15,672	$2,760	$18,431
Delivering the intervention: training teachers
Program
Arranging facilities, meals	10	$165		$165
Facility costs		$82		$82
Meal costs		$1,584		$1,584
Preparing materials, supplies	81	$1,335		$1,335
Supply costs		$304		$304
Staff coordination	35	$577		$577
Project administration	29	$963		$963
Curriculum
Staff training materials		$733		$733
Dinosaur curriculum		$11,898		$11,898
Teachers
3 Days training: substitute	320	$9,152		$9,152
3 Days training: stipends	320	$6,864		$6,864
Project staff
Training preparation	83	$3,810		$3,810
Mileage		$1,109		$1,109
Conducting workshops	116	$5,325		$5,325
Travel	31	$1,423		$1,423
Consultation
Staff time (with travel)	36	$1,652		$1,652
Mileage		$294		$294
Sub‐total	1061	$47,269	$0	$47,269
Delivering the intervention: classroom time
Homework (teacher during training)	114		$5,442	$3,443
Classroom social skill training
SP preparation	113		$2,974	$2,974
SP implementation	261		$12,459	$12,459
Classroom behavior plans
CB preparation	64		$1,684	$1,684
CB implementation			$0	$0
Individual behavior plans
IB preparation	40		$1,053	$1,053
IB implementation			$0	$0
Consultation
Teacher time	19		$500	$500
Sub‐total	611	$0	$24,111	$22,113
Total costs	1816	$62,941	$26,871	$87,813
% Total costs		72%	31%
Costs per child		$206	$88	$287
Costs per high‐risk child		$1,907	$814	$2,661

Table IV.

Sensitivity Analyses

	Costs
Cost value scenarios	Budgetary costs	Other costs	Totals
Actual results: teacher training (from Table II)
1. Recruitment of schools and teachers	$15,672	$2,760	$18,431
2. Delivering the intervention: training teachers	$47,269	$0	$47,269
3. Delivering the intervention: classroom time	$0	$24,111	$22,113
Total	$62,941	$26,871	$89,812
Per child	$206	$88	$294
Per high‐risk child	$1,907	$814	$2,722
% of Total	70%	30%	100%
A. Alternative scenario: teachers teach multiple (5) years before needing retraining (shaded cells are those that differ from those in the actual results)
1. Recruitment of schools and teachers	$3,134	$552	$3,686
2. Delivering the intervention: training teachers	$9,454	$0	$9,454
3. Delivering the intervention: classroom time	$0	$24,111	$22,113
Total	$12,588	$24,663	$37,251
Per child	$41	$81	$122
Per high‐risk child	$381	$747	$1,129
% of total	34%	66%	100%
B. Alternative scenario: no opportunity costs of classroom time
1. Recruitment of schools and teachers	$15,672	$2,760	$18,432
2. Delivering the intervention: training teachers	$47,269	$0	$47,269
3. Delivering the intervention: classroom time	$0	$0	$0
Total	$62,941	$2,760	$65,700
Per child	$206	$9	$215
Per high‐risk child	$1,907	$84	$1,991
% of total	96%	4%	100%
Actual results: parent training (from Table III)
1. Recruitment of parents	$4,972	$0	$4,972
2. Delivering the intervention	$49,337	$3,073	$52,411
Total	$54,309	$3,073	$57,382
Per child	$1,646	$93	$1,739
% of total	95%	5%	100%
C. Alternative scenario: parent time is valued at $10.00 per hour
1. Recruitment of parents	$4,972	$0	$4,972
2. Delivering the intervention	$49,337	$4,232	$53,570
Total	$54,309	$4,232	$58,541
Per child	$1,646	$128	$1,774
% of Total	93%	7%	100%
D. Actual results: no compensation (babysitting and incentives) for parents
1. Recruitment of parents	$4,972	$0	$4,972
2. Delivering the intervention	$40,912	$11,499	$52,411
Total	$45,884	$11,499	$57,382
Per child	$1,390	$348	$1,739
% of total	80%	20%	100%

Note that “other” costs represent the costs not appearing on project budgets but borne by children, teachers, parents or other members of society. Predominantly, these are participants’ time costs valued in terms of their opportunity costs. As discussed below, however, some of these costs represent out‐of‐pocket expenditures (e.g., travel costs). Since these are generally small, we use the terms “other,” “time,” and “opportunity” costs interchangeably.

The Costs of the Classroom Intervention

Table II displays the costs of the classroom intervention. As discussed above, this component involved training teachers in classroom management and the social problem‐solving curriculum, delivering the curriculum in the classroom, and implementing individual behavior plans for challenging students. Intervention costs also included expert consultation between teachers and project staff. The costs of these components can be grouped into three broad categories—school recruitment, training teachers, and delivering the intervention in the classroom. We discuss each of these in turn.

Recruitment of schools and teachers

Recruitment of schools and teachers for a comprehensive and complex project like Success is time‐intensive. As indicated in the top panel of Table II, recruitment involved teacher and principal time, project staff time, and a stipend paid to each school ($750 for each pair of intervention and nonintervention teachers in each of this cohort’s 8 project schools). Recruitment also involved opportunity costs—the time teachers and principals spent attending meetings with project staff. As indicated in Table I, we valued principal and teacher work time using information on their salaries and fringes and assuming a 2,080‐hour work year. (We assume that with preparation time a full‐time teacher job is equivalent to a year‐round 40‐hour/week job, though the hours tend to concentrated at certain times of the year.) Teacher salary data were calculated using average salary figures available for five comparable rural‐suburban school districts in Oregon. Information on respective principal salaries was not gathered, so we averaged the annual average salaries for principals across three representative school districts (urban, suburban, and rural) in Oregon to derive an estimate.

Delivering the intervention: Training teachers

Training teachers in classroom management and in the classroom‐based social problem‐solving curriculum involved teacher and project staff time as well as materials, such as supplies and curricula. Teacher and project staff time involving intervention activities were tracked through self‐report on time sheets and valued using budgetary information on salary and fringes (as reported in Table I). Other direct costs were related to teacher attendance—teachers were paid stipends for workshops held on weekends ($429 per teacher on average for two days), and substitutes were paid to cover their classes for workshops attended on school days ($28.60 per hour). As a result, teachers experienced no additional (opportunity or) time costs.

Delivering the intervention: Classroom time

The social costs of classroom time involved foregone learning opportunities for children. We assumed that policy makers pay for hours of classroom contact, a subset of the time teachers spend working. Per‐unit costs for classroom time for delivery of the social problem‐solving curriculum, therefore, were larger than for teacher preparation time or other professional activities outside of direct classroom instruction $47.73 per hour v. $26.32 per hour, respectively. The latter is smaller for two reasons. First, as noted, it reflects the larger number of hours used to calculate it. Second, this figure is post‐tax.¹¹ As noted previously, estimating the costs in this way is conservative; the intervention may have improved learning efficiency, eliminating any opportunity costs. Note that we assumed that the time costs of actually implementing classroom‐wide and individual behavior plans are zero; specifically, we assumed that teachers are able to integrate the implementation of these plans into their regular school day.

During the course of teacher training, teachers were expected to complete homework assignments by trying out various practices in their classrooms. In addition, teachers’ delivery of the curriculum during the school day required some preparation by teachers. The time involved with these activities was estimated based on monthly surveys of participating teachers. Teachers reported roughly 20 hours per teacher in total preparation time, apart from time spent delivering the classroom intervention. This time was valued based on the value of leisure (i.e., the appropriate salary divided by hours of work plus the corresponding fringes minus taxes).

Bottom Line

Table II reveals that the overall costs of training 16 teachers in the classroom intervention were $87,813. On a per‐child basis, this figure represents $287 for each child in a teacher’s classroom. Another perspective is to calculate per‐child costs in terms of the at‐risk children targeted for the parenting intervention—in that light, the per‐child costs were higher but still relatively modest $2,661.

Time costs represent a substantial proportion (31%) of total costs. About half of these costs involved the opportunity costs of classroom time devoted to the social problem‐solving curriculum. Of course, an agency may ignore these costs‥ If so, the costs per child fall to $206 and $1,907 for all and at‐risk children, respectively.

The Costs of Parent Training

Recruitment of parents

Table III presents the costs associated with the parent training component of the intervention. The families of at‐risk children had to be recruited for the study. This activity involved inputs such as postage and mileage as well as the time of project staff and teachers. Teachers were involved in screening children to identify those families most appropriate for participating in the project and were paid for their time. Parent and staff time were valued as discussed above.

Table III.

The Costs of Parent Training

	Time and costs
Parent training	Hours	Budgetary costs	Other costs	Totals
Recruitment of parents
Program staff	72	$3,305		$3,305
Teachers		$1,450		$1,450
Postage and mileage		$217		$217
Sub‐total	72	$4,972	$0	$4,972
Delivering the intervention
Intervention staff
Facilitator hiring, supervision
Facilitators	44	N/Aa
Clinical supervision	111	$5,222		$5,222
Preparing, conducting training	728	$16,143		$16,143
Consultation (with parents)	28
Travel time	41	$1,929		$1,929
Travel mileage		$700		$700
Administrative support
Arranging facilities, meals	66	$1,088		$1,088
Hiring, training, supporting child care	131	$2,159		$2,159
Preparing materials, supplies	100	$1,648		$1,648
Staff coordination	102	$1,681		$1,681
Project administration	81	$2,689		$2,689
Facility costs		$863		$863
Meal costs		$5,252		$5,252
Supply costs		$117		$117
Curriculum
Staff training materials		$359		$359
Parent training materials		$1,064		$1,064
Child care services	651	$6,445		$6,445
Parent stipends (incentives)		$1,980		$1,980
Parental time costs
Attendance (less incentives and child care)	1044		$1,368	$1,368
Homework	119		$1,116	$1,116
Training consultation	46		$431	$431
Travel time	11		$103	$103
Travel mileage			$55	$55
Sub‐total	3303	$49,337	$3,073	$52,410
Total costs	3375	$54,309	$3,073	$57,382
% of total		95%	5%
Per child costs		$1,646	$93	$1,739

^aTime costs incorporated into weekly 6 hrs. for preparing and conducting parent group training.

Delivering the Intervention

The key project costs of the parent training component were staff and related costs in conducting the parent training sessions. These costs included facilitators’ time preparing for and leading groups and consulting with parents, their travel expenses, group meals before meetings, facilities, curriculum, administrative support, and supplies. The project also provided child care during parenting group sessions, and parents attending six or more sessions received a stipend incentive of $60. This cost information was tracked and recorded through monthly payroll and project budget reports.

Parental time costs

These costs were calculated based on attendance reports for parent training sessions and parent surveys of intervention‐related activities outside of the sessions (e.g., homework and consultation with parenting group facilitators). Parent time was valued using Oregon’s current minimum wage.¹². We used this rate for two reasons. First, we feared that parental reports of earnings might be inaccurate; and second, we were concerned that some unemployment in the sample was in fact involuntary. Parental reports of their occupations suggest that the minimum wage may be fairly close to what the majority of the sample actually earned (after taxes).

As noted above, parents were partially reimbursed for this time in the form of incentive payments and the provision of free child care, and we subtracted those budgetary costs from parents’ time costs. Parents also reported travel costs related to program involvement, including additional time costs.

Bottom Line

Table III reveals that the overall costs of parent training were $57,382. On a per‐child basis, these costs were $1,739. As a percentage of total costs, the time costs for parents were relatively modest (5%) and much smaller in proportional terms than time costs for teacher training. One explanation for why time costs were lower for parent training than for the teacher intervention is that we valued parent time at a much lower rate. This difference also reflected the fact that the classroom intervention required more time than did the parent intervention.

Sensitivity Analyses

The analyses above rely on several key assumptions, and a thorough economic analysis should consider their influence on the sensitivity of key findings. Table IV presents four such analyses.

First, we consider the sensitivity of the teacher intervention to the key assumption that teachers would need retraining each year. An alternative approach would be to treat teacher training as an input that depreciates with time. For that reason, panel B of Table IV presents alternative figures under the assumption that teachers would be retrained every five years. (We applied straight‐line depreciation within this period. For comparison purposes, panel A repeats the figures from Table II.) As a result, all costs except those related to intervention delivery were reduced by 80%. The effect on per‐child budgetary costs was quite substantial, reducing these costs from $1,907 to $381 per child. The opportunity costs of teacher and classroom time, however, fell relatively little because most of those costs are repeated every year ($814 to $747). As a result, these time costs represented the majority (66%) of the recalculated total costs in panel B.

A second sensitivity analysis (panel C) focused on the opportunity costs of classroom time. A social problem‐solving skills curriculum and improved behavior management practices may improve the efficiency of time spent in other activities, and as a result, time available for academic instruction may not be reduced. Adjusting our figures to eliminate opportunity costs for classroom time delivering the intervention eliminated nearly all of the opportunity costs across the board, reducing them to a mere 4% of the total costs.

The third and fourth sensitivity analyses involved the parent training intervention. As discussed above, the original analyses valued parental time costs using the minimum wage. Panel D and E compare the original parent cost figures with those valuing parental time at $10.00 per hour ($13.33 with fringes included) (panel C). As one would expect, there was no impact on the direct budgetary costs of the intervention. Parental time (opportunity) costs, however, increased proportionately and represent 7% of overall costs in panel C.

A fourth sensitivity analysis considered the role of parental compensation (panel D). In the real world, parents likely would not be paid for participating nor would child care necessarily be provided. In that case, the related time costs were shifted into the opportunity costs (“other”) column. Under those circumstances, the nonbudgetary parent time costs increased from $93 to $348 per child and now represent 20% of total costs in panel F.

Discussion

This article outlines the principles by which the costs of a program can be valued and illustrates consistent procedures for doing so using data from an implementation of a well‐known preventive intervention. In general, analyses of the budgetary costs highlight the relatively low‐cost nature of both the Incredible Years classroom and parenting interventions, especially the former. If we consider all children exposed as having benefited, then the costs of the classroom intervention are particularly low, perhaps as low as $41 per child (panel B of Table IV).

Parent training is considerably more expensive per child (roughly $1,500), reflecting the greater intensity of the intervention delivered to a smaller group. One way to reduce these costs involves shifting more of the costs onto parents themselves (e.g., by not providing child care). Furthermore, costs could be reduced still further by not providing meals—those meals represented over 10% of the budgetary costs. Doing so, however, may well negatively affect parents’ participation rates (Dumka, Garza, Roosa, & Stoerzinger, 1997; Dumka, Roosa, Michaels, & Suh, 1995; Metzler & Taylor, 2002; Taylor & Biglan, 1998). In this study, parental participation was fairly high, and whether such rates could be achieved without stipends, child care, and meals is doubtful.

In addition, the gap in costs between the parent and classroom intervention is narrowed when we focus on at‐risk children, the targets of the parenting intervention. In that case, the costs per at‐risk child of the classroom intervention are actually greater than for the parenting intervention. This perspective is reasonable for thinking about the classroom intervention—the highest risk children are likely to benefit the most, and these children are most likely to generate high public and social costs if untreated.

Whether and how to pick between the different modalities of intervention depends on key issues yet unresolved. These include the long‐term outcomes for the at‐risk children as well the size of any spillover effects from their behavior onto other children in their classrooms or families.

Our analyses suggest that time costs are potentially substantial. For the Incredible Years parenting intervention, those costs ranged from 5% to 20% of total costs; for the teacher training and the social problem‐solving skills curriculum, the costs ranged from 4% to as much as 66% of the total costs. A key reason for the relative importance of opportunity costs for classroom interventions is that those costs are realized every year; other costs, such as training the teachers themselves, may be amortized over time.

Many readers may feel that the perspective that matters most is that of the funding agencies, and as result, he or she may find our results novel but not particularly useful. However, ignoring these non‐budgetary time costs may lead to bad policy decisions. For example, if agencies do not value parents’ time, they will overuse that resource as they would any free resource. For several of the situations we consider, including our baseline case, the ratio of direct budgetary costs for the parenting versus the classroom intervention is greater than that for the total costs (that is, with time costs included). These figures imply that public policy may over‐rely on classroom‐based interventions. Whether this is in fact the case depends on whether policy makers actually ignore the opportunity costs of classroom time. In an era of accountability, it may well be that principals are quite cognizant of these costs and so incorporate them into their decisions.

These findings have a variety of implications. First, efforts to disseminate evidence‐based interventions into community settings, especially schools, should be sensitive to these time costs. While the project compensated schools for training and materials, the effort to implement the core features of the program still fell heavily on teachers. The ability to forecast such impacts is very important in planning and negotiating the dissemination of evidence‐based practices ethically and responsibly in school settings. Our findings also highlight that since classroom teachers were expected to do the most uncompensated work for the project, it made sense that the recruitment strategy required teachers to decide—without pressure from administration—whether to participate.

We also find that the opportunity costs of classroom time devoted to intervention are potentially large. Whether those costs are real, however, depends on future analyses that consider the effect of the intervention on academic performance. Whether those costs are real or not may not matter. Policy makers may believe them to be, and in that sense, the amount of time devoted to the intervention may represent a barrier to implementation.

Second, parent time costs are also substantial. They have less bearing on the overall costs of the parenting intervention; to some extent, this finding reflects the relatively low value we placed on parental time. Still, the amount of time involved is substantial, approaching 40 hours per family. As a result, interventions may require more of families than families can reasonably deliver, and in that case, program impact may be reduced.

This possibility is ironic given that one justification for the programs is to reduce the burden of illness on families. These findings suggest that the costs of participation in interventions may partially offset any benefits that stem from improvements in the child’s mental health. Ideally, the time costs of intervention would be realized once, reducing a stream of future time costs related to the child’s illness. Whether this is the case remains to be seen.

For that reason, whether the families enjoy intervention activities for their own sake is important. If the intervention reduces the time parents attend to unpleasant problems created by their child, then the time costs reported here may be too high. In that case, intervention activities may be a rather pleasant substitute for other more difficult problem‐solving experiences and not a net cost. By assuming that the intervention displaced leisure or employment, our findings likely represent an upper bound for true time costs. To truly understand this issue, one would have to embed a study like Success in a broader effort to measure the costs of illness for children and their families.

Finally, these findings also have important implications for how psychologists and other program developers design their interventions. The recognition that time costs can be substantial need not imply that programs minimize their use of that time. The program that requires the least of parents is not necessarily the most effective or even cost‐effective program. Rather, our findings imply that program designers need to optimize their use of that resource as they do others—by balancing the costs and benefits. At this point, we do not have enough information on how parental time affects treatment outcomes to provide interventionists with concrete guidance. In order to examine this issue more carefully, evaluations of treatments and interventions need to routinely ask youth, parents, teachers, or others how much of their own time they devote to program activities. Other information might be collected on other aspects of time use, such as whether and how the implementation of behavioral plans disrupts or improves the learning environment.