Understanding Proximal-Distal Economic Projections of the Benefits of Childhood Preventive Interventions

Abstract

This paper discusses the steps and decisions involved in proximal-distal economic modeling, in which social, behavioral, and academic outcomes data for children may be used to inform projections of the economic consequences of interventions. Economic projections based on proximal-distal modeling techniques may be used in cost-benefit analyses when information is unavailable for certain long term outcomes data in adulthood or to build entire cost-benefit analyses. Although examples of proximal-distal economic analyses of preventive interventions exist in policy reports prepared for governmental agencies, such analyses have rarely been completed in conjunction with research trials. The modeling decisions on which these prediction models are based are often opaque to policymakers and other end-users. This paper aims to illuminate some of the key steps and considerations involved in constructing proximal-distal prediction models and to provide examples and suggestions that may help guide future proximal-distal analyses.

One of the obstacles to assessing the economic value of child-focused preventive interventions is the long period of time that must elapse from the point of intervention until economic outcomes occur. Conventional cost-benefit analysis requires data on economically salient outcomes in adulthood, especially labor market earnings, criminal justice system contacts, healthcare utilization and receipt of public program supports. However, with few exceptions, follow-up data are not collected beyond childhood or early adolescence in research trials of child-focused interventions (for reviews, see Barnett, 1995; Farrington & Welsh, 2003; Karoly, 1998). Consequently, in many of these research trials, conventional cost-benefit analyses are either not feasible or are based on limited knowledge of future economic outcomes.

Proximal-distal projections are predictions of long-term future or “distal” economic consequences that are based on near-term or “proximal” social, academic or behavioral outcomes in childhood, adolescence, or early adulthood. Such projections can be used to provide policymakers and advocates easy-to-interpret estimates of the potential future economic benefits of childhood preventive interventions (e.g., Foster & Jones, 2006), even when data on economic outcomes in adulthood are unavailable or incomplete. They could also be used to give research funding agencies preliminary evidence on an intervention’s long-term economic benefits. Although proximal-distal projection methods have been used for more than a decade in studies of injury prevention (e.g., Graham, Thompson, Goldie, Segui-Gomez, & Weinstein, 1997), healthcare interventions (e.g., Silverstein et al., 1999), preschool education (e.g., Rolnick & Grunewald, 2003), and other health fields, they have, with a few exceptions (Aos et al., 2004; Aos, Lieb, Mayfield, Miller, & Pennucci, 2004; Aos, Miller, Drake, & Washington State Institute for Public Policy, 2006; Foster & Jones, 2006), rarely been used in child-focused behavior problem prevention studies.

Analysts have much discretion in how they design proximal-distal prediction models, and such models have not been held to the same methodological standards that govern the reporting of other types of results in research studies. Proximal-distal models are rarely subjected to validation tests, there is no consensus around reporting results, and the inner mechanics of proximal-distal modeling are often opaque to the end-users of their results. This article discusses methods used to construct proximal-distal projections. Its primary aim is to clarify the importance of modeling decisions commonly made in proximal-distal projections for preventive interventions. Its secondary aim is to highlight the importance of collecting in research trials those types of proximal outcomes data that can be used in modeling future economic effects.

Proximal-Distal Modeling

Before introducing some of the key decision points of proximal-distal models, it is useful to characterize their basic design (for an in-depth discussion of modeling strategies, see Pearl, 2000). A proximal-distal model relating the proximal (i.e., near- and intermediate-term) effects of an intervention to distal (i.e., future) economic benefits provides a structure into which empirical estimates can be substituted. The analyst selects proximal outcomes G_k, k=1,2,…,K, which are mutually exclusive observed outcomes that are known to be predictive of future (i.e., distal) outcomes, and a set of distal outcomes H_l, l=1,2,…,L, that are associated with economic benefits B_l. Most proximal-distal models of the expected benefits of an intervention I can then be written in the following form:

$$\text{Expected}\text{Benefits}(I)={\sum }_l{\sum }_k({\delta }_{k\mid I=1}-{\delta }_{k\mid I=0}){\gamma }_{l\mid k}{B}_l$$ (1)

In Equation 1, Σ_l indicates summation over all distal outcomes H_l; Σ_k indicates summation over all proximal outcomes G; δ_k indicates the likelihood of G_k given exposure either to the intervention (I=1) or exposure to a comparison condition (I=0); γ_l|k indicates the probability of distal outcome H_l given G_k. Equation 1 states that the expected benefits of an intervention (I) can be calculated as the sum over k and l of the effects of I on the likelihood of proximal outcomes G_k multiplied by the changes in the conditional probabilities of distal outcomes H_l (conditional on G_k), multiplied by the economic values B_l of distal outcomes H_l.

B_l represents appropriately discounted present valued benefits. To obtain these present values, future economic effects are “discounted” or multiplied by a discount factor—a value less than 1—to reflect the fact that a dollar received in the future is not worth as much as a dollar received now (Drummond, O’Brien, Stoddart, & Torrance, 2000). The discount factor can be written as 1/(1+r)^t, where r is a positively valued “discount rate,” and t is the number of time periods until a future cost or benefit occurs. Thus, for example, the present value of a winning lottery ticket (or of any other source of income) that pays the holder of the ticket $100 five years from now has a present value of $100/(1+r)⁵. By convention, r is often specified as 3% or 5%, and may be varied higher or lower in a sensitivity analysis.

In Equation 1, the benefits of an intervention are expressed as a summation of the product of direct and indirect effects. The direct effects of intervention, represented by δ_k in Equation 1, are on the proximal outcomes G_k, which might be indicators of behavior, social functioning, or academic functioning (e.g., satisfactory elementary school attendance). Equation 1 assumes a structural relationship between these proximal outcomes and the likelihood of the distal outcomes (H_l), such that the intervention will have an impact on distal outcomes if these proximal outcomes are altered. Importantly, the intervention effects on distal outcomes are assumed to be fully mediated via their effects on these proximal outcomes. Effects that are not explicitly modeled in Equation 1 are, for better or worse, disregarded.

It is also worth emphasizing that the proximal outcomes do not necessarily cause the distal outcomes. Rather, they may only be near-term markers of developmental trajectories that result in adverse future outcomes. The critical assumption underlying the choice of proximal outcomes in Equation 1 is that an intervention that reduces the likelihood of the proximal outcome will also reduce the likelihood of future distal outcomes that have associated economic consequences. Thus, proximal and distal outcomes must be located along a common causal pathway. For example, Lee and Aos (Lee & Aos, 2011) draw indirect linkages between reduction of child abuse and neglect and several distal outcomes, including reduced participation in crime, substance use, and depression as well as increased achievement test scores and high school graduation rates. These distal outcomes are monetized in order to estimate the economic benefits of preventing a case of child abuse or neglect

To avoid double-counting benefits, each proximal and distal outcome represented in Equation 1 should represent the effects of separable causal pathways. That is, the effects of proximal outcomes on distal outcomes should be distinct and “separable.” Separability means that each effect represented in Equation 1 does not duplicate another effect in the model. For example, suppose G included measures of both alcohol use and illicit drug use. This setup would imply the assumption that alcohol use and illicit drug use independently contribute to distal economic outcomes. The analyst would consequently need an estimate of the marginal associations of each factor with distal outcomes H. However, if the proximal variables have non-separable (i.e., non-independent) effects, or if the independent contributions of alcohol and drug use to distal outcomes cannot be separately measured, the assumed model may result in a double-counting of the future benefits of the intervention.

Adherence to this conceptual standard is made difficult in practice because preventive interventions often act upon numerous risk and protective factors and because intervention impacts are potentially dynamic (National Research Council and Institute of Medicine, 2009). On the one hand, at present, the field of cost-benefit analysis lacks any consensus regarding what set of causal pathways could be considered unique and separable, and, consequently, analysts have few guidelines to follow on this issue. On the other hand, to increase the transparency of the assumed causal pathways, it would seem that prevention researchers could refer explicitly to a theoretical framework to help guide the specification of a proximal-distal model. A developmental theory may, for example, indicate a valid distinction between an aggressive behavior pathway and a substance abuse pathway. Proximal indicators and distal outcomes could be specified in relation to each of these conceptually indicated pathways. Regression coefficients estimated from a regression of distal outcomes on a set of proximal outcomes can be used to infer orthogonal weights for each proximal outcome.

Another empirically thorny issue is the specification and measurement of secondary and tertiary effects of interventions. These are indirect impacts on other individuals that may occur as a result of an intervention’s impacts on an intervention participant’s outcomes. For example, younger siblings of children who participate in a school-based behavioral intervention may benefit from improvements in those children’s behavior at home. Such effects should be considered for inclusion in the prediction model and their omission can be considered a limitation of the resulting predictions.

To illustrate how Equation 1 could be used, suppose that the results of a preventive intervention research study indicate proximal effects for suspensions and school attendance in the 5^th grade, as shown in the top panel of Table 1. Suppose further that a separate longitudinal database has information on the associations between elementary school attendance and disciplinary events on the one hand and high school outcomes on the other. Using both datasets together, the probabilities for the distal outcome of high school non-completion can be estimated for the intervention and comparison groups, given the proximal outcomes of school attendance and suspensions. The second panel of Table 1 shows assumed values for these outcomes. Finally, based on representative earnings profiles for high school graduates and non-graduates, suppose a high school graduate will earn approximately $2.83 million during their employment career (ages 18–65) compared to $2.02 million for a person who does not complete high school. Substituting these numbers into Equation 1, exposure to the intervention results in an expected $40,000 increase in lifetime earnings, from ($2.64 million to $2.68 million). Expansions of this example could incorporate other distal outcomes, such as adult criminal incarcerations and their associated costs, or other proximal outcomes, such as grade retention.

Table 1.

Example of Proximal-Distal Projection using Outcomes in 5th Grade

	Intervention Group	Comparison Group
Proximal School Outcomes in 5th Grade (fraction)
A1: Student suspended at least once & student attended school less than150 days in the prior school year	0.05	0.14
A2: Student suspended at least once & student attended school at least 150 days in the prior school year	0.07	0.17
A3: Student never suspended & student attended school less than 150 days in the prior school year	0.09	0.07
A4: Student never suspended & student attended school at least 150 days in the prior school year	0.79	0.63
Distal Outcomes
Fraction not expected to complete high school if:
A5: Student suspended at least once & student attended school less than150 days in the prior school year	0.55	0.55
A6: Student suspended at least once & student attended school at least 150 days in the prior school year	0.30	0.30
A7: Student never suspended & student attended school less than 150 days in the prior school year	0.25	0.25
A8: Student never suspended & student attended school at least 150 days in the prior school year	0.15	0.15
Lifetime earnings ($ millions, present value) if:
Does not complete high school (A9)	2.02	2.02
Completes high school (A10)	2.83	2.83
Expected earnings ($ millions, present value)
[(A1×A5)+(A2×A6)+(A3×A7)+(A4×A8)] × A9 + [A1×(1-A5)+A2×(1-A6)+A3×(1-A7)+A4×(1-A8)] ×A10	2.68	2.64

Equation 1 can also be easily re-written to express expected net benefits, which are defined as expected benefits minus expected costs. Expected costs can be expressed in a manner similar to equation 1. Costs, which can be thought of as negative benefits, would include the costs of the intervention as well as future costs related to the intervention (e.g., increased tuition costs for interventions that increase college attendance). An expression for expected costs can be subtracted from Equation 1 to obtain an expression for expected net benefits.

There are at least three key caveats to this proximal-distal prediction approach. First, the method introduces imprecision into the estimates of intervention effects in comparison to estimation based on data obtained through actual follow-ups into adulthood. Second, the accuracy of the estimates depends critically on the validity of the model assumed in Equation 1. If important intervention effects are not reflected in the model, or if the analyst misspecifies the relationships between proximal and distal outcomes, resulting estimated benefits may misrepresent the actual effects of intervention. Third, the accuracy of the predictions depends on whether the proximal-distal associations that were estimated using the external database can be validly extended to the intervention study sample.

The validity and reliability of the effect-sizes used in a proximal-distal model should be discussed and contextualized by, for example, characterizing the ages and socio-demographic characteristics of the source studies and characterizing the level of agreement in effect sizes across studies. In addition, various empirical methods for characterizing effect size heterogeneity and bias, such as those used in meta-analysis (Sutton, A.J., Higgins & Thompson, 2002; Sutton et al., 2000), may be used to develop an empirical basis for an assumed effect size or effect-size range. It may also be important to consider whether the statistical approaches used in source studies accounted appropriately for possible non-independence of observations due to clustering in the original sample. When clustering is not accounted for, care should be taken by the analyst to contextualize the robustness of identified effect sizes and significance levels.

Steps in Proximal-Distal Modeling

To form proximal-distal projections, the analyst must complete a series of modeling steps. Here we focus on the following steps: 1) selecting intervention and comparison conditions; 2) selecting proximal and distal outcomes; 3) selecting a time horizon; 4) constructing a model linking proximal to distal outcomes; and 5) reporting the results. Below, we highlight the importance of key decisions at each of these steps, drawing examples from published cost-benefit analyses and proximal-distal projections.

Step 1. Defining the Conditions

The first step in the modeling problem is to define the intervention and its alternative(s). As in a randomized controlled trial, an intervention’s effects are contrasted with the effects of the intervention’s primary alternative. Alternatives may include another program with similar aims, “services as usual” (i.e., the status quo), or no intervention at all. The future costs and benefits of the intervention are conceptually well-defined only in relation to the costs and benefits of the primary alternative(s). Consider two economic evaluations of the High/Scope Perry Preschool program, which provided preschool education and teacher home-visits to low-IQ, disadvantaged African-American children living in Ypsilanti, Michigan (Heckman, Moon, Pinto, Savelyev, & Yavitz, 2010). In one economic evaluation, outcomes for children who received the High/Scope curriculum were compared to the outcomes of children who received no preschool services (Schweinhart & Weikart, 1989). Cost-benefit analyses demonstrated substantial net benefits (i.e., total benefits net of total costs) of the High/Scope curriculum relative to the no-preschool comparison (Nores, Belfield, Barnett, & Schweinhart, 2005). However, in another study from the Perry Preschool experiment (Schweinhart & Weikart, 1997), outcomes at age 23 for children who had received the High/Scope curriculum were compared to outcomes for children who participated in traditional child-centered “Nursery School,” which focused on social and emotional development. Results indicated that High/Scope and Nursery School resulted in similar positive outcomes across most outcome categories. In contrast to the earlier High/Scope study, these results suggested that adding the (more expensive) High/Scope curriculum to a traditional Nursery School curriculum may not result in substantial net economic benefits. This example illustrated that the relative economic benefits of an intervention may vary according to the comparison condition.

Step 2. Selecting Proximal and Distal Outcomes

In theory, proximal outcomes could include almost any outcome measure in a prevention study as long as the outcome is predictive of distal outcomes that have a quantifiable economic value. Candidate proximal measures for researchers to consider would include epidemiological indicators of persistent delinquency problems and antisocial behavior, child abuse, school suspensions and expulsion, grade retention or low academic achievement in elementary school, early participation in criminal acts, tobacco smoking, substance use disorders or mental illness, and educational attainment. Each of these outcomes has been linked empirically to future labor market earnings, criminal activities, healthcare utilization, contact with criminal justice systems and/or other outcomes with associated public and/or private costs and benefits (Ensminger & Slusarcick, 1992; Foster & Jones, 2005; McLeod & Kaiser, 2004; Scott, Knapp, Henderson, & Maughan, 2001). Consequently, projected changes in the likelihood of adverse outcomes in these distal domains can, in principle, be assigned a future economic value. In practice, analysts have to consider the scope of measurable economic effects, and some potentially relevant effects are deemed outside the scope of the analysis. Here, several proximal and distal outcomes and their potential use in projections of future economic benefits are summarized.

Educational attainment

Often the economic benefit from greater educational attainment benefit is summarized as an internal rate of return, the discount rate that equates the present value of two lifetime earnings streams. The present value of lifetime earnings refers to the total value of employment compensation (net of taxes but inclusive of fringe benefits) over one’s career discounted to the starting age of paid work (usually assumed to be 18). Thus, if the present value of lifetime earnings for persons who do not complete high school is Y and λ is the internal rate of return associated with high school completion, the present value of lifetime earnings for persons who complete high school is (1+λ)×Y and the benefit of completing high school versus not is λ×Y. Although internal rates of return to additional years of schooling have been estimated in many studies (see Ashenfelter, Harmon, & Oosterbeek, 1999), no consensus has developed around a particular value or set of values. Nevertheless, there is good evidence that internal rates of return to completing high school versus not completing high school exceed 30%, and the returns to completing college versus completing high school (with no college experience) are 13–16% (Heckman, Lochner, & Todd, 2006). Completing either of the first two years of college has a return of 8–12% (Heckman et al., 2006).

Increased educational attainment may have other private and social benefits besides improvements in lifetime earnings, such as better health and higher quality of life (Gilleskie & Harrison, 1998; Wolfe & Haveman, 2002). However, these additional “non-market” benefits (i.e., beneficial consequences that are not commercially exchanged in an economy) can be more difficult to quantify than improvements in earnings and therefore are not counted in many cost-benefit analyses, an important limitation that should be considered in discussions of proximal-distal analyses. The inclusion of those economic benefits resulting from the non-market effects of education would tend to increase the return to education, and by implication, would increase the benefits of child-focused interventions that improve educational outcomes.

Crime and criminal justice system involvement

Prevention science has access to a large repository of information on child and adolescent predictors of participation in criminal activity and criminal justice system involvement (e.g., Loeber et al., 2005; Loeber & Farrington, 2000), and substantial efforts have been made to quantify the costs associated with these outcomes (e.g., Cohen, 1998; Cohen, Rust, Steen, & Tidd, 2004; Donohue & Siegleman, 1998; Scott et al., 2001; Welsh, Farrington, & Sherman, 2001). Estimated savings from reductions in the expected costs of crimes often comprise a large proportion of the overall benefits of prevention programs. For example, in a cost-benefit evaluation of the High/Scope Perry Preschool program (Belfield, Nores, Barnett & Schweinhart, 2006), benefits resulting from reduced crime costs accounted for more than two-thirds (71%) of program benefits per participant ($69,758 of $98,767).

To form benefit estimates for crimes prevented, average costs associated with particular types of crimes are multiplied by a measure of the expected reduction in numbers of particular types of crimes and/or criminal justice system outcomes in a population of interest. Consequently, to form a proximal-distal estimate, an analyst would need predictions of the numbers of future crimes and/or criminal justice system contacts, as predicted by childhood (i.e., proximal) outcomes, such as externalizing behaviors, school suspensions, and/or violent and delinquent acts. Such estimates can be straightforwardly obtained from any of several longitudinal databases, such as one of the National Longitudinal Surveys (www.bls.gov/nls) databases or Cohort 1 and 2 databases from the Hopkins Center for Prevention and Early Intervention (www.jhsph.edu/prevention).

Salkever et al., 2008 provides an empirical illustration for how a proximal-distal prediction of the benefits from future reduction of criminal incarceration costs could be formed. The model developed by Salkever et al., 2008 uses information from the Teacher Observation of Child Adaptation-Revised (TOCA-R) scale and other academic performance indicators to predict future savings from prevention of criminal incarceration in adulthood, and shows how these expected benefits vary across individuals depending on their risk profiles in childhood. Predictions from this model could be linked to intervention trials outcomes data if intervention effects were available for these same childhood (proximal) risk factors for future incarceration. In other words, proximal-distal modeling using this approach would require an intervention effect size for the TOCA-R and other academic performance outcome measures in childhood.

Labor market productivity and earnings

There is at present a limited evidence base available for constructing proximal-distal projections of future productivity or average earnings based on behavioral outcomes in childhood. Limited evidence is available regarding the relationship between behavioral problems in children and future earnings trajectories, at least for those behavioral behaviors that are typically measured in child-focused prevention research studies (e.g., externalizing and internalizing behavior problems). However, lifetime earnings losses from illness morbidity or premature mortality have been used in some studies to value outcomes such as initiation of substance use for tobacco, alcohol, or illicit drugs (Aos et al., 2004). There have also been a number of studies of the effects of clinical depression on labor market productivity and work days missed (e.g., Lerner et al., 2004). These projections rely on an assumption that substance use in childhood is causally related to future use and/or dependence, outcomes which can result in lost work time due to impaired work functioning, disability, and premature mortality.

Other outcome domains

Other outcome domains that could yield distal outcome measures include healthcare utilization, public disability income program receipt, and receipt of cash, services, or other benefits from public programs (e.g., food stamps). However, empirical evidence linking children’s behavioral outcomes to these outcomes is somewhat sparse and may have limited generalizability. For example, estimates of the future public costs of conduct disorder derived from the Fast Track project sample (Foster & Jones, 2005) may be one of the only reliable sources of information on long-term costs associated with aggressive or violent behavior in children who grew up in the U.S.

Using data from a U.S. cohort of 664 youths from 4 communities, Foster and Jones estimated public sector costs during the study period of 6^th through 12^th grade, and compared costs for youth with conduct disorders to costs for youth with other types of conduct problems or no conduct problems. Public costs for the conduct disorder group were $76,795 per youth in year 2000, nearly $60,000 greater than public costs for the no-problems group ($16,940 per youth). This estimate of the incremental cost of conduct disorder could be used to provide a benchmark of savings through 12^th grade for intervention studies that prevent onset of conduct disorder, if properly discounted. However, as this estimate includes public sector costs only through 12^th grade (approximately age 18), it may substantially underestimate the lifetime benefits of prevention and it is unknown whether public costs in this sample are representative of costs for all U.S. youths with conduct disorder.

In a proximal-distal analysis of the costs and benefits of the Good Behavior Game intervention, Aos and colleagues (Aos et al., 2004) combine estimates of the average effect of the Good Behavior Game on males’ probability of “smoking initiation” with projections of the lifetime costs of tobacco smoking to estimate the economic benefits of the Good Behavior Game, a Kindergarten classroom-based 2-year-long preventive intervention led by teachers. Smoking initiation was the only outcome considered in the analysis, even though the Good Behavior Game was designed to reduce aggressive behavior in children and several related outcomes (Kellam, Rebok, Ialongo, & Mayer, 1994; Kellam et al., 2008). The exclusion of other outcomes may have been conservative, insofar as the benefits of smoking reduction alone exceeded the full costs of the intervention. If other benefits had been included, presumably the estimated net benefits would have been even greater.

In summary, practical proximal-distal models that provide estimates of future economic benefits resulting from particular outcomes that may be measured in prevention studies are a work-in-progress. Although models are available for some combinations of proximal and distal outcomes, these models do not yet encompass the full breadth of outcomes that may be measured in intervention trials. To form a proximal-distal model for any particular intervention trial, the intervention trial’s database and the longitudinal database used for predicting future economic outcomes must both contain some of the same proximal outcomes. Consequently, the selection of effects to be included in a proximal distal model may be constrained by the existence of longitudinal data for the proximal outcomes of interest.

Step 3. Selecting a Relevant Time Horizon

In specifying Equation 1 (above), the analyst must choose a relevant time horizon, which means a timeframe over which the intervention’s effects on economic outcomes are projected. In general in proximal-distal models, there is a tradeoff between uncertainty inherent in modeling more distal future outcomes and bias caused by the omission of more distal future effects. Effects occurring far into the future may be sensitive to unexpected future changes in public programs or secular changes in personal incomes, health habits, or economic circumstances, which could alter an intervention’s benefits or costs. For example, compared to the year 1990, today there are approximately 20 percent fewer current smokers (American Lung Association, 2011). This suggests that projections made circa 1990 of the potential future savings from smoking prevention might overestimate the actual realized savings of these programs. Further, some evidence suggests that secular declines may be a general phenomenon for certain unhealthy behaviors (e.g., Pinker, 2011). In this case, the proximal-distal analyst could account for these secular trends within the proximal-distal model by specifying a higher discount rate.

On the other hand, shorter time horizons may result in failure to account for economic effects that are large in magnitude but that do not accrue until a person reaches an age when those effects can emerge. For example, the accumulated lifetime earnings of college graduates net of their education expenses on average exceed those of high school graduates (Heckman et al., 2006), even though college attendance has near-term costs (e.g., tuition, fees, supplies, and foregone earnings) that may exceed the increase in accumulated earnings during the first several post-college work years. Consequently, in order to capture the potentially large costs and benefits associated with outcomes such as employment, criminal justice, public welfare/disability, and mental health care utilization outcomes, the time horizon used in making most proximal-distal projections must extend at least into adulthood.

Step 4. Linking Proximal and Distal Effects

Perhaps the most speculative component of a proximal-distal projection involves modeling the causal links between proximal and distal effects. Few, if any, outcomes in childhood are known with certainty to result in adverse economic outcomes twenty or thirty years into the future. Rather, proximal outcomes may be informative regarding the true probability distribution of future outcomes for an individual. The proximal-distal analyst uses empirical estimates of the conditional probabilities of future outcomes (γ_l|k in Equation 1)—conditional on the proximal outcomes—to model the future economic effects of proximal outcomes. Analyses of longitudinal databases, such as long term follow-up studies of epidemiological samples, may provide information on the conditional probabilities of distal outcomes, conditional on various levels of proximal outcomes. For certain proximal and distal outcomes, an extensive body of evidence has been developed, and estimates of association are considered reliable. For other proximal and distal outcomes, evidence on proximal-distal associations may be less extensive and may not be considered robust.

To give a concrete example, Aos and colleagues (Aos et al., 2004) used estimates of the probability of becoming a regular tobacco user in adulthood (the distal outcome) conditional on age of first tobacco use (the proximal outcome) to link a previously published effect of the Good Behavior Game with future smoking costs. Future smoking costs were those resulting from foregone lifetime earnings due to premature mortality and smoking related morbidity and greater medical care costs. Estimates of these costs were obtained from previously published estimates for the U.S. population. An effect of the Good Behavior Game on the likelihood of initiating tobacco use by age 14 (Kellam & Anthony, 1998) was linked to these future costs indirectly through the implied impact of the Good Behavior Game on the likelihood of becoming a regular smoker in adulthood. Transition probabilities were derived from published estimates of the prevalence of regular smoking in adulthood among persons who had started smoking at various ages in adolescence. One assumption imbedded in this use of published transition probabilities is that published estimates for the U.S. population generalize to the population which was studied in the original Good Behavior Game trials. This assumption may or may not be reliable. The Good Behavior Game trial included predominantly children from low-income, urban, African American families. Smoking transition probabilities for that population might deviate substantially from transition probabilities for a representative U.S. population.

Similar modeling approaches can be used to construct estimates of lifetime costs attributable to alcohol use, drug use, depression, or other mental illnesses. All of these models require an assumption that the intervention of interest affects only the likelihood of the proximal outcomes, and does not directly affect the conditional probabilities of distal outcomes. Stated differently, the probability distribution of γ_l|k in Equation 1 (above) is independent of the intervention. For example, the proximal-distal approach implicitly assumes that if exposure to an intervention of interest results in a delay in age of first tobacco use, the same exposure does not also affect the likelihood of becoming a regular smoker in adulthood, conditional on the effect being modeled indirectly through age of first tobacco use. If this assumption is wrong, the resulting proximal-distal estimate of intervention benefit will be biased, and the direction of this bias depends on how exposure impacts future probabilities for the distal outcomes.

Step 5. Reporting Results

Summary measures of net benefits are often reported in isolation, yet, as we discuss in the sections that follow, the interpretation and utility of economic projections can be enhanced by providing contextual information.

Sensitivity analyses

Conducting sensitivity analyses is the standard way of representing uncertainty in an economic projection. In addition to sampling variability, estimates of net benefits are uncertain due to uncertainty about the assumptions of the projection model and the modeling decisions made in its construction. In a sensitivity analysis, critical parameters of the projection are varied (e.g., from best to worst case), and the net benefits of intervention are recalculated (Gold, Siegel, Russell, & Weinstein, 1996). The resulting variability in net benefits (or lack thereof) indicates how robust the projection may be to possible deviations of parameter values from their assumed values, and helps the reader judge the robustness of the net benefit estimate to variations in the analysis’ key assumptions. Among all the assumptions of a model, the assumed intervention effect size, which in most cases is itself an estimate based on the average effect reported in one or more research studies, is usually a critically influential value in the prediction model. Consequently, the implications of plausible variation in the intervention effect size should, as a general rule, be a focus of a sensitivity analysis. The overall cost-benefit findings of proximal-distal analyses are also sensitive to transition probability values used to link proximal with distal effects. These probabilities are multiplied by future costs and benefits in an equation like Equation 1 (above). Consequently, any proportional increase or decrease in a transition probability will increase or reduce overall expected costs and benefits by an equal proportion. This suggests that transition probabilities should always be subjected to a sensitivity analysis. Full presentations of sensitivity analysis methods are available from several sources (see, for example Gold et al., 1996; Stinnett & Mullahy, 1998).

Distribution of effects

The distribution (or accrual) of costs and benefits across program clients and the broader public and their distribution across public and private sectors of the economy is important because it has implications for program financing and program participation. The willingness of public officials to finance new public programs may depend on the extent to which the public benefits of the program outweigh its public costs. In contrast, program participation is likely to depend on the extent to which the private benefits to participants outweigh any private costs of participation. Belfield et al.’s 2006 cost-benefit analysis of the Perry Preschool program provides a good example of how to contextualize the distribution of effects in a cost-benefit analysis (Belfield et al., 2006). Estimated overall program net benefits were $229,645 per participant. Participants were estimated to receive $49,190 (21%) of this total, and the general public was estimated to receive $180,455 (79%). The vast majority of the gross public benefit ($171,473 or 87%) was attributable to savings resulting from fewer violent crimes committed, whereas nearly all of the gross private benefit to participants ($50,448 or 97%) was from increased earnings. A similar distribution of public benefits can be found in evaluations of other prevention studies, as criminal justice system expenditures dwarf most other categories of public expenditures for children and young adults.

Implementation issues

One of the least developed aspects of economic projection methodology concerns the effects of program implementation. It is often assumed that a program, once implemented, will produce an average effect size equal in magnitude to the effect obtained in a prior randomized trial. That assumption may be unrealistic, for two reasons. First, when testing an intervention’s efficacy, researchers usually dedicate considerable time and resources to support high-fidelity implementation through intensive training, supervision, expert consultation, and fidelity monitoring. However, similar levels of support may not be available post-implementation of the experimental intervention (National Research Council and Institute of Medicine, 2009; Crowley, Jones, Greenberg, Feinberg, & Spoth, 2012). An economic evaluation of an intervention trial should consequently include a realistic estimate of the costs associated with providing training and support for the delivery the intervention at a level of fidelity needed to produce a desired effect size (Foster et al., 2007).

Second, in real-world implementations, scale effects and regulatory requirements may cause a prevention program’s costs and benefits to deviate from those observed in experimental settings. In relation to scale effects, economic predictions often assume that the average benefit of a program, as estimated from a prevention trial, will remain constant when the program is implemented on a much larger scale (i.e., when the intervention is scaled-up). However, larger implementation may necessitate greater administrative and other overhead expenses that are not usually considered when a program is experimental in nature. Some of these costs may be offset by savings as a result of being able to spread fixed overhead costs over a larger client base, and these effects will vary across programs. In addition to scale effects, regulatory mandates, such as mandates around training, insurance, maximum client-staff ratios, and reporting requirements, may also raise program costs and/or limit average benefits per program client. Consequently, estimates of the likely administrative overhead costs associated with maintaining a necessary or required level of training, performance monitoring, administrative duties, and so forth may be an informative accompaniment to estimates of projected costs and benefits. In depth discussion of these and related implementation issues can be found in a National Research Council and Institute of Medicine (2009). An experimental approach to addressing fidelity and sustainability issues is presented in Crowley, Jones, Greenberg, Feinberg, & Spoth (2012).

Program targeting

In an era of tightening government budgets, the question of how best to target available prevention program financing is becoming more germane. An intervention’s overall net benefits may be maximized by targeting enrollments in programs that offer the intervention to individuals for whom the intervention has the highest marginal net benefits. For example, Foster and Jones (Foster & Jones, 2006) have pointed out that the likelihood that the Fast Track intervention (designed for prevention of conduct problems) is considered sufficiently cost-effective by policymakers is positively related to participants’ baseline risk for conduct problems. Under the assumptions of their analysis, although the FastTrack intervention did not surpass a cost-effectiveness threshold when applied to lower-risk program participants, it did satisfy this same threshold when applied to higher-risk participants.

Salkever and colleagues formalized this notion of targeting in an empirical model (Salkever et al., 2008), in which individual-level information on children’s baseline risk level for a costly adverse future outcome (incarceration) is used to predict for each individual the expected net benefits of a hypothetical program. They then showed by example how the net benefits of the program can be maximized by enrolling additional program participants in order of greatest to least marginal expected net benefits until the marginal expected net benefit for the next enrollee equals the marginal cost of participation. Empirical models based on this approach could be used to help guide enrollment criteria and program size limits in future implementations of prevention programs.

The Good Behavior Game proximal-distal analysis by Aos and colleagues (Aos et al., 2004) is an example of an ad hoc targeting analysis. Girls were not included in the analysis, even though girls had participated in the Good Behavior Game intervention, which had been used in general education classrooms that included both boys and girls. Consequently, girls’ participation contributed to the overall costs of the intervention. However, for girls, the Good Behavior Game was not associated with any reduction of future smoking (Kellam et al., 1994). As a result, intervention benefits for girls were probably less than benefits for boys, on average. Aos and colleagues’ estimates consequently may overestimate the net benefits of the Good Behavior Game when implemented in classrooms containing both boys and girls.

Summary and Conclusions

This paper’s twofold purpose was to help readers become more informed consumers of cost and benefit projections developed from the results of prevention research studies and to highlight that these models utilize specific types of childhood outcomes data from intervention trials. Standardized achievement test scores, special education participation, grade repetition, high school completion, juvenile criminal arrests, substance use, and indicators of psychiatric morbidity all can provide a basis for proximal-distal modeling. Another key point is that proximal-distal analysts have much discretion in selecting the types of benefits that are to be included in projections of future economic benefits. Analysts should characterize these decisions in text accompanying an economic projection, so that readers can identify these decisions and assess their implications. A third key point is that in choosing a time horizon, proximal-distal analysts implicitly are balancing their uncertainty about how the evidence supporting the model’s assumptions might change in the future, which argues for a shorter time horizon, against the bias that may result from leaving out important future effects, which argues for a longer time horizon. Further methodological research on how to address this tradeoff optimally may be worthwhile.

It is also important to keep in mind the limitations of proximal-distal modeling vis a’ vis more conventional cost-benefit analyses of preventive interventions that are based on data from longer-term follow-ups of prevention study cohorts after they reach adolescence and adulthood (e.g., Belfield et al., 2006). In comparison to proximal-distal projections, such analyses do not require the analyst to make stringent assumptions about the empirical relationships between proximal and distal outcomes or to assign average monetary values to future outcomes that are not directly measured. As a result, the conclusions of such studies may be considered more reliable. In this regard, it bears emphasis that proximal-distal projections are based on limited and preliminary information, and consequently do not replace the need for longer-term follow-up of prevention study participants. In some instances, a proximal-distal projection may indicate that under conservative assumptions, including possible underestimation of the intervention’s benefits, the intervention’s forecast benefits far exceed the costs of intervention. Such a finding would strongly support the need for a longer-term follow-up of participants and could be sufficient to support further implementation of the intervention.

In some instances, the proximal-distal projection may offer mixed evidence on the likelihood that an intervention has benefits in excess of costs. For example, predicted benefits may be substantial for participants who had a high aggression rating at the study baseline but minimal for others. In such instances, a reassessment of the findings may suggest the need to gather, in future follow-ups, data for an expanded set of proximal measures—which could be used to expand the range of domains included in future benefit projections—or may suggest the need for a more detailed assessment of possible proximal benefits, especially for the subgroup for whom the benefits appear to be largest.

Some prevention scientists may not realize that economic projections of net benefits have already been published for many experimental childhood interventions (see Aos et al., 2004). Point estimates of net benefits, such as those that have been reported in policy studies, may disguise underlying variability in intervention effect sizes, associations between proximal and distal outcomes, or the economic value of distal outcomes. In addition, some parameter values used in existing economic prediction models may be based on empirical data that are outdated or that are now understood to contain bias, based on more recent findings or methods. Consequently, reappraisals and regular updates economic projections are also critical to the development of a cost-benefit evidence base in prevention.

One reason why proximal-distal economic modeling techniques may not be more frequently applied to results from child-focused preventive intervention trials is that key pieces of information needed to construct these models are unavailable or difficult to acquire. Transition probabilities needed to project educational attainment, labor market outcomes, healthcare utilization, criminal justice system involvement, and other outcomes in later adolescence and early adulthood from social, behavioral, and academic outcomes in childhood and adolescence may be obtainable only from one or two existing longitudinal cohort studies, and those databases often are difficult to access and/or considerable experience in their use is required. Further evidence of robust proximal-distal associations with economic outcomes is needed to expand the array of proximal endpoints that can be used for proximal-distal analysis.

Finally, to enhance the science of proximal-distal economic modeling, greater collaboration between intervention scientists and analysts who have expertise in modeling long-term economic costs and benefits may be essential. Such collaborations have been uncommon, and, consequently, data from research intervention trials have been infrequently available for use in cost-benefit analysis or for proximal-distal modeling of economic effects. As was demonstrated in a cost-effectiveness analysis of the Fast-Track intervention data (Foster & Jones, 2006), some interventions may be more cost-effective for some subgroups of intervention participants than for others. Further evidence on modifiers of interventions’ economic benefits opens up the possibility of designing empirically-based implementation strategies that enhance the expected economic benefits of preventive interventions. More frequent exploratory economic assessment of findings from intervention trials using proximal-distal modeling techniques may increase the likelihood of uncovering other nuanced economic interpretations of results in experimental preventive interventions.