How Has the Economic Downturn Affected Communities and Implementation of Science-Based Prevention in the Randomized Trial of Communities That Care?

Abstract

This study examined implications of the economic downturn that began in December 2007 for the Community Youth Development Study (CYDS), a longitudinal randomized controlled trial of the Communities That Care (CTC) prevention system. The downturn had the potential to affect the internal validity of the CYDS research design and implementation of science-based prevention in study communities. We used archival economic indicators and community key leader reports of economic conditions to assess the extent of the economic downturn in CYDS communities and potential internal validity threats. We also examined whether stronger economic downturn effects were associated with a decline in science-based prevention implementation. Economic indicators suggested the downturn affected CYDS communities to different degrees. We found no evidence of systematic differences in downturn effects in CTC compared to control communities that would threaten internal validity of the randomized trial. The Community Economic Problems scale was a reliable measure of community economic conditions, and it showed criterion validity in relation to several objective economic indicators. CTC coalitions continued to implement science-based prevention to a significantly greater degree than control coalitions 2 years after the downturn began. However, CTC implementation levels declined to some extent as unemployment, the percentage of students qualifying for free lunch, and community economic problems worsened. Control coalition implementation levels were not related to economic conditions before or after the downturn, but mean implementation levels of science-based prevention were also relatively low in both periods.

What happens to a longitudinal randomized controlled trial when a major exogenous event occurs in its midst? This was the question faced by the Community Youth Development Study (CYDS), the first randomized controlled trial of Communities That Care, a prevention system aimed at reducing problem behavior in youth (Hawkins, Catalano et al., 2008). The economic downturn, the most significant economic event since the Great Depression of the 1930s, began in December 2007, 7 months before the end of a 5-year CYDS implementation phase and the start of a 5-year sustainability phase, with potential implications for key study outcomes such as the implementation of science-based prevention, and the internal validity of the study itself.

The downturn is of particular concern because its effects on individuals and families have been broad, deep, and unyielding. At the end of 2010, over 14 million workers were unemployed, nearly double the annual average during the prior decade, and the unemployment rate exceeded 9% (Bureau of Labor Statistics, 2011b, 2011c). Housing prices in major markets have declined more than 30% from their 2006 peaks, sharply reducing the value of the major asset of many Americans (Standard and Poors, 2011). At the same time, families have struggled to stay current on mortgage payments, with nearly 14% of loans either delinquent or in the foreclosure process at the end of 2010 (Mortgage Bankers Association, 2010). Not surprisingly, estimates of annual median household income have declined in real terms (i.e., after the effects of inflation are removed so that income levels can be compared over time) by $4,000 since 2007 (United States Census Bureau, 2011c). Poverty has increased from 12.5% to 15.1%for all persons and from 18.0% to 22.0% for persons under age 18 (United States Census Bureau, 2011b). In this period of heightened need, state governments have been unable to provide sufficient support due to fiscal constraints. Federal stimulus dollars helped stabilize state and local governments immediately after the downturn, but most states have since reduced budgets below their 2008 peaks (National Governors Association and the National Association of State Budget Officers, 2011). The Federal Reserve recently lowered its forecast for GDP growth and unemployment through the end of 2013 (Federal Reserve Board of Governors, 2012). The significant economic stress on individuals and families is likely to continue.

The downturn has the potential to affect the CYDS in several ways. In this paper we describe economic trends in CYDS communities before and after the downturn using multiple economic indicators, assess potential threats to the internal validity of the CYDS arising from the downturn, explore criterion validity between subjective and objective economic indicators, and examine the relationship between economic conditions and implementation of science-based prevention following the downturn.

Communities That Care and the CYDS

Communities That Care (CTC) is a coalition-based prevention system designed to assist communities in applying prevention science to reduce prevalent youth problem behaviors including substance use, delinquency, and violence (Hawkins, Catalano et al., 2008). CTC uses a risk- and protection-focused approach to prevention. The premise is that when communities decrease risk factors and increase protective factors affecting youth, problem behavior can be reduced. Communities using CTC start by establishing a broad-based coalition of key stakeholders from diverse community sectors (e.g., local government, education, human services, health services, business, media, police, etc.). The coalition collects data, including surveys from youth, to understand the nature and extent of exposure to risk and protective factors as well as youth health and behavior problems in the community. The coalition uses the results to develop a community-specific profile of elevated risk factors, depressed protective factors, and problem behavior. Coalitions prioritize two to five risk and protective factors for improvement. They then select scientifically tested and effective prevention programs to address their priorities, ensure programs are implemented with fidelity, and monitor progress towards goals. Community-wide effects on risk and protection are expected to be observed within 2 to 4 years, and effects on youth outcomes within 4 to 10 years.

The initial CYDS study period was from 2003 to 2008, and a sustainability phase spanning 2008 to 2013 is ongoing. Results have shown CTC to be effective. Four years after the start of the trial, CTC communities were significantly more likely to have adopted a science-based approach to prevention (Brown, Hawkins, Arthur, Briney, & Fagan, 2011), to have implemented the core components of the CTC system (Arthur et al., 2010), to have faithfully implemented evidence-based programs addressing prioritized risk and protective factors (Fagan, Hanson, Hawkins, & Arthur, 2009), and to be monitoring prevention program effects (Fagan, Arthur, Hanson, Briney, & Hawkins, 2011). In a panel of students followed from Grade 5 in randomly assigned CTC and control communities, prioritized risk factors increased significantly less rapidly in CTC communities from Grade 5 to the spring of Grade 7 (Hawkins, Brown et al., 2008). One year later, in the spring of Grade 8, youth in CTC communities were 33% less likely to have initiated cigarette smoking, 32% less likely to have initiated alcohol use, and 24% less likely to have initiated delinquent behavior than youth from control communities (Hawkins et al., 2009). The prevalence of youth problem behaviors, including past-month alcohol use and binge drinking in the past 2 weeks, was reduced in CTC communities as well (Hawkins et al., 2009). Effects on delinquency and substance use generally have been universal, regardless of gender and baseline risk differences in youth (Oesterle, Hawkins, Fagan, Abbott, & Catalano, 2010). CTC also was found to be cost beneficial, generating $5.30 in returns to society for every dollar invested (Kuklinski, Briney, Hawkins, & Catalano, 2012). Effects on prevention system and youth outcomes have been sustained through 2009 when panel youth were in Grade 10, and one year after study-provided financial support and technical assistance to CTC communities ended (Fagan, Hanson, Briney, & Hawkins, 2012; Gloppen, Arthur, Hawkins, & Shapiro, 2012; Hawkins et al., 2012; Rhew, Brown, Hawkins, & Briney, in press).

The Economic Downturn and the CYDS

The economic downturn could threaten the internal validity of the CYDS. At the study’s outset, care was taken to ensure internal validity by matching communities within states on demographic characteristics and community use of science-based prevention approaches before random assignment of communities to conditions within each matched pair. Analyses of outcomes have been conducted using an “intent to treat” approach, and there has been no evidence of differential attrition in samples of youth from CTC and control communities (Brown et al., 2009; Hawkins et al., 2012). Measures have demonstrated reliability and have been used consistently in a planned repeated measures research design (Brown et al., 2009; Hawkins, Catalano et al., 2008). However, the design could not anticipate the onset of a major exogenous historical event like the economic downturn. If the downturn differentially affected CTC versus control communities, economic effects and intervention effects could be confounded, compromising the ability to reach conclusions about the efficacy of CTC.

Objective economic data are available publicly at state and national levels, less so at county levels. They are rarely disaggregated below the county level, making it difficult to examine economic changes following the downturn within communities. The relevant unit of economic analysis may, however, be broader than the community. For example, many individuals work outside the communities they live in, rendering county a potentially meaningful unit of analysis for employment data. Data on unemployment and income are publicly available annually at the county level, and poverty data are available at the county and community levels. We supplemented these data sources with annual home mortgage performance indicators measured at the community level. Volatility in the housing market suggested these indicators could further illustrate community-level economic concerns. Objective data offered the ability to chart economic trends before and after the downturn in sample communities and counties and could be used to assess threats to internal validity. Even if internal validity were preserved, economic conditions could vary significantly in sample communities over time. Economic indicator data also could be used to examine this variability before and after the downturn.

The degree of economic downturn in CYDS communities could affect implementation of science-based prevention in CTC and control coalitions. Prior research on the sustainability of prevention coalitions indicates that the ability to obtain funding is critical to long-term sustainability (Feinberg, Bontempo, & Greenberg, 2008), yet potential state, county, and community funding sources have diminished substantially in the face of lower revenue streams (National Governors Association and the National Association of State Budget Officers, 2011). Increasing levels of unemployment may also have negative impacts on the availability of human resources needed to implement science-based prevention activities carried out by CTC and other prevention coalitions. Implementation might suffer further because of reduced support and political will in the face of increasingly scarce resources. However, research on community budgeting suggests that groups that are more effective in advocating for their own interests are better able to maintain resources in difficult budgetary times (Skidmore & Scorsone, 2011). CTC’s strong prevention infrastructure, including the support of key community leaders in CTC coalitions, as well as its emphasis on implementing evidence-based programs, might help CTC coalitions maintain implementation levels even in the face of deteriorating economic conditions.

Objective economic indicators are likely to be associated with prevention system implementation levels, but subjective reports of economic conditions as perceived by those within communities may also help explain implementation following the downturn. Key community leaders typically have the power and authority to direct resources to prevention activities. Their perceptions of community economic conditions are likely to affect these decisions, as they may be less inclined to allocate resources to CTC and other prevention efforts during a time of perceived economic difficulty. Although prevention implementation levels may be related to objective economic data, particularly those that are well publicized like unemployment rates, key leader perceptions of community economic conditions may also be important predictors of prevention implementation.

Subjective measures of economic conditions are rare, but in their efforts to understand the relationship between poverty and economic conditions and family functioning, Conger and colleagues developed a seven-item Community Economic Problems scale (Statistical and Measurement Unit, 1993) asking about various aspects of community economic conditions, including business failure, unemployment, and people leaving the area. Their scale showed good reliability (Cronbach’s standardized alpha = 0.82). Though it has not been published, the Community Economic Problems scale could prove useful in depicting community economic conditions and in understanding science-based prevention implementation following the downturn.

The Present Study

We examined three issues related to the economic downturn and the CYDS: (1) We estimated trends in economic conditions in CYDS communities before and after the downturn using multiple objective economic indicators (unemployment, poverty, income, and home mortgage performance). We assessed threats to the internal validity of the CYDS by examining whether economic trends in CTC and control communities differed, especially after the downturn. We also examined evidence for variability in economic trends across individual CYDS communities. (2) We explored the utility of the Community Economic Problems scale (Statistical and Measurement Unit, 1993) as a subjective measure of community economic conditions. We examined the scale’s reliability in a sample of key community leaders and also evaluated its criterion validity against objective economic indicators. (3) We examined the relationship between economic conditions and implementation of science-based prevention in 2009, approximately 2 years into the economic downturn period. We wanted to understand whether economic conditions (objective and subjective) were associated with prevention system implementation levels in 2009, as well as whether CTC communities were better able to maintain implementation of science-based prevention than control communities in this period of general economic distress.

Method

The CYDS sample consists of 24 communities in Colorado, Illinois, Kansas, Maine, Oregon, Utah, and Washington, matched in pairs within state on population, racial and ethnic diversity, economic indicators, and crime rates prior to the start of the intervention. One community from each matched pair was assigned randomly to the intervention (CTC) or control condition. At the study’s start none of the communities had advanced in the use of science-based prevention to the point of selecting and using tested, effective preventive interventions to address prioritized community risks (Brown et al., 2009; Hawkins, Catalano et al., 2008). Youth in CTC and control communities did not differ at baseline on key outcome variables including substance use and delinquency (Hawkins, Catalano et al., 2008). Participating communities are small- to moderate-sized incorporated towns with their own governmental, educational, and law enforcement structures (population range: 1,500 - 50,000, M = 17,270, SD= 10,594).

Measures

Archival data

We collected annual data on unemployment, home loan performance, poverty, and income from 2004, when CTC prevention programs were first implemented, as far forward as possible, in order to compare trends during the relatively stable economic period at the start of the study through the post-downturn period. Data were available at different levels of aggregation. They were collected at the community level whenever possible, identified in data sources by zip code, city/town name, or school district. However, county was the smallest available data unit for unemployment, income, and some poverty measures. In these cases communities were assigned economic data based on their county location.

Unemployment

Annual average county unemployment rates for 2004 to 2010 were from the Bureau of Labor Statistics (2011a).

Home loan performance

Home mortgage performance data were from Lender Processing Services, a large national proprietary loan-level database estimated to cover approximately two thirds of first-lien mortgages originated after 2005. The database includes information on delinquency and foreclosure status for both prime and subprime loans. We developed an annual time series of loan performance data at the zip code level for first-lien, owner-occupied mortgages, measured in April for 2005 to 2011. Loan performance measures included the number of home loans in the zip code, number of loans on which payments were current, number of loans that were 90 days or more delinquent, and number of homes in the foreclosure process. Most CYDS communities were represented by one zip code. Data from multiple zip codes were combined when necessary to reflect the entire community. We converted raw data to percentages to adjust for differences in the number of loans across communities.

Poverty

We examined three poverty measures. Data on the percentage of students qualifying for free lunch were from each participating state’s October Survey reports for 2004 to 2010, which document eligibility for free and reduced-price lunch on the last operating day of October (Colorado Department of Education, 2011; Illinois State Board of Education, 2011; Kansas State Department of Education, 2011; Maine Department of Education, 2011; Oregon Department of Education, 2011; State of Washington Office of Superintendent of Public Instruction, 2011; Utah State Office of Education, 2011). Colorado and Utah data were available only from 2005 forward, so we used National Center for Education Statistics data (2011) for 2004 estimates. County poverty rates for people of all ages and for children ages 5 to 17 are from the Small Area Income and Poverty Estimates program of the United States Census (2011a).

Income

County median household income levels were also obtained from the Small Area Income and Poverty Estimates program. We converted nominal estimates to constant 2004 dollars using the implicit price deflator for personal consumption expenditures (Bureau of Economic Analysis, 2011) to estimate real income trends, that is, so that we could examine changes in income after removing the effects of inflation on prices.

Key leader perceptions of economic conditions

These data were from the Community Key Informant Survey (CKI) conducted by trained interviewers over the telephone. In 2009, 348 key leaders from the 24 CYDS communities were surveyed (range: 12 - 15 key leaders per community). They consisted of positional leaders (e.g., mayors, city managers, police chiefs, school superintendents, heads of social service agencies) and up to five additional referred leaders most frequently identified by positional leaders due to their knowledge of prevention. This study uses reports of 313 key leaders who had complete data; missing data analyses revealed no differential attrition in CTC and control communities or clustering of missingness by community or demographics. Key leaders in CTC and control communities were similar with respect to age, leadership status, gender, education, and time they had resided in the community. They represented 11 different employment sectors (8% to 13% from each): business, churches, civic organizations, community coalitions, human services, judiciary, law enforcement, media, recreational organizations, health care, and schools. Key leaders in CTC communities were significantly more likely to represent the community coalition sector (CTC = 10.6%, Control = 4.0%, F(1, 22) = 5.07, p< .05), but otherwise representation was the same.

The 2009 survey contained questions about community economic conditions following the downturn. Respondents completed six of seven items from the Economic Problems subscale of the Community Economic Problems instrument (Cronbach’s standardized alpha = 0.82). Questions addressed concerns about business failures, rundown conditions in businesses and homes, financial problems for people in the community, people leaving the area, unemployment, and jobs paying too little to raise a family (e.g., “There have been a lot of business failures in our area in recent years.”). We removed the item “Too many women with children have to work around here in order to have enough money to live on.” We added the item “There have been a lot of home foreclosures in our area in recent years” because of sharply increasing home foreclosure rates since the downturn began. Response options ranged from 1 “strongly agree” to 5 “strongly disagree.” Items were coded so that higher scores indicated greater economic problems. A key leader’s score was the mean of his/her item scores. Key leader scores were averaged to yield a community-level score to assess variation in perceived economic conditions across communities. Positional versus referred leader status, gender, age, education, years resided in the community, and sector represented were included as covariate controls in analyses of key leader scores because individuals with different characteristics, backgrounds, and positions within the community might have different sensitivity and exposure to community economic conditions (Riggs, Feinberg, & Greenberg, 2002).

Coalition Implementation of Science-Based Prevention

We examined implementation of science-based prevention among prevention coalitions identified in CTC and control communities operating in 2007 and 2009 (N =25, 11 CTC, 14 control); sample recruitment is described in Arthur et al. (2010). Coalition chairs were surveyed over the telephone about the degree to which their coalitions had achieved 15 benchmarks shown in Table 1 indicating adoption of a science-based approach to prevention (Arthur et al., 2010). Designed originally to assess adherence to the CTC prevention system, the 15 benchmarks more generally assess the degree to which coalitions implement science-based prevention. Coalition chairs were interviewed in 2007 and again in 2009 about whether they had completed each benchmark. Completed benchmarks were assigned a score of 1. A coalition’s total science-based prevention implementation score was calculated by summing individual benchmark scores (range 0-15).

Table 1.

Science-Based Prevention Implementation Survey Benchmarks

Does the coalition use a risk and protection framework? Has the coalition received training in the risk and protection framework? Have 75% or more of coalition members been trained in the risk and protection framework? Has the coalition assessed risk and protective factors in the community? …using student surveys? …using archival indicators? Has the coalition focused on any specific risk or protective factors? Has the coalition assessed prevention resources in the community? Did the coalition develop an explicit, written action plan? Were specific programs implemented to address these factors? Did the coalition sponsor at least two tested and effective programs? Has the coalition evaluated or monitored the results of these programs? …using pre/post surveys? …using changes in participant outcomes (e.g., delinquency, school performance, etc.)? …using changes in participant risk and protective factors?

Intervention condition

A dummy variable coded 0 for control communities and 1 for CTC communities was used to indicate intervention condition.

Data Analytic Strategy

Economic trends and the internal validity of the CYDS

We used multilevel linear regression to estimate economic indicator trends before and after the economic downturn began. Multilevel techniques were appropriate because data were nested, with time at Level 1 nested within community at Level 2. We evaluated whether (1) trends in economic indicators differed before and after the downturn, (2) trends differed significantly across individual communities, and (3) trends differed systematically in CTC versus control communities. Economic data were centered so that estimated intercepts reflected economic indicator levels in 2004/05.

Piecewise regression (Raudenbush & Bryk, 2002) was used to model separate trends in economic indicators prior to 2007 (pre-downturn period) and from 2007 forward (post-downturn period), estimated using the statistical analysis software package HLM 6: Hierarchical Linear and Nonlinear Modeling (HLM). To understand the range of economic conditions in the CYDS sample, particularly after the downturn, we evaluated variability between communities in intercepts and pre- and post-downturn slopes. Time (at Level 1) and community (at Level 2) were the only predictors in these models. We assessed variability with the Chi-square statistic provided by the HLM program. When variation in trends across communities was significant, we retained the Empirical Bayes community-specific trend estimates generated by the HLM program so that we could compare trends in different communities. Community-specific trends in the post-downturn period were interpreted as representing the degree of economic downturn effect, with higher values (i.e., greater annual change in indicators) indicating stronger effects.

We next added an intervention condition dummy variable at Level 2 to assess whether there were systematic differences in intercepts (i.e., indicator levels in 2004/05), pre-downturn slopes (i.e., annual changes in economic indicators from 2004/05 to 2007), or post-downturn slopes (i.e., annual changes in economic indicators from 2007 forward) in CTC compared to control communities. Significant differences between CTC and control communities would indicate threat to internal validity. Nonsignificance would indicate any threats were minimal.

Reliability and criterion validity in the Community Economic Problems scale

We evaluated the reliability of the Community Economic Problems scale in the sample of 313 key leaders using Cronbach’s alpha, and we also examined item intercorrelations. Descriptive statistics portraying community-level means and variation in key leader perceptions of community economic problems were also reviewed.

We assessed criterion validity in the Community Economic Problems scale against objective economic indicators. We used multilevel linear regression analysis to predict individual key leader perceptions of community economic problems because key leaders were nested within communities. We started by estimating fully unconditional two-level models (Level 1: individual key leader perceptions, Level 2: community) to understand the degree of variation between communities in key leader perceptions, as represented by the intraclass correlation coefficient. We then estimated a second set of regressions controlling for key leader covariates at Level 1 and intervention status and objective economic indicators at Level 2. This specification allowed us to understand the role of the Level 2 predictors, the focus of our study, after the effects of individual differences in key leaders were controlled. We explored whether 2009 levels of economic indicators, measured at the same time as key leader perceptions, explained variation in Community Economic Problems scores reported by key leaders. We also evaluated whether change in economic indicators from 2007 forward, represented by community-specific post-downturn trend estimates (derived from prior longitudinal analysis of economic indicators), explained key leader reports. The intervention condition dummy variable offered an additional assessment of internal validity. Predictors were not centered. Coefficients on Level 1 dummy variables represented the shift in Community Economic Problems scores associated with being a referred leader, female, or highly educated, with other predictors held constant. Sectorial coefficients represented the shift relative to the mean, again with other predictors held constant.

The economic downturn and coalition implementation of science-based prevention

We evaluated relationships between economic conditions and coalition implementation of science-based prevention using regression analysis in which the 2009 implementation score was the dependent variable. We started by examining correlations between economic indicators and implementation levels in 2007 and 2009, before and after the downturn began. We used Fisher’s z transformation (Howell, 2010) to evaluate whether relationships between implementation levels and economic indicators in CTC communities differed from those in control communities after the downturn began; transformed correlations follow a z distribution, making assessment of differences straightforward. When significant differences were found, we used regression analysis to see if relationships between economic indicators and 2009 implementation levels held after controlling for 2007 implementation levels. Regressions were not multilevel because all predictors were measured at the community level. Economic conditions were measured by their 2009 levels or annual change following the downturn. Implementation scores were centered around their coalition group mean, but economic indicators were centered around the overall mean. Intercepts reflected expected 2009 implementation scores for the mean CTC or mean control coalition experiencing overall mean levels of economic conditions.

Results

Economic Indicator Trends and the Internal Validity of the CYDS

Results of fully unconditional piecewise regression models of economic indicator trends are summarized in Table 2. The table includes predicted values for each indicator at the start of the study period in 2004/05, as well as pre-downturn and post-downturn slope estimates (i.e., annual change in indicators) for the entire CYDS sample. The table also contains information about whether intercept and slope estimates varied significantly across individual CYDS communities. Standard deviation, associated chi square statistics, and degrees of freedom are presented. Because of our emphasis on understanding economic conditions in the post-downturn period, we also reported the range in post-downturn slope estimates for indicators that differed significantly across individual communities. These values summarized differences in the degree of deterioration in economic conditions following the downturn across sample communities.

Table 2.

Economic Indicator Trends Before and After the Economic Downturn

	Parameter estimates from multilevel piecewise regression a						Community/county variation in parameter estimates						Range: Post-Downturn Annual Change
			Annual Change						Annual Change
	Start: 2004/05		Pre-Downturn: Before 2007		Post-Downturn: After 2007		Start: 2004/05		Pre-Downturn Before 2007		Post-Downturn After 2007
	Intercept	SEa	Slope	SE	Slope	SE	SD	Chi-Square a	SD	Chi-Square	SD	Chi-Square	Low	High
Unemployment (%)	5.85	0.34 ***	-0.53	0.08 ***	1.72	0.15 ***	1.54	142.20 ***	0.19	30.10	0.64	106.74 ***	0.55	2.73
Home mortgage performance (%)
Current loans	94.16	0.51 ***	-0.28	0.17	-1.48	0.12 ***	2.25	127.79 ***	0.41	26.65	0.52	70.25 ***	-0.72	2.5
90+-day delinquencies	0.64	0.12 ***	0.07	0.08	0.58	0.05 ***	0.20	21.56	0.05	15.27	0.17	38.02 *	0.24	0.84
Foreclosures	0.64	0.12 ***	-0.04	0.04	0.56	0.06 ***	0.52	102.88 ***	0.11	28.16	0.29	242.51 ***	0.14	1.26
Poverty rate (%)
Free lunch	33.91	2.70 ***	0.06	0.38	2.64	0.27 ***	13.08	921.39 ***	1.50	70.65 ***	0.81	36.84 *	1.54	3.87
County rate (all ages)	12.28	0.58 ***	0.10	0.11	0.39	0.16 *	2.67	207.68 ***	0.24	25.17	3.48	28.19	na	na
County rate (ages 5-17)	14.41	0.79 ***	0.06	0.11	0.82	0.21 ***	3.74	324.11 ***	0.19	11.36	0.67	36.62 *	0.06	1.91
County median household income ($2004, 000s)	42.43	1.17 ***	0.56	0.19 ***	-0.41	0.23 +	5.61	523.79 ***	0.68	50.81 **	0.65	34.22 +	-1.30	0.57

^aUnemployment, Free Lunch: 2004-10, Home Mortgage Performance: 2005-11, County Poverty & Income: 2004-09. Pre-downturn: 2004/05-07, Post-downturn: 2007-forward. Range pre-downturn annual change in Free Lunch (%) -2.96% - 2.15, County Median Household Income ($2004) -265 – 2,346. There are 23 degrees of freedom in all analyses (parameter estimates, variation in parameter estimates).

^***p < .001,

^**p < .01,

^*p < .05,

⁺p <.10.

At the start of the study, economic conditions were more favorable than in later years. For instance, as shown in Table 2, the intercepts for unemployment, home mortgage foreclosures, and delinquencies of 90 days or more were low. Indicators were stable during the pre-downturn period; the year-to-year change in most indicators did not differ significantly from zero. With respect to variation in these estimates, communities started the study at significantly different levels of economic health, as expected. However, pre-downturn slopes generally did not differ across communities. Results indicated a stable pattern of differences in CYDS communities in this period, with most indicators remaining at essentially the same levels from 2004/2005 to 2007. There were some exceptions. Prior to the downturn, unemployment decreased each year at a similar rate across communities. Median income also improved, but the magnitude varied significantly across communities. The percentage of children qualifying for free lunch increased to varying degrees across communities.

Findings from the post-downturn period, 2007 forward, were quite different. The slope, or year-to-year change in most indicators, differed significantly from zero, indicating worsening economic conditions each year after the start of the downturn. Results also showed significant community-level variation in most indicator trends, meaning that communities experienced significantly different rates of economic deterioration. The range in post-downturn slopes indicated that communities at one end of the sample experienced relatively large economic downturn effects, while at the other extreme, relatively small shifts occurred. Two variables followed a different pattern: The post-downturn slope in all-age county poverty was similar across counties, and the post-downturn slope in county median household income varied only marginally (p < .10) across counties in the sample. Results suggested that most economic indicators moved in community- or county-specific ways after the downturn began.

We next added a dummy variable representing intervention status (CTC v. control) at Level 2 to assess whether economic indicators differed in CTC compared to control communities on average at the intercept in 2004/05, in pre-downturn slopes, and in post-downturn slopes. Relationships were nonsignificant for all eight indicators. For unemployment, for example, Wald t-test statistics are all small and nonsignificant: intercept t(22) = 0.11, p = ns, pre-downturn slope t(22) = 0.25, p = ns, post-downturn slope t(22) = -0.22, p = ns. These analyses provided no evidence of systematic differences in economic conditions in CTC compared to control communities before or after the downturn began, suggesting the economic downturn did not threaten the internal validity of the CYDS.

Reliability and Criterion Validity of the Community Economic Problems Scale

Reliability

Item correlations based on the ratings of 313 key community informants are presented in Table 3. All were significant at the p < .01 level. Cronbach’s alpha for the seven-item scale containing the foreclosure item was .85, evidence of reliability in the Community Economic Problems scale containing this additional item (and with the item “Too many women with children have to work around here in order to have enough money to live on” removed).

Table 3.

Community Economic Problems Scale Item Correlations

	Scale items
	2	3	4	5	6	7
1 Lots of business failures	.572**	.636**	.346**	.616**	.279**	.591**
2 Businesses & homes are more run down		.474**	.447**	.483**	.352**	.444**
3 Lots of people have financial problems			.306**	.711**	.294**	.575**
4 People have left the area				.385**	.270**	.268**
5 Lots of people are unemployed					.359**	.605**
6 Jobs pay too little						.257**
7 Lots of home foreclosures

Note. N =313.

^**p < .01

Community-level variability in community economic problems

When aggregated at the community level, Community Economic Problems scale scores in 2009 averaged 3.54 (SD = .50), half way between a neutral score of 3 and a score of 4 indicating economic problems. Scores ranged from 2.5 to 4.4, but the mean score was less than 3, or better than neutral, in only two communities. Mean scores did not differ statistically between CTC and control communities (F(1, 22) = 0.24, p = ns), further supporting the internal validity of the CYDS after the downturn. However, the degree of economic problems did differ across individual CYDS communities.

Criterion validity of the Community Economic Problems scale with objective economic indicators

The fully unconditional hierarchical model with individual key leader Community Economic Problems scores as the dependent variable indicated significant community-level variation in key leader reports (ICC = .35, χ² (23, N = 330) = 197.45, p < .001), so we tested more complex models to determine whether key leader scale scores were associated with intervention condition and/or with objective economic indicators (both Level 2 variables). We also added key leader characteristics as control variables at Level 1.

Because of the limited number of communities (N = 24), we first estimated separate regression equations for each of the economic indicators. The final model contained all indicators found significant in the separate regressions: county unemployment, community 90+-day home loan delinquencies, and county poverty in children ages 5 to 17. Zero-order correlations between these economic indicators ranged from -.02 to .47. Descriptive statistics for all predictors in the final models are shown in Table 4. One set of regression analyses evaluated the role of indicator levels in 2009. A second set included community-specific estimates of community-level trends (annual rates of change) following the downturn. Results are reported in Table 5. Notably, variation in Community Economic Problems scores was not related significantly to intervention status in either model, providing further evidence that economic problems did not differ systematically between CTC and control communities.

Table 4.

Descriptive Statistics for Variables in Multilevel Regression Equations Predicting Community Economic Problems

	Mean	SD	Mean	SD
Community-level variables (N = 24 communities)
CTC v. control community status	0.50	0.51
Economic indicators	2009 Values		Post-downturn annual change
County unemployment (%)	8.80	2.88	1.72	0.59
90+-day loan delinquencies (%)	2.26	0.76	0.58	0.14
County poverty, ages 5 to 17 (%)	16.78	4.90	0.82	0.47
Key leader characteristics (N = 313 key leaders)
Referred leaders (%)	32.59	47.10
Female (%)	37.38	48.36
MA degree or higher (%)	43.77	49.72
Age (years)	51.38	10.15
Years in community	18.04	17.98
Employment sector (%)
Business	8.31	27.63
Churches	7.03	25.56
Civic	9.58	29.51
Community coalitions	7.35	26.06
Human services	13.10	33.79
Judiciary	9.90	29.91
Law enforcement	10.22	30.31
Media	7.67	26.70
Recreation organizations	7.99	27.17
Health care	8.95	28.52
Schools	9.90	29.91

Table 5.

Multilevel Regression Results Predicting Community Economic Problems

Predictor	2009 Economic indicator values		Post-downturn annual change
	Regression coefficient	SE	Regression coefficient	SE
Community-level predictors (N = 24 communities, df = 19)
CTC v. control community status	-0.022	0.115	0.091	0.118
County unemployment (%)	0.113	0.022 ***	0.470	0.117 ***
90+-day loan delinquencies (%)	0.254	0.077 ***	1.184	0.473 **
County poverty, ages 5 to 17 (%)	0.014	0.013	0.292	0.134 **
Individual-level predictors (N = 313 key leaders, df = 293)
Referred leaders	-0.017	0.080	-0.019	0.080
Female	0.309	0.083 ***	0.305	0.083 ***
MA degree or higher	-0.009	0.081	-0.004	0.081
Age	-0.001	0.004	-0.001	0.004
Years in community	0.000	0.000	0.000	0.000
Employment sector:
Business	-0.435	0.106 ***	-0.428	0.106 ***
Churches	0.178	0.117	0.172	0.117
Civic	-0.309	0.105 ***	-0.312	0.105 ***
Community coalitions	0.159	0.130	0.161	0.130
Human services	0.294	0.093 ***	0.296	0.093 ***
Judiciary	-0.051	0.103	-0.061	0.104
Law enforcement	0.117	0.101	0.121	0.101
Media	-0.011	0.114	-0.006	0.114
Recreation organizations	0.105	0.102	0.108	0.102
Health care	-0.066	0.103	-0.072	0.103
Schools	0.019	0.102	0.022	0.107
Intercept	1.665	0.313 ***	1.692	0.330 ***

^**p < .05,

^***p < .001.

In the final models containing 2009 indicator levels, county unemployment rate and 90+-day delinquencies remained significant. Higher levels of these indicators in 2009 predicted more economic problems reported by key community leaders in 2009. Results using community-specific post-downturn trends were consistent with these findings, except that the county poverty rate in children ages 5 to 17 also was significant in the final equation. Higher rates of growth in unemployment, poverty in children ages 5 to 17, and 90+-day delinquent loan payments after the downturn began were significantly related to more economic problems reported by key community leaders in 2009. The significant objective economic indicators together accounted for 75% to 80% of the variance in community-level Community Economic Problems scores. Unemployment explained 61% to 66% of the community-level variance, 90+-day home loan delinquencies added an additional 7% to 14%, and poverty in children ages 5 to 17 an additional 7% to models containing indicator trends post downturn. Objective economic indicators explained a large part of the variation in key community informant reports of economic problems in their own communities, supporting the criterion validity of community key informant survey data using the revised Community Economic Problems scale.

Table 5 also shows that some additional variation was explained by key leader characteristics. Women reported significantly higher community economic problems, as did those who worked in the human services sector. Those who worked in the business and civic sectors reported significantly lower community economic problems.

The Economic Downturn and Coalition Implementation of Science-Based Prevention

In 2007, before the economic downturn began, CTC coalitions had significantly higher science-based prevention implementation scores than coalitions in control communities (CTC: M = 14.09, SD = 0.83 range: 12-15; control: M = 8.07, SD = 3.47, range: 2-13; F(1, 23) = 31.33, p < .001). They remained higher in 2009 (CTC: M = 12.82, SD = 2.32, range: 9-15; control: M = 7.00, SD = 4.85, range: 0-13; F(1, 23) = 13.34, p < .01), some evidence that CTC coalitions were better able to implement science-based prevention in a time of overall economic decline. Implementation scores were somewhat lower in 2009 than in 2007. Nine of 14 control coalitions had lower implementation scores in 2009. Only 4 of 11 CTC coalitions scored lower.

Correlations between economic indicators and implementation scores in 2007 and 2009 are presented in Table 6 separately for CTC and control coalitions. In 2007, all correlations were nonsignificant and small. Implementation levels were unrelated to economic conditions at this time. By 2009, correlations between economic indicators and science-based prevention implementation levels in CTC coalitions were negative, and moderate to strong in magnitude for all indicators (regardless of whether assessed via 2009 levels or annual rate of change following the downturn). However, correlations achieved significance only with respect to county unemployment rates, the percentage of children qualifying for free lunch, and Community Economic Problems scale scores. As these economic indicators deteriorated in CTC communities, coalition implementation of science-based prevention declined, though levels largely remained above those found in control communities. For control coalitions, correlations remained nonsignificant in the post-downturn period. Tests of transformed correlations confirmed that several correlations found in CTC coalitions were significantly stronger than those found in control communities, though differences were at the trend level for free lunch percentage and Community Economic Problems.

Table 6.

Correlations Between Prevention System Implementation Scores and Economic Variables

	County Unemployment Rate	Home Mortgage Loans (%)			Poverty (%)			County Median Household Income ($2004)	Community Economic Problems Score
						County Poverty
		Current	90+ Days Delinquent	Foreclosure	Free Lunch	All Ages	Ages 17-May
2007 Implementation Score with 2007 Economic Conditions
CTC Coalitions (N = 11)	.165	-.039	-.070	.101	-.077	-.073	-.068	.251
Control Coalitions (N = 14)	.105	-.144	.070	.070	.086	.003	.020	-.100
2009 Implementation Score with 2009 Economic Conditions
CTC (N = 11)	-.678 *	.303	-.324	-.223	-.437	-.030	-.101	-.325	-.546 +
Control (N = 14)	.164	-.120	-.002	.126	.218	.333	.295	-.358	-.008
z-score – difference between transformed correlations	-2.13 *				-1.49 +				-1.30 +
2009 Implementation Score with Annual Change in Economic Conditions – Post-downturn
CTC (N = 11)	-.722 *	-.254	-.085	-.228	-.689 *	na a	-.133	-.245
Control (N = 14)	.092	-.071	-.028	.089	-.057	na a	.190	-.068
z-score – difference between transformed correlations	-2.16 *				-1.70 *

^aGrowth in County poverty (all ages) not included because trends did not differ significantly across communities after the downturn began.

^*p < .05,

⁺p < .10, one-tailed test of whether CTC correlation was significantly stronger than control correlation.

We used multiple regression analysis to assess whether patterns endured after controlling for implementation level in 2007. Results are reported in Table 7 separately for CTC and control coalitions because bivariate correlations suggested important group differences in relationships between economic indicators and implementation scores. For CTC coalitions, 2009 implementation levels were not related to 2007 levels (r(9) = .17, p = ns). Rather, high levels and increases in unemployment, increases in free lunch eligibility, and high levels of key leader reported community economic problems all predicted decreases in coalition implementation of science-based prevention in 2009. In contrast, for control coalitions, 2007 implementation level was the only significant predictor of 2009 implementation level (r(12) = .55, p < .05). Economic indicators did not play a significant role, even after controlling for 2007 implementation levels. However, regression analyses indicated that even at the means of economic indicators (8.4% unemployment, 40.4% qualifying for free lunch, 3.6 Community Economic Problems score, 1.7% annual increase in unemployment, 2.7% annual increase in percentage qualifying for free lunch), the mean CTC coalition was estimated to maintain much higher levels of implementation (12.6 - 13.1 intercept) than the mean control coalition (7.0 - 7.1 intercept). Moreover, estimated 2009 implementation scores in CTC communities at the extremes of economic difficulty ranged from 9.6 to 12.8, still well above mean implementation level of 7.0 in control coalitions (analyses not shown).

Table 7.

Regression Models Predicting 2009 Implementation of Science-Based Prevention from Economic Variables

Indicator:	County Unemployment Rate (%)				Students Qualifying for Free Lunch (%)				Community Economic Problems
	2009 Level		Annual Change – Post Downturn		2009 Level		Annual Change – Post Downturn		2009 Level
Predictor a	B	SD	B	SD	B	SD	B	SD	B	SD
CTC Communities (N = 11, df = 8)
Intercept	13.05	0.56 ***	13.01	0.54 ***	12.78	0.69 ***	12.96	0.56 ***	12.56	0.65 ***
Implementation level in 2007	0.60	0.70	0.33	0.67	0.41	0.79	0.20	0.72	0.51	0.81
Economic indicator	-.61	0.20 *	-2.82	0.95 *	-0.07	0.05	-5.42	2.06 *	-2.62	1.37 +
R2		.51		.54		.21		.48		.33
Control Communities (N = 14, df = 11)
Intercept	7.11	1.16 ***	7.10	1.16 ***	6.98	1.15 ***	6.95	1.15 ***	7.00	1.18 ***
Implementation level in 2007	0.78	0.34 *	0.81	0.35 *	0.75	0.35 *	0.83	0.35 *	0.77	0.35 *
Economic indicator	0.36	0.48	1.80	2.51	0.05	0.08	-2.37	3.08	-0.01	2.55
R2		.34		.34		.33		.34		.31

^aImplementation scores are centered around the mean within coalition type. Economic indicators are centered around the overall mean across all coalitions.

Post hoc analysis of declines in science-based prevention implementation in CTC coalitions

From 2007 to 2009, implementation scores declined by 2 to 5 points for 4 of 11 CTC coalitions. All four faced very difficult economic conditions following the downturn. For example, 2009 unemployment rates were between 9.9% and 13.8% in the communities in which these coalitions operated. Notably, all four coalitions ceased making explicit written action plans in 2009, though all had completed them in 2007. Three coalitions declined in their use of data to evaluate changes in youth outcomes and risk and protective factors following prevention program implementation. Two also stopped reviewing archival indicators to assess elevated risk factors and depressed protective factors in community youth. One offered fewer evidence-based prevention programs to youth.

Discussion

This study advances understanding of economic conditions and implementation of science-based prevention in CYDS communities in the face of deteriorating economic conditions. It used multiple objective economic indicators to describe economic conditions in CYDs communities from 2004 forward, including changing conditions after the economic downturn began. The study also established the reliability and validity of the Community Economic Problems scale (Statistical and Measurement Unit, 1993) as a subjective measure of community economic conditions. Both data sources were used to evaluate internal validity of the CYDS. Finally, the study assessed the relationship between economic conditions and implementation of science-based prevention in CTC and control coalitions after the downturn.

Economic Indicator Trends and Internal Validity of the CYDS

CYDS communities experienced significant change in economic conditions following the downturn that started officially in December 2007. Archival economic data and reports of key community leaders indicated economic deterioration, on average, with respect to unemployment, home mortgage performance, poverty, income, and perceived community economic problems. This deterioration followed a period of economic stability from 2004 to 2007, in which indicators generally changed little from year to year.

Importantly, analyses did not reveal systematic differences in the degree of economic downturn in CTC versus control communities among the broad set of indicators surveyed and analyses conducted. We did not find evidence of threats to the internal validity of the CYDS research design in any objective economic indicators, or in key leader perceptions of community economic problems. Analyses indicated the economic downturn affected CTC and control communities equally on average.

Given the magnitude of the downturn nationally, it is not surprising that effects were seen in CYDS communities. Effects on sample communities were not equal, however. They ranged from mild (e.g., unemployment up 0.6%/year, foreclosures up 0.14%/year, poverty at ages 5 to 17 up 0.1%/year) to more severe (e.g., unemployment up 2.7%/year, foreclosures up 1.3%/year, poverty at ages 5 to 17 up 1.9%/year). Key leader reports documented concern about economic problems, yet they showed variability across communities, as with the objective indicators. This pattern of overall economic concern in CYDS communities in the post-downturn period, with variability as to degree, is consistent with national patterns (Bureau of Labor Statistics, 2011b).

Reliability and Criterion Validity of Community Economic Problems Scale

Analyses supported the reliability of the Community Economic Problems scale as a survey of community economic conditions. They also documented criterion validity in relation to several objective economic indicators. The consistency between key leader perceptions of community economic problems and unemployment (both the 2009 level and change since the downturn started), was not surprising, given its prominence in public reports. Key leader reports were also consistent with the 2009 level and post-downturn annual change in 90+-day home loan delinquency rates. Loan performance problems have been frequently cited in the media and foreclosures can be a “visible” community problem, but this variable may also be a proxy for community-specific conditions and concerns not picked up in the county-level unemployment variable. Community Economic Problems scores were also significantly related to post-downturn annual change in county poverty in children ages 5 to 17, but not in the 2009 level, suggesting that key leaders were aware of worsening child poverty conditions, though not absolute magnitudes. Objective indicators together accounted for 75% to 80% of community-level variability in key informant reported Community Economic Problems, further supporting this scale as a valid measure of community economic conditions.

Some individual-level covariates also explained variation in key leader reports of Community Economic Problems. Women perceived significantly more community economic problems than men, consistent with research showing greater empathy and community attachment on the part of women (Riggs et al., 2002). Those employed in the human services sector also perceived more community economic problems, which could reflect greater exposure in their work to individuals and families affected by the downturn. Business and civic sector employees reported significantly lower community economic problems, a surprising finding. Results extend gender- and sectorial-based differences in perceptions of community prevention activity found in Riggs et al. (2002) to perceived community economic conditions.

The Economic Downturn and Coalition Implementation of Science-Based Prevention

Regression analyses examining coalition implementation of science-based prevention in 2009, well into the economic downturn period, revealed important differences between CTC and control coalitions. First, CTC coalitions retained higher levels of science-based prevention implementation in general, scoring 12.8 of 15 possible points on average compared to only 7.0 in control coalitions in 2009. However, deteriorating economic conditions challenged the ability of some CTC coalitions to maintain high implementation levels. For example, the four coalitions faced with unemployment rates of 10% or more lost ground in science-based prevention implementation from 2007 to 2009. Post hoc analysis of benchmarks completed before but not after the downturn indicated that activities requiring human and perhaps financial resources, such as developing an explicit written action plan or evaluating the effects of prevention programs using survey data, suffered as economic conditions worsened. In control coalitions, science-based prevention implementation was not related to economic indicators, but implementation levels were generally low for these coalitions both before and after the downturn. Economic conditions may not have had a measurable impact on these coalitions because not much science-based prevention work was going on even when economic conditions were more favorable.

Results further suggested that science-based prevention implementation by many CTC coalitions was resilient in the face of deteriorating economic conditions, as 7 of 11 coalitions maintained 2007 implementation levels through 2009. Although our analyses do not focus on the mechanisms responsible for sustained high implementation levels, CTC’s strong theoretical framework, focused prevention approach, reliance on tested and effective prevention programs, and presence of key community leaders on the CTC board may have helped these CTC coalitions continue science-based prevention activities even as economic conditions worsened.

Limitations

Conclusions are limited by the ability of available data to adequately portray an economic event that continues to affect CYDS communities. Archival indicators were available for different time periods (2004 to 2010, 2005 to 2011, and 2004 to 2009), based on the reporting schedules of both private and governmental organizations. Some indicators were available only at the county level, and some were available for a relatively short post-downturn period. Our sample of communities was small, limiting the power of some statistical tests. Nonetheless, results support study hypotheses regarding downturn effects, criterion validity in the Community Economic Problems scale, and downturn implications for science-based prevention.

Coalition implementation of science-based prevention provides one lens on prevention efforts in CTC and control communities. It is possible that increasing economic hardship affected other aspects of prevention in CTC and control communities, including coalition functioning, available budgets for prevention activity, and various barriers to implementing prevention activity. Further analyses are needed to portray the many ways in which the economic downturn may have influenced prevention activities in CTC and control communities.

Implications for future analyses

The internal validity of the CYDS does not appear compromised by the economic downturn that began in December 2007. However, CYDS communities experienced a range of macroeconomic effects that have bearing on the community and family contexts in which youth develop. In this study, economic conditions were linked to science-based prevention implementation by CTC coalitions, with potential consequences for youth risk and behavioral outcomes through diminished prevention activity. Economic indicators also portrayed sharply increasing hardship for families, in the form of rising unemployment, poverty, loan payment delinquencies and foreclosure rates since the downturn began, which may have additional consequences for youth. Future analyses will consider the role of changing economic conditions in communities on additional dimensions of coalition functioning and risk, protection, and youth outcomes.