Knowledge, Attitudes, and Commitment Concerning Evidence-Based Prevention Programs: Differences between Family and Consumer Sciences and 4-H Youth Development Educators

Abstract

In this article, we describe the results of a study designed to assess knowledge, perceptions, and attitudes towards evidence-based and other prevention programs among county Extension educators. We examined differences across educators from the Family and Consumer Sciences (FCS) and 4-H Youth Development program areas. Analyses based on a multi-state sample of educators revealed significant differences across program areas such that, compared to their 4-H counterparts, FCS educators were more knowledgeable of evidence-based programs and had more favorable perceptions and attitudes towards evidence-based and other pre-packaged prevention programs. These findings suggest that Cooperative Extension administrators should work to encourage the use of evidence-based and other prevention programs, particularly within the 4-H program area.

Over the past several decades, increasing numbers of preventive interventions targeting youth and families have been developed and packaged for distribution. Many, but not all, of these interventions or programs are considered “evidence-based” in that they are built on a sound theoretical base, and their effectiveness has been assessed using rigorous evaluation methodologies (Catalano, Berglund, Ryan, Lonczak, & Hawkins, 2004; Flay et al., 2005; National Research Council and Institute of Medicine, 2009; Olson, 2010; Small, Cooney, & O’Connor, 2009). While the relative effectiveness of individual programs varies, we can have general confidence that implementing evidence-based interventions or programs will lead to stated outcomes when they are implemented with fidelity and with similar populations. This reasoning is clearly illustrated in that funding to implement prevention programs from most government sources is contingent on selecting programs from government supported “best practice” lists (Hill & Parker, 2005; Olson, 2010).

In light of trends toward the use of pre-packaged evidence-based and other prevention programs, various scholars have called on Cooperative Extension administrators, faculty, and staff, especially within the 4-H Youth Development and Family and Consumer Sciences (FCS) areas, to follow suit. These scholars argue that adopting evidence-based practices (EBPs) will help FCS and 4-H educators strengthen the effectiveness of their outreach efforts, be more accountable to their funders, and demonstrate positive outcomes to stakeholders (Dunifon, Duttweiler, Pillemer, Tobias, & Trochim, 2004; Fetsch, MacPhee, & Boyer, 2012; Hill & Parker, 2005; Molgaard, 1997; Spoth & Greenberg, 2005; 2011). Cooperative Extension’s involvement in delivering EBPs can also have a positive effect on its reputation as an important resource for children, youth and families in the community. (Mincemoyer, et al., 2008). Furthermore, implementing an existing, packaged prevention program, whether evidence-based or not, may be more efficient than developing a new program (Olson, 2010).

In light of these trends, the purpose of the research described in this paper was to examine differences and similarities across county-level 4-H and FCS Extension educators in attitudes towards and knowledge of two separate, but related types of interventions. Specifically, our research focused on attitudes and knowledge related to both evidence-based practices at a general level, and also pre-packaged prevention programs that may or may not be evidence-based.

Use of Evidence-Based and/or Prevention Programs within Cooperative Extension

While research on evidence-based and other prevention programming within Extension is limited, several studies have examined the extent to which Extension professionals are open to incorporating such efforts into their work. For instance, Hill and Parker (2005) surveyed a sample of FCS and 4-H faculty and educators to determine their attitudes toward the use of evidence-based and other prevention programs. The results revealed a majority of study participants perceived a need for prevention programming to address issues such as: substance abuse, school dropout, and suicide. Moreover, between one-half and two-thirds of participants believed that evidence-based prevention programming could be effective in addressing negative outcomes. However, nearly as many participants expected that traditional Extension programs would be equally effective. Participants perceived numerous barriers to implementing evidence-based and/or prevention programs, which included: limited resources, collaboration difficulties, fear of losing traditional Extension programs, and a perception that prevention programs focus on deficits of program participants (Hill & Parker, 2005).

A similar study with a sample from Pennsylvania and Iowa compared attitudes and knowledge of prevention programs among FCS and 4-H educators and community-based human service providers (Perkins, Mincemoyer, & Lillehoj, 2006). In this study, results indicated both Extension educators and human service providers felt connected to their communities, and both groups had similar experiences with collaborations. However, while Extension educators reported more knowledge of the risk and resiliency model, they reported less awareness of actual community-based prevention programs. These findings may be the result of educators’ biases toward traditional Extension programs, as indicated in Hill’s and Parker’s (2005) findings. Similarly, they may reflect the common 4-H focus on positive youth development, which could be viewed as not encompassing a risk-reduction approach employed in some prevention programs. Whatever the causes, these results suggest several factors that may discourage Extension professionals from implementing pre-packaged prevention programs, whether evidence-based or not.

Differences in Evidence-Based Approaches Across Program Areas

Although most calls for increased use of evidence-based programs have been directed at the FCS and 4-H program areas, there is some evidence that receptiveness towards such approaches differs between them. Hamilton, Chen, Pillemer and Meador (2013) examined use of research by Extension educators from different program areas in New York State. Their findings indicate that compared to educators based in the agriculture and family and consumer sciences program areas, those from 4-H reported more difficulty accessing relevant research, less involvement in conducting research, and less familiarity with evidence-based programs.

While the reasons for differences in usage of research-based information among Extension educators is not entirely known, a perception exists that 4-H educators are more likely to administer traditional club programming than to implement packaged evidence-based youth or family focused programs (Hill & Parker, 2005; Scholl & Paster, 2011). While 4-H club activities are usually based on solid educational theory and can be tailored to the unique needs of stakeholders, these youth development programs are typically not subject to rigorous evaluation.

While a study by Bikos and colleagues (2011) indicated that 4-H faculty, educators, and volunteers expressed a general interest in using research to inform program development, they reported numerous barriers that limited 4-H personnel capacity to act on such interests. For example, study participants reported partial knowledge on where to access relevant research, limited time to incorporate research-based knowledge into their work, and limited funding. Furthermore, some saw value in continuing their traditional extension programming as they felt it could be more responsive and tailored to the unique needs of their stakeholders. Pre-packaged prevention programs, whether evidence-based or not, may be viewed as less flexible and thus less in line with stakeholder needs.

Research on the use of evidence-based programs among FCS educators is limited. However, in recent years, several indicators suggest that FCS educators have been relatively successful in creating and implementing evidence-based strategies. For example, the common Extension practice of using age-paced newsletters for parents has been found to effectively promote positive parenting in several outcome evaluations (Bogenschneider & Stone, 1997; Dworkin, Gonzalez, Gengler, & Olson, 2011; Riley, Meinhardt, Nelson, Salisbury & Winnett, 1991). In addition, FCS educators have developed and evaluated evidence-based marriage curricula (Goddard & Olsen, 2004), and recent efforts of FCS faculty have established evidence-based programs to support military families (Carroll, Robinson, Orthner, Matthews, & Smith-Rotabi, 2008).

Purpose of Current Study

Prior research provides some preliminary evidence that differences exist between FCS and 4-H educators in their comfort with and/or use of evidence-based and/or prevention programs. As outlined above, FCS educators may have had more experience developing and implementing evidence-based programming than 4-H educators. Furthermore, 4-H educators may experience specific barriers that make them less likely than their FCS colleagues to adopt evidence-based and/or prevention programs. To date, however, there have been few studies that directly compare FCS and 4-H educators.

This study builds on a report of Extension system readiness for EBP implementation and dissemination (Ralston et al., 2011; Spoth, Ralston et al., 2013) that was part of a series of projects aimed at developing strategies for scaling-up the PROSPER partnership-based delivery system (Spoth & Greenberg, 2011). The purpose of this preliminary research was to assess readiness of all state Extension systems for potential implementation of the PROSPER model, which encompasses a system that supports the implementation of evidence-based prevention programs within local community settings. The survey addressed five readiness areas, including understanding the degree to which state Extension system staff in the youth and family program areas were ready for, knowledgeable about, engaged in, and committed to implementing evidence-based and other prevention programs.

In this study, we analyzed data from a national sample of FCS and 4-H educators. The primary purpose of the study was to discover the unique knowledge, attitudes, and perceptions of barriers to implementing EBPs and other prevention programs among county-level Extension educators within each program area. Our primary hypothesis was that compared to their 4-H counterparts, FCS educators would be more likely to support using evidence-based and/or prevention programs because FCS educators would have more knowledge regarding these practices, would report fewer barriers to employing them, and would be better prepared to collaborate with community agencies, schools and groups on implementing EBPs. This hypothesis was based in part on the formal training that FCS educators receive and use in their practice.

Method

Participants

During fall, 2009, Cooperative Extension administrators, faculty, coordinators, specialists, educators, and assistants involved in the youth and family programming areas across the United States were invited to participate in a web-based survey to assess their attitudes and knowledge regarding prevention, evidence-based programs, and collaboration and partnership activities. Out of a possible 4,181 participants, 958 (23%) completed the web surveys, a rate consistent with similar web-based surveys (Hamilton, 2009).

In the current study, our analyses were limited to the 225 educators in the sample whose title identified them as a FCS or 4-H/Youth Development Educator. These educators (97 Family; 128 4-H/Youth) were distributed among 30 states for an average of eight participants in each state (Min = 1; Max = 34). On average, participants had been in their current position for 9.5 years (SD = 8.6), and their tenure with their state’s Extension system averaged 11.0 years (SD = 9.4). Ninety-six percent of the sample had full-time positions. The educational status of the sample was high: 30.3% had a minimum of a college degree, 63.8% had a master’s degree, and 3.6% had a terminal degree.

Procedures

The potential respondent pool was created through compiling a full list of FCS and 4-H personnel from each state’s (and Washington DC’s) web-based employee rosters. Once rosters were collected (N=5,072), they were examined for completeness and size. To balance the sample, employee names were randomly selected from state systems that had over 100 names on their roster so that there was a maximum of 100 potential participants from each state. This process resulted in the sampling frame of 4,181 participants.

Once the sampling frame was finalized, participants were recruited through a series of letters to State Extension Directors and regional administrators of Extension within each state. A $2000 incentive was offered to three state Extension Systems that each had the highest response rate in one of three different size categories: relatively small (< 100 employees), medium (between 100- 250 employees), and large (approximately greater than 250 employees). In addition, one $500 award for professional development was offered to one randomly selected respondent within each state’s Extension System. State Extension Directors were asked to send a notification letter to their staff about the survey, before the survey invite emails were sent.

Participants were officially invited to take the survey via an email that came directly from data collection staff. This email included a consent letter, a survey link, and an individual access code. Surveys and reminders were sent to all potential participants over a 12-week period in fall 2009.

Measures

Eight measures that assessed participant attitudes toward and knowledge of evidence-based programming, prevention programming (which may or may not be evidence-based), and collaboration and partnership activities were included in this paper. Information on each measure is included in Table 1.

Table 1.

Self-Report Measures of Attitudes and Knowledge Towards Prevention Programming

Scale name	Description	# of items	Sample item	Range	Reliability
Focus on prevention	Perceptions of importance of prevention	5	How important are each of the following areas of prevention for the communities in your state? (Delinquency, obesity, etc.)	1 = not important 5 = very important	α = .85
Support for prevention	Perceptions of Extension’s support for prevention programming	3	Your state Extension system is committed to planning and conducting prevention programming.	1 = strongly disagree 5 = strongly agree	α = .75
Obstacles to prevention programming	Perceptions of the barriers to implementing prevention programs	3	There are no financial resources to support prevention programming.	1 = strongly disagree 5 = strongly agree	α = .65
Individual knowledge of Evidence-based programs (EBPs)	Knowledge or attitudes of evidence-based programs	2	I know where to go to find information on evidence- based programs.	1 = strongly disagree 5 = strongly agree	r = .69
Requirements and appeal of EBPs	Importance of factors when deciding on new EBPs	3	It was being successfully implemented by colleagues.	1 = not important 5 = very important	α = .57
Extension commitment to EBPs	Perceptions of Extension’s commitment to using EBPs	3	Most of the children, youth and families programs offered by Extension use evidence-based models.	1 = strongly disagree 5 = strongly agree	α = .58
Resources and support for collaboration and partnerships	Perceptions of support available to engage in collaborative efforts	2	Extension program staff is provided with the monetary support necessary to engage in collaborative efforts that serve children, youth, and families.	1 = strongly disagree 5 = strongly agree	r = .48
Collaboration and partnership activity	Perceptions of collaborative activities of Extension program staff	5	Extension program staff has successfully collaborated on prevention programming with public schools.	1 = strongly disagree 5 = strongly agree	α = .77

Analysis Plan

The data had a 2-level hierarchical structure; individuals (Level 1) were nested within states (Level 2). Moreover, the number of respondents within each state varied creating an unbalanced sample. As a result, multilevel mixed models utilizing proc mixed in SAS Version 9.2 were used to conduct analyses. Preliminary analyses were conducted to test for significant between (Level 2) variance (i.e., significant agreement among responses from different individuals within the same state) in order to decide if a random Level 2 intercept should be estimated in the analyses. The variance due to state (i.e., Level 2 between variance) for all scales ranged from 0.02 to 0.09 with one exception - the between variance of the requirements and appeal of EBPs scale was zero. As a result, all models included estimating a random Level 2 intercept, with the exception of the requirements and appeal of EBPs scale. Estimating a Level 2 random intercept accounts for the shared variance among participants within the same state; therefore, is a more appropriate and conservative analysis (Singer & Willett, 2003). Final statistical models tested for the fixed effect of educator type (4-H/Youth vs. Family) while estimating a random intercept for state. Models were assessed with the Restricted Maximum Likelihood (Singer & Willett, 2003) utilizing the Satterthwaite degrees of freedom method.

Results

Descriptive statistics and participation numbers are in Table 2. Family and youth educators perceive a relatively strong level of the importance of prevention and support for prevention programs for the communities in their state. Endorsements from their colleagues and supervisors and strong outcomes were also important considerations when selecting a new EBP. Individuals seemed to perceive a moderate level of obstacles to implementing prevention programs and felt they had a moderate level of knowledge of EBP. Levels of collaboration and partnership activities and Extension’s commitment to EBP were moderate to high. However, participants seemed to perceive a lack of available resources for collaboration and partnerships.

Table 2.

Descriptive Statistics of Attitude and Knowledge Scales

Scale	N	Mean	SD	Min	Max
Focus on Prevention	225	4.22	0.69	1.40	5.00
Support of Prevention	225	3.88	0.69	2.33	5.00
Obstacles to Prevention Programming	225	2.78	0.74	1.00	5.00
Knowledge of EBPs	225	3.49	0.98	1.00	5.00
Requirements and Appeal of EBPs	222	3.89	0.70	1.00	5.00
Extension Commitment to EBPs	225	3.63	0.69	1.67	5.00
Resources and Support for Collaboration and Partnerships	225	2.58	0.86	1.00	5.00
Collaboration and Partnership Activities	224	3.54	0.62	1.80	5.00

Results of the multi-level mixed models, including the least squares means, the 95% Upper Confidence Level (UCL) and Lower Confidence Level (LCL), and significance values are presented in Table 3. Significant differences in the expected direction were found in five out of the eight scales. Specifically, FCS educators compared to 4-H educators had higher levels on focus on prevention, perceived agency support of prevention, knowledge of EBPs, Extension’s commitment to EBPs, and collaboration and partnership activities. One additional scale, resources and support for collaboration and partnerships, was in the expected direction and approached significance levels. Means of the two groups were not significantly different on the last two scales, obstacles to prevention programming and requirements and appeal of EBPs.

Table 3.

Results From Multi-Level Mixed Models Assessing Differences in Attitudes and Knowledge of Family Versus Youth Educators

Scale	Least Squares Mean	LCL	UCL	Significance
Focus on Prevention				p = .02
Family	4.33	4.15	4.52
Youth	4.11	3.95	3.95
Support of Prevention				p = .0002
Family	4.07	3.91	4.23
Youth	3.71	3.57	3.85
Obstacles to Prevention Programming				p = .27
Family	2.84	2.65	3.02
Youth	2.71	2.57	2.88
Knowledge of EBPs				p < .0001
Family	3.80	3.56	4.04
Youth	3.25	3.04	3.45
Requirements and Appeal of EBPs				p = .66
Family	3.86	3.72	4.00
Youth	3.90	3.78	4.03
Extension Commitment to EBPs				p = .002
Family	3.78	3.62	3.94
Youth	3.48	3.34	3.61
Resources and Support for Collaboration and Partnerships				p = .06
Family	2.71	2.52	2.90
Youth	2.49	2.33	2.65
Collaboration and Partnership Activities				p = .02
Family	3.64	3.50	3.78
Youth	3.43	3.31	3.55

Discussion

The results of this study complement those of previous investigations in that a majority of Extension educators reported that they value EBPs and prevention programs. Descriptive data in the form of mean scores reported in Table 2 suggest that educators generally support using evidence-based and/or prevention programs within their own communities. 82.8% felt that prevention of substance abuse among youth was important or very important, and 70.3% felt the same way about delinquent behaviors. However, only 55% of participants agreed or strongly agreed that their Extension leaders are committed to evidence-based programming.

Furthermore, the unique contribution of our findings is that we were able to use a national sample to directly compare the attitudes and knowledge about evidence-based and/or prevention programs between FCS educators to 4-H educators. The results of these comparisons were consistent with the New York state-wide findings of Hamilton and colleagues (2013) reviewed earlier. Specifically, we found that compared to 4-H educators, FCS educators were significantly more likely to focus on prevention, to support prevention efforts, to report knowledge of and commitment to evidence-based programs, and to engage in collaborative programming efforts. These findings support our hypothesis that FCS educators are more familiar with and more willing to implement evidence-based programs and/or prevention strategies.

The reservation of 4-H educators to implement evidence-based programs, and particularly pre-packaged prevention programs, may be related to their commitment to positive youth development (PYD) programming. Historically, tension has existed among professionals advocating a prevention approach to youth programming and those in favor of PYD approaches. In recent years, such tensions have diminished as youth development professionals have found common ground that stresses risk reduction as well as promotion of strengths and competencies (Catalano, Hawkins, Berglund, Pollard, & Arthur, 2002; Lerner et al., 2011; Perkins & Caldwell, 2005). However, remaining biases toward traditional prevention practices may, at least partially, explain our current results. Another potential explanation for this difference may have to do with the nature of Extension programming; 4-H programs often are a collection of distinct activities and events. Finally, as noted earlier, the education and training of FCS educators involves emphases on evidence-informed decision-making and programming.

We were surprised to find no differences in reports of barriers to using prevention programs. Given prior research (Bikos et al., 2011; Hill & Parker, 2005), we had expected to find more barriers reported by 4-H educators relative to their FCS counterparts. On the contrary, the findings indicate that experiences with and attitudes toward prevention programming may be more important predictors of willingness to use them than obstacles.

Our findings signal that FCS educators might be more ready and/or willing than their 4-H colleagues to integrate evidence-based programs and prevention programs into their programming dossiers. As noted earlier, this integration comes with benefits such as an increased chance of program effectiveness, accountability to funders, and simplicity of ready-made curricula. Nevertheless, limitations exist, depending on the specific program selected: the possibility of less flexibility in program implementation, less control over programming details, and less ability to tailor content. Also, costs may be associated with acquiring an established evidence-based or other prevention curriculum; although, such financial investments could be recovered by saving time ordinarily dedicated to program development. The evidence base of many of these programs also may result in more program effectiveness, which could offset costs by reducing the need for future programs.

Limitations

This study has several limitations. First, while the response rate to our survey was consistent with previous web-based surveys, less than 25% of eligible respondents participated. Thus, the sample may be biased in unknown ways concerning the perceptions of Extension educators. Similarly, the sample was drawn from only 30 states, and 13 states had 3 or fewer respondents. Perceptions and attitudes may have been different in states not represented in this study, and they may not be entirely representative of the perceptions and attitudes of those that did not respond to the survey. Finally, several of our constructs were assessed with two-item measures, and others had relatively low internal consistency. Therefore, these results should be considered preliminary, pending replication with stronger instruments. Nevertheless, the strengths of this study, in terms of reach and depth, outweigh the limitations.

Implications for Practice

Given the realities of the movement towards evidence-based practice and using pre-packaged prevention programs, increased demands for accountability by stakeholders, and shrinking numbers of Extension professionals who can be dedicated to program development, the use of established evidence-based and other prevention programs will likely increase in the coming years. Based on the results of this study, Extension administrators and stakeholders may be able to help facilitate this process within Cooperative Extension by supporting efforts to remove barriers and provide additional support and resources in order to create a culture that encourages collaborative efforts to use evidence-based and other pre-packaged prevention programs.

Although the differences were not large, our results indicate that 4-H educators seem less likely to be prepared to implement evidence-based and/or prevention programming as compared to their FCS counterparts. Given this finding, Extension program leaders and administrators may wish to identify ways to incorporate the best aspects of existing 4-H culture (e.g., parent involvement and project-based learning) into evidence-based and/or prevention programming. Then, Extension administrators can convey these ideas and values when communicating about evidence-based and/or prevention programs with 4-H educators. For example, previous authors have noted the benefits of 4-H educators’ flexibility, experience, and connections with local communities (Dunifon et al., 2004; Fetsch et al., 2012). Rather than eliminating these strengths when implementing a pre-packaged evidence-based and/or prevention program, we recommend that 4-H educators integrate such characteristics into the selected programming. While in some ways they might lose a degree of flexibility in their programming, they can and should engage community members in the selection, implementation, and evaluation of these strategies, and considerable flexibility exists regarding how these processes will work best in different communities. There are examples where 4-H educators are leading community-based implementations of such initiatives (see Spoth, 2007; Spoth, Greenberg, Bierman, & Redmond, 2004). Because of their skills and experiences, 4-H educators are well positioned to be successful at engaging community members in this process, which may improve resiliency of youth and families in their communities.

We may also see more acceptance of packaged evidence-based and other prevention programs if educators understand that many contemporary prevention programs incorporate strengths-based principles and are developed for all youth; few are strictly deficit-based targeting specific populations of youth. By limiting existing biases and removing some of the above-mentioned barriers, we may create an environment in which FCS and 4-H educators maintain many of the valued traditions of existing Extension programming, while maximizing the likelihood of success in addressing stakeholder needs.

Conclusions

As Cooperative Extension budgets continue to tighten, serving wider geographical areas and securing external funding are new standards for program administration and support. Thus, with funding becoming increasingly reliant on using evidence-based and other pre-packaged programming, implementing these programs is one way that Extension professionals can be successful and use limited resources wisely. Although some see drawbacks to using established evidence-based and other prevention programs, most professionals agree that these initiatives can be efficient and effective (Catalano et al., 2004). By helping to remove current barriers and encouraging educators from FCS and 4-H program areas to incorporate evidence-based strategies into their current efforts, we can help Extension effectively meet the needs of youth and families, while being able to document positive outcomes.