Impact of a National Rural Youth Health and Safety Initiative: Results from a Randomized Controlled Trial

Abstract

Objectives. We conducted a comprehensive evaluation of a rural youth health and safety initiative implemented in 4000 National FFA (formerly Future Farmers of America) chapters across the United States.

Methods. Data were collected from high school students and their FFA advisers at 3 time intervals (preintervention, immediate postintervention, and 1 year postintervention) with a 3-group (standard, enhanced, and control), cluster-randomized, controlled trial design.

Results. Matched data from 3081 students and 81 advisers revealed no significant effect of this initiative on agricultural health and safety knowledge, safety attitudes, leadership, self-concept, and self-reported injuries of project participants. Data from 30 public health nurses following the intervention confirmed the program’s failure to develop sustainable community partnerships.

Conclusions. This nationally coordinated initiative was funded with more than $1 million donated by agribusinesses. Program implementation was inconsistent, and desired outcomes were not achieved. Future efforts should better guide effective use of private sector resources aimed at reducing agricultural disease and injury among rural youths.

The National FFA (formerly known as Future Farmers of America) is the largest youth-serving organization in the United States, with more than 7000 chapters across all 50 states and nearly one-half million youth members aged 12–21 years.1 FFA is a school-based agricultural education program in public schools that is chartered by Congress through the US Department of Education. The FFA’s mission is “Premier leadership, personal growth, and career success through agricultural education.”2 In 1997 the National FFA Foundation announced it raised more than $1 million from major agribusinesses to implement a new initiative, Partners for a Safer Community (henceforth referred to as the Partners Program), aimed at promoting agricultural health and safety for adolescents through education, community partnerships, and youth leadership. Many FFA chapters had historically conducted agricultural safety programs but not via a nationally coordinated system.

FFA’s desire to undertake this initiative was driven by its leaders, who noted that the farm press and rural communities were increasingly more aware of traumatic childhood agricultural injuries and hazardous work conditions, starting with a lengthy, 1989 feature story, “We Kill Too Many Farm Kids,” published in a national farm magazine.3 By 1996, the National Action Plan for Childhood Agricultural Injury Prevention was developed and adopted by the US Congress as a blueprint for a public health response to this problem.4 FFA adopted this cause, intending to promote agricultural health and safety with financial support from major tractor manufacturers, chemical producers, and seed dealers. The marketing value of this initiative was obvious to corporate donors that would be acknowledged in promotional materials and educational resources. Yet there were questions regarding its effect on the knowledge, attitudes, and practices of FFA members. The question was raised: is the Partners Program an effective use of a $1 million private sector investment aimed toward adolescent agricultural disease and injury prevention?

The stated goals of the FFA’s Partners Program were to promote students’ agricultural health and safety knowledge and attitudes, increase students’ leadership skills, and develop sustainable community partnerships that promote health and safety. The Partners Program involved national and state-level training of FFA advisers who used a modified version of the Teaching Agricultural Safety to Kids curriculum, which trains youths to become junior teachers, mentors, and role models for young children.5 All participating FFA chapters were given a multivolume educational kit of videotapes, CD-ROMS, printed resources, and instructions (Table 1 ▶). Ongoing communications, quarterly newsletters, and encouragement to already trained FFA advisers were provided by the Partners Program director. Across the United States, about 4000 FFA chapters were enrolled in this highly marketed initiative. The National FFA hired a program evaluator to monitor activities and assess state trainers’ perspectives during the start-up phase. No mechanisms were employed to ensure that the program was implemented according to the detailed protocol.

TABLE 1—

Components of Partners Program, by Study Group

Activity	Standard	Enhanced	Control
Marketing, promotion of Partners Program via newsletter, conferences, videotape	X	X	X
Formal interactive training (4 hours) of FFA adviser and student team members with travel expenses reimbursed	X	X
Printed instruction guides
Core Team Training Guide describing 7-step program model, purpose, methods, responsibilities (35 pages)	X	X
Student Team Training Guide including modules on agricultural safety and health facts, community leadership, teaching children about safety and health, and assessing program effectiveness (74 pages)	X	X
Resource Guide with Resource Collection (“treasure chest”; 52 pages)	X	X
Encouragement to implement program from local agribusinesses	X	X
“Treasure chest” of educational CD-ROM, videotapes, booklets, and so forth for teaching agricultural health and safety content to adolescents (e.g., safe handling of farm equipment, livestock, and firearms; hearing protection) via leadership methods (e.g., teaching 4-H groups, exhibits at health fairs, competing for safety awards)	X	X
Opportunity to discuss activities at national conventions	X	X
Opportunity to highlight local health and safety events in national newsletter	X	X
“Refresher” on-site in-person training session on implementation of Partners Program	X
Biweekly telephone contact with Partners Program facilitator		X
Quarterly mailings of topic-specific guides regarding core content and program options and resources (e.g., wall posters on all-terrain vehicle safety)		X
Free supplies of personal protective equipment (e.g., sunscreen, earplugs) to accompany lesson plans and projects on specific topics		X
Personal contact with local public health office, with $300 incentive for community nurse involvement in program activities with FFA adviser and students		X

Note. FFA = Future Farmers of America. “Standard” indicates the program provided to 4000 FFA chapters across the United States. “Enhanced” indicates the experimental group that received 4 interventions beyond the standard program. “Control” chapters did not engage in any program interventions.

Our research was aimed at evaluating the effect of this Partners Program. Through a cooperative arrangement with FFA, investigators were provided access to conduct this evaluation, nested within the nationwide implementation of the Partners Program. The FFA’s implementation timeframe impeded the ideal strategy of confirming the program’s efficacy before its full implementation and evaluation.6–10

METHODS

This study used a cluster-randomized, controlled trial design with a sample of rural-based FFA chapters in 10 states (California, Iowa, Kansas, Kentucky, Minnesota, Ohio, Oklahoma, Tennessee, Texas, and Wisconsin). Sample size was designed to provide at least 80% statistical power to detect group differences as small as 10% on a relative basis after adjustment for the correlation within chapters. The FFA chapters were randomly assigned to 1 of 3 groups: the standard group, which were treated the same as nearly 4000 chapters receiving the program nationally; the enhanced group; or the control group. Our study design used local nurses to build relevant community partnerships, following the success of the National Institute for Occupational Safety and Health’s community partners’ initiative with agricultural nurses.11 For the enhanced intervention chapters, project staff facilitated contacts between the FFA chapter and the local public health department, offering a $300 financial gratuity to serve as a local resource. Factors (e.g., knowledge, safety consciousness, participation in safety campaigns) that demonstrated weak or nonsignificant changes following a standard program were expected to demonstrate statistically significant changes following an enhanced, more intense program. Table 1 ▶ depicts interventions for each group.

Study Population

Our intent was to sample FFA chapters that were representative of sites where agricultural health and safety issues were relevant. States with the highest number of farms were identified from the US Census of Agriculture.12 Inclusion criteria included FFA chapters located in cities of < 50 000 people that had more than 10 female and male chapter members in the ninth and tenth grades. Within each FFA chapter, the adviser enrolled agricultural students in the ninth, tenth, and eleventh grades, allowing for follow-up at the conclusion of their subsequent academic year. Given turnover among FFA advisers, along with changing patterns of courses as class schedules changed by semester and year, modest dropout rates were anticipated (students were not necessarily enrolled in subsequent classes when data were collected).

Instruments

Four data collection instruments were developed for this study: a student instrument for measuring program effect on attitudinal, knowledge, and behavioral domains; an FFA adviser instrument for measuring program implementation (process) and effect; an FFA chapter activity report form for quantifying program activities; and a community nurse instrument for assessing program integration. The student instrument required the most extensive testing and modification; the adviser form was developed with many of the same subscales. All scales were pilot tested with factor analyses and measures of internal consistency, allowing for elimination of items that poorly reflected constructs of interest.13

The student and adviser assessment instrument was administered in its entirety 3 times: at the beginning of academic year 1 (preintervention), at the end of academic year 1 (postintervention), and at the end of academic year 2 (postintervention 2). Unless stated otherwise, respondents rated scale items on a 5-point scale ranging from strongly disagree (1) to strongly agree (5). Self-esteem was assessed with items from the Rosenberg scale (Cronbach α= .83).14 Leadership self-concept was assessed by 9 items representing a person’s perceived leadership level (Cronbach α= .88). Safety consciousness was assessed by 7 items tapping general attitudes toward acting safely, such as, “I always take extra time to do things safely” (α= .83). A risk-taking scale was included with 5 items (e.g., “I’d rather take risks than be overly cautious”) with a Cronbach α of .73. A safety knowledge scale was based on self-reported learning across several areas relevant to agricultural safety (α= .82). Safety campaign participation had 7 items assessing participation in various activities that promote safety (α= .82). Self-reported injuries were assessed, with 2 items measuring the extent to which the participant experienced injuries within the last 3 months (α = .71). Results from confirmatory factor analyses demonstrated that the above concepts represented construct valid latent variables.13 Demographic information was also gathered.

The adviser instrument comprised the same subscales as used for students, minus the safety campaign scale. This instrument assessed advisers’ guidance in helping youths get involved in agricultural safety and health activities. For the final collection of data, advisers in the standard and enhanced groups were asked about their perspectives on the implementation and effect of the Partners Program.

To assess fidelity to program implementation guidelines, a 1-page “Relevant Chapter Activities” query was developed, addressing implementation recommendations. The form was updated monthly and submitted to FFA chapters in the standard and enhanced groups, with a request to complete and return it within 2 weeks. Responses diminished over time, despite multiple telephone reminders. After 5 months, evaluation data regarding implementation of the Partners Program were no longer collected because of inadequate response rates.

A 12-item telephone interview tool was used with community nurses working with the enhanced intervention FFA chapters. The purpose was to assess the extent to which new partnerships between FFA chapters and community health leaders were addressing local agricultural health and safety concerns (a stated goal of the Partners Program).

Matched Data

To optimize the likelihood of honest responses from students, the anonymity of respondents was protected. A self-generated, anonymous identification code was used to allow investigators to match responses from individuals at 3 points in time.15 The self-generated identification code, located in the final section of the instrument, asked respondents for the following information: first 2 letters of US state of birth, first letter of month of birth, first 2 letters of mother’s first name, and number of older brothers. A computer algorithm was developed to match the identification codes of study participants at times 1, 2, and 3. Data from the subset of matched cases formed the basis for evaluating the effect of the Partners Program.

Data Collection

Data were collected through a detailed system with written protocols for FFA chapter advisers. Investigators sent the advisers an appropriate number of student survey forms and 1 adviser form, with preaddressed return labels for boxed materials. Telephone and fax contacts were made if completed forms were not returned within 1 month.

Data entry was conducted by trained staff, who used a glossary of problem resolutions to deal consistently with aberrant responses. Forms that had < 50% responses complete or that were not handled completely seriously were discarded ( < 1% of the surveys processed). Quality assurance for all data entry was conducted for each of the 3 time periods with > 99.7% accuracy.

For measuring community partnership activity, trained telephone interviewers contacted the designated public health nurse 6 months after intervention interactions between the nurses and FFA chapters were likely to have occurred. Following the structured interview, nurses provided general comments regarding the project.

In addition to methods described here, the National FFA hired an evaluator to monitor the pilot testing and nationwide implementation of the Partners Program. Reports from this evaluator provided additional documentation to guide FFA’s decisions regarding program maintenance.

Statistical Methods

To assess the effect of the Partners Program, we measured specific constructs, using the same measurement tool at 3 points in time. To demonstrate support for the initiatives, we expected a statistically significant difference between the standard and enhanced treatment groups relative to respondents in the control group. Further, we expected the differences between the enhanced and the control group to be greater than between the standard and the control group, given the increased resources devoted to the enhanced manipulations.

To statistically examine the efficacy of experimental conditions to the control condition, each dependent variable was examined with analysis of variance (ANOVA) and analysis of covariance (ANCOVA); the latter also was used to control for demographic variables (i.e., gender, age, working status, rural residences, and farm residence) and pretest scale scores. Both analyses yielded the same conclusions in this study concerning comparisons across standard, enhanced, and control conditions. In these analyses, each dependent variable was represented by its scale score (i.e., average composite across items, none of which deviated sufficiently from normal distributions to preclude the use of parametric analyses). The between-subjects factor was represented by 1 of 3 levels, corresponding to the standard, enhanced, and control experimental conditions, respectively. Analyses were conducted at both the individual and aggregate cluster level, which accounts for important group-level effects.15 Because both analyses yielded exactly the same conclusions, and because no significant cluster effects were manifest (P > .10), individual-level data are reported below, giving greater precision in mean point estimates across the youth participants.

RESULTS

Sample Characteristics

In September 1998, 123 FFA chapters were enrolled in this study. Of these, 111 provided completed student surveys for preintervention data collection. Postintervention surveys were collected from 98 FFA chapters, and 1 year later, 104 chapters submitted postintervention 2 data. By May 2000, matched data were secured from 92 FFA chapters at 3 time intervals. The control chapters were most likely to sustain participation for the 3-year duration, with the greatest dropout rate noted by enhanced intervention chapters.

Preintervention data were collected from 8068 students, and of these, 68% were male, 42% were farm residents, 83% were active FFA members, and 84% were aged 17 years or younger. Postintervention data were collected from 5925 students in 98 FFA chapters. Because of class enrollment and student attrition, only 3081 were matched with the same individual who provided preintervention data. This data set formed the basis for the assessment of the immediate effect of the Partners Program. One year later (postintervention 2), data were collected from 6604 students, with 1167 being matched to the same individual responding 2 previous times. Ultimately, the program’s effect could be assessed by these 1167 students, along with 60 advisers whose data were matched at 3 time intervals (Figure 1 ▶). There were no significant differences among the students or advisers in the standard, enhanced, or control groups relative to gender, residency on or off farm, or hours of work. We report the findings from the preintervention and first postintervention data (n = 3081 students) to reflect the immediate program effect on the students whose characteristics are depicted in Table 2 ▶. Results from the second postintervention data collection were the same as for the first postintervention findings reported in this article. ANCOVA results are presented in Table 2 ▶, given their convergence with ANOVA results for the between-subjects factor.

FIGURE 1—

Flow diagram of study participants.

Note. Study participants provided self-generated, anonymous identification codes on their survey forms, allowing data to be matched at different time intervals.

^aFFA advisers matched from preintervention to postintervention data.

^b Students matched from preintervention to postintervention data.

^c FFA advisers matched across preintervention, postintervention, and postintervention 2 data.

^d Students matched across preintervention, postintervention, and postintervention 2 data.

TABLE 2—

Description of 3081 Matched Students with Preintervention and Immediate Postintervention Data

	No. (%)
	Standard Intervention (n = 1059)		Enhanced Intervention (n = 683)		Control (n = 1339)		Total (n = 3081)
Gender
Male	714	67.4	906	67.7	906	67.7	2079	67.5
Female	345	32.6	433	32.3	433	32.3	1002	32.5
Age, y
≤14	230	21.7	283	21.1	283	21.1	652	21.2
15	273	25.8	368	27.5	368	27.5	843	27.4
16	288	27.2	339	25.3	339	25.3	805	26.1
≥17	268	25.3	348	26.0	348	26.0	776	25.2
Unknown	0	0.0	1	0.1	1	0.1	5	0.2
Where child lives
Farm or ranch	472	44.6	537	40.1	537	40.1	1294	42.0
Rural area	322	30.4	413	30.8	413	30.8	944	30.6
City or town	258	24.4	382	28.5	382	28.5	821	26.6
Unknown	7	0.7	7	0.5	7	0.5	22	0.7
Hurt in accident last 3 months
Yes	231	21.8	318	23.7	318	23.7	709	23.0
No	823	77.7	1017	76.0	1017	76.0	2359	76.6
Unknown	5	0.5	4	0.3	4	0.3	13	0.4
Currently member of FFA
Yes	912	86.1	1148	85.7	1148	85.7	2588	84.0
No	144	13.6	190	14.2	190	14.2	485	15.7
Unknown	3	0.3	1	0.1	1	0.1	8	0.3

Note. FFA = Future Farmers of America.

Youth Outcomes

Results following the intervention indicated no significant difference between the standard, enhanced, or control groups on factors under study (see Table 3 ▶). This was true for both ANOVA and ANCOVA analyses.

TABLE 3—

Comparison of Standard, Enhanced, and Control Conditions in Adolescent Self-Reported Knowledge, Cognitions, and Experiences Following Intervention

	Mean (SD)
	Standard	Enhanced	Control	P
Safety knowledge	2.7 (.75)	2.8 (.88)	2.7 (.82)	.43
Safety consciousness	2.9 (.72)	3.0 (.69)	3.0 (.77)	.47
Self-reported leadership	3.3 (.84)	3.3 (.79)	3.3 (.81)	.53
Dangerous risk taking	3.2 (.73)	3.2 (.72)	3.2 (.78)	.38
Self-esteem	3.8 (.73)	3.8 (.78)	3.9 (.78)	.69
Safety campaign participation	2.1 (.93)	2.1 (.91)	2.0 (.90)	.67
Injury experiences	1.8 (.36)	1.7 (.40)	1.7 (.38)	.44

Note. Variables represent average scores on scaled constructs. All P values are nonsignificant. Nonsignificant findings were also replicated with nonparametric analyses.

Safety knowledge.

A moderate level of safety knowledge was manifest across conditions (mean = 2.7; SD = 0.82). Descriptive results across all groups indicated that students reported the greatest amount of learning during agricultural class activities and that the persons most likely to motivate them to learn agricultural safety were their parents. Safety knowledge was also positively correlated with number of work hours (r = 0.27; P< .001). Extensive details of the descriptive and correlational results across all 3 conditions (cross-sectional and longitudinal) have recently been reported.13

Safety consciousness.

Comparative findings indicated that safety consciousness did not differ significantly across groups, according to both analysis of covariance and analysis of variance tests (overall mean = 3.0; SD = 0.73). However, safety consciousness was a relatively strong negative correlate of self-reported injury (r= −0.15; P< .001).

Self-reported leadership.

There was no significant difference between the groups on self-reported leadership (overall mean = 3.3; SD = 0.81), and across conditions, youths with strong leadership self concepts were more likely to be safety conscious (r = 0.28; P < .001).

Dangerous risk taking.

Results following the intervention revealed no significant difference between the 3 groups on participants’ dangerous risk-taking tendencies (overall mean = 3.2; SD = 0.74). Across all conditions, dangerous risk taking was correlated with gender (i.e., males were more dangerous risk takers; r= 0.12; P< .001) and self-reported injury (r= 0.18, P< .001).

Self-esteem.

Self-esteem did not differ significantly over time as a function of group assignment (overall mean = 3.8; SD = 0.76). Across conditions, self-esteem was significantly associated with safety consciousness (r= 0.22; P< .001).

Safety campaign participation.

Results indicated no significant difference in safety campaign participation between the groups. The overall participation level in safety activities was relatively low (overall mean = 2.1; SD = .91). Those students participating in safety campaign activities primarily did so through 3 mechanisms: helping teach young people, planning and working at a community event, or participating in a health promotion activity. Safety campaign participation was also strongly correlated with safety knowledge (r= 0.34; P< .001) and safety consciousness (r= 0.34; P< .001) across groups.

Injury experiences.

About one fourth of respondents reported they had been “injured in an accident” in the last 3 months. However, there were no significant differences between the 3 groups on self-reported injury (overall mean = 1.7; SD = 0.38). Injury was most strongly related to safety consciousness and dangerous risk taking across conditions both before and after the intervention.

In summary, all the analyses testing the standard, enhanced, or control groups (at both the individual and aggregate level both with and without additional control variables) revealed no substantive effect of the intervention conditions on participants’ self-reported knowledge, attitudes, activities, leadership, community participation, or injury.

FFA Adviser Outcomes

With each time period, the FFA adviser completed an assessment instrument similar to that of the students. Because of chapter attrition and employment changes, matched adviser responses were not secured from all participating chapters. Preintervention responses were secured from 108 FFA advisers, followed by 90 responses immediately after the intervention. Of these, 81 matches were confirmed. One year later, we received 85 FFA adviser responses, providing 60 FFA advisers with matched identification codes over 3 points in time.

Data revealed minor differences between the 3 treatment groups. The FFA advisers in the enhanced intervention group reported a statistically significant (P= .045) increase in the number of safety campaign activities over both the standard and control groups. However, this finding is not consistent with the students’ survey responses on the same factors (at both the individual and aggregate level). Another unexplainable finding was that following the intervention, FFA advisers in the control group reported significantly higher (P= .037) self-esteem over both the standard and enhanced intervention group advisers. No other significant differences were manifest on criterion variables.

When asked about perspectives on the National FFA’s Partners Program, most FFA advisers qualitatively indicated their support. However, time constraints limited their ability to implement the program as it was designed.

Community Nurse Outcomes

All but 2 of the original 35 public health agencies participated in the follow-up interviews, even if it was to report “nothing occurred.” Of the 30 nurses who provided data (1 nurse covered 3 different FFA chapters), 37% indicated they had no contact with the FFA chapter. Of those nurses who had contact, 57% spent 1–5 hours with student activities, and 5% spent from 6 to 10 hours with student activities, typically giving a class lecture on a health topic. The majority of nurses (67%) deemed the Partners Program to be valuable. Regarding the $300 incentive for the public health agency, 33% of nurse respondents indicated it was “not at all important.” For the most part, funds were put into the general operating account or used to purchase resources for the FFA chapter.

Implementation

Efforts of the National FFA’s program evaluator to assess the nationwide start-up phase of the Partners Program were severely hampered by low response rates to mail surveys and telephone interviews. Of survey respondents, less than half indicated they intended to implement the program. Among those starting the program, many reported adopting only selected aspects of the designed program (Sharon Dorfman, ScM, CHES; President of SPECTRA, Damond Beach, Fla; oral communication; February 2004). Our own efforts to collect monthly implementation data were terminated midway through the study because responses on a single-page questionnaire dropped to < 20%, despite repeated telephone reminders.

DISCUSSION

We examined the effect of the Partners Program on students’ agricultural health and safety knowledge, safety attitudes, leadership skills, community partnership, and self-reported injuries. Anecdotal reports indicated that the program was implemented inconsistently across the United States, although within our study sites, valid implementation data were not available. Results showed no positive effect of the intervention conditions across a series of important dependent variables: safety knowledge, safety consciousness, self-reported leadership, dangerous risk taking, self-esteem, safety campaign participation, and injury experiences. Even allowing for varying degrees of fidelity to implementation protocols, we expected to measure a positive effect in the enhanced chapters, which benefited from 8 months of encouragement and resources superior to standard chapters.

Our study also assessed the effect of the Partners Program on building sustainable community partnerships. Although the FFA advisers in the enhanced intervention group reported an increased involvement in community-based safety campaign activities, results from students and community nurses did not support the advisers’ data. Furthermore, no other differences were manifest.

Ideally, an education initiative would undergo testing for efficacy before being rolled out on a national scale. Our study findings reflect outcomes of a program that was untested for efficacy and implemented inconsistently across sites. We cannot state unequivocally that the program was a failure, but our results show that its effect as actually implemented was negligible.

Educational initiatives often promise greater results than those that can actually be achieved. Study results led to the conclusion that the National FFA’s Partners Program initiative, with more than $1 million in corporate support, failed to achieve the expected outcomes. Although local FFA chapter advisers appreciated receiving the Partners Program curriculum with its extensive resources and training, only a few FFA advisers had time and interest sufficient to implement the full program as prescribed. In addition, although community nurses expressed support for the Partners Program, the money and time invested in building a community partnership between FFA chapters and nurses yielded minimal results. Program sponsors, such as the agribusinesses supporting the Partners Program, would benefit from feedback on evaluation studies so that they can more effectively direct future contributions.

These results are consistent with other educational approaches for agricultural health and safety, and by indicating that educational programs alone are insufficient to prompt desired changes in agricultural health and safety, they contribute to the body of knowl-edge.17–20 Systematic reviews of health promotion and injury control strategies have revealed mixed results.21–24 A recent review of 35 successful prevention programs highlighted key characteristics of program content matched to target populations.25 The addition of some of these characteristics might have improved the outcomes of the Partners Program.

Low-cost evaluation methodologies that provide dependable results are limited, and the ideal randomized-controlled trial design can be challenging because of its inherent difficulties, including logistics, budgets, and politics.26 Although our evaluation study used a cluster-randomized–controlled trial design, we were evaluating an intervention that was designed by national program leaders and implemented at the local level beyond the control of its originators. In the absence of controlled study results, decisions regarding future initiatives should consider factors such as plausibility, adequacy, and maintenance.27–29

FFA continues to be a leading youth-serving organization in the United States, and efforts to determine the most effective methods for promoting agricultural health and safety to its young rural members should be made.

EPILOGUE

The executive director of the National FFA was provided preliminary results of this evaluation study approximately 2 years after the organization implemented the Partners Program. These results were discussed among organizational leaders and youth programming staff. Although there was disappointment that the data did not reveal a positive effect of the Partners Program, staff sincerely appreciated receiving evaluation results. The National FFA office subsequently discontinued training and technical support for the program’s maintenance. Further, the National FFA Foundation that had raised the $1 million to implement the program was no longer obliged to generate funds for its continuation.