The “ins” and “outs” of physical activity policy implementation: Inadequate capacity, inappropriate measures, and insufficient fund

Abstract

BACKGROUND

Despite broad public support and legislative activity, policies intended to promote physical activity in schools have not produced positive outcomes in levels of physical activity or student health. What explains the broad failure of Physical Activity Policies (PAPs)? Thus far, PAP research has used limited quantitative methods to assess PAP outcomes. New paradigms of qualitative policy implementation research can make important contributions to explaining the causes of policy failure and to the future design of more efficacious PAP legislation.

METHODS

This analysis is a case study of South Carolina’s 2005 Student Health and Fitness Act (SHFA). Written documents, investigators’ observation and experience, and an interview with a key stakeholder were analyzed to for themes based on theoretical frameworks from education implementation research including: (1) bottom-up and top-down perspectives; (2) conceptualizing policy as practice; and (3) the implementer as learner.

RESULTS

“Weak policy signals” in SHFA undermined the implementation of PAP in three problematic areas: inadequate capacity development for implementers; inappropriate measures of implementation; and insufficient funding.

CONCLUSIONS

These findings illustrate the contributions of qualitative research and establish the need for further qualitative research into PAP implementation processes. To ensure successful future physical activity policies, policymakers and stakeholders need to consider implementation, evaluation and funding from the beginning phases of policy development.

Many children do not meet recommended levels of physical activity¹. As a result, childhood obesity levels² are increasing and children’s health³ is declining. In South Carolina, almost one out of 3 adolescents are overweight or obese and half do not meet physical activity recommendations.⁴ Many biological, social, and environmental factors influence physical activity,⁵ including policy.⁶ Several states have passed school-related physical activity policies (PAPs) in an attempt to stem rising childhood obesity rates. Between 2003 and 2005, 717 childhood obesity prevention bills were introduced at the state level across the country,⁷ indicating a broad public recognition of the problems and support for taking action to combat obesity and physical inactivity.

Unfortunately, translating this support into positive measureable outcomes has proven to be difficult. Effective physical activity promotion practices are uncommon, and quantitative studies of outcomes reveal that physical activity rates have not improved.⁴ Why are PAPs failing? To understand policy failure, it is important to examine the policy process, particularly implementation. This theoretical analysis of South Carolina’s Student Health and Fitness Act of 2005 (SHFA) reveals how reconceptualizing implementation helps to understand PAP failure and may contribute to better policy design and health outcomes.

Previous PAP studies focused on policy adoption and outcomes.⁸ Other research has focused on individual state’s policies in Utah,⁹ Texas,¹⁰ North Carolina,¹¹ and Colorado.¹² Few have examined implementation more in-depth; implementation is putting policy into action or turning policy into practice.¹³ The few PAP studies that have studied implementation reveal that ‘real-world’ implementation practices differ widely in practice and deviate from official policy documents resulting in implementation gaps.¹⁴ PAP research has largely overlooked the implementation phase of the policy process.

While PAP implementation research is emergent, implementation research in education and political science has been ongoing since the 1960s.¹³ Preliminary implementation research analyzed the dichotomy of implementation: whether a policy was implemented or not. For example, in their pivotal study of a federal program in Oakland, Pressman and Wildavsky¹⁵ emphasized the almost inevitable failure of policies as a result of the complexity of implementation. Later implementation researchers have used diverse theoretical frameworks to understand policy failure, including: (1) bottom-up and top-down perspectives; (2) conceptualizing policy as practice; and (3) the implementer as learner. Whereas top-down analyses emphasize the role of policymakers in designing policy, bottom-up implementation analyses, termed “backward mapping,”¹⁶ emphasize the desired distal outcomes. Policy is a complex social process, and both perspectives are needed to understand implementation.¹⁷ Secondly, to understand the implementation process, policies cannot be separated from practices and thus must be investigated in tandem.^{14, 18} Finally, early implementation analysis viewed policy failure as the result of non-compliance by the implementers or deliberate attempts to sabotage a policy.¹³ In contrast, an implementer as learner approach understands failure as resulting from a non-deliberate lack of understanding and/or capacity to successfully implement the policies. These 3 theoretical approaches, including bottom-up perspectives, policy as practice and implementer as learner, provided a framework for this critical analysis of the implementation of a physical activity policy.

Summary of Student Health and Fitness Act of 2005

The Student Health and Fitness Act of 2005, seeks to “provide every elementary student with the equivalent of thirty minutes of physical activity daily,” or a total of 150 minutes, which must include at least 90 minutes of physical education per week.¹⁹ The legislation also stipulates additional quality measures of physical education and school health. Funding has primarily supported salaries for physical education specialists and school nurses.¹⁹

SHFA requires each elementary school to report physical education and physical activity minutes annually. The SC Department of Education collects and publishes information from district representatives about students’ opportunities for physical education and physical activity between kindergarten through 5th grade.²⁰ According to the legislation, schools who get an “unsatisfactory program effectiveness score,” as defined by the Department of Education, must receive additional professional development.

METHODS

In contrast to deductive empirical studies which test an a priori hypothesis, this qualitative study took an inductive approach, allowing themes to emerge from a research process guided by theoretical frameworks.²¹ This study, examined SHFA from unique theoretical perspectives developed in education implementation research.

Investigator experience and systematic data collection were integrated into a comprehensive case study analysis. Data sources included document analyses, investigators’ observations, and informal interviews. Primary analysis was of the Student Health and Fitness Act of 2005 document and supplemented by additional materials.^{22, 23} An expert historical account came from a physical education faculty member who was directly involved with the policy process and evaluation.

RESULTS

The annual state report on SHFA suggests a degree of failure in implementation; not all schools are meeting requirements. Even according to self-reported data, which reliability will be discussed further, only 52% of reporting schools comply with state-mandated physical education minutes.

Three main flaws in implementation of SHFA emerged: (1) inadequate capacity building for implementers; (2) inappropriate measures of implementation; (3) and insufficient funding for implementation (Figure 1).

Figure 1.

Conceptual Model of Factors Related to Policy Implementation Failure

Inadequate Capacity

Whereas the SHFA recommends professional development to enhance knowledge and skills; adequate training has not been provided. When a policy fails, critics often assume that it is a result of direct teacher sabotage.²⁴ Previous research has shown that policies are often poorly implemented due to characteristics of the implementers rather than higher level policy makers.²⁵ The implementer as learner framework recognizes the need to develop capacity, including essential knowledge and skills, for successful implementation.^{24, 24} Thus, implementation failure is often the result of lack of capacity development to help the implementers acquire the skills and knowledge they need to implement the policy. School administrators and teachers are not trained in physical activity and often have little physical activity experience.

Although the SHFA recommends professional development to enhance knowledge and skills, adequate training has not been provided. A partnership between the SCDOE and South Carolina’s education television network, ETV, produced an informational video to inform administrators, district staff, teachers and school boards about the SHFA.²⁶ In coordination with the state obesity plan, additional training is theoretically provided through the South Carolina Healthy Schools Summer Leadership Institute and School Health Leadership Academy.²⁷ The leadership institute held by the South Carolina Association of School Administrators (SCASA) has not included a healthy schools component to date. The lack of intended comprehensive training severely limits the ability to build capacity and provide technical assistance.

Inappropriate Measures and Enforcement of Implementation

Currently, only the immediate outcome (minutes of physical activity and physical education) of the SHFA is being roughly evaluated. The current evaluation of the SHFA does not thoroughly assess implementation of the policy. The survey used for evaluation of SHFA is closed response, with “yes/no” and “check all that apply” responses and reporting of minutes of opportunity by classroom.²⁰ A complete analysis of implementation requires an accurate evaluation of changes in practice, as practice is an inseparable piece of policy.²⁸ Thus, to assess the process of implementation, additional evaluation is necessary.

Currently, the outcome evaluation is flawed. First, principals may not accurately and reliably self-report these minutes. Secondly, reporting minutes of physical education may not accurately represent quality opportunities for physical activity. Physical education does not necessarily entail quality physical activity,³⁰ and national physical education standards emphasize that at least 50% of time spent in physical education should be at a moderate-to-vigorous physical activity level.³¹ Teachers may have students waiting in lines, or standing while doing drills, which are not opportunities for physical activity. One study found that only 9% of PE class time involved moderate-to-vigorous physical activity.³² Whereas schools may be complying with the time requirement for opportunities for physical activity, students may not be receiving quality physical activity.

The focus on outcomes influences how a policy is implemented. Requiring principals to report the number of minutes shifts the focus from increasing physical activity levels to complying with policy reporting requirements. According to Barrett and Fudge,^{14(p. 21)} often policies with strict outcome evaluations result in “achieving performance albeit at the expense of the original intentions.” Outcome evaluation results in compliance with the specific a priori targets, such as minutes of physical education, rather than achieving successful performance focuses on achieving ultimate policy goals such as increasing physical activity of students.¹⁴ Implementation evaluation is needed to assess performance.

Few PAP evaluations include assessment of implementation. When they do, the primary focus has been reports from district-level administrators (not the implementers) in a top-down approach. From a policy as practice perspective,^{14, 18} evaluations will remain incomplete until those who are directly involved in enacting or implementing the policy (teachers) and those who directly experience it (students) are included in evaluations.

A more bottom-up approach of including teachers, the ultimate implementers in the entire policy process has been left out of the PAP process despite being critical to the success of implementation.^{13, 33, 34} Plaut and Sharkey^{35(p. 1)} attribute the failure of many policies to “the lack of connection between K-12 policy and practice.” The perspectives of both top-down administrators and bottom-up teachers in a multisource approach are essential for meaningful evaluation and successful implementation.³³ Qualitative interviews, such as focus groups directly with implementers (teachers), would provide a detailed account of implementation. ²⁹ Physical activity policy evaluation needs to integrate both quantitative and qualitative information.¹³

Insufficient Funding for Implementation

While there are limited funds appropriated in SHFA, they are not adequate for the PE staffing needs and resources in South Carolina. Section 59-10-370 provides a funding loophole that schools’ efforts are contingent upon adequate funding from the state. Furthermore, in 2009, Joint resolution H3352 allows local districts to reallocate this limited funding to academic and arts instruction. Therefore there are limited budgets for improved programs, capacity development and evaluation of implementation. In comparison, schools receive funding for improving standardized test scores, and there are financial consequences when scores are unsatisfactory. There is not equivalent funding and accountability requirements for physical activity in the SHFA. The lack of funding for implementation suggests that the SHFA is not as highly valued as other competing priorities in education.

The lack of funding sends weak policy signals to school districts and policy implementers. Wisconsin Superintendent Mark Lichete^{36(p. 164)} eloquently summed up a common attitude towards unfunded mandates: “Are we meeting [state physical education] requirements? No. Do I care? No. Until they start allocating what they’re mandating, we can’t meet their mandates.” Whereas bills with small budgets are more likely to be enacted, changes in practice will not occur without strong policy signals from adequate funding and enforcement.

DISCUSSION

A re-examination of the implementation of the SHFA revealed three failures in the implementation of the SHFA. These flaws are ultimately a result of ‘weak policy signals’²⁴ in the design of the PAP. While these findings are unique to SHFA, the findings may inform analyses of other PAPs. Future research of PAP needs to further examine implementation of these policies.³⁷

The ‘weak policy signals’ found in SHFA have been found in evaluations of other states’ PAPs. The importance of capacity development, appropriate measures for evaluation and sufficient funding have been found in Utah, Texas and Colorado. Despite meeting quantitative standards for guidelines in Utah, a study found that many schools were just meeting the bare minimum and lacked essential components of school wellness programs to create meaningful change.⁹ A policy analysis of PAP in Texas concluded that schools need sustained resources for implementation and evaluation of policies, especially in particular disadvantaged regions.¹⁰ Belansky et al¹² cited the lack of accountability and community resources as reasons for the lack of implementation of a PAP in Colorado. Together with the findings of the current study, these studies suggest that changes are needed to create effective PAP policies.

Many of the weak policy signals found in South Carolina’s SHFA as well as other states’ policies could be addressed with new legislation. New PAPs in all states should include the following characteristics: require “quality physical activity” in schools, not making the assumption that physical education is equivalent to physical activity; provide adequate and continuing training for all implementers; provide for effective and accurate assessment of the quality of implementation; and provide sufficient funding for implementation. The policy process is complex; simply ameliorating these three policy design failures would not lead to definitive success. Additionally, PAP is only one component of the comprehensive initiative needed to increase physical activity and decrease obesity in youth. But beginning with a strong policy design may be one step closer to successful implementation. Ultimately, policymakers need to consider future implementation when designing PAP to ensure that the policy produces the desired outcome.

Despite this pessimistic analysis, South Carolina’s SHFA is a single example of many flawed PAPs. South Carolina should be lauded as an early adopter in establishing state school PAPs. As policies continue to be adopted, PAP research must also aim to evaluate the entire policy process. With the wide variety of policy analysis methods available from other disciplines, PAP implementation analysis has been monocular thus far. More work is needed to compose a complete picture of PAP implementation and create effective school-based PAPs.

IMPLICATIONS FOR SCHOOL HEALTH

These findings have multiple implications for school health. Policy makers and educators need to construct and implement policies that create broad physical activity opportunities for students beyond physical education. These can include active classroom activity breaks or lessons, intramural sports and after school programs, and additional time and encouragement for active recess. Policies need to support capacity development for teachers to provide quality physical activity opportunities, such as professional development opportunities that explain the importance of physical activity, give example physical activity in school, and give teachers the skills and practice to be able to implement these opportunities. School administrators should be an integral part of evaluating policies for effectiveness which may include objective measures of physical activity as well as qualitative information from teacher and students on the process of implementation. Administrators may partner with researchers, community coalitions or health agencies to receive assistance with data collection. Policymakers and government agencies need to realize the importance of implementation and provide sufficient funding for these policies and practices as well as their evaluation. These policies can provide the infrastructure for school staff to increase quality physical activity opportunities in schools. This will give all students the potential to receive the physical, psychological, and social benefits of physical activity and become healthier, happier, and higher achieving students.

Human Subjects Approval Statement

In accordance with the University of South Carolina’s Institutional Review Board, this study received exempted status due to the anonymity of interviews and use of publicly available documents and records.