Economic Analysis of Physical Activity Interventions

Abstract

Background

Numerous interventions have been shown to increase physical activity, but have not been ranked by effectiveness or cost.

Purpose

This study provides a systematic review of physical activity interventions and calculates their cost-effectiveness ratios.

Methods

A systematic literature review was conducted (5,579 articles) and 91 effective interventions promoting physical activity were identified with enough information to translate effects into MET-hours gained. Cost-effectiveness ratios were then calculated as cost per MET-hour gained per day per individual reached. Physical activity benefits were compared to U.S. guideline–recommended levels (1.5 MET-hours per day for adults and 3.0 MET-hours per day for children, equivalent to walking 30 and 60 minutes respectively).

Results

The most cost-effective strategies were for point-of-decision prompts (e.g., signs to prompt stair use), with a median cost of $0.07/MET-hour/day/person; these had tiny effects, adding only 0.2% of minimum recommended physical activity levels. School-based physical activity interventions targeting children and adolescents ranked well with a median of $0.42/MET-hour/day/person, generating an average of 16% of recommended physical activity. Although there were few interventions in the categories of “creation or enhanced access to places for physical activity” and “community campaigns”, several were cost effective. The least cost-effective categories were the high-intensity “individually-adapted behavior change” and “social support” programs; with median CE ratios of $0.84/ and $1.16/MET-hour/day/person. However, they also had the largest effect sizes, adding 35%–43% of recommended physical activity, respectively. Study quality was variable, with many relying on self-reported outcomes.

Conclusions

The cost-effectiveness, effect size, and study quality should all be considered when choosing physical activity interventions.

Introduction

Regular physical activity has positive effects on both physical and mental health^1,2 and numerous interventions have successfully increased physical activity. Yet differences in both the types of interventions and how they are measured make comparisons difficult. Cost-effectiveness analysis (CEA) could aid decision makers in allocating resources efficiently³ through synthesizing information about the effectiveness, costs and benefits of interventions. Of course, decision makers have to take other criteria into account, such as the distribution of benefits and costs, perceptions of fairness, and political support,⁴ but choosing the most cost-effective interventions maximizes the total benefits of limited resources.

A small number of physical activity interventions have been evaluated from a cost-effectiveness perspective, but in isolation and generally not in a way that allows comparisons across studies. One reason is that the relationship between physical activity interventions and long-run outcomes remains fairly speculative and seemingly minor changes in assumptions (such as how physical activity effects are sustained over time) have a larger impact on cost-effectiveness estimates than the intervention itself. No study so far has compared the cost effectiveness of dozens of effective physical activity interventions now published and recommended for general use.⁵

Ideally, CEA follows the reference case of the U.S. Panel on Cost-Effectiveness in Health and Medicine ⁶, which uses a nonspecific health outcome (quality-adjusted life-years, or QALYs), takes a social perspective on costs to include all costs (including the opportunity costs of participants), and discounts long-run costs and outcomes. At this point, this may be too demanding for physical activity interventions, partly because relating changes in physical activity resulting from an intervention to QALYs involves assumptions about health benefits and partly because important short-term benefits (such as mood improvement or decrease in musculoskeletal complaints^{7, 8}) are often not adequately captured in standard QALY methodology.

In this paper, therefore, a less-ambitious approach is taken by using an outcome that is specific to physical activity, namely the quantity of physical activity produced among the population reached, measured in MET-hours. Similarly, costs are limited to actual intervention costs, which leaves out the more speculative (although not necessarily unimportant) effects of the intervention on healthcare costs, productivity, or participants’ opportunity costs. This allows us to compare a larger set of physical activity interventions, even though it does not allow comparisons to other types of interventions like smoking cessation. To provide a benchmark, a crude calculation was used based on the estimated medical costs of inactivity ^{9, 10}. In that case, any intervention with a cost-effectiveness ratio below this benchmark may actually be cost-saving.

Methods

Both existing systematic reviews were used and a public database search was conducted to retrieve candidate studies for inclusion. All interventions listed in two systematic reviews were selected: Kahn et al.⁵ and van Sluijs et al.¹¹ A systematic literature search was then carried out using seven databases (MEDLINE, Sportdiscus, PsychInfo, Transportation Research Information Services, Enviroline, Sociological Abstracts, and Socio Sci Search) and included interventions published between 2000 and June 2008. The review was restricted to published trials (controlled trial, pre–post trial, or postmeasure-comparison approach) designed to promote physical activity excluding those focused on individuals with a specific disease or health conditions. To be included, interventions had to have a sample size equal to or larger than 50 participants and report a significant effective physical activity outcome which could be translated into MET-hours gained per person per day (described below). Interventions were grouped into 6 broad categories ⁵: (1) community-wide education campaigns, (2) “point-of-decision” prompts to encourage use of stairs as an alternative to elevators or escalators, (3) individually adapted behavior change programs, (4) school-based physical activity intervention targeting children and adolescents, (5) social support in community context, (6) creation or enhanced access to places for physical activity. Two independent reviewers (Shi and Pearson) reviewed the results from the initial search of the title then the abstract and finally the full paper. When opinions differed consensus was reached through discussion.

The reviewers also rated rigor or quality of each study by the presence/absence of nine dichotomous criteria: ^12–14 (1) existence of a control group (2) participants were randomly recruited and the response rate greater than 60%, (3) baseline characteristics between control and comparison groups were similar, (4) attrition was less than 30%, (5) the period of assessment was more than a single day, (6) the follow-up was at least 6 months after the intervention, (7) an objective measure of physical activity was used, (8) the measurement tool was shown to be reliable and valid in previously published manuscripts, (9) participants’ baseline physical activity was less than the national physical activity guidelines. These criteria were summed to arrive at a total quality score, ranging from 0 to 9. The quality scores were not used to determine the inclusion of the studies, but as an aid to judge the strength of the study evidence.

General Approach to Cost-Effectiveness Estimation

To compare the cost efficiency of different interventions as originally published, effectiveness was standardized by estimating the time spent in activities with higher MET intensities (METhours) per person per day as a result of the intervention, here referred to as “MET-hours gained” from the intervention. A MET represents the ratio of energy expended divided by resting energy expenditure, either measured or estimated from body size. MET-hours gained are derived by multiplying the METs associated with the type and intensity of the activity promoted by the intervention by the time spent performing the activity using hours as the unit of analysis. Estimating MET-hours as effectiveness measures accounts for the major parameters of physical activity including frequency, duration, and intensity. Validated classification systems were followed, including the Compendium of Physical Activity, to code different types and intensity of activities into METs.^15,16 Moderate physical activity (MPA) was assigned 3.0 METs, moderate-to-vigorous physical activity (MVPA) 4.5 METs, and vigorous physical activity (VPA) 6.0 METs.

The benchmark used for adequate physical activity for an adult was 1.5 MET-hours gained per day, equivalent to a half hour of moderate physical activity. For children, the benchmark used was 3.0 MET-hours gained per day. Intervention effectiveness is also expressed as percentage change of adequate physical activity per day, or MET-hours gained per person per day divided by 1.5 MET-hours for adults and 3.0 MET-hours for children.

Table 1 shows the default formula to translate various physical activity outcomes into MET-hours gained per person per day. A differences-in-differences approach was applied when a control group was available. In assessing the effectiveness of controlled interventions that measured only post-intervention outcomes, the corresponding control group was treated postmeasurement as its baseline.

Table 1.

Formula for physical activity outcome translation

Reported measure	MET hr gained per day translation formula
kcal/kg/min	MET-hour = (kcal/kg/min)*(average weight)*(6/7)
kcal/min	MET-hour = (kcal/kg/min)*(6/7)
kcal/week	MET-hour = (kcal/week)/70/7
steps/day on walking	MET-hour = (steps/10,000)*4.25*(1/3)*3MET
30 min blocks in physical activity per day	MET-hour = [(30 min block)/4]*MET assigned
min/day on physical activity	MET-hour = [(min/day)*MET assigned]/60
% people meeting guideline	MET-hour = (% people)*(1.5 MET-hour for adults or 3.0 MET-hour for children)
MET min/wk	MET-hour = (MET min/wk)/60/7
active days(at least 3 MET- hour) per week	MET-hour = (active days)*(3.0 MET-hour)/7

Definitions/default values:

If the study outcome is time spent on MVPA, MET assigned is the average of MPA and VPA = (3+6)/2 = 4.5 (all people)

MPA = 3.0 MET, VPA = 6.0 MET

Walking speed: 20min/mile. 10,000 steps = 4.25 miles (3.79 mile for women and 4.73 mile for men). To get a reasonable baseline, subtract 5000 steps

School recess time: morning = 15 min, lunch = 30 min

Furthermore, a calculation was made of how much each of the interventions brought the participants toward the minimum USDHHS recommended physical activity guidelines¹⁷. To accomplish this the baseline level of participants’ average physical activity levels were also rated in terms of the percentage of recommended minimum physical activity guidelines achieved.

Estimates of Costs

Program costs were considered as the total cost to the public health system to implement the intervention, regardless of sources of funds. Costs incurred for the development of the intervention and for research purposes were excluded in order to capture the cost of replicating proven interventions. The final cost parameter used is the total program cost for all people reached in the program, not only those who participated in the evaluation. All costs are reported in 2007 dollars. For some interventions, these costs were available from published cost analyses. For others, costs were imputed based on resource utilization ³ including program personnel costs, supplies and materials, equipment, transportation costs and travel expenses of program personnel, training costs, outside consultant cost services, and program overhead costs. The median national wage was used for program personnel salary imputation based on Bureau of Labor Statistics data ¹⁸. For some comprehensive interventions (community-wide physical activity education campaign), study authors were contacted to obtain the relevant costs, for example, of dollars spent on media buys. Not included are potential opportunity costs of physical activity, which are particularly hard to assess. In some cases, there are no opportunity costs (e.g., using stairs versus elevator), whereas interventions that change time allocation may entail some substantial opportunity costs. Potential effects on healthcare costs or productivity are not included.

Comparisons of Cost Effectiveness

First, the cost effectiveness of each intervention was calculated using the population in the selected published study. If an intervention reported physical activity outcomes at multiple follow-ups, the outcome with the best cost-effectiveness ratio was kept. Because the duration of interventions and follow-up time varied widely and the majority [56/91(61.5%)] had follow-up at 12 months or later, to make interventions comparable, the duration of the intervention was standardized to 1 year for a potential 10,000 target population. The effect reported in the study end point at earlier times was assumed to be sustained for 1 year. Total population physical activity benefit in this hypothetic year was simply MET-hours gained per person per day multiplied by 1-year duration and then by 10,000. Similarly, the total standardized annual intervention cost to reach 10,000 people was calculated as the cost per person per month multiplying by 12 months and then by 10,000. In standardizing interventions using “point-of-decision” prompts to encourage use of stairs as an alternative to elevators or escalators, stair use signs were assumed to have a 1-year service life. For those interventions involving playground painting enhancements, a 3-year service life was assumed, and costs were amortized to 1 year.

Benchmark for Cost Effectiveness

The cost of sedentary behavior has been estimated to account for 2.4%–5% of annual healthcare costs^{9, 10}. Spending for health care was $7,681 per capita in 2008 ¹⁹, so an expenditure of 2.4%–5% is $184–$384 per capita. An annual investment of this level of funding may be reasonable, if one assumes meeting physical activity guidelines has the potential to avert the equivalent of healthcare costs. The current minimum recommendation for physical activity is to engage in moderate physical activity at least 150 minutes per week, which translates to 390 MET-hours per year for adults and 780 MET-hours for youth ¹⁷. Therefore, for those who are sedentary, an expenditure of approximately $0.50 to $1.00 per MET-hour gained (for adults) and 25–50 cents per MET-hour for youth could be used as a benchmark of cost effectiveness. In fact, if avoided medical costs were the only cost component omitted from the current analysis, interventions with a cost-effectiveness ratio lower than the benchmark might be cost-saving from a public health perspective.

Results

Figure 1 provides a flow chart of the systematic literature review. A total of 91 studies with 141 intervention arms were identified meeting the current inclusion criteria (1.6% of articles screened). Many studies that were excluded did not specify either the intensity or duration of the activity gained, but offered outcomes like an increased frequency of activity only. Of the included studies, 48 used RCT or controlled trial (CT) study design. While the primary goal of 81 interventions was to promote general physical activity or walking, 10 studies focused on weight control, lifestyle modification, or cardiovascular disease risk factor reduction, with physical activity as a secondary outcome. Thirty-five studies (38%) used objective measurement instrument including direct physical activity observation, accelerometer, and pedometer. Fifty-six studies (62%) used subjective measures, usually self-report of physical activity over a given time period.

Figure 1.

Flow chart for article screening

Among the selected studies, both within and across the intervention categories, the costs of interventions varied considerably as did the number of people targeted and the amount of physical activity that was gained. Table 2 summarizes the intervention cost per person reached, the measured effectiveness in terms of the additional MET-hours the intervention added to routine physical activity, and the cost effectiveness in each intervention category. The last column shows the annual costs of a hypothetic scenario to provide the physical activity interventions to a community of 10,000 people. The most cost-effective strategy was point-of-decision prompts (e.g., signs to prompt stair use), with a median of $0.07/MET-hour/day/person, because of its low cost and large population reached. However, the benefit is limited to meeting only 0.2% of guideline recommended physical activity levels. School-based physical activity interventions targeted at children and adolescents (e.g., physical activity education, promotion of out-of-school physical activity) were relatively cost effective (median $0.42/MET-hour/day/person) when no additional school staff's labor costs were required. These interventions generated a median of .48 MET-hours, a quantity equivalent to 16% of the guideline recommended physical activity for youth.

Table 2.

Summary of intervention effectiveness, total cost, and CE ratio, median and range

Type of intervention (#= 141)	Costs / person	MET-hours gained / day / person	C/E ratio as $ per MET- hour gained/person	Annual Costs for 10,000 population reached
Point-of-decision Prompts	$0.0025 ($0.001 –$1.34)	0.0026 (0.007– 0.0142)	0.07 (0.0022– 4.72)	$58 ($58 – $13,441)
Community campaign (4 studies)a	$0.14; $14.93; $0.46; 55.86	0.44; 0.01; 0.10; 0.48	0.009; 1.50; 0.01; 1.90	$1432, $74,655, $4563, $3,351,369
Individually adapted behavior change (all)	$55.27 ($0.25 – $422)	0.50 (0.09–2.76)	0.41 (0.01–7.25)	$1,166,667 ($4970 – $10,938,000)
Low-intensity	$11.04 ($0.25–$274)	0.50 (0.15–1.26)	0.10 (0.01–5.95)	$545,000 ($4970 –$6,632,903)
High-intensity	$64.80 ($1.69–$422)	0.53 (0.09–2.76)	0.84 (0.02–7.25)	$1,452,089 ($142,204–10,938,000)
Social support (all)	$107.15 ($5.25– $1,609)	0.65 (0.05–2.89)	1.14 (0.07–60.2)	$2,520,000 ($317,581–16,932,192)
Low-intensity	$21 ($5.25–$167.90)	0.77 (0.11–2.39)	0.47 (0.07–5.17)	$2,099,500 ($630,000–$5,648,275)
High-intensity	$153.49 ($10.72~$1609)	0.65 (0.05–2.89)	1.16 (0.13–0.22)	$3,040,625 ($317,581–16,932,192)
School-based physical activity intervention	$48.86 ($0.00 – $947)	0.48 (0.06–1.41)	0.42 (0.00–8.77)	$398,717 ($0 – $12,626,263)
Creation or enhanced access to places for physical activitya (3 studies)	$15.08; $5.07; $137.46	0.62; 0.98; 0.26	0.40; 0.17; 4.47	$50,273, $16,914, $458,207

^aThis category contains less than 5 studies, so individual studies are listed instead of deriving their means.

Programs that are more costly per MET-hour gained include high-intensity individually adapted behavior change and social support programs with median $0.84 and $1.16/MET-hour/day/person, respectively. They are less cost effective primarily because physical activity interventions in these categories are much more intensive to deliver to individuals (involving more face-to-face counseling or interaction) compared with others. While more intensive interventions generate larger effect sizes (0.53 and 0.65 MET-hours/day, respectively, or about 35% and 43% of the guideline recommended physical activity for adults), the increase in effect size does not match the increase in costs compared to low-cost interventions.

Table 3 summarizes changes in physical activity achieved by interventions and most are relatively small. Although 38 of the 141 intervention arms studies appear to facilitate >5 MET-hours per week of added physical activity, only 7 of these studies were measured objectively and only 4 of these would be below the $1/MET-hour benchmark. Three of these interventions were for adults ^20–22 and employed pedometers, while the fourth was an intervention that simply painted playgrounds with multicolored markings ²³. The great majority of interventions increased physical activity a modest amount and on average, the reported increases in physical activity were smaller from studies that used objective assessment methods compared to studies that relied on self-reported outcome measures (median of 0.30 MET-hours gained in objectively measured studies versus a median of 0.68 MET-hours gained in subjectively measured ones, p<0.05). Among the 141 study arms, 36 (26%) were, on average, able to increase the physical activity of participants who did not meet guidelines at baseline to achieving 100% or more of recommended physical activity.

Table 3.

Magnitude of study effects and summary of standardized intervention cost per 10,000 population reached

Category of intervention (# of interventions)	# adding <1 MET hr/wk /person	# adding 1–3 MET hr/wk /person	# adding 3–5 MET hr/wk /person	# adding >5 MET hr/wk /person	Median (range) annual cost for 10,000 people to add 3–5 MET hr/wk
Point-of-decision prompts (28)	28	0	0	0	N/A
Community Campaign (4)	2	0	2	0	$3,350,000; $1,431
Individual adapted behavior change (49)	2	20	11	16	$688,000 ($71,000 – $11,000,000)
Social support (31)	5	7	5	14	$9,500,000 ($700,000 – $14,780,000)
School-based physical activity intervention (26)	5	10	4	7	$300,000 ($188,000 – $3,586,000,000)
Creation or enhanced access to places for physical activity (3)	0	1	1	1	$50,000

Figure 2 graphs the estimated annual cost of the interventions for reaching 10,000 individuals against the MET-hours gained per year for that population. (Community campaigns and environmental approaches are combined as one group due to the small number of interventions). Because of the large differences among studies, both costs and MET-hour gained were log-transformed to fit in the figure. The cost-effectiveness benchmark of $1 per MET-hour is drawn diagonally in the figure to show the relative cost effectiveness of the interventions. Interventions that produce at least an additional 3.5 MET-hours per week (equivalent of walking 1 hour more per week, or 8.5 minutes/day) are on the right side of the vertical bar. The graph demonstrates that point-of-decision prompts are very inexpensive, yet produce very little in the level of physical activity, while interventions relying on social support and individually adapted behavior change are the most expensive yet produce greater levels of physical activity.

Figure 2.

Standardized costs and effectiveness of physical activity interventions

Note: Any intervention that falls in the region below the diagonal line is cost effective using the adult intervention benchmark of $1/Met. Interventions that fall to the right of the vertical line produce at least 3.5 extra METs per week (equivalent of walking 1 hour per week or about 8.5 minutes more per day).

Appendix A (available online at www.ajpm-online.net) includes a data supplement that summarizes the interventions included in the analysis, including basic demographics of study population, intervention duration, type of measurement instrument, translated physical activity outcome (in MET-hours gained per person per day), total program cost, standardized 1-year cost, CE ratio, and study quality-rating score. For articles with multiple intervention arms or multiple follow-ups, those having the best CE ratio were reported.

Cost-Effective High-Quality Programs

Based on a benchmark of $1.00/MET-hour, 97/141 arms of the 91 interventions could be considered cost effective. Most of interventions targeting adults cost considerably less than $1.00/MET-hour, with 62/115 arms costing ≤ $.50/MET-hour. For school-based interventions targeting youth (for whom twice the level of adults physical activity is recommended), only 7/26 study arms cost less than $.25/MET-hour and another 6/26 cost between 25 and 50 cents/MET-hour gained. Table 4 lists the 17 studies meeting at least 7 or more of the quality criteria. Ten of these studies cost 50 cents/MET-hour gained or less and only three of them used objective measures of physical activity.

Table 4.

Physical activity interventions that meet seven or more quality criteria

Author	Intervention	Cost of Intervent ion/ pop reached	MET-hours gained/ day	Cost per MET- hour gained
Objectively measured
Sallis (2003)28	School-based PE intervention, nutrition intervention (provide low-fat foods) and environmental, policy, and social marketing interventions (effect on boys only)	$508,913/13,308 boys	0.42	$0.50
Stratton(2005)23	Environmental Change—Playgrounds painted with multicolored markings	$5,779/1139 Children	0.98	$0.17
Verstraete (2006)29	Provide game equipment to children in school during recess	$1840/122 Children	0.62	$0.40
Subjectively measured
Aittasalo (2006)30	Physical activity self-monitoring using pedometer and physical activity log for 5 consecutive days	$3427/62 adults	0.28	$5.95
Aittasalo (2006)30	Physician individual counseling (one time)	$8334/130 adults	0.28	$1.71
Arao (2007)31	Individual counseling for 15 min at the goal-setting session and 5 sessions monthly individual consultations for 10 min, plus environmental and social support	$8073/84 adults	0.82	$0.70
Haerens (2006)32	School-based intervention combining environmental changes with computer-tailored feedback (plus parental involvement)	$1,106/2232 boys	0.29	$0.11
Haerens (2007)33	School-based computer tailored intervention to increase physical activity provided by CDs	$525/139 youth	0.23	$0.27
Halbert (2000)34	20-min individualized physical activity advice by an exercise specialist in general practice, reinforced at 3 and 8 months	$4771/149 seniors	0.79	$0.14
Kolt (2007)35	8 telephone counseling sessions	$5578/93 seniors	0.59
Manios (2005)36	School-based health education, School PE, parental involvement	$534,300/4171 youth	1.25	$0.05
Marshall (2003)37	Mailed stage-targeted print intervention, consisted of a single mailing of a letter and full-color stage-targeted booklets	$1192/227 adults	0.20	$0.17
McKenzie (1996)38	School-based CATCH intervention included school policy changes, food service intervention, a physical education program, cardiovascular health and tobacco curriculum, home/family component.	$400,113/5,352 children	1.37	$0.33
Pazoki (2007)39	Community-based lifestyle modification: audio-taped activity instructions with music and practical usage of the educational package were given in weekly home-visits	$1919/179 women	0.76	$0.24
Rhudy (2007)40	20 personal phone calls from a nurse	$5336/70 Veterans	0.21	$1.17
Rhudy (2007)40	10 randomly interspersed personal and 10 automated phone calls	$3818/70 Veterans	0.21	$0.84
Shirazi (2007)41	Home-based exercise prescription consisted of strength and balance training that was progressive, individually tailored and included a walking program	$4962/61 Iranian women	0.95	$0.94

Discussion

The most cost-effective interventions provide the highest benefit for each dollar invested, which in some cases might lead to overall savings from a societal perspective. In this study, the most cost-effective interventions reached a large number of people with low-intensity (and low-cost) efforts. Interventions like stair climbing prompts may be extremely cost effective, but because they increase physical activity by only minuscule amount, they alone could not greatly increase the proportion of individuals who meet physical activity recommendations.

Within each of the categories studied, some interventions appeared more cost effective than others and could be models for replication. Community-based campaigns and school-based interventions have the greatest potential to be scaled up at the lowest costs. Although many individually adapted behavior change programs and social support programs had larger effect sizes, the more effective programs in these categories tended to be more intensive and require personal coaches and multiple sessions. These demanding programs may be difficult to scale up to reach the large numbers of people who could benefit from increased physical activity given the costs. Even though they are effective, they may not be the most cost-effective interventions for changes at the population level. Instead, these might be interventions better used for targeted groups, possibly in clinical settings.

While the studies were analyzed at face value, there is a large potential for measurement error and biases. In fact, the cost-effectiveness ratios varied dramatically even across interventions in the same category, targeting similar populations. This highlights another benefit of the comparative approach used here, which enabled the identification of a systematic methods effect. Interventions using subjective measures typically appeared more effective than those using objective measures. This is probably because self-report measurement is subject to biases from imprecise measurement tools, social desirability, recall problems, and definition/interpretation problems (e.g., what activity is considered exercise). Of course, objective measures like accelerometers can underestimate physical activity as some do not capture certain types of motion and activity exertion, like weight lifting, cycling and swimming. Such biases are probably small compared to biases from subjective measures ^{24, 25}.

Although physical activity can offer health benefits along a continuum of activity, the effects are probably nonlinear and there may be a threshold effect, possibly around the recommended physical activity levels. In that case, interventions with very small effect sizes may not get enough individuals into the most beneficial range, while interventions with large effects may have fewer benefits, if many individuals exceed the threshold. In 37/141 of the study arms the average level of physical activity exceeded the national physical activity guidelines at baseline, from 113% to 371%. Of course, this raises other questions about the credibility or at least the generalizability of those studies, as the majority of the population is not moderately active ¹.

Our analytic approach also has limitations. The first is that translating different original measurement tools to a common metric may not achieve comparability. For example, the 7-day recall instruments give a full half-hour credit for exercise even when the duration of exercise could have been less than 30 minutes ²⁶. Pedometers also tend to overestimate the amount of moderate and vigorous activity, as they register all movement, including light physical activity ²⁷. The time horizon for the evaluation also differs across studies (and none of them would come close to capturing lifespan effects). The length of an intervention and the duration effect (how long the effect of an intervention will last) will influence the total number of MET-hours that can be attributed to the intervention. The 1-year cost standardization was based on the assumption that all interventions will be conducted in a 1-year period and the impact will be constant over this time period. But this assumption may not be valid, and the intervention duration effect is unknown. Effectiveness usually decays over time, and it is not possible to be sure how this decay differs across interventions¹⁰.

Studies with insignificant results were excluded. The intent here was not to conduct a meta-analysis to determine whether interventions are effective; rather, a cost-effectiveness comparison was conducted of interventions claimed to be effective. This can be problematic when there are systematic publication biases. When studies have low statistical power (and therefore precision), significant findings that are due to chance (which means about 1 in 20 studies at standard significance levels) tend to widely overestimate true effect sizes. This could be the real reason for some surprisingly low cost-effectiveness ratios.

There is variation in the quality of the underlying intervention evaluations. Some studies used rigorous, RCTs, while others were natural experiments or not randomized. Rigorous study designs using objective measures tended to have a smaller physical activity effectiveness and higher cost-effectiveness ratios compared to pre–post or postmeasurement comparisons and subjective measures, but they provide greater confidence that results are genuine.

The reference case CEA uses a measure like the QALY that is comparable across very different interventions, but there is no reliable and valid method to convert all the various health benefits of physical activity into a single health metric. A less-ambitious approach was used here, with the choice to use an outcome specific to physical activity interventions, namely MET-hours. This limits comparisons with very different types of interventions (say smoking cessation), but it remains useful to contrast physical activity interventions. Future CEA studies may be able to address this shortcoming.

Despite these limitations, the methods used in the present study do provide a means to understand the general pattern of relative cost effectiveness of various physical activity interventions, and provide a meaningful metric to gauge immediate outcomes of the interventions. Interventions with low cost-effectiveness ratios were found in all six intervention categories, although only a few were of the highest quality and also had objective measures. Nevertheless, these studies comprise evidence that increasing physical activity at a population level is likely to be feasible.

Recommendations for Evaluating Future physical activity Interventions

Future interventions to promote physical activity should take care to report resource utilized and costs of intervention. They should also measure the frequency, intensity, and duration of physical activity gained, so that their costs and effectiveness can be ranked against other interventions to help better inform public health policies. Many studies in the current review had relatively small samples. Future studies should have sufficient sample sizes (statistical power) to reliably detect more realistic effect sizes. Underpowered studies in combination with publication biases that favor significant results create overly optimistic assessments of intervention effects and their cost effectiveness. In reporting the effects, the authors should include not only the average change in physical activity, but the SE as well. Baseline physical activity of participants should also be compared to recommended guidelines, so researchers may have the opportunity to tailor their interventions to the target group, determine whether an intervention is indeed warranted, and whether the measures to be used are sufficiently sensitive. As there are an increasing number of studies promoting physical activity now being published, more attention should be paid to study design to ensure high-quality studies. Given the large task of increasing physical activity among a sedentary population, a wide variety of options are needed to enhance the likelihood of adoption among disparate target populations.