School-based obesity-prevention interventions in low- and middle-income countries: do they really work?

See corresponding article on page 415.

Obesity is the most common nutrition-related health problem around the world (1), especially among children (2). Hundreds of studies have been conducted to test approaches to prevent obesity, and many were in children in schools (3). Most of these studies were conducted in higher-income countries. An article in this issue of the Journal by Verstraeten et al (4) is the first to review these interventions in low- and middle-income countries.

Some reviews of this literature concluded that it was virtually impossible to determine intervention efficacy or effectiveness because there was huge variability across studies in conceptual foundations, selection of intervention procedures, measures of outcomes, study designs, sample sizes, quality of the measures selected, and other factors (5). Others have been quoted as “there was now so much evidence about the impact of interventions in children ages 6 to 12 that further such trials in this age group seemed unnecessary” (6). The latter individuals appear to seek justification for policy and other community actions to begin to deal with the pressing obesity problems; however, the use of inadequately supported procedures may cause problems of their own.

Evaluations of interventions early in development should test for “efficacy”—ie, assess whether the intervention worked under ideal circumstances; later evaluations should test for “effectiveness”—ie, whether an intervention worked under “real world” circumstances (7). Tests of efficacy need to document a conceptual foundation on the basis of the best-available evidence for predicting or changing behavior (8) and use adequately powered randomized controlled trial (RCT) designs to minimize threats to internal validity. Efficacy investigators need to select (or develop) the following: 1) intervention procedures consistent with the conceptual framework and shown to strongly influence targeted mediators; 2) mediating variables that assess key constructs from the conceptual framework, shown to causally and strongly relate to the behavior or behaviors, with the use of psychometrically tested and validated instruments; 3) behaviors causally related to adiposity, with instruments validated in the targeted population; 4) adiposity indicators assessed by using the best-available validated procedures; and 5) process evaluation to verify whether the intervention was delivered with high fidelity in adequate quantities to the targeted population (9). Statistical analysis procedures need to be applied that reflect the complexity of the study (eg, account for the clustering by schools) and test the effect of the experiment both on adiposity and on mediating (psychosocial and/or environmental) (10) and moderating (11) effects.

Even under the best of circumstances, substantial problems exist in documenting these effects, including psychometric inadequacies in commonly used mediating variables (12), low validity in commonly used dietary food-frequency measures (13), limitations in the use of BMI as an indicator of adiposity (14), disagreements about the most appropriate conceptual foundation for such interventions (8), and lack of evidence of causal relations (15). Few mediating variable analyses have been reported (10); lack of significant findings of mediation in our own work has inhibited our submission of such manuscripts for publication. Interventions shown to affect adiposity outcomes without affecting the targeted mediating variables (behavioral, environmental, or psychosocial) provide little evidence that they can be replicated, because there is little evidence about how or why they worked.

Another concern is the behavior or behaviors targeted for change. Many obesity prevention interventions have targeted increasing fruit and vegetable intake and decreasing sweetened beverage intake. Systematic reviews, however, showed no consistent evidence that increased fruit and vegetable intake protected against obesity (16–18) or that sweetened beverage intake contributed to it (19). Furthermore, severe problems in measurement of diet (20) with substantial underreporting of intake among the obese (21) challenge investigators’ abilities to test these pathways of effects. There has been substantial interest in involving parents in obesity-prevention interventions, but systematic reviews have shown little evidence that involving the family has influenced children’s diet (22) or physical activity (23) behaviors; we only found effectiveness to vary by the intensity of the intervention.

Most evaluations of obesity-prevention interventions have not met the above standards (15), in part because they require large samples and are costly to deliver. One study, which was adequately funded to reach a large sample (n = ≥6000) with documentation of mostly ≥90% fidelity, did not attain its primary outcome (percentage of children initially above the 85th percentile of BMI who lowered their percentile category ranking), with only a weak effect among children initially above the 95th percentile of BMI who lowered their percentile category ranking (24). Few obesity prevention interventions with evidence of being effective on first implementation and evaluation (25) have been tested in a second implementation; and when a second implementation has been tested, these have generally not had the desired effects (26–28). Such patterns of findings should be sobering to the child obesity prevention community.

It is not clear which of the reviewed studies were efficacy trials and which were effectiveness trials (4). Effectiveness trials do not have to meet the same rigorous standards because they were originally conceived as monitoring implementation and outcomes to identify the more adverse “real world” circumstances under which they worked (7) (eg, lower fidelity due to low training of implementers, more challenging target populations). The demonstration of effectiveness, however, should not be done until efficacy has been shown: if an intervention does not work under ideal circumstances, how could it work under worse conditions?

It was reassuring that some evidence indicated that the obesity-prevention interventions attained some desired effects in low- to middle-income countries (4), suggesting these investigators were pursuing important leads, and the effects documented in high-income countries are not circumscribed by their socioeconomic conditions. This review, however, also documented the same diversity in conceptual foundations, intervention procedures, and measures of mediators, behavior, and outcomes and found small sample sizes: there were few schools (a clustering unit) and few students. High within-cluster similarity of students shrinks the effective sample from the number of students to the number of schools (29). Not correcting for the clustering can lead to a misrepresentation of outcomes (usually thinking an effect was significant, when it was not).

This systematic review of obesity-prevention intervention programs in low- to middle-income countries focuses attention on this issue of growing importance. The findings provide hope that answers can be found. Much research remains to find documented effective solutions for large numbers of children throughout the world.