Childhood Obesity Evidence Base Project: Building Translational Capacity through Meta-Analytic Methods

Abstract

Purpose: The purpose of this article is to demonstrate the need for and utility of using a taxonomic approach for evidence aggregation and meta-analyses, with focus on prevention and reduction of childhood obesity in very young children. As evidence has been generated through heterogeneous efforts, it is important that the field makes use of all available evidence to learn what works, for who, and in what circumstances.

Methods: The Childhood Obesity Evidence Base (COEB) project conducted a taxonomic meta-analysis, using Grounded Theory to code elements present in reports of existing studies and initiatives, of diverse design and evaluation approaches, which were then mapped onto the levels of the socio-ecologic model. This article is the fourth in a series that describes the COEB project overall. It discusses both generally and specifically how taxonomies contribute to traditional meta-analytic methods, what questions can and cannot be answered, the method's contribution to translational (implementation) capacity, and ability to inform future efforts.

Results: The COEB project illustrates how the taxonomic meta-analytic approach broadens the evidence base, increases translational capacity for effective intervention components, and evaluates the influence of contextual elements to inform future initiatives. How the method is used to establish associations between varying intervention components, contextual elements, and outcomes is discussed.

Conclusions: Taxonomies generated through this process can be used for meta-analysis, serving to generate topic-specific questions associated with intervention approaches and outcomes in context, which is adjunctive to traditional meta-analytic methods and can inform public health approaches.

Introduction

Early childhood has been shown to be a period of risk for the development of obesity that can persist into adulthood.¹ Despite significant funding spent to try to reduce the development of childhood obesity through various approaches and institutional lenses (person based, institution based, environmental manipulation, policy development, and implementation), the prevalence among populations most in need appears to be slowing, but remains a significant public health concern.^2,3 The National Collaborative on Childhood Obesity Research Childhood Obesity Evidence Base (COEB) project was undertaken for three primary reasons: (1) to use a novel taxonomic-based method of meta-analysis, (2) to identify successful approaches used to prevent childhood obesity in children 2 to 5 years old, and (3) to provide evidence to inform future efforts to reduce rates of early childhood obesity. Particular attention was paid to identifying active components that could be implemented in the intended setting with intended recipients through mapping of elements and categories within the socioecologic model (SEM).⁴ An outcome measure of BMI was adopted.

What Information Do Traditional Meta-Analytic Methods Provide? What Questions Remain Unanswered?

Meta-analytic approaches are undertaken at different stages of discovery and answer different questions: initial formulation of the state of the efforts, establishing the quality of the evidence in the field, and in traditional meta-analyses, testing a “package” of intervention components to establish the size of intervention effects.⁵ Systematic reviews utilize both quantitative and qualitative information and are comprehensive in use of the available literature to prevent selection bias. Such reviews can incorporate ratings of study rigor to help understand robustness of the evidence, and to whom it does and does not apply, especially if a comparison population is identified who do not receive the intervention.^6,7 However, systematic reviews may be affected by bias wherein studies achieving statistically significant change in the designated main outcome are more likely to reach publication.^8,9

Both systematic reviews and traditional meta-analyses use PRISMA criterion: preferred reporting items for systematic reviews and meta-analyses, which provides transparency in both decision process and criteria in formulating the analyses.¹⁰ This mapping of decisional criteria allows replication of results and information, making transparent which initiatives and studies can and cannot be included in the review. Initiatives that may not have had an a priori evaluation schema, have negative primary outcome results or demonstrate only mediational effects are less likely to be published^7,8 or included in meta-analyses. Importantly, the outcomes from these excluded studies could help move knowledge forward regarding what does and does not work (and the pathway), for whom, and under what circumstances, that is, contribute evidence that increases “translational capacity.”¹¹

Meta-analysis has traditionally been understood to mean the generation of a numeric estimate of the relationship, derived from aggregated effect sizes, from a set of studies similar in design characteristics, using a priori designated evaluation schemas, and the same targeted outcome.¹² Thus, this traditional form of meta-analysis helps to understand the extent to which highly specified and resourced interventions, usually delivered in tightly controlled conditions with fidelity, are demonstrated to achieve the desired outcome.

The studies that provide this information are typically randomized controlled trials (RCTs), using randomization to control for inherent bias and enhance internal validity. Inclusion criterion for participants is highly selective. Evidence from such studies is limited to intervention participants who may or may not include intended recipients in community or clinical settings, reducing generalizability. External validity may be further compromised because RCTs by design are uniformly implemented, which often cannot be replicated in situ.

Component analysis is usually absent in efficacy trials, primarily because the study is powered on the main study outcome only. Thus traditional meta-analytic statistical approaches constrain the types of studies that are included and evidence that is evaluated in the analysis. Recent advances in RCT design include optimization designs in the intended setting for the intended population, and Hybrid 2 designs that first test the efficacy of the intervention and then effectiveness in the intended population,^13,14 that have been formulated to address these limitations. However, foundational efficacy studies modeling a main outcome are most often used in traditional meta-analyses.

The underlying assumption in efficacy RCTs has been that an intervention works uniformly with selected recipients when delivered with fidelity. Which components are active drivers of the effects are usually not evaluated. Examining translational capacity, particularly for behaviorally based interventions, requires modeling of intervention elements such as mechanisms, resources used as part of the implementation process, that is, who (type of implementer) delivers what components to whom (intended recipients), what are intervention costs (e.g., cost of the curriculum, healthy foods, and exercise equipment), and what are other “contextual costs” (e.g., intervention site and personnel requirements).¹⁵ Evaluation of discrete components can potentially explain mechanistic effects and efficacy, independent of well-accepted considerations such as fidelity and intervention dose. Component analysis might show that various elements of the intervention negate the overall effects of the packaged intervention. Component analysis could potentially explain why a disproportionate number of clinical trials show negative results.¹⁶ These considerations are increasingly incorporated into adaptive designs and implementation trial design such as Hybrid 2 designs, after efficacy has been established.^13,14 This information could potentially be obtained from natural experiments, that is, those that do not randomize participants and conduct the intervention in the intended context: the community or health care setting. These are studies not traditionally included in meta-analyses.

Taxonomic Meta-Analysis: How Do Taxonomies Add Evidence to “Tell a Useful Story” to Inform Future Community-Based Research, Program Development, and Policy Implementation? What Questions Remain Unanswered?

Broadly defined, taxonomy is a classification approach that names and categorizes elements of systems. (Miriam Webster, https://www.merriam-webster.com/dictionary/taxonomy#:~:text=1%20%3A%20the%20study%20of%20the,to%20their%20presumed%20natural%20relationships) Thus taxonomies are specific to the system being evaluated. Taxonomy has previously been used to evaluate interventions utilizing behavior change techniques, contributing to our understanding of which approaches are more-or-less successful. The behavior change taxonomy establishes consensus naming conventions and hierarchically organizes behavior change methods.¹⁷ Alternatively, taxonomies can provide a broad structure for relevant elements of interventions, including context, intended setting, and recipients, as well as mechanisms of action.¹⁸ The taxonomic meta-analytic approach has the potential to inform implementation strategies for health practices, systems, and policies, as well as community initiatives. Importantly, taxonomic meta-analysis has the potential to use all available evidence pertinent to change in targeted outcomes.

Taxonomic meta-analysis makes use of traditional meta-analytic methods: scoping review, use of PRISMA criterion, and statistical modeling. See Methods article¹⁹ in this supplement that presents an illustrative example of the taxonomic meta-analytic process. However, rather than using a packaged intervention as the unit of analysis, taxonomic meta-analysis uses a component approach to evaluate effects. Effect sizes can be tested modeling mechanistic components, methods of intervention delivery in the intended environment, taking into account the influence of characteristics of the intended recipients and contextual elements, as long as these elements are described in reports included in the evidence base. Evidence from heterogeneous reports can be aggregated through the coding and categorization of like elements that are not uniformly described but have the same intended consequence. This meta-analytic approach can examine efforts through a wide lens that also include weighting for the rigor of research design and factor effects, and can incorporate a framing schema for evidence such as the SEM. (See Methods article for chart of mapping COEB taxonomies onto the SEM.) In contrast to traditional meta-analysis, results of taxonomic meta-analysis describe the relationships of the tested elements. Varying questions can be modeled using a taxonomic database, which can be expanded as new evidence emerges. Although taxonomic meta-analysis does not test hypotheses, it generates hypotheses about what appears to be working with who in what circumstances, and provides a basis for replicability of effects. See Rational article in this issue.²⁰

Contributions of the COEB Taxonomic Meta-Analysis to the Field of Childhood Obesity

The COEB taxonomy-based meta-analysis is an example of use of the approach described in the Methods article in this supplement.¹⁹ As described in the COEB Results article in this supplement,²¹ the taxonomic meta-analysis was initiated with a scoping review to identify the state of initiatives and evidence from efforts to prevent obesity in 2- to 5-year-old children in the United States, informed by stakeholder consultation. All relevant published literature that could be identified through exhaustive methods was considered. Meta-analysis was conducted using foundational statistical meta-analytic approaches, PRISMA criterion was applied, mean component effect sizes were calculated from elements derived from reports/studies meeting inclusion criteria of BMI outcome, and individual study risk of bias established.²² A foundational set of taxonomic elements was coded using Grounded Theory²³ from primary sources. Secondary sources were then used to supplement descriptions of missing elements such as population characteristics, intervention components, and contextual elements, resulting in taxonomies representative of the field of obesity prevention in children 2–5 years from 2005 to 2019. Literature containing an outcome measure of child BMI contributed to the cumulative evidence base. Adoption of this outcome measure permitted comparison of results of the taxonomic meta-analytic method with traditional methods.²⁴

The scoping of the literature, in preparation for taxonomic analysis, found 51 applicable studies, from 18,335 unique records of efforts being made to prevent childhood obesity, in our age group. The majority of interventions for children who are overweight or have obesity occur in preschool and school settings, and examine children 4 years and older.²⁵ Upon closer examination, there are proportionately fewer studies targeting the at-risk age group—children 2 to 5 years—and fewer still in infants. This may be the result of the previously held belief that a child's weight trajectory was not a reliable predictor of future weight status until after the age of 2 years. Although this assumption has been refuted, that is, babies' growth rate in the first 6 months of life is predictive of future weight status,²⁶ conducting RCTs with young babies and new mothers outside the school or daycare setting is difficult and imprecise. To date for example, RCT studies to prevent childhood obesity supported by national funders of childhood obesity prevention efforts using mother–baby pairs as the unit of analysis have been equivocal or unsuccessful.^27,28 Besides contextual and measurement issues, there are a number of potential explanations for lack of success, including the efficacy RCT presumption that “one size fits all,” that is, that a manualized intervention will affect all pairs similarly, that tightly controlled conditions can be established, and that randomization accounts for unexplained factors that are working to mediate intervention effects. However, a recent Cochrane review (2019) aggregating RCT results measuring z-BMI outcomes indicated that multimodal interventions addressing children birth to 5 years old showed significant reduction in the risk of obesity. Importantly, single approach studies were not effective in this age group,²⁹ thus supporting the need to identify effective components in multimodal interventions.

The results of the COEB taxonomic meta-analysis generated hypotheses to be further tested, identifying effective strategies that had population-based effects for >29,000 children, and provided a preliminary answer to the question: what approaches work best with whom, using which intervention components, in what context or setting. We found that many community-based initiatives, policy initiatives, health department reports of community efforts, and other stakeholder efforts do not report or are not tracking child weight status. Rather, approaches and curriculums targeting behaviors aimed at increasing healthy eating or amounts and intensity of exercise were often described through administrative and organizational reports, listing parents, health care providers, teachers, or research team implementers as the agents of education or behavioral change strategies. The agents of policy change were less likely to be identified, although the policy could have been identified (e.g., Special Supplemental Nutrition Program for Women, Infants and Children, or Supplemental Nutrition Assistance Program). It is noteworthy that implementation took place in varying venues, through multiple vectors, that is, multiple people were involved in the implementation process per se, and implementation processes targeted multiple levels, following the SEM. In most instances, study participant characteristics were available regarding race/ethnicity, gender, geography, and delivery site, and in some cases the characteristics of the implementers were described (such as a physician or nurse in a doctor's office). Although not directly targeting weight change, educational and behavioral approaches were presumed to impact child weight status that is reduce rates of childhood obesity. However, unless a formal evaluation schema was prespecified, including before and after measurements, weight status of children over the course of community-based initiatives was not often available for data synthesis. It is important to remind that this is the type of documentation usually needed for inclusion in meta-analyses. The taxonomic approach can, however, use population-based estimates for comparison group purposes. Direct causality cannot however be established.

We further found that community-based non-research-based initiatives less often track intervention elements on a level granular enough to do a component analysis. Reports did, however, provide a structure upon which taxonomies could be developed, including a description of approach, content, and delivery mode. Based on this specific taxonomic evaluation of the extant literature, it was difficult to determine exactly what intervention components are and are not working to prevent or reduce prevalence of obesity in this age group (2–5 years) across settings.

In summary, statistically significant “signals” did emerge,²¹ suggesting that interventions delivered by health care providers in health care settings provide an opportunity for intervention success, and that teaching parents about reduction in screen time for their children appears to have effects on reducing rate of increase in BMI in very young children. These approaches appear to be both pragmatic and feasible to pursue. Most very young children are routinely monitored by health care professionals who are likely to measure weight and height. However, because we wanted to be able to compare taxonomic analysis with more traditional methods of meta-analysis, the inclusion criteria of a reported BMI significantly reduced the number of nonresearch reports/efforts/initiatives that could be analyzed. This may have artificially constrained significant “signals” simply by the fact that the majority of community-based interventions do/did not measure height and weight over the course of the initiative, intervention, or implementation of a policy. Surprisingly, preschools were not found to be significant venues, but that finding might also be due to the choice of BMI as our primary outcome.

Of note are the implications of these findings for building implementation capacity. Use of successful behavior change strategies in young children such as counseling provided to parents by health care providers in health care settings, seem highly feasible. The parents of very young children routinely receive preventive health care guidance from their health care providers, who are usually considered trusted sources of information. Finally, the identification of reduced screen time as a successful approach to weight management in this age group mirrors findings of studies in older children.³⁰

Moving Implementation Science Forward

Implementation science focuses on external validity.³¹ Interventions carried out in the community often target multiple stakeholders and outcomes, are multilevel and multicomponent-core elements of implementation processes. Although reports of community initiatives may document context and recipient characteristics, implementation strategies (e.g., channels of delivery and cost) and recipient costs (e.g., time off work, intervention burden), which could impact adoption of proven intervention approaches, may or may not be specified in primary sources. These descriptive elements may, however, be available through other documentation (e.g., county health clinic reports). Elements derived in this manner can be incorporated into a taxonomic database to be used in meta-analyses through the use of natural language coding.²³ They would, however, be precluded from traditional meta-analyses. Thus, use of taxonomies has the potential to inform implementation science methods, is a useful tool for meta-analysis, and can increase the translational capacity of evidence. Obtaining evidence from diverse initiatives, from tightly controlled to quasi-experimental studies from community-based initiatives to policy implementation, is needed to “tell a useful story.”

As an example, the COEB taxonomic analysis was able to illuminate specific approaches and components related to reductions in BMI in very young children in situ, thus identifying promising strategies to be further tested in pragmatic trials. Evidence to do so is aggregated in the taxonomy database developed from the COEB project, which can be expanded as new evidence emerges. As there were no constraints on taxonomy-based evidence, only that which was reported in diverse reports, this test of the taxonomy method utilizing data from initiatives aimed at preventing childhood obesity provides a blueprint or guide to test implementation approaches. Adding coding to account for rigor of design and evidence, and presence or absence of inherent bias, further improves translational capacity.

Conclusions

Implementation science is ripe for the use of taxonomic meta-analysis, which could facilitate translation of effective intervention components into clinical and community use.³² Manualizing and training of the methods of a proven intervention are always helpful for the implementer in the service of optimizing fidelity; however, external validity depends on the usefulness of the intervention approach with the intended population in the intended setting in free living conditions, thereby improving public health. It is often virtually impossible to replicate, in situ, the controlled conditions under which delivery of interventions and efficacy of the dose were originally established. Thus, in the absence of the ability to measure dose and fidelity for strategies implemented in the community, identifying the presence or absence of components, exposures, setting, resources utilized, and mode of delivery becomes more relevant to implementation. Use of traditional meta-analytic statistical approaches to estimate component effect size using taxonomic elements anchors taxonomic meta-analytic findings in accepted public health metrics: how much effect will implementing core elements have on improving the health outcome of the intended recipients?

Reproducibility of effect is central to implementation science. Construction of a reusable evidence base using taxonomic elements provides information not otherwise available through traditional meta-analytic methods and is expandable as new evidence emerges. The COEB database is available for others to use for future examination of different questions regarding prevention of childhood obesity in this age group. This is a significant addition to traditional meta-analytic approaches.

Funding Information

The research reported in this article was supported by the Office of Behavioral and Social Science Research of the National Institutes of Health (NIH) under contract no. GS-00F-0007M. The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the NIH or the CDC.

Author Disclosure Statement

No competing financial interests exist.