Abstract
Objective
The dissemination of effective obesity interventions requires the documentation of key elements of the intervention. But outcome papers and other published manuscripts often lack detail that allow the replication of the intervention. The Behavior Change Technique (BCT) Taxonomy (BCTTv1) is a widely used approach to identify key elements of an intervention study. This study compares the extent to which BCTs and domains identified in studies' intervention protocol are concordant with detail from corresponding intervention design and study outcome papers.
Methods
Data come from four obesity interventions with complete intervention protocols as well as published intervention design and outcome papers. The number of domains and BCTs was calculated for each treatment arm and stratified by coding source. Emphasis of domains and BCTs was determined using an Analytical Hierarchy Process (AHP).
Results
A review of each study's intervention protocol showed the mean number of domains and BCTs used in treatment arms as 11.8 and 26.7, respectively. Primary outcome papers had a mean loss of 34% of the reported domains and 43% of BCTS as compared with intervention protocl. Design papers showed a loss of 11% and 21% of domains and BCTs, respectively.
Conclusions
The results confirm the limitations of using the BCTTv1 coding of outcome papers to describe obesity‐related interventions. The results also highlight the need for mechanisms that allow for a full description of intervention content such as inclusion in a supplemental section of an online journal or the use of intervention‐focused consort guidelines.
Keywords: behavior change techniques, obesity interventions
1. INTRODUCTION
The need to create obesity interventions (including interventions designed to promote weight loss as well as prevent unhealthy weight gain) that have long term effectiveness and can be implemented with fidelity, sustained through existing systems, and disseminated widely remains a significant public health challenge. While progress has been made in designing behaviorally‐based obesity interventions that are effective in successful weight loss in the short term 1 , 2 intervention approaches that result in sustained weight loss or healthy weight maintenance continue to be elusive. 2 , 3 Those interventions found to be efficacious in rigorously conducted trials need to be disseminated more broadly to have public health impact. 4 , 5 , 6
Enhancing the replication and dissemination potential of interventions found to be effective requires a clear and complete understanding of the intervention strategies tested. But complete descriptions of interventions are often not widely available. Most typically, some description of the intervention is included in the publication of the primary outcome paper of the intervention trial but lack of space limits the amount of detail available. An intervention design paper may be published that adds additional detail, but not all studies publish a design paper. Obtaining the intervention protocol, including the manual of procedures and other related intervention materials from the original investigators is often difficult either because of challenges in connecting with the original investigators or, perhaps, because the intervention protocol was not created with dissemination in mind, or created at all.
The level of intervention detail needed for dissemination includes information on the determinants targeted for change, the mode of delivery, intended dose to be delivered, the intervention components and the intervention strategies designed to promote change. 1 , 7 Michie and colleagues 8 , 9 created a behavior change taxonomy to code intervention descriptions, explore techniques used across different types of interventions, and help identify techniques that are most effective. Work developing the taxonomy began by having experts identify a set of behavior change techniques (BCTs) commonly reported as used by intervention studies, particularly interventions focusing on impacting change at the individual level through educational, counseling, motivational or skills‐based approaches. The taxonomy identified and categorized 93 BCTs embedded in 16 broader categories or domains. These BCTs and domains are detailed in the Behavior Change Technique Taxonomy (BCTTv1) which also includes a website providing detailed training on how to code interventions. 8 This approach has spurred a good deal of research in the past decade with the goal of identifying the “active ingredients” in interventions. To date, the work has been conducted post hoc, often relying on descriptions of interventions that are included in the primary outcome paper or, if available, the intervention design paper as the source of information for the coding.
While there is evidence that the coding process shows good inter‐rater reliability, 10 , 11 , 12 , 13 , 14 evidence for criterion validity is less strong. Criterion validity assesses the level of agreement between a “gold standard” or criterion measure and another source of information purporting to measure the same construct. The “gold standard” for describing an intervention is found in the intervention protocol, including the intervention manual of procedures and the specific intervention materials that document the details of the intervention as planned. However, the most commonly used document to code BCTs is the study's primary outcome paper. Primary outcome papers typically include a short description of the intervention but focus more on study design, outcome measures, analysis examining the impact of the intervention, and a description and discussion of the results. Coding the outcome paper, rather than the intervention protocol, may threaten criterion validity and result in an incomplete and potentially skewed picture of the intervention. If an intervention design paper is published, more detailed information on the intervention may be included but it is unlikely to be as rich and complete as the intervention protocol.
Minimal work has been done to investigate the loss of intervention detail between the gold standard (intervention protocol or manuals) and subsequent publications that mention the intervention and the majority of work has been with smoking cessation interventions. De Bruin and colleagues conducted a review of interventions for smoking cessation 14 that included 142 randomized controlled trials spanning the years 1995–2015. The research involved making comparisons between the BCTs identified from coding the published materials related to the intervention and unpublished information on the intervention obtained from the study authors. While inter‐rater reliability of the coding was high (Kappa = 0.98) only 35% of the BCTs from intervention groups and 26% from control groups were identified in the published studies. 14 In the field of obesity interventions more of the work has focused on establishing the reliability of the coding method rather than examining the validity of the inclusions of key domains and BCTs in published papers. 10
In addition, while the BCT taxonomy process allows identification of specific BCTs and domains used in an intervention, the approach does not address the relative emphasis of one domain or BCT over others in intervention activities. Rather, the taxonomy process functions as a binary evaluation of the presence or absence of a domain or BCT. In order to replicate an intervention or understand how it works some sense of which domains and BCTs were emphasized would be useful; emphasis may reflect the interventionists' expectations about the potential impact of specific approaches.
The EARLY trials 15 provide a unique opportunity to examine the extent to which BCTs and domains identified in studies' intervention protocol are concordant with detail from corresponding intervention design and study outcome papers. The EARLY trials were funded by the National Heart, Lung and Blood Institute (NHLBI) and the National Institute of Child Health and Human Development (NICHD) with the goal of developing, refining and testing innovative behavioral approaches for weight control in young adults at high risk for weight gain. Each of the seven funded universities were charged with conducting a unique 2 year intervention that included technology to appeal to young adults. Sites also agreed to a common primary outcome and a set of common elements and measurement protocol. Of the seven EARLY trials, four trials (two weight loss trials 16 , 17 and two weight gain prevention trials 18 , 19 ) included a published intervention design or methods paper, 16 , 17 , 18 , 19 a published outcome paper reporting on the effectiveness of the trial, 20 , 21 , 22 , 23 and produced a complete intervention protocol.
The purpose of this paper is to compare the number and specific BCTs and domains identified in a full review of the intervention protocol (the criterion measure) with BCTs and domains mentioned in the intervention design or methods paper and in the final outcome paper. Additionally, using a process that allows comparison of the emphasis between domains and BCTs used in each intervention, an examination of which domains and BCTs were most and least emphasized in each intervention was conducted and compared across the three sources of materials.
2. METHODS
2.1. Study selection
The four EARLY trials that included complete intervention protocols, a published intervention design paper, and a published outcome paper reporting on the effectiveness of the trial were included in the current analysis. The EARLY interventions included in this study are: (1) the Cellphone InTervention for You (CITY) study, 16 (2) the Social/Mobile Approaches to Reduce weighT (SMART) study, 17 (3) the Choosing Healthy Options In College Environment Settings (CHOICES) study, 18 and (4) the Study of Novel Approaches to weight gain Prevention (SNAP). 19 Table 1 provides an overview of these four studies. Two studies (CITY and SMART) focused on weight loss and two studies (CHOICES and SNAP) targeted weight gain prevention. Two of the studies (CHOICES and SMART) had a control and a single intervention arm. CITY had a control arm and a cell phone intervention arm (SMART CP) and a personal coaching intervention arm (SMART PC). SNAP had a control arm and a small change intervention arm (SNAP SC) and a large change intervention arm (SNAP LC). Two of the four studies focused on college students (CHOICES and SMART) while the other two (CITY and SNAP) recruited young adults from the general population. The sample size for the studies ranged from 365 to 600; White/Caucasians and females represented the majority of the sample. A variety of technologies were used to engage young adults randomized to the intervention conditions while young adults randomized to the control condition received usual care or general health information.
TABLE 1.
Description of individual studies
| Study | CHOICES | CITY | SMART | SNAP |
|---|---|---|---|---|
| Institution(s), PI(s) | U Minnesota, L Lytle | Duke U, L Svetkey | U California—San Diego, K Patrick | Brown U, R Wing; U North Carolina, D Tate |
| Primary outcome | Change in BMI | Change in weight | Change in weight | Change in weight |
| Focus | Weight gain prevention | Weight loss | Weight loss | Weight gain prevention |
| Target population | Community college students | Overweight/obese young adults | Overweight/obese 4‐year college students | Young adults |
| Sample size | 441 | 365 | 404 | 600 |
| Recruitment by race | 72.8% white/Caucasian | 56% white/Caucasian | 41.8% white/Caucasian | 75.4% white/Caucasian |
| 15% Black/African American | 36% Black/African American | 3.7% Black/African American | 11.4% Black/African American | |
| 6.1% Asian | 3% Asian | 23.8% Asian | 4.3% Asian | |
| 4.8% multiple | 0% multiple | 9.2% multiple | 8.7% multiple | |
| 1.3% other | 1% other | 21.5% other | 0% other | |
| 0% Unknown | 4% Unknown | 0% Unknown | 0.2% Unknown | |
| Recruitment by gender | 67.6% female | 70% female | 70% female | 78.3% female |
| Technologies used | Online curriculum; web‐based social network | Cell phone; bluetooth‐enabled bathroom scale | Cell phone (text messages, smartphone apps); Facebook, web, email | Cell phone; Internet |
| Brief intervention descriptions | Students are randomized to two conditions: Intervention and control. Intervention arm begins with a one‐credit college course focused on behaviors important in weight control. A web‐based social network site designed for this research and focusing on weight and behavioral tracking and goal setting is introduced during the class and continues for 24 months. The control group receives standard public health information on maintaining a healthy weight | Participants are randomized to one of three conditions 1 : cell phone (CP) based intervention 2 ; personal coaching (PC) plus cell phones for self‐monitoring 3 ; control group. The cell phone technology includes self‐monitoring of weight, diet and physical activity. Both group and personalized coaching are used in the coaching condition. The control group receives usual care | Students are randomized to two conditions: Intervention and control. Intervention students receive theory‐based content on physical activity, diet, calories and weight management strategies through text messaging, emails, Facebook, websites, and apps. Apps are developed by college‐age tech designers. Control students receive access to a study website with general health information | Participants are randomized to one of three conditions 1 : large change intervention 2 ; small change intervention; and 3 control. Two intervention groups test the differences in making large changes (LC) or small changes (SC) in diet and physical activity to avoid unhealthy weight gain. The goal of the large change group is to lose 5–10 lb to buffer against the weight gain that often occurs during young adulthood. The small change group makes small changes in diet and activity to reduce the chance of weight gain. The control group receives usual care |
2.2. Intervention coding
A full description of the coding process and results for all the EARLY trials is published. 11 Briefly, four coders with at least Master's level training in behavioral science were trained in BCTTv1 using the website (https://www.bct‐taxonomy.com/) 24 and app created by Michie and colleagues, as well as through practice coding exercises. Coding plans were developed through meetings held with two of the initial taxonomy developers, Drs. Charles Abraham and Susan Michie.
For the full review (the coding of the criterion measure) each of the participating EARLY studies provided the intervention protocol including intervention manuals of procedures, all intervention materials, and screen shots or logins for direct access to technology components. Each treatment arm was coded independently by two or three raters. After coding, a consensus meeting was used to identify discrepancies and additional documents were requested from the sites. Raters independently re‐coded those BCTs for which there was disagreement. Following this second coding, structured interviews with study teams were completed to clarify questions and the coding team met to reach consensus. Following these interviews, the coded BCTs were sent to sites for their review and consensus. In every case, the study team indicated that additional BCTs should be coded and they were asked to provide documentation (e.g., lessons, podcasts, campaign documents) to demonstrate how the BCT was used. An average of 3.2 (range 0–12) BCTs were added to the coding following study team review. A domain was coded as present if an intervention included at least one BCT from the domain.
For the present study, additional coding was performed using the published outcome and intervention design papers from each team. Two independent coders with extensive training in BCT coding and at least Master's level education reviewed each manuscript coding for presence or absence of the BCTs. When at least one BCT from a domain was used, the domain was marked present. After independent coding, coders compared results with each other and with a lead investigator (DT) to resolve any discrepancies and reach consensus on the coding.
2.3. Analysis
Coding results were entered in Excel and imported into Stata (Version 15), with each domain and BCT dummy‐coded as 1 = present and 0 = absent and source coded as 1 = full review (representing a review of the intervention protocol), 2 = published intervention design paper, and 3 = published outcome paper. The number of domains and BCTs was calculated separately for each intervention arm (n = 6) and control arm (n = 4), as well as averaged across all arms, stratified by intervention versus control. Next, for each intervention arm and coding source, lists of the specific instances of missing domains and BCTs were generated. To determine if domains and BCTs were missing according to their degree of emphasis in the intervention, missingness was compared using the Analytical Hierarchy Process (AHP), 25 a method for analyzing complex decisions that uses pairwise comparisons to determine relative emphasis or importance. Briefly, each EARLY study team was trained on how to apply the AHP during a multi‐day face‐to‐face meeting. Each study team received the list of domains and BCTs used in their intervention arms with examples of how they were employed. Pairwise comparisons of the domains were made on an anchored scale where one indicated equal emphasis, and values two to nine represented progressively divergent emphasis. Results were presented as the percentage emphasis of each domain for each intervention arm allowing an estimation of the extent to which each domain used in the intervention was emphasized in relationship to other domains used. For the present analysis, based on examination of the AHP distributions, a domain was defined as having “High” or “Low” emphasis if it was rated in the AHP as having ≥10% emphasis or <5% emphasis in the intervention, respectively.
3. RESULTS
Figure 1 shows the mean number of domains and BCTs coded across the intervention and control arms by coded source. While the taxonomy contains 16 domains, the unique number of domains used in the intervention arms ranged from 7.8 to 11.8. Far fewer domains were used in the control arms (range 2.3–4.0). Likewise, even though there are 93 BCTs included in the taxonomy, the number of unique BCTs coded in the intervention arms ranged from 15.3 to 26.7 with far fewer BCTs coded for the control arms (range 2.8–5.3).
FIGURE 1.
Mean number of domains (A) and behavior change techniques (BCTs) (B) among intervention and control arms, by coding source
There was loss of domains and BCTs across both intervention and control arms based on source with the greatest loss occurring between full review and outcome papers (Figure 1). Compared to the full review, the average percent loss of domains as counted across the six intervention arms was 33.9% for outcome papers and 11.0% for design papers. When the coding results of both the outcome and design papers were combined, the loss was slightly attenuated to 8.5% compared to the full review. The loss of the number of BCTs identified was greater than the loss of domains. Compared to the full review, the average percent loss of BCTs across the six intervention arms was 42.7%, 21.3%, and 18.4% for outcome papers, design papers, and combined coding of design and outcome papers, respectively. Loss of domains and BCTs also occurred among control arms, but there was little differentiation in the loss that occurred among design and outcome papers, as compared to full review.
When examined by study, the pattern of loss for domains was variable across the six intervention arms (Figure 2). There was zero percent loss between the design paper and the full review for four intervention arms (CHOICES, SNAP SC, SNAP LC, and CITY CP), 30.8% loss for CITY PC and 33.3% loss for SMART. The percent loss of domains between the outcome paper and the full review was much greater than for the design papers and the full review, with the exception of the intervention arm in SMART. The percent loss between the outcome paper and full review was 22.2% (CITY CP), 25% (SMART), 25% (SNAP SC), 38.5% (CHOICES), 41.7% (SNAP LC) and 46.2% (CITY PC).
FIGURE 2.
Number of domains among intervention arms, by coding source and study
Table 2 displays the domains having high and low emphasis according to the AHP for each of the six intervention arms. “Feedback and Monitoring” was a domain emphasized in all six intervention arms and “Goals and Planning” was emphasized in five of the six intervention arms. The next most commonly emphasized domain was “Social Support” with half of the intervention arms emphasizing this domain. A wide range of domains had low emphasis. All domains missing in the design paper, as compared to the full review, were those domains categorized as low emphasis according to the AHP (Table 3). There was no instance in which a domain was missing in the design paper yet highly emphasized according to the AHP. While the majority of domains missing in the outcome paper were also those categorized as low emphasis according to the AHP, there were exceptions (Table 4). SMART was missing a domain in the outcome paper that was categorized as having high emphasis. In addition, SNAP SC, SNAP LC and SMART were each missing one domain in the outcome paper that was rated as having neither high nor low emphasis, that is, these domains had >5% emphasis, but less than 10% (data not shown).
TABLE 2.
Summary of domain emphasis according to the Analytical Hierarchy Process (AHP), by intervention arm
| Intervention arm | High emphasis (≥10% according to AHP) | Low emphasis (<5% according to AHP) | ||
|---|---|---|---|---|
| Domain name | % Emphasis | Domain name | % Emphasis | |
| CHOICES | 2‐Feedback and monitoring | 24.4 | 7‐Associations | 4.9 |
| 1‐Goals and planning | 21.3 | 11‐Regulation | 4.2 | |
| 10‐Reward and threat | 11.0 | 9‐Comparison of outcomes | 4.2 | |
| 3‐Social support | 10.6 | 8‐Repetition and substitution | 4.1 | |
| 5‐Natural consequences | 2.2 | |||
| 12‐Antecedents | 1.6 | |||
| 6‐Comparison of behavior | 1.5 | |||
| 15‐Self‐belief | 1.1 | |||
| CITY PC | 2‐Feedback and monitoring | 22.3 | 10‐Reward and threat | 4.8 |
| 1‐Goals and planning | 20.4 | 5‐Natural consequences | 3.9 | |
| 3‐Social support | 16.6 | 6‐Comparison of behavior | 3.0 | |
| 15‐Self‐belief | 2.6 | |||
| 8‐Repetition and substitution | 2.3 | |||
| 13‐Identity | 2.3 | |||
| 9‐Comparison of outcomes | 2.1 | |||
| 11‐Regulation | 2.0 | |||
| CITY CP | 2‐Feedback and monitoring | 36.0 | 10‐Reward and threat | 3.7 |
| 7‐Associations | 25.4 | 3‐Social support | 2.4 | |
| 12‐Antecedents | 11.9 | 5‐Natural consequences | 2.4 | |
| 9‐Comparison of outcomes | 2.1 | |||
| SMART | 2‐Feedback and monitoring | 22.4 | 7‐Associations | 3.8 |
| 1‐Goals and planning | 20.3 | 5‐Natural consequences | 3.4 | |
| 3‐Social support | 16.1 | 4‐Shaping knowledge | 2.5 | |
| 10‐Reward and threat | 11.2 | 15‐Self‐belief | 2.3 | |
| 12‐Antecedents | 1.9 | |||
| 8‐Repetition and substitution | 1.7 | |||
| SNAP SC | 2‐Feedback and monitoring | 30.5 | 4‐Shaping knowledge | 4.3 |
| 1‐Goals and planning | 23.6 | 7‐Associations | 4.3 | |
| 8‐Repetition and substitution | 10.3 | 15‐Self‐belief | 4.3 | |
| 5‐Natural consequences | 2.7 | |||
| 6‐Comparison of behavior | 2.7 | |||
| 9‐Comparison of outcomes | 1.8 | |||
| 12‐Antecedents | 1.8 | |||
| SNAP LC | 2‐Feedback and monitoring | 30.5 | 4‐Shaping knowledge | 4.3 |
| 1‐Goals and planning | 23.6 | 7‐Associations | 4.3 | |
| 8‐Repetition and substitution | 10.3 | 15‐Self‐belief | 4.3 | |
| 5‐Natural consequences | 2.7 | |||
| 6‐Comparison of behavior | 2.7 | |||
| 9‐Comparison of outcomes | 1.8 | |||
| 12‐Antecedents | 1.8 | |||
TABLE 3.
Domains missing in design papers versus full review, by intervention arm and level of emphasis according to the Analytical Hierarchy Process (AHP)
| Emphasis of domain in AHP | |||
|---|---|---|---|
| Intervention arm | Name of missing domain | High a | Low b |
| Weight gain prevention trials | |||
| CHOICES | 5‐Natural consequences | X | |
| SNAP SC | None missing | ‐‐ | ‐‐ |
| SNAP LC | None missing | ‐‐ | ‐‐ |
| Weight loss trials | |||
| CITY PC | 5‐Natural consequences | X | |
| 6‐Comparison of behavior | X | ||
| 10‐Reward and threat | X | ||
| 13‐Identity | X | ||
| CITY CP | 5‐Natural consequences | X | |
| SMART | 5‐Natural consequences | X | |
| 6‐Comparison of behavior | X | ||
| 8‐Repetition and substitution | X | ||
| 12‐Antecedents | X | ||
Rated as having ≥10% emphasis in the intervention according to the AHP.
Rated as having <5% emphasis in the intervention according to the AHP.
TABLE 4.
Domains missing in outcome papers versus full review, by intervention arm and level of emphasis according to the Analytical Hierarchy Process (AHP)
| Emphasis of domain in AHP | |||
|---|---|---|---|
| Intervention arm | Name of missing domain | High a | Low b |
| Weight gain prevention trials | |||
| CHOICES | 5‐Natural consequences | X | |
| 6‐Comparison of behavior | X | ||
| 7‐Associations | X | ||
| 8‐Repetition and substitution | X | ||
| 12‐Antecedents | X | ||
| SNAP SC | 5‐Natural consequences | X | |
| 6‐ Comparison of behavior | X | ||
| 10‐Reward and threat c | |||
| SNAP LC | 5‐Natural consequences | X | |
| 6‐Comparison of behavior | X | ||
| 7‐Associations | X | ||
| 10‐Reward and threat c | |||
| 12‐Antecedents | X | ||
| Weight loss trials | |||
| CITY PC | 5‐Natural consequences | X | |
| 6‐Comparison of behavior | X | ||
| 8‐Repetition and substitution | X | ||
| 10‐Reward and threat | X | ||
| 11‐Regulation | X | ||
| 13‐Identity | X | ||
| CITY CP | 5‐Natural consequences | X | |
| 10‐Reward and threat | X | ||
| SMART | 6‐Comparison of behavior c | ||
| 10‐Reward and threat | X | ||
| 12‐Antecedents | X | ||
Rated as having ≥10% emphasis in the intervention according to the AHP.
Rated as having <5% emphasis in the intervention according to the AHP.
Rated as having >5% and <10% emphasis in the intervention according to the AHP.
The pattern of loss for BCTs by study is shown in Figure 3. The loss between the full review and the design papers ranged from 42.4% (CHOICES) to 15.6% for both SNAP SC and SNAP LC. Unexpectedly, there was one intervention arm (CITY CP) in which more BCTs were coded in the design paper than the full review, resulting in a negative percent change (−13.3%). The pattern of loss between the full review and the outcome paper also showed a large range of between 63.6% loss for CHOICES and 20% loss for the CITY CP intervention arm. Similar to domains, there was no instance in which a BCT was missing in the design paper or outcome paper yet highly emphasized according to the AHP (Electronic Supplemental Material).
FIGURE 3.
Number of behavior change techniques (BCTs) among intervention arms, by coding source and study
4. DISCUSSION
The results of this study suggest that much detail about both the domains and BCTs used in intervention studies is lost in published manuscripts. At the same time, findings from the AHP emphasis analysis suggest the domains and BCTs identified by coding the published papers are picking up the intervention approaches most emphasized in the intervention. Adding this type of analysis to the taxonomy coding process provides reassurance that the overall essence of the intervention is being captured in published papers. However, the ability of other intervention teams to successfully replicate and disseminate an effective intervention using the limited information on domains and BCTs, even with information on the emphasis, is highly unlikely.
This research also shows that only a fraction of potential domains and BCTs were used in these weight‐related interventions. Although Michie 9 identifies 16 domains and 93 BCTs, the review of these four obesity interventions show only 75% of possible domains and 29% of BCTs are used. This finding may suggest the field is not taking advantage of the full repertoire of intervention approaches available, or may suggest the utility of domains and BCTs has some content‐specificity; BCTs appropriate for use in smoking cessation trials may be less appropriate in obesity trials. Likewise, interventionists may be choosing a parsimonious set of BCTs to use based on their perceived effectiveness and to avoid creating an intervention that is overly complicated.
The results of our study may also reflect the possibility that the taxonomy used in the Michie approach 9 is one that has not fully been adopted by behavioral interventionists. With the exception of SMART, the interventions designed for these four EARLY trials did not evolve from a domain/BCT approach; rather, the interventions were mapped onto the Michie BCT taxonomy 9 post hoc. It is possible that interventionists used different terms to describe the strategies they used in their interventions and these terms do not map on well to the BCT taxonomy.
In order to speed up the rate of discovery and dissemination of effective weight‐related interventions, intervention details must be more easily accessible to other interventionists and behavioral scientists in ways that move beyond publication in scientific journals. There are examples of repositories of interventions that have been shown to be effective and the content of the intervention made available on public websites. 7 For example, the Evidence‐Based Cancer Control Programs of the National Cancer Institute (http://ebccp.cancercontrol.cancer.gov) identify core elements of interventions found to be effective and provide direct links to the intervention developer, intervention protocol and materials, and training and technical support for implementing the intervention. 7 This type of access is typically available only for interventions that have been rigorously studied and found to be effective.
Other mechanisms for making intervention details more widely available are needed for all stages of intervention development. 26 Some journals are offering a Supplemental Material section, often available only online with the electronic version of the article, as an option for providing more detailed information. In addition, there is a push to expand the CONSORT guidelines by adding a checklist of details that should be included on aspects of the intervention. Both TIDieR (Template for Intervention Description and Replication) 27 and the CONSORT‐SPI 2018 extension 28 call for additional details to be provided when reporting on any intervention study. Montgomery et al. (2018) suggested that in addition to funders and publishers, there are other stakeholders who would find the revised CONSORT diagram and checklist helpful, including the study designers as they consider ways to ensure the quality of their studies, as well as policy makers and practitioners.
The limitations of this research include that this exercise was conducted with just four weight‐related intervention trials. In addition, the focus of this research was on the intervention as designed, not as actually delivered; no attempt was made to incorporate available process data to document the actual domains or BCTs delivered to participants. The discrepancies that exist between the domains and BCTs identified in the intervention protocol versus the outcome paper may, in part, reflect the gap between what was intended and the intervention that was actually delivered. However, to the extent that most design and outcome papers include a description of the intervention as designed, these findings suggest that an incomplete picture is being portrayed.
One of the strengths of the study is the use of four intervention trials that had complete sets of intervention protocol, a published design paper, and primary outcome paper. In addition, the use of interventions from the EARLY consortium also provides some standardization across the interventions with regard to the length, intensity, and intent of the intervention, allowing a more direct comparisons between trials with regard to how they are described. Previous research attempting to examine the correspondence between BCTs and domains cited in outcome papers as compared to intervention protocol have noted the low response to requests for intervention protocol manuals and the actual numbers of manuals received. This poor response limits the generalizability of the findings and also suggests that intervention manual of procedures may not be commonly prepared or in a format that allows easy sharing with other intervention teams. 12
This is one of the first studies to examine the criterion validity of the description of obesity interventions in the published research as compared to the intervention protocol using the Michie taxonomy and coding process. In addition, the four EARLY studies provide unique insights into the reporting of domains and BCTs in obesity interventions with a strong technology focus. To our knowledge, this is the first study, across any behavioral content area, to examine if the intended emphasis of the domains and BCTs, as reported by study interventionists, was related to the domains and BCTs reported in the design and primary outcome paper. The findings provide some reassurance that those domains and BCTs considered especially important to the interventionists are reflected in the primary outcome and intervention design papers.
5. CONCLUSION
In order to make progress on creating and disseminating effective interventions to reduce obesity risk as well as other behaviorally‐related health risks, the field needs to recognize and appreciate the importance of describing interventions in detail and in being more transparent about intervention design. Descriptions of interventions in published manuscripts need to be recognized for what they are: brief overviews of the general intervention approaches used. While the Michie taxonomy 9 attempts to document one important element of the intervention—the domains and BCTs used as intervention strategies—detail is not available on dose, delivery method, intended recipients, or the specific content of the strategies used. In addition, no information on the overall intervention design process is provided in the taxonomy including the determinants that were targeted by the intervention strategies; how those determinants and related intervention approaches and strategies were chosen to address the needs of the target population; and the role of various stakeholders involved in designing, delivering, and evaluating the intervention. Without providing details on the intervention and the intervention design process used to create an intervention, attempts to refine, adapt, and disseminate interventions are stymied. 5 , 6 , 7 , 29
AUTHOR CONTRIBUTIONS
Leslie A Lytle and Deborah F Tate were the grant holders and developed the idea for the research, providing oversight for the study throughout, including the design and implementation of the coding process. Heather M Wasser conducted the data analyses and created all figures and tables for the manuscript. All authors contributed to writing, commenting on, and approving the final manuscript.
CONFLICT OF INTEREST
DFT reports being a member of the Scientific Advisory Board for Weight Watchers International. The other authors declare no conflict of interest.
CLINICAL TRIAL REGISTRATION
All of the trials are registered as clinical trials. ClinicalTrials.gov identifiers are as follows: NCT01134783 (CHOICES), NCT01092364 (CITY), NCT01200459 (SMART), NCT01183689 (SNAP).
ACKNOWLEDGMENTS
This work was supported by the National Heart, Lung, and Blood Institute research grants HL122144 (Tate/Belle), HL096767 (Lytle), HL096715 (Patrick/Godino), HL096720 (Svetkey/Lin), HL090864 (Wing/Tate) and the National Institute of Diabetes and Digestive and Kidney Diseases DK111793 (Wasser)
Lytle LA, Wasser HM, Godino J, Lin P‐H, Tate DF. Identifying the behavior change techniques used in obesity interventions: an example from the Early trials. Obes Sci Pract. 2023;9(2):179‐189. 10.1002/osp4.633
REFERENCES
- 1. Tronieri JS, Wadden TA, Chao AM, Tsai AG. Primary care interventions for obesity: review of the evidence. Curr Obes Rep. 2019;8(2):128‐136. 10.1007/s13679-019-00341-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Hutchesson MJ, Rollo ME, Krukowski R, et al. eHealth interventions for the prevention and treatment of overweight and obesity in adults: a systematic review with meta‐analysis. Obes Rev. 2015;16(5):376‐392. 10.1111/obr.12268 [DOI] [PubMed] [Google Scholar]
- 3. Lau Y, Chee DGH, Chow XP, Cheng LJ, Wong SN. Personalised eHealth interventions in adults with overweight and obesity: a systematic review and meta‐analysis of randomised controlled trials. Prev Med. 2020;132:106001. 10.1016/j.ypmed.2020.106001 [DOI] [PubMed] [Google Scholar]
- 4. Glasgow RE, Lichtenstein E, Marcus AC. Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy‐to‐effectiveness transition. Am J Public Health. 2003;93(8):1261‐1267. 10.2105/ajph.93.8.1261 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. McCrabb S, Lane C, Hall A, et al. Scaling‐up evidence‐based obesity interventions: a systematic review assessing intervention adaptations and effectiveness and quantifying the scale‐up penalty. Obes Rev. 2019;20(7):964‐982. 10.1111/obr.12845 [DOI] [PubMed] [Google Scholar]
- 6. Wiltsey Stirman S, Baumann AA, Miller CJ. The FRAME: an expanded framework for reporting adaptations and modifications to evidence‐based interventions. Implement Sci. 2019;14(1):58. 10.1186/s13012-019-0898-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Lytle LA. Designing Interventions to Promote Community Health: A Multilevel, Stepwise Approach. American Psychological Assocation; 2022. [Google Scholar]
- 8. Abraham C, Michie S. A taxonomy of behavior change techniques used in interventions. Health Psychol 2008;27(3):379‐387. 10.1037/0278-6133.27.3.379 [DOI] [PubMed] [Google Scholar]
- 9. Michie S, Richardson M, Johnston M, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med. 2013;46(1):81‐95. 10.1007/s12160-013-9486-6 [DOI] [PubMed] [Google Scholar]
- 10. Michie S, Ashford S, Sniehotta FF, Dombrowski SU, Bishop A, French DP. A refined taxonomy of behaviour change techniques to help people change their physical activity and healthy eating behaviours: the CALO‐RE taxonomy. Psychol Health. 2011;26(11):1479‐1498. 10.1080/08870446.2010.540664 [DOI] [PubMed] [Google Scholar]
- 11. Tate DF, Lytle L, Polzien K, et al. Deconstructing weight management interventions for young adults: looking inside the black box of the EARLY consortium trials. Obesity. 2019;27(7):1085‐1098. 10.1002/oby.22506 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Lorencatto F, West R, Stavri Z, Michie S. How well is intervention content described in published reports of smoking cessation interventions? Nicotine Tob Res. 2013;15(7):1273‐1282. 10.1093/ntr/nts266 [DOI] [PubMed] [Google Scholar]
- 13. Cradock KA, ÓLaighin G, Finucane FM, Gainforth HL, Quinlan LR, Ginis KAM. Behaviour change techniques targeting both diet and physical activity in type 2 diabetes: a systematic review and meta‐analysis. Int J Behav Nutr Phys Activ. 2017;14(1):18. 10.1186/s12966-016-0436-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. de Bruin M, Black N, Javornik N, et al. Underreporting of the active content of behavioural interventions: a systematic review and meta‐analysis of randomised trials of smoking cessation interventions. Health Psychol Rev. 2021;15(2):195‐213. [DOI] [PubMed] [Google Scholar]
- 15. Lytle LA, Svetkey LP, Patrick K, et al. The EARLY trials: a consortium of studies targeting weight control in young adults. Transl Behav Med. 2014;4(3):304‐313. 10.1007/s13142-014-0252-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Batch BC, Tyson C, Bagwell J, et al. Weight loss intervention for young adults using mobile technology: design and rationale of a randomized controlled trial ‐ cell Phone Intervention for You (CITY). Contemp Clin Trials. 2014;37(2):333‐341. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Patrick K, Marshall SJ, Davila EP, et al. Design and implementation of a randomized controlled social and mobile weight loss trial for young adults (project SMART). Contemp Clin Trials. 2014;37(1):10‐18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Lytle LA, Moe SG, Nanney MS, Laska MN, Linde JA. Designing a weight gain prevention trial for young adults: the CHOICES study. Am J Health Educ. 2014;45(2):67‐75. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Wing RR, Tate D, Espeland M, et al. Weight gain prevention in young adults: design of the study of novel approaches to weight gain prevention (SNAP) randomized controlled trial. BMC Publ Health. 2013;13:300. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Wing RR, Tate DF, Espeland MA, et al. Innovative self‐regulation strategies to reduce weight gain in young adults: the study of novel approaches to weight gain prevention (SNAP) randomized clinical trial. JAMA Intern Med. 2016;176(6):755‐762. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Lytle LA, Laska MN, Linde JA, et al. Weight‐gain reduction among 2‐year college students: the CHOICES RCT. Am J Prev Med. 2017;52(2):183‐191. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Svetkey LP, Batch BC, Lin P.‐H, et al. Cell phone intervention for you (CITY): a randomized, controlled trial of behavioral weight loss intervention for young adults using mobile technology. Obesity. 2015;23(11):2133‐2141. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Godino JG, Merchant G, Norman GJ, et al. Using social and mobile tools for weight loss in overweight and obese young adults (Project SMART): a 2 year, parallel‐group, randomised, controlled trial. Lancet Diabetes Endocrinol. 2016;4(9):747‐755. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Welcome ‐ BCT taxonomy training [internet]. [cited 2020 Dec 21]. Available from:
- 25. Saaty TL. The Analytic Hierarchy Process: Planning, Priority Setting, Resource Allocation. McGraw‐Hill International Book Co.; 1980.c1980 [Google Scholar]
- 26. Michie S, West R, Sheals K, Godinho CA. Evaluating the effectiveness of behavior change techniques in health‐related behavior: a scoping review of methods used. Transl Behav Med. 2018;8(2):212‐224. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Hoffmann TC, Glasziou PP, Boutron I, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348:g1687. [DOI] [PubMed] [Google Scholar]
- 28. Montgomery P, Grant S, Mayo‐Wilson E, et al. Reporting randomised trials of social and psychological interventions: the CONSORT‐SPI 2018 Extension. Trials. 2018;19(1):407. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Eldredge LKB, Markham CM, Ruiter RAC, Fernández ME, Kok G, Parcel GS. Planning health promotion Programs: an intervention mapping approach (Jossey‐Bass public health).