Abstract
Randomized controlled trials (RCTs) are considered the gold standard used to assess the efficacy of treatment. While a well implemented RCT can produce an unbiased estimate of the relative difference between treatment groups, the generalizability of these findings may be limited. Specific threats to the external validity include treatment preference. The purposes of this study were to: (i) assess whether receiving one's treatment preference was associated with weight loss and retention and (ii) whether receiving one's treatment preference modified the relationship between the treatments and weight loss. Treatment preference was assessed in 250 subjects prior to but independent of randomization into either low-carbohydrate or low-fat diets. Treatment preference was a predictor of weight loss (P = 0.002) but not retention (P = 0.90). Participants who received their preference lost less weight (–7.7 kg, 95% confidence interval (CI): –9.3 to –6.1) than participants who did not receive their preference (–9.7 kg, 95% CI: –11.4 to –8.1) and participants who did not report a strong preference at baseline (–11.2 kg, 95% CI: –12.6 to –9.7) (P = 0.04 and P = 0.0004, respectively). Treatment preference did not modify the effect of the treatment on weight loss. Contrary to conceptual predictions, this study failed to identify an interaction between treatment preference and weight loss in the setting of a randomized trial. Until treatment preference effects are definitively ruled out in this domain, future studies might consider stratifying their randomization procedure by treatment preference rather than excluding participants with strong treatment preferences.
Introduction
Randomized controlled trials (RCTs) are considered the gold standard used to assess the efficacy of treatment. Despite an RCT's ability to produce an unbiased estimate of the relative diference between treatment groups, its external validity has been called into question (1). Specific threats to the external validity (i.e., generalizability of results to other participants and settings) include doctor–patient relationships, placebo effects, and patient preference (1). Blinded treatment allocation and placebo control have been adopted widely into research designs to mitigate the first two threats but it is less clear how to handle participant preference in the context of random assignment.
Not considering these preferences may lead to an inaccurate assessment of relative efficacy in a RCT. According to behavioral choice theory, depending on the type of behavior that is being modified, participants who receive their preferred treatment should have better outcomes than those who receive their nonpreferred treatment (2,3). Although random assignment should evenly distribute preference across the treatment groups, if it does not and if preference interacts with treatment, then the treatment's effect estimate will be biased (3). Researchers wishing to account for this threat to internal validity have generally excluded participants from studies who expressed strong treatment preferences; however, this choice compromises external validity (1).
Despite the potential for treatment preference to bias the results of an RCT, few studies have explored the role of preference in the context of obesity treatment trials. Moreover the effect of treatment preference is unclear because of conflicting data between studies. Some studies observed no effect of preference on weight loss (4,5), while others observed less (6) weight loss in participants who received their preferred treatment relative to participants who did not receive their preferred treatment. These studies were characterized by short duration (<6 months) (4,5) and small sample sizes (n = 12–75 subjects) (4,5). Using data obtained from a large, 2-year RCT of low-carbohydrate and low-fat diets on adults (7), the purposes of this secondary study were to: (i) assess whether receiving one's treatment preference was associated with weight loss and retention (main effect of preference category) and (ii) whether receiving one's treatment preference modified the relationship between the treatments and weight loss (preference category × treatment interaction).
Methods and Procedures
Data were obtained from a 2-year, multi-center randomized evaluation of the effects of 2-year treatment with a low-carbohydrate or low-fat diet, each of which was combined with a comprehensive lifestyle modification program. The design and results from this original study have been described previously (7). Diet preference data, which are reported here, form a secondary analysis of this main study. Participants were recruited, enrolled, and followed from March 2003 to June 2007. After a scripted phone screen, eligible patients attended a behavioral evaluation during which the study's purpose, requirements, and treatments were fully discussed, eligibility was confirmed and written informed consent was obtained. The study was approved by the institutional review boards of each of the three participating institutions.
Among the 307 study subjects, baseline preference data were not collected from 57 subjects (18.6% of the full sample), leaving 250 participants for the current study. There were no differences between the subset with preference data (n = 250) and the subset with no preference data (n = 57) in terms of gender (P = 0.66), treatment assignment (P = 0.20), race (P = 0.46), age (P = 0.44), BMI (P = 0.09), or weight (P = 0.06) at baseline.
Treatment preference
Treatment preference was assessed prior to, but independent of, randomization into a low-carbohydrate or low-fat diet. As described previously (7) approximately half of the participants were assigned to a low-carbohydrate diet, which limited carbohydrate intake with unrestricted consumption of fat and protein. During the first 12 weeks of treatment, participants assigned to the low-carbohydrate diet were instructed to limit carbohydrate intake to 20 g/day in the form of low-glycemic index vegetables. After the first 12 weeks, participants gradually increased carbohydrate intake (5 g/day per week) by consuming more vegetables, a limited amount of fruits, and eventually small quantities of whole grains and dairy products, until a stable and desired weight was achieved. The remaining participants were assigned to consume a low-fat diet, which consisted of limiting energy intake to 1,200 –1,500 kcal/day for women and 1,500 –1800 kcal/day for men, with ∼55 percent of calories from carbohydrate, 30 percent from fat, and 15 percent from protein. All participants received comprehensive group behavioral treatment (8,9) weekly for 20 weeks, every other week for 20 weeks and then every other month for the remainder of the 2-year study period. Body weight was measured at each treatment visit on calibrated scales while participants wore light clothing and no shoes. Height was measured by a stadiometer at baseline. Assessors were not blinded to treatment condition.
Both treatments were described to participants who were told that they would not be able to choose their diet (low-carbohydrate or low-fat) but were instructed to state their preference anyway. Preference was assessed using a 6-point Likert questionnaire (1 = strongly prefer low-carbohydrate, 2 = prefer low-carbohydrate, 3 = slightly prefer low-carbohydrate, 4 = slightly prefer low-fat, 5 = prefer low-fat, 6 = strongly prefer low-fat). In addition, space was provided on the questionnaire for participants to qualitatively explain their preference selection.
After participants were randomly assigned to either the low-carbohydrate or low-fat condition, treatment preference scores for the low-fat group were revalenced by subtracting each score from 7. After revalencing, scores ranged from 1 (“received strong preference” to 6 (“did not receive strong preference”) (Figure 1). For example, a participant who was randomly assigned to the low-fat condition who had circled a 1 on the Likert scale (“strongly prefer low-carbohydrate diet”) would receive a 6 (“did not receive strong preference”) after revalencing. If that same individual were randomly assigned to the low-carbohydrate group, they would be assigned a 1 (“received strong preference) after revalencing. This new scoring placed both groups' (low-carbohydrate and low-fat) diet preferences on the same scale and effectively captures whether a subject received their preferred treatment. Using this scoring, participants were trichotomized into one of the following three categories: those who received their preference (treatment congruence) (score of 1 or 2), those who did not report a strong preference (score of 3 or 4), and those who did not receive their preference (treatment incongruence) (score of 5 or 6) (Figure 1). This choice was guided by the qualitative rationale respondents submitted when submitting their preference, and it refected a compromise between utilizing all available information (i.e., maintaining all six categories and making 15 statistical comparisons) and simplicity (i.e., dichotomization and making one statistical comparison). In addition, collapsing categories into trichotomies effectively doubled the sample size of each of the preference groups, which increased power to detect preference effects.
Figure 1.
The trichotomy of revalenced diet preference scores.
Analysis
Prior to addressing the two specific aims of this study, diet preference group equivalence was assessed at baseline using χ2 tests for categorical variables and analysis of variance for continuous outcomes. Differences in weight loss by treatment group were also assessed at each main time point (6 months, 12 months, and 24 months) using the longitudinal models discussed below.
To address specific aim 1, whether receiving one's treatment preference was associated with weight loss and retention, a repeated measures mixed-effects model was used to assess change, since weight was assessed at various time points (baseline, 6 months, 12 months, and 24 months). This analysis included data on all participants, regardless of drop out. The longitudinal mixed-effects approach to handling missing data (i.e., creating person-specific trajectories determined by the observed outcomes and covariates in the model) relaxes the missing completely at random assumption required of more traditional approaches to missing data (i.e., baseline- or last observation carried forward) (9).
To account for the nonindependence of observations nested within study participants, the residuals of all 250 study participants (regardless of early dropout) were allowed to covary across measurement occasions. Akaike's information criterion was used to determine the most appropriate way to address the nonindependence of observations within individuals. Different variance–covariance matrices were tested. The model with the lowest Akaike's information criterion (i.e., indicating better model ft (10)), which was the unstructured variance–covariance matrix, was selected. In addition, because intact groups of participants received group behavioral treatment (7), group was included as a random effect in all analyses to account for this clustering of subjects (data not shown). Sensitivity analyses were performed with absolute weight as an outcome. Results obtained on changes in weight and absolute weight were similar (same direction and significance). The results obtained from the analyses on change are reported herein.
The key parameter of interest to address specific aim 1 was treatment preference category (i.e., treatment congruent, treatment incongruent, and those who did not receive their preference). The following fixed effects were controlled for in the model (data not shown): time, treatment (i.e., low-carbohydrate and low-fat), a time × treatment interaction, baseline weight, and research site. An interaction between treatment preference and time was included to assess whether the relationship between treatment preference and weight loss changed over time. Chi square tests were used to assess whether there was differential attrition across treatment preference categories at 6, 12, and 24 months.
To address specific aim 2, whether receiving one's treatment preference modified the relationship between the treatments and weight loss, an interaction term between treatment preference and treatment was included in the repeated measures model specified in specific aim 1. All analyses were conducted using Proc MIXED in SAS version 9.2 (Cary, NC) with an α level of 0.05 as a barometer of statistical significance.
Results
Participant characteristics
Participant characteristics at baseline are reported in Table 1. The 250 participants were predominantly female (67.2%) with a mean age of 45.7 ± 9.8 years, BMI of 36.2 ± 3.6 kg/m2, and weight of 104.2 ± 14.8 kg. Slightly more than half of the sample (145, 58.0%) preferred the low-carbohydrate diet over the low-fat diet at baseline. Baseline treatment preference was not predicted by sex (P = 0.97), race (P = 0.18), age (0.14), baseline weight (P = 0.39) or BMI (P = 0.54).
Table 1. Participant characteristics at baseline.
| Variable | |
|---|---|
| Sex, n (%) | |
| Male | 82 (32.8) |
| Female | 168 (67.2) |
| Race, n (%) | |
| White | 187 (74.8) |
| African American | 57 (22.8) |
| Other | 6 (2.4) |
| Age (years) | 45.7 ± 9.8 |
| BMI (kg/m2) | 36.2 ± 3.6 |
| Weight (kg) | 104.2 ± 14.8 |
| Treatment | |
| Low carbohydrate | 129 (51.6) |
| Low fat | 121 (48.4) |
| Diet preference, n (%) | |
| Strongly prefer low carbohydrate | 49 (19.6) |
| Prefer low carbohydrate | 45 (18.0) |
| Slightly prefer low carbohydrate | 51 (20.4) |
| Slightly prefer low fat | 47 (18.8) |
| Prefer low fat | 33 (13.2) |
| Strongly prefer low fat | 25 (10.0) |
N = 250.
Participant characteristics by preference trichotomy (treatment congruent, those who did not report a strong preference, and treatment incongruent) are reported in Table 2. As expected, roughly half of the participants that reported more than a slight preference at baseline (152, 60.8%), received their preference (treatment congruent) (74, 48.7%) and half (78, 51.3%) did not receive their preference (treatment incongruent). The remaining participants (98, 39.2%) reported only a slight preference at baseline (Table 2). With the exception of treatment assignment (P < 0.01), which was included as a covariate, there were no statistically significant differences between the three preference groups on any of the baseline demographic variables. Since more subjects preferred the low-carbohydrate (n = 145, 58.0%) over the low-fat (n = 105, 42.0%) diet at baseline, more subjects that were randomly assigned to the low-carbohydrate diet received their preference.
Table 2. Participant characteristics at baseline by treatment preference group.
| Preference congruent | Did not report a strong preference | Preference incongruent | |
|---|---|---|---|
| n | 74 (29.6) | 98 (39.2) | 78 (31.2) |
| Sex, n (%) | |||
| Male | 23 (31.1) | 34 (34.7) | 25 (32.1) |
| Female | 51 (68.9) | 64 (65.3) | 53 (68.0) |
| Race, n (%) | |||
| White | 51 (68.9) | 76 (77.6) | 60 (76.9) |
| African American | 22 (29.7) | 19 (19.4) | 16 (20.5) |
| Other | 1 (1.4) | 3 (3.1) | 2 (2.6) |
| Treatment | |||
| Low Carbohydrate | 49 (66.2) | 47 (48.0) | 33 (42.3) |
| Low Fat | 25 (33.8) | 51 (52.0) | 45 (57.7) |
| Age (y) | 47.1 ± 9.9 | 45.4 ± 9.6 | 44.7 ± 9.8 |
| BMI (kg/m2) | 35.7 ± 3.6 | 36.6 ± 3.8 | 36.2 ± 3.3 |
| Weight (kg) | 101.8 ± 12.4 | 106.1 ± 15.5 | 104.1 ± 15.8 |
Between-group differences assessed using ANOVA for continuous and χ2 tests for categorical outcomes. Similar results were obtained when participants were dichotomized by treatment preference.
Groups did not differ on any variable at baseline except for treatment (P < 0.01). N = 250.
Weight loss and treatment
The mean ± s.d. weight loss was 11.9 ± 6.8 kg at 6 months, 11.2 ± 8.6 kg at 12 months, and 7.3 ± 9.9 kg at 24 months. There were no statistically significant differences between the two treatment groups (low-carbohydrate and low-fat) in weight loss at any time during the study.
Weight loss and treatment preference
After treatment preference was trichotomized into treatment congruent, those who did not report a strong preference, and treatment incongruent, there was a statistically significant main effect of receiving one's treatment preference on weight loss (P = 0.002). The treatment congruent group lost less weight (–7.7 kg, 95% confidence interval (CI): –9.3 to –6.1) than the treatment incongruent group (–9.7 kg, 95% CI: –11.4 to –8.1) and participants who did not report a strong preference (–11.2 kg, 95% CI: –12.6 to –9.7) (P = 0.04 and P = 0.0004, respectively). There was no difference (P = 0.14) in 24 month weight loss between the treatment incongruent group and participants who did not report a strong preference. A preference × treatment interaction was included to assess whether the treatment was more or less effective for different preference groups. This interaction was also not statistically significant (P = 0.85) and was removed from the analysis. Similarly, the preference × time interaction was not statistically significant (P = 0.62) indicating that the effect of treatment preference on weight loss persisted over the duration of the trial (Figure 2).
Figure 2.
Change in weight (kg) by treatment preference group (those who received their preference, preference congruent, those who did report a strong preference, and those who did not receive their preference, preference incongruent). Data are from a repeated measures mixed effects model that included fixed effects for time, treatment (i.e., low-carbohydrate and low-fat), a time × treatment interaction, baseline weight, preference category (i.e., received preference, did not report a strong preference, and did not receive preference), and research site. A random effect was included for participant cohort. Over the course of the 24 month study, the preference congruent group lost less weight than the preference incongruent group (P = 0.04), and the group that did not report a strong preference (P = 0.0004). There was no difference in mean weight loss between the group that did not report a strong preference and the preference incongruent group (P = 0.14). The effect of preference on weight loss persisted across the 24 month study (the preference × time interaction was not statistically significant).
Treatment preference and retention
Of the 250 participants assessed at baseline, 216 (86.4%), 185 (74.0%), and 157 (62.8%) participants remained in the study at 6, 12, and 24 months, respectively. There was no relationship between receiving one's treatment preference and retention over the duration of the study (P = 0.90). Similar rates of retention were observed in all three groups: those who received their preference (64.9%), those who did not report a strong preference (62.2%), and those who did not receive their preference (61.5%) across the 2 year study.
Discussion
In the current study, we observed that while all groups lost weight, the treatment congruent group lost less weight than the treatment incongruent group or the group who did not report a strong preference at baseline. This effect was observed throughout the duration of the study (a preference × time interaction was not statistically significant). There were no differences between preference groups with respect to attrition. Previous studies on treatment preference in the context of weight loss studies were mixed (4–6). Burke et al. (6), observed greater weight loss in participants who did not receive their preferred treatment. Other studies that also dichotomized preference found no effect of preference on weight loss (4,5).
An explanation for these discrepant findings (4,5) may be the oversimplification of preference (i.e., received preference vs. did not receive preference). Subjects who have strong preferences may behave differently than subjects who expressed no strong preferences at baseline. Treating preference as a trichotomy may clarify this important qualitative distinction between preference and no preference and was a statistically significant predictor of weight loss.
In the current study, that the treatment congruent group loss less weight than the treatment incongruent group or the group who did not report a strong preference at baseline may seem counter intuitive. According to behavioral choice theory, participants who received their preference would have better outcomes than those who received their nonpreferred treatment (2). This theory; however, assumes that the behavior being modified is a deficit behavior. In the case of a weight loss study, where the desired outcomes are weight loss and retention, the behavior that participants desire to be changed is a dependent or preferred behavior (i.e., a preferred way of eating). In the context of a weight loss study, dietary modifications, such as low-carbohydrate or low-fat diets are offered to participants as alternative reinforcers. Whether participants in the current study made their preference selection based on their preferred way of eating or as an alternative to their preferred way of eating is unknown.
In the current study, participants were asked to qualitatively explain their preference selection; however, they were not asked whether their selection was based on their preferred diet or a preferred alternative to their dependent behavior (i.e., what they already enjoyed eating). It is, therefore, unknown whether participants who received their preference lost less weight because they selected a diet that was consistent with what they enjoyed eating. The qualitative data suggest that participants selected their preferred or desired diet, which may explain why participants in this group lost the least amount of weight. For example, the participants that provided the following qualitative explanations for their preference, “Because I am a carbohydrate addict,” “Because I am the BBQ King,” preferred the low-fat and low-carbohydrate diets, respectively. Future studies of preference may want to confirm this by explicitly asking subjects whether their preference was based on a dependent behavior or a preferred alternative.
Alternative explanations for the observed relationship between preference and weight loss may also be considered. For example, participants who received their preference may have greater expectations for it to produce weight loss (6). Alternatively, participants who did not express a strong preference at baseline or those who do not receive their preference may internalize the responsibility of weight loss. This internalization was refected in the qualitative data collected when the respondents submitted their diet preference (i.e., “No preference because I was willing to give either diet 100%.”).
In the current study, receiving one's treatment preference was associated with differential weight loss; however, it did not modify the effect of the treatments (low-carbohydrate and low-calorie diet) on weight loss, which was consistent with previous studies (4–6). While more participants who received the low-carbohydrate diet received their preference (38.0 vs. 20.7%), the groups were fairly comparable with respect to preference. A greater threat to validity may exist if participants widely preferred one treatment over another treatment, which would result in an imbalance in receiving a preferred treatment across groups.
This study had several strengths including the duration (24 months), multi-center design (three sites), sample size (N = 250), and an a priori analytic plan to assess preference effects. In addition, the two treatments (low-carbohydrate and low-fat diets) produced equal weight loss between groups. This allowed a more sensitive assessment of the relationship between preference and weight loss without treatment as a confounder. Participants were also approximately evenly split into preference congruent (29.6%), did not report a strong preference (39.2%), and preference incongruent (31.2%) groups. Finally, our assessment of treatment preference using a 6-item Likert scale allowed us to build on the work of previous studies in the area of weight loss and treatment preference (4–6). We chose to trichotomize treatment preference, which refected a compromise between utilizing all available information (i.e., maintaining all 6 categories) and simplicity (i.e., dichotomization).
A limitation, which is consistent with most studies of weight loss, is the under representation of male participants. Attrition may be another limitation. Of the 250 participants assessed at baseline, 216 (86.4%), 185 (74.0%), and 157 (62.8%) participants remained in the study at 6 months, 1 year and 2 years, respectively.
In summary, in this large, 2-year study of weight loss, receiving one's treatment preference was a predictor of successful weight loss. While all groups lost weight, participants who received their preference lost less weight than participants who either did not receive their preference or who did not report a strong preference at baseline. This effect persisted across the duration of the study. Receiving one's treatment preference did not modify the effect of the treatment on weight loss. Future studies should assess treatment preference at baseline and explore whether preference modifies the effect of the treatment on the main outcomes. In addition, to help ensure that treatment preference is equally distributed between groups, when the external validity of results is a goal, studies should stratify the randomization procedure by preference as an alternative to excluding participants with strong treatment preferences.
Acknowledgments
This study was made possible by: National Institutes of Health Grant AT1103 (primary funding source). ClinicalTrials.gov Identifier: NCT00143936
Footnotes
Disclosure: G.D.F. serves on the following scientific advisory boards: Amylin, Arena, Con Agra Foods, and Nutrisystem. He has received research grants from Orexigen and the Almond Board of California and has served as a consultant to Orexigen, Nutrisytem and GlaxoSmithKline. H.R.W. serves on the following scientific advisory boards: Arena, Wellspring, Pfizer. J.O.H. serves on the following scientific advisory boards: Nutrisystem, General Mills, Merck, Novo-Nordisk. S.K. is the director of the Veronica Atkins Center for Excellence in Obesity Medicine at Washington University School of Medicine, which provides unrestricted funds for community outreach and obesity-related activities.
References
- 1.Rothwell PM. External validity of randomised controlled trials: “to whom do the results of this trial apply?”. Lancet. 2005;365:82–93. doi: 10.1016/S0140-6736(04)17670-8. [DOI] [PubMed] [Google Scholar]
- 2.Epstein LH. Role of behavior theory in behavioral medicine. J Consult Clin Psychol. 1992;60:493–498. doi: 10.1037//0022-006x.60.4.493. [DOI] [PubMed] [Google Scholar]
- 3.Halpern SD. Evaluating preference effects in partially unblinded, randomized clinical trials. J Clin Epidemiol. 2003;56:109–115. doi: 10.1016/s0895-4356(02)00598-x. [DOI] [PubMed] [Google Scholar]
- 4.Murray DC. Preferred versus nonpreferred treatment and self-control training versus determination raising as treatments of obesity: a pilot study. Psychol Rep. 1976;38:191–198. doi: 10.2466/pr0.1976.38.1.191. [DOI] [PubMed] [Google Scholar]
- 5.Renjilian DA, Perri MG, Nezu AM, et al. Individual versus group therapy for obesity: effects of matching participants to their treatment preferences. J Consult Clin Psychol. 2001;69:717–721. [PubMed] [Google Scholar]
- 6.Burke LE, Warziski M, Styn MA, et al. A randomized clinical trial of a standard versus vegetarian diet for weight loss: the impact of treatment preference. Int J Obes (Lond) 2008;32:166–176. doi: 10.1038/sj.ijo.0803706. [DOI] [PubMed] [Google Scholar]
- 7.Foster GD, Wyatt HR, Hill JO, et al. Weight and metabolic outcomes after 2 years on a low-carbohydrate versus low-fat diet: a randomized trial. Ann Intern Med. 2010;153:147–157. doi: 10.1059/0003-4819-153-3-201008030-00005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Foster GD, Makris AP, Bailer BA. Using behavioral approaches to treat obesity. Am J Clin Nutr. 2005;82(Suppl):230S–235S. doi: 10.1093/ajcn/82.1.230S. [DOI] [PubMed] [Google Scholar]
- 9.Wadden TA, Butryn ML, Wilson C. Lifestyle modification for the management of obesity. Gastroenterology. 2007;132:2226–2238. doi: 10.1053/j.gastro.2007.03.051. [DOI] [PubMed] [Google Scholar]
- 10.Singer JD, Willett JB. Applied longitudinal data analysis: modeling change and event occurrence. Oxford University Press; New York: p. 2003. [Google Scholar]