Early Weight Loss and Treatment Response: Data From a Lifestyle Change Program in Clinical Practice

Abstract

Introduction

The purpose of this study was to develop and validate a predictive model for the early identification of non-responders to a 12-month lifestyle change program in clinical practice.

Methods

Investigators identified lifestyle change program participants in the electronic health records of a large healthcare delivery system between 2010 and 2017. Non-response was defined as weight gain or no weight loss at 12 months from program initiation (baseline). Logistic regression with percentage weight change at 2–12 weeks from baseline was used as an independent predictor of non-response. Baseline demographics and clinical characteristics were also tested as potential predictors. The authors performed tenfold cross-validation for model assessment and examined model performance with the area under the receiver operating characteristic curve, sensitivity, specificity, and positive and negative predictive values. Analyses were conducted in 2019.

Results

Among 947 program participants, 30% were classified as non-responders at 12-months. The model with the best discrimination of responders from non-responders included weight change at 12 weeks from baseline as the sole predictor (area under the receiver operating characteristic curve, 0.789). Sensitivity and positive predictive value were maximized at 0.56 (specificity and negative predictive value, 0.81 each).

Conclusions

In a cohort of lifestyle change program participants from clinical practice, percentage weight change at 12 weeks from baseline can serve as a single indicator of non-response at the completion of the 12-month program. Clinicians can easily apply this algorithm to identify and assess participants in potential need of adjunctive or alternative therapy to maximize treatment outcomes.

INTRODUCTION

More than 70% of Americans are overweight or have obesity, and an elevated risk for diabetes and cardiovascular disease.¹ Evidence-based lifestyle interventions, such as those aligned with Centers for Disease Control and Prevention (CDC) recommendations, are efficacious in promoting weight loss and reducing cardiometabolic risk.^2–7 Although many participants are successful in achieving clinically meaningful weight loss, variability is typically high.⁸ Successful strategies for promoting weight loss and maintenance remain elusive for numerous individuals,⁹ as heterogeneity in treatment response is poorly understood,^10,11 particularly within clinical practice settings.

Variation in treatment response is multifactorial, including genetic, behavioral, and psychological factors. Nevertheless, these characteristics have not successfully predicted weight outcomes.⁸ Initial weight loss, however, is recognized as a strong predictor of future weight loss.¹² In the landmark Diabetes Prevention Program trial, participants attaining the 7% weight loss goal at 16-week follow-up were three times as likely to meet this goal at 3-year follow-up versus those who did not meet it initially.^8,13–15

An understanding of who is likely to attain weight loss goals is important for local and national efforts to combat the obesity epidemic and to reduce an individual’s cardiometabolic risk. The early identification of lifestyle change program (LCP) participants who are unlikely to respond to treatment presents an important opportunity to assess the need for adjunctive or alternative therapy (e.g., behavioral, pharmacological, or surgical) to maximize treatment outcomes. Such an assessment of potential non-responders can inform the development of novel interventions (e.g., integration of other behavioral or pharmacological treatments) for those that find traditional behavioral lifestyle intervention alone insufficient.

No studies have developed and tested an algorithm to identify potential non-responders to LCPs. The ability to accurately identify non-responders in real time could be used to promote a pragmatic and more patient-centered approach to weight loss and cardiometabolic risk reduction.

The purpose of this study was to develop and validate a predictive model to identify non-responders to a CDC-aligned LCP using electronic health record (EHR) data. Specifically, the model was designed to identify non-responders early, during the first 12 weeks of the program, corresponding to the end of the core phase of the evidence-based curriculum. A range of predictive models with different independent variables were examined. Model performance over time was assessed to determine when, during the first 12 weeks of follow-up, the model could confidently predict non-responder status.

METHODS

This study was conducted at Sutter Health, a community-based and not-for-profit healthcare delivery system in northern California. Sutter Health provides medical services across the continuum of care within 23 state counties, comprising both urban and rural communities. In this study, a Sutter Health EHR research database was used. Sutter Health’s IRB reviewed and approved this study.

The LCP evaluated in this study is aligned with CDC recommendations for diabetes prevention, and is derived from the evidence-based Diabetes Prevention Program lifestyle intervention.¹⁶ The program is a structured 12-month and in-person curriculum with three phases. The general composition of the program is as follows. The initial core phase includes 12 sessions conducted weekly for the first 12 weeks. The transition phase includes four sessions conducted once to twice monthly over an additional 12 weeks. The support phase includes six sessions conducted monthly over the remainder of the year. Per CDC recommendations, the target population of structured LCPs is individuals with clinical prediabetes or an elevated risk for type 2 diabetes; however, at Sutter Health the program is open to patients with a range of cardiometabolic risk factors, including those with diabetes, given that the focus of the program is on weight loss.

Study Sample

Program participants were identified in the EHR database between January 1, 2010 (year of first implementation of the program at Sutter) and December 31, 2017 (end of study database). Participants were required to be aged ≥18 years at the first encounter (baseline) and have EHR activity in the 12–36 months prior to capture medical history. Participants were required to have a weight measurement recorded in the EHR ≤30 days prior to baseline and within at least three of four time intervals (3-week segments) in the 12-week period after baseline, corresponding to the core phase of the program. Participants were further required to have a weight measurement at 12 months from baseline (±3.0 months) and to have overweight/obesity as determined by the baseline BMI. Patients were excluded with a history of conditions associated with substantial weight change, including metastatic cancer, pregnancy, gastric bypass surgery, and end-stage renal disease based on ICD 9/10 encounter or billing diagnoses.

Measures

Among eligible participants, demographic data were obtained from the EHR database, including date of birth, sex, race/ethnicity, and preferred spoken language, which are self-reported by patients, as well as primary insurance payer. Additional baseline characteristics were collected ≤12 months prior to baseline. Census tract median household income was extracted based on participants’ home addresses as a proxy of SES. Patient clinical characteristics were also extracted, including comorbidities (pre-diabetes, diabetes, hypertension, dyslipidemia, metabolic syndrome, atherosclerotic cardiovascular disease, and depression based on ICD-9/10 diagnosis codes), BMI, and smoking status. Participants were classified as having a high risk for diabetes if they had clinical evidence of prediabetes or were considered high risk based on a validated screening tool developed by the American Diabetes Association.¹⁷ For each participant, a Charlson Comorbidity Index was calculated as a measure of overall disease burden.¹⁸ Medication orders were identified as of baseline. Information was collected on whether participants had an established Sutter Health primary care provider and the number of outpatient ambulatory encounters and telephonic/electronic encounters before baseline. To assess potential health engagement and motivation, the authors assessed whether patients had a preventive visit or influenza immunization prior to baseline. In prior analyses, a history of a preventive visit was associated with more pronounced weight loss.^19,20 In unpublished work, history of a preventive visit and influenza immunization were positively associated with LCP adherence (RJR, unpublished observation, 2019).

Weight is recorded in the EHR at routine clinical encounters and at each LCP visit. Weekly mean percentage weight change between 2 and 12 weeks from baseline and at 12 months from baseline were calculated as the difference in weight at each time point from baseline divided by baseline weight. Missing weight measurements were imputed between Weeks 2 and 12 using linear interpolation, if the missing value was between two other values. Otherwise, missing values were imputed using the last observation carried forward method. Weight at 12 months was not imputed.

Statistical Analysis

All analyses were conducted in SAS, version 9.4. Logistic regression was used to assess the probability of non-response to the LCP, defined as weight gain or no weight loss at 12 months from baseline. This definition of non-response, rather than a clinical cut off (e.g., ≥5% or 7% weight loss), was used to maximize the probability of identifying participants who would most benefit from adjunctive or alternative therapy. For each week during the 12-week core phase, four nested candidate models of increasing complexity were considered:

1. percentage weight change from baseline

2. Model 1 + percentage weight change from baseline and weight change through the prior week(s)

3. Model 2 + baseline weight; and

4. Model 3 + participant baseline demographics and clinical characteristics (Table 1).

Table 1.

Baseline Lifestyle Change Program Participant Characteristics

Characteristics	Participants (n=947)
Demographics
Mean age, years ± SD	54.12 ± 12.66
Female, n (%)	737 (77.8)
Race/ethnicity, n (%)
African American	39 (4.1)
Asian	65 (6.9)
Hispanic	122 (12.9)
Non-Hispanic white	627 (66.2)
Other	10 (1.1)
Unknown	84 (8.9)
Established PCP, n (%)	905 (95.6)
English preferred, n (%)	920 (97.1)
Clinical characteristics
Mean systolic blood pressure, mmHg ± SD	126.6 ± 15.2
Mean diastolic blood pressure, mmHg ± SD	76.8 ± 9.5
Mean BMI, kg/m2 ± SD	36.2 ± 6.6
BMI categories,a n (%)
Overweight	131 (13.8)
Obesity	586 (61.9)
Severe obesity	230 (24.3)
Smoking status
Current	37 (3.9)
Ever	258 (27.2)
Never	648 (68.4)
Unknown	4 (0.4)
Comorbidities
High risk for diabetes, n (%)	520 (54.9)
Type 2 diabetes, n (%)	204 (21.5)
Hypertension, n (%)	450 (47.5)
Depression, n (%)	217 (22.9)
Dyslipidemia, n (%)	504 (53.2)
Metabolic syndrome, n (%)	272 (28.7)
Overweight/obesity without other diabetes risk, n (%)	220 (23.2)
ASCVD, n (%)	85 (9.0)
CCI Score, n (%)
0	515 (54.4)
1-2	425 (44.9)
3-4	7 (0.7)
Medications
Mean total prescription, count ± SD	4.9 ± 4.0
Weight loss/appetite suppressants,a n (%)	74 (7.8)
GLT (with weight loss),a n (%)	157 (16.6)
Other GLT drugs, n (%)	75 (7.9)
Prior health resource utilization
Mean outpatient visits, count ± SD	11.3 ± 11.7
Mean telephone/electronic, count ± SD	16.7 ± 16.0
Preventive visit, n (%)	427 (45.1)
Immunization, n (%)	324 (34.2)
Socioeconomic characteristics
Insurance payer, n (%)
Commercial FFS/PPO	483 (51.0)
Commercial HMO	192 (20.3)
Medicare FFS	144 (15.2)
Medicare HMO	58 (6.1)
Medicaid/Medi-Cal	17 (1.8)
Self	3 (0.3)
Unknown	50 (5.3)
Census median household income, n (%)
<$50,000	83 (8.8)
≥$50,000 to <$75,000	384 (40.5)
≥$75,000 to <$100,000	290 (30.6)
≥$100,000 to <$200,000	190 (20.1)
≥$200,000	0 (0)
Program characteristics
Year of program initiation, n (%)
2010	7 (0.7)
2011	95 (10.0)
2012	103 (10.9)
2013	226 (23.9)
2014	176 (18.6)
2015	182 (19.2)
2016	139 (14.7)
2017	19 (2.0)
Season of program initiation, n (%)
Spring	301 (31.8)
Summer	241 (25.4)
Fall	338 (35.7)
Winter	67 (7.1)

^aWeight loss drugs included phentermine-based products (86%), naltrexone-bupropion (7%), orlistat (3.5%), and lorcaserin (3.5%). GLTs included metformin (85%), metformin-combination therapies (6%), GLP-1 receptor agonists (7.5%), and the sodium-glucose co-transport 2 inhibitor canagliflozin (1.5%).

The equation below describes the logistic regression model, where Pr is the predicted probability of non-response conditional on the set of n predictor variables, x through k.

$$\log (\frac{Pr}{1-Pr})=b_0+b_{1}x+\cdots b_{n}k$$

Models 1–4 for Weeks 2–12 from baseline (44 models in total) were fit using tenfold cross-validation. Specifically, the sample was randomly divided into ten roughly equally sized subsamples. Nine subsamples were then aggregated and used as a training data set and the remaining subsample was used as a validation data set. For each model and cross-validation split, predicted probabilities of non-response for the validation sample were generated. This process was repeated ten times until each subsample was used as a validation set. The area under the receiver operating characteristic curve (AUROC) was calculated as a diagnostic of model discrimination.²¹ The AUROC ranges from 0.5 to 1.0, and is interpreted as “failure to discriminate” (0.5–0.6) or “poor” (0.6–0.7), “fair” (0.7–0.8), “good” (0.8–0.9), or “excellent” (0.9–1.0) discrimination. AUROC values were averaged across the ten validation samples to obtain the final AUROC for each model at each week from baseline.

Once a model with optimal discrimination was selected, model precision (positive predictive value [PPV]) and recall (sensitivity) were examined across the range of predicted probabilities to determine the best cut point for distinguishing non-responders from responders. Model specificity, negative predictive value, and total predictive value were also calculated at different predicted probability cut points. Percentage weight change was calculated at 2–12 weeks from baseline from the corresponding predicted probabilities, based on the logit function of the logistic regression model.

The robustness of outcomes to imputation of model fit (i.e., AUROC) and patterns in weekly weight change weight values between Weeks 2 and 12 were examined by using only available weight data. For the model with optimal performance, analyses were repeated on a larger sample of patients with weight measurements available at baseline, the optimal time between 2 and 12 weeks of follow-up, and at 52 weeks (12 months) from baseline.

RESULTS

A total of 947 (21.2%) of 4,463 LCP participants met study eligibility criteria. Most patients were excluded owing to insufficient weight measurements between 2- and 12-week follow-up (n=1,914) (Appendix Figure 1). Participants had a mean age of 54 years; 78% were female and 66% were non-Hispanic white (Table 1). More than 85% had obesity or severe obesity (mean BMI=36 kg/m²). Fifty-five percent of participants had pre-diabetes or a high risk for diabetes and 21.5% had evidence of type 2 diabetes. The remaining participants had overweight/obesity in the absence of other diabetes risk factors. Participants attended a mean of 12.2 program sessions during follow-up (median=11).

Among 947 participants, 6,747 weight measurements were recorded between Weeks two and 12. A total of 3,670 missing weight values were imputed during this period for a total of 10,417 weight measurements.

Mean percentage weight change at 3, 6, and 12 weeks from baseline was –1.5% (SD=1.3), – 2.5% (SD=2.1), and –4.2% (SD=3.4), respectively. The direction and magnitude of mean changes in weight were robust to imputation (data not shown). At 12 months from baseline, mean percentage weight change was –3.8% (SD=7.0) and 286 (30.2%) of 947 participants were non-responders. Approximately 34% of patients (n=318) achieved clinically meaningful (≥5%) weight loss at 12 months from baseline.

Model 1, with percentage weight change as the sole predictor, had an AUROC of 0.640 at Week 2 from baseline, which incrementally increased to 0.789 at Week 12 from baseline (Appendix Figure 2). The inclusion of prior weight change (Model 2) and baseline weight (Model 3) as independent covariates did not improve model performance. The inclusion of baseline demographics and characteristics (Model 4) as covariates decreased the performance of the model at each time point. Model performance was robust to imputation of missing weight (data not shown). The ROC curves for Model 1 at each week from baseline are shown in Appendix Figure 3.

Based on these findings, Model 1, with percentage weight change at 12 weeks from baseline, was considered optimal. Both the sensitivity and PPV of Model 1 were maximized (sensitivity and PPV=0.56) at a predicted probability of non-response of 0.41 (Figure 1). For the chosen model, the relationship between the conditional probability of long-term non-response (Pr) and percentage weight change at week 12 from baseline (x) is expressed as follows (Figure 2):

$$\log (\frac{Pr}{1-Pr})=0.4255+0.4200x.$$

Figure 1.

Recall and precision as a function of predicted probability of 12-month non-response. Notes: Recall (sensitivity) and precision (positive predictive value; PPV) shown for Model 1 for percent weight change at week 12 from baseline. Non-response is defined as weight gain or no weight loss at 12 months from baseline.

Figure 2.

Predicted probability of 12-month non-response as a function of percent weight change at 12 weeks from baseline.

Notes: Predicted probability of non-response (Pr) versus percent weight change at week 12 from baseline (x) is derived from the logit function:$\log (\frac{Pr}{1-Pr})=0.4255+0.4200x$. Values of percent weight change corresponding to predicted probabilities within the range of 0.3 to 0.5 are show above graph for illustration. Non-response is defined as weight gain or no weight loss at 12 months from baseline.

When repeating the analysis on the larger sample of patients with weight measurements at baseline, Week 12, and Week 52 (n=1,625), similar results were observed. For example, the larger sample was similar to the main cohort (n=947) in terms of baseline demographics and clinical characteristics (data not shown). The model with the best discrimination included percentage weight change at 12 weeks from baseline as the sole predictor (AUROC=0.750). The inclusion of baseline weight (AUROC=0.750) or baseline demographics and clinical characteristics (AUROC=0.735) did not improve model discrimination. With the optimal model, sensitivity and PPV (sensitivity and PPV=0.52) were maximized at a probability cut off of 0.39 with a negative predictive value of 0.73 and specificity of 0.97.

The goal of this predictive model is to develop a clinical tool to identify, early in the course of an LCP, non-responders who require assessment for adjunctive or alternative therapy. As an illustration, a range of predicted probabilities of non-response (0.30–0.50) with acceptable model performance were considered (Table 2). Using a cut point of 0.41, where sensitivity and precision are both maximized, among 100 program participants, the algorithm would flag 30 potential non-responders who lost ≤1.87% body weight at 12 weeks from program initiation. Approximately 17 (56.7%) of the 30 participants would be true non-responders and the remaining 13 (43.3%) would be false positives (likely to respond regardless of being flagged); another 13 participants would be false negatives or “missed opportunities” (i.e., true non-responders, but missed by the algorithm). With a predicted probability cut point <0.41, the model identifies more true non-responders (increased sensitivity), but at the cost of more false positives (decreased precision). Conversely, with a predicted probability cut point higher >0.41, the model identifies fewer non-responders (increased precision), but also fewer true positives (decreased sensitivity).

Table 2.

Model Performance by Predictive Probability of Long-Term Non-Response

	Predicted probability of long-term non-response assessed at 12 weeks from baseline
Model performance	0.30	0.41	0.50
Sensitivity	0.759	0.563	0.409
Specificity	0.666	0.808	0.893
Positive predictive value	0.495	0.559	0.622
Negative predictive value	0.864	0.810	0.777
Total predictive value	0.684	0.723	0.736
Among 100 hypothetical patients
Weight loss at 12 weeks, %	≤3.02	≤1.87	≤1.01
Total patients flagged, N	46	30	19
Correctly identified, n (%)	23 (50.0)	17 (56.7)	12 (0.40)
Incorrectly identified, n (%)	23 (50.0)	13 (43.3)	7 (0.60)
Missed opportunities, n	7	13	18

PCP, primary-care provider; ASCVD, atherosclerotic cardiovascular disease; CCI, Charson comorbidity index; GLT, glucose-lowering therapy; FFS, fee for service; PPO, preferred provider organization.

DISCUSSION

In this study, a simple predictive algorithm was developed and validated to identify non-responders in a 12-month, CDC-aligned LCP, using real-world data from a large healthcare delivery system. Among several tested models, percentage weight change alone demonstrated the highest predictive performance, especially at 12 weeks from baseline. The results of this study show that early weight loss alone can predict non-responders at the completion of a LCP, with fair model discrimination (AUROC=0.789).

Prior studies have identified treatment response heterogeneity as a major challenge for behavioral weight management and diabetes prevention interventions.^10,22,23 Variability in treatment response is influenced by a confluence of behavioral, biological, environmental, and psychosocial factors.²³ However, an understanding of the impact of these factors on individual treatment response is currently limited, given that these predictors are often difficult to measure, inadequately captured, or remain unknown.

In a review of pre-treatment weight loss predictors, Carraca et al.²⁴ identified several factors, including BMI, eating self-efficacy, and previous weight loss attempts, which are associated with achievement of weight loss/maintenance. Although there was no information on eating self-efficacy and previous weight loss attempts in this study, the addition of patient baseline characteristics did not improve the performance of the predictive algorithm. In fact, together, they decreased discrimination, potentially because of model overfitting.

Early weight loss during the course of an intervention is well established to correlate with both short- and long-term treatment response.^11,24–29 For example, in one study, weight loss at 3 weeks from baseline was associated with weight loss at the completion of an 8-week low-calorie diet intervention.²⁸ Unick and colleagues²⁹ found that, for participants in the LookAhead Trial (N=2,290), weight loss at 1 and 2 months from baseline was associated with clinically significant (≥5%) weight loss annually through 8 years post-intervention. Importantly, weight loss is a modifiable and actionable factor, especially when monitored in real time.

No studies to date have used early weight loss to develop a predictive algorithm for use in clinical practice. The application of an algorithm intended to identify those who are unlikely to respond to treatment in the early stages of an intervention, implemented in clinical practice, offers the opportunity for adjunctive or alternative treatment and targeted efforts to enhance engagement and effectiveness. Adjunctive or alternative treatment may include 1:1 motivational interviewing, other group-based behavioral or education programs, pharmacotherapy, or bariatric surgery.

In this study, the authors intentionally sought to identify LCP participants who were likely to gain weight or not lose any weight at the conclusion of the 12-month program, rather than those who did not meet a more clinically meaningful cut point (such as 5% or 7% weight loss). This was a pragmatic decision to ensure that efforts to identify non-responders in routine clinical practice and, subsequently, to perform an assessment for adjunctive or alternative therapy would be focused on those with the greatest need. For example, in an evaluation of a National Diabetes Prevention Program registry, Ely et al.³⁰ found that 35.5% of patients achieved ≥5% weight loss. Similarly, in this study, about 34% achieved this goal. Thus, this definition of non-responders would capture approximately 65% all program participants. In real-world clinical settings, there is a need to allocate limited resources efficiently among patients who need them most.

Limitations

This study has several limitations. The observational nature of the study limits causal inferences. Many patients were excluded owing to insufficient weight measurements. Sensitivity analyses showed similar demographics, weight outcomes, and model performance with and without imputation. Although the predictive algorithm developed in this study holds much promise, the addition of other predictors, not routinely collected in clinical practice, may improve its accuracy. Such predictors should be identified and tested in future studies. Though tenfold cross-validation of the algorithm was performed, this study did not externally validate the algorithm. Moreover, this study included a small cohort of patients from single healthcare delivery system. Findings may not be generalizable to those who did not have sufficient weight measurements, those who did not complete the program, or to other populations in different healthcare systems. With regard to program non-completers, other studies have shown that individuals with poor early progress in weight loss tend to drop out of such behavioral weight-loss programs.^31–36

The utility of a predictive algorithm and the specific predicted probability cut point that is applied in clinical settings depends on a number of factors, including the number participants in the program, clinic resources for assessing the needs of potential non-responders, and available adjunctive or alternative treatment strategies within the setting.

CONCLUSIONS

In a cohort of LCP participants from clinical practice, percentage weight change at 12 weeks from baseline can serve as a single indicator of non-response at the completion of the 12-month program. Clinicians can easily apply this algorithm to identify and assess participants in potential need of adjunctive or alternative therapy to maximize treatment outcomes.