Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Jul 1.
Published in final edited form as: Contemp Clin Trials. 2019 May 4;82:9–16. doi: 10.1016/j.cct.2019.05.003

Rationale and design for a pragmatic effectiveness-implementation trial of online behavioral obesity treatment in primary care

Hallie M Espel-Huynh 1,2,3,*, Rena R Wing 2,3, Carly M Goldstein 2,3, J Graham Thomas 2,3
PMCID: PMC6606329  NIHMSID: NIHMS1529672  PMID: 31063870

Abstract

Background

Current guidelines recommend behavioral intervention to achieve a modest weight loss (e.g., 3–5%) as a first-line obesity treatment. Online behavioral obesity treatment, delivered via the Rx Weight Loss (RxWL) program, produces clinically significant initial weight losses. However, the program’s pragmatic utility in routine medical care has yet to be tested. Further, additional research is needed to determine how best to extend the RxWL program to facilitate weight loss maintenance. This paper summarizes methods for a pragmatic trial aimed at identifying optimal methods for implementation of RxWL in primary care and evaluating relative effectiveness of two approaches to weight loss maintenance.

Study Design

RxWL will be implemented in a network of approximately 60 primary care clinics. Implementation outcomes (program uptake and completion metrics) will be compared between a Basic Implementation Intervention consisting primarily of access to RxWL, and an Enhanced Implementation Intervention with additional training in strategies for motivating and supporting patients in their use of RxWL and online clinician support tools for tracking patient progress. Second, two intervention approaches (Monthly Lessons versus Refresher Courses) within the RxWL patient platform will be tested against an educational control condition, and effects on 1-year weight loss maintenance will be compared.

Conclusion

This study will provide essential information about the feasibility and utility of online obesity treatment in primary care. It will provide novel information on two approaches to weight loss maintenance for patients in this setting. This project fills key gaps in evidence regarding best practices for obesity treatment in primary care.

Keywords: Obesity, Weight loss, Primary care, Hybrid implementation-effectiveness trial

Introduction

Current guidelines recommend behavioral intervention to achieve modest weight loss (e.g., 3–5%) as a first-line obesity treatment [1, 2]. Primary care providers are optimally positioned to facilitate obesity management given their regular contact with much of the United States population, particularly patients of diverse ages, racial/ethnic backgrounds, and socioeconomic status [3, 4]. Moreover, extensive evidence supports the efficacy of behavioral weight loss treatment delivered to primary care patients [5].

Despite the guidelines, only 29–42% of patients with obesity are advised to lose weight in primary care [6, 7]; far fewer receive counseling of appropriate content and intensity [8]. Medical teams (including physicians, nurses, and care coordination staff) face several barriers to delivery of in-person behavioral obesity treatment, including insufficient time, lack of training, and inadequate payment [9–11]. We therefore developed an interactive, online behavioral obesity treatment—Rx Weight Loss (RxWL)—designed to be more accessible than in-person treatment by limiting costs and time burden [12]. When tested with primary care patients in a randomized controlled trial [13], RxWL produced mean weight loss of 5.5±4.4 kg at 3-months, which was largely maintained by 6-months (5.4±5.4 kg). Given this evidence of efficacy, we are moving towards dissemination but face two primary challenges and questions: First, what are the most sustainable methods for implementing RxWL in routine care? Second, given that weight regain prevention remains a critical challenge for patients who successfully lose weight, what can be done to maximize long-term clinical effectiveness of RxWL?

Most research evaluating the efficacy of online and/or primary care-based obesity treatment has been conducted in randomized controlled trials with limited pragmatic utility (i.e., involving few clinics, delivering treatment outside of usual clinical care, or utilizing researchers extensively for care delivery or participant recruitment and retention) [5]. Sustainable implementation requires that primary care medical teams support their patients in initiating and completing RxWL in usual care. Thus, providers may require additional training in how to discuss weight and weight loss with patients, motivate them to lose weight, and support them in a weight loss attempt. This protocol takes a pragmatic approach to identify training and implementation methods that most effectively promote active provider engagement, more frequent patient referral, and greater patient uptake of the program.

Even if the hurdles of provider referral and patient follow-through are successfully addressed, the critical challenge of preventing weight regain after treatment remains [14]. Weight loss maintenance is facilitated when treatment includes extended contact after active weight loss [15–17]. Beyond additional treatment contact, ideal format and frequency are unknown. Preliminary evidence from in-person treatment trials suggest that providing a maintenance intervention in periodic bursts, or “refresher campaigns,” may have advantages compared to providing a less intensive intervention more consistently over time [18–21]. This study evaluates differential effectiveness of these approaches.

In this paper, we summarize the rationale, research design, and methods for a study protocol that will (1) evaluate how intensity of technology-enhanced clinical support for primary medical providers affects implementation success, and (2) compare effectiveness of two approaches for long-term weight loss maintenance.

Methods

Study Aims and Design

This study seeks to extend the reach and significance of online obesity treatment in primary care using a pragmatic type II effectiveness-implementation hybrid design [22]. This allows for simultaneous testing of treatment effectiveness and implementation success in the applied clinical setting. The study was conceived in response to a call for trials involving “Pragmatic Research in Healthcare Settings to Improve Diabetes and Obesity Prevention and Care” (NIDDK: PAR-17–177), which sought to encourage research on practical and sustainable improvements to routine care for obesity and emphasized broadly disseminable treatment approaches. The resulting funded project (R18: DK114715) involves two primary aims, testing hypotheses regarding implementation success and clinical effectiveness, respectively. This pragmatic design makes use of staff, resources, and facilities in routine care, thus maximizing practical relevance of study results. Co-primary study aims are as follows:

Primary effectiveness aim:

Compare three different approaches to weight loss maintenance treatment in the RxWL platform. We will test whether 12-month patient weight losses produced by a Refresher Course condition are greater than those of a Monthly Lessons approach, and whether both outperform a Newsletter control condition.

Secondary effectiveness aims:

(1) Test relative effectiveness of the three approaches to weight loss maintenance for longer-term (24-month) weight outcomes. We will test whether 24-month patient weight losses produced by a Refresher Course condition are greater than those of the Monthly Lessons approach, and whether both outperform the Newsletter control condition. (2) Determine whether differential weight losses are also associated with improved overall health. We will compare the RxWL intervention and control groups on change in weight-related cardiovascular disease risk factors over the month weight loss and maintenance period, and at 12-months following RxWL program completion (24-month follow-up).

Primary implementation aim:

Test whether an Enhanced Implementation Intervention that fully integrates and supports RxWL use in primary care serves to improve patient engagement with the platform. We will compare whether providers receiving the Enhanced Implementation have a greater proportion of eligible patients who: (a) are directed to RxWL; (b) enroll in RxWL, and (c) complete the RxWL initial weight loss program at 3-months and the weight loss maintenance program at 12-months. The Enhanced Implementation Intervention will be compared to a Basic Implementation Intervention with minimal implementation support (i.e., simply makes the program available).

Secondary implementation aims:

Examine effects of the Enhanced versus Basic Implementation programs on provider engagement. We will test whether (a) providers who receive the Enhanced Implementation are more satisfied with the RxWL program; and (b) Enhanced Implementation providers spend more time and effort supporting their RxWL-enrolled patients than do Basic Implementation providers.

Effectiveness of Rx Weight Loss Intervention

The Rx Weight Loss program.

RxWL is a 3-month, interactive, online weight loss program that patients can access via the Web [13]. This initial weight loss intervention was tested in our prior efficacy trial [13] and includes: (a) 12 weekly multimedia video lessons addressing empirically-supported self-regulation skills to promote weight loss, including self-monitoring, stimulus control, problem solving, and obtaining social support [23, 24]; (b) online submission of self-monitored daily weight, caloric intake, and physical activity; and (c) weekly automated feedback to the patient reinforcing success to date and suggesting strategies to further enhance outcomes. Prescribed program goals are consistent with those of prior, efficacious in-person behavioral weight loss programs [23, 24]. These include: (1) weight loss of 1–2 lbs. per week, working toward a total weight loss of 10% of initial body weight; (2) caloric intake of 1200–1500 kcal/day for patients weighing ≤ 250 lbs., and 1500–1800 kcal for patients weighing > 250 lbs.; and (3) gradual increases in physical activity toward a final goal of ≥ 150 minutes of physical activity across 5 days per week.

Improvements have been made upon the originally tested program to increase disseminability. Specifically, the program has been expanded for use on smartphone and tablet computers, given increasing rates of reliance on these devices for Web access [25]. Patients are also now able to choose to follow a more flexible, low-fat diet plan or a Mediterranean diet, which emphasizes a plant-based diet and is more structured. Patients’ providers have access to records of patients’ enrollment, weight loss, and progress in RxWL to date. The specific information made available to providers, and the way it is presented, depends on the implementation condition of a patient’s provider (described in the Implementation Methods section). To maximize pragmatic relevance of trial results, direct contact between researchers and patients will be limited to phone-based support for technical issues with the platform. Appreciating that patient retention may prove challenging, occasional postcard mailings will be sent to enrolled patients (e.g., at the start of the New Year) to remind them of their commitment to health and to encourage them to re-engage with the program if they have not used the intervention platform recently. Such contact is considered feasible to be continued by a primary care practice network if RxWL implementation were sustained beyond completion of the randomized trial.

Weight loss maintenance conditions.

Given our aims to evaluate the differential effects of three weight loss maintenance interventions, all patients will receive the same initial RxWL intervention for the first 3 months (i.e., active weight loss phase). Upon enrollment into the initial weight loss program, patients will be randomized to one of three weight loss maintenance conditions. However, patients will not be informed of their randomization until they complete the initial 3-month weight loss program to avoid potential for differential attrition by condition during the weight loss period. Upon progressing to the weight loss maintenance phase, patients will be provided with one of the following interventions:

Refresher Course maintenance condition.

One-third of patients will be assigned to the Refresher Course maintenance program, which consists of an online introductory session at the beginning of month 4, and two, 4-week “courses” occurring in months 7 and 10. In the introductory session, patients are instructed to gradually increase physical activity beyond that achieved during weight loss. If a patient has not yet reached the program’s initial goal of 150 minutes/week, they are encouraged to continue increasing toward that point. Patients at or above 150 minutes/week gradually work toward 200 and ideally 250 minutes/week for the duration of the maintenance period. Small weight gains (relative to starting maintenance weight) are addressed through a color-coded self-regulation program [17]. Weight loss maintenance or loss falls in the “Green/GO” zone. Patients in the “Yellow/CAUTION” zone (1–3 lbs. gained) are asked to make small dietary changes to reduce intake until reaching the Green zone again. Patients who reach the “Red/STOP” zone (≥ 4 lbs. gained) resume full weight loss efforts until returning to maintenance goal weight.

During the Refresher Courses in months 7 and 10, patients are instructed to self-monitor their weight, caloric intake, and physical activity every day. A 5-lb. weight loss goal is also prescribed for each 4-week period. At the end of each week during the Refresher course, automated feedback is provided to patients, similar in content to that provided during the initial 3-month weight loss period.

Each refresher course focuses on a new topic and introduces new behavioral challenges specific to longer-term maintenance. Examples include coping with the potential boredom of routine diet and physical activity patterns, managing cravings and urges, and connecting daily weight management efforts with one’s broader personal values (e.g., health or family). Such strategies form the basis of long-term interventions in lifestyle trials such as the Diabetes Prevention Program and Look AHEAD [23, 24, 26]. Lessons within a month-long “course” build upon one another progressively. For example, one course takes a meal-by-meal approach to improving dietary intake. One session focuses on the importance of eating breakfast. It provides examples of specific breakfast plans and instructs patients to eat breakfast on all 7 days of the week (while still achieving the daily calorie goal). Subsequent sessions focus on the other meals, snacks, and calories in beverages and provide specific, topic-related behavioral challenges. A schedule of session topics is outlined in Table 1.

Table 1.

Lesson Content and Delivery Schedule for Monthly Lessons and Refresher Course Maintenance Conditions.

Lesson Topic  Week Lesson is Delivered During Maintenance Period
 Monthly Lessons  Refresher Courses
Introduction to Maintenance Month 1, Week 1 Month 1, Week 1
Commitment to Weight Maintenance and Personal Values Month 2, Week 1 Month 4, Week 1
Understanding Metabolism and Increasing Lifestyle Physical Activity Month 3, Week 1 Month 7, Week 1
Eating a Healthy Breakfast Month 4, Week 1 Month 7, Week 2
Changing “Yes, but” to “Yes, and” Month 5, Week 1 Month 4, Week 2
Eating a Healthy Dinner Month 6, Week 1 Month 7, Week 3
Managing Cravings and Urges Month 7, Week 1 Month 4, Week 3
Eating Mindfully and Choosing Healthy Snacks Month 8, Week 1 Month 7, Week 4
Assertive Communication for Weight Management Month 9, Week 1 Month 4, Week 4
Open in a new tab

Note. Month numbers listed here refer to the maintenance phase of the program only. Thus, Month 1 content is delivered upon completion of the 12-week weight loss phase (4 months after starting the program).

Monthly Lessons maintenance condition.

One-third of patients will be assigned to the Monthly Lessons maintenance program. Each month, patients receive a new video lesson, submit one week of daily self-monitoring records, and receive automated feedback on those data. Video lesson content is similar to the Refresher course, but it is delivered in 9 monthly online sessions rather than in short bursts. See Table 1 for a summary of lesson content and timing relative to the Refresher Course condition. As with the Refresher condition, patients follow the color-coded self-regulation plan for weight loss maintenance and have the same physical activity prescription as in the Refresher condition [17].

Newsletter control maintenance condition.

One-third of patients are assigned to the Newsletter maintenance (control) condition, which consists of monthly online content aimed at facilitating weight loss maintenance. This content, delivered via traditional Web pages with text, images, and links, provides general educational information about healthy eating, physical activity, the health benefits of weight loss, and recipes for healthy meals. Content reinforces skills taught in the initial 3-month program.

Effectiveness assessment measures and data collection.

All patients who enroll in RxWL will continue to be followed by their providers for routine care. Consistent with protocols for optimal care at the clinics in which RxWL is implemented, patients who are directed to attempt weight loss are routinely scheduled for 3-and 6-month follow-up visits. They also typically see their providers at least annually following completion of an active weight loss attempt. In alignment with a pragmatic trial design, all patient outcome data will be drawn from electronic medical record (EMR) documentation of these routine care visits, or from the RxWL platform itself.

Primary outcomes will be compared across the three RxWL maintenance condition groups and include:

  • Body weight. Patient weight at baseline and throughout treatment and follow-up will be obtained from the EMR. Patient weight is measured and documented at every clinic encounter. Weight obtained at the visit when a patient is directed to the RxWL program will serve as the baseline weight. All post-baseline weights recorded in the EMR over the year of intervention (12-months) will be used in group comparison analyses.

  • Proportion achieving ≥ 5% weight loss. RxWL maintenance conditions will be compared on proportion of enrolled patients losing ≥ 5% of baseline weight (i.e., a clinically significant amount) at 12-months after referral.

Given their potential relevance to weight-related behavior change and ultimate weight loss achieved, the following secondary outcomes will also be evaluated:

  • 24-month weight outcomes. Similar to 12-month primary outcome analyses, groups will be compared on body weight and proportion achieving ≥ 5% weight loss at 24-months.

  • Weight-related patient health indices. Data from the EMR will be used to determine changes in the health indices expected to improve concurrently with weight loss and successful maintenance [27, 28], including body mass index, lipids, blood pressure, and glycemic control over the year of intervention and the year of follow-up. Changes in use and dosing of relevant medications (e.g., for hypertension, glucose dysregulation, and hyperlipidemia) will also be compared across groups.

  • RxWL engagement and adherence. The RxWL platform automatically records all aspects of utilization, including logins, video lessons viewed, and self-monitoring records submitted.

  • Patient satisfaction. Patients will complete user satisfaction questionnaires upon finishing the 3-month weight loss program and at the end of the 9-month maintenance intervention (12-months post-enrollment). The questionnaire consists of 21 items to assess patients’ overall perspective on program usability and effectiveness (e.g., RxWL is a good fit for my weight loss needs), rated on a 5-point Likert scale ranging from “Strongly Agree” to “Strongly Disagree.”

Implementation Process and Outcomes

Organizational setting.

Implementation will occur in the context of routine care provided at the Rhode Island Primary Care Physicians Corporation (RIPCPC). RIPCPC is a large, multi-specialty association of independent physicians serving approximately 160,000 patients across 58 primary care practices in the state of Rhode Island. All practices in the network utilize the patient-centered medical home care model [29]. Clinical teams consist primarily of physicians and nurse care managers (NCMs) coordinating care within a patient-centric model. This model is designed to address the healthcare needs of medically complex, “high-risk” patients, such as those who have obesity comorbid with cardiovascular and other chronic health conditions [29].

Providers.

RIPCPC employs 16 NCMs assigned to work with high-risk patients through in-person and remotely delivered care. Each NCM serves 3–4 of the 58 total practices with no overlap (i.e., no practice is served by more than one NCM). In typical practice, NCMs work closely with patients’ physicians and, consistent with their role in this trial, are generally tasked with guiding patient weight loss efforts. Thus, NCMs will be the unit of focus for the RxWL implementation trial.

Patients and RxWL program eligibility.

RIPCPC patient demographics are representative of the state of Rhode Island and include individuals with diverse racial, ethnic, and socioeconomic backgrounds. Participants in the current trial are those served by the aforementioned NCMs and include patients with a wide range of medical comorbidities. In prior RxWL research, women and Whites have constituted approximately 80% of participants [13]. This trial aims to increase recruitment of men and patients of racial/ethnic minority backgrounds (≥ 30% men and ≥ 30% minorities), thus increasing likelihood that the sample is representative of the RIPCPC patient population.

Eligibility criteria are kept broad to maximize patient inclusion and increase practical utility of study findings. Patient eligibility criteria include: age 18 −75, body mass index ≥ 25 kg/m2, deemed medically appropriate for weight loss by a RIPCPC physician, and access to an Internet-connected device such as personal computer, smartphone, or tablet. For safety reasons, patients are excluded if they are: taking weight loss medication; utilizing a liquid diet for weight loss; currently pregnant, lactating, <6 months post-partum, or plan to become pregnant during the next 12 months; or unable to walk 2 blocks without stopping.

Implementation methods.

NCMs will be recruited from RIPCPC and matched on the size, racial/ethnic composition, and socioeconomic status of the practices they serve. Matched pairs will then be randomized to receive the Basic or Enhanced Implementation intervention in a 1:1 ratio.

Basic Implementation.

All NCMs randomized to the Basic Implementation condition undergo informed consent procedures in-person with research staff. Consenting NCMs then complete a brief in-person or online training, which describes the rationale for the RxWL program and provides instruction for identifying and directing eligible patients to the program. Upon completing the training, clinicians have access to the following Basic program components:

  • Eligible Patient Lists. Lists of potentially eligible patients in each practice are generated from the electronic medical record (EMR) on a regular basis and provided to clinicians, along with date of next clinical contact. Physicians at each practice will assess patient appropriateness for weight loss and confirm final eligibility. NCMs are encouraged to discuss the RxWL program with eligible patients during any routine care interaction.

  • Warm Handoff “Prescriptions” for RxWL. Physicians will invite potentially eligible patients to speak with an NCM at the end of their clinic visit to discuss RxWL. Patients who are interested in learning more about the program will be given a printed “prescription” during their visit with the NCM. The “prescription” will contain a unique, one-time-use code for access to the RxWL platform. Each code is linked to one RIPCPC practice. Upon activating the code, patients complete informed consent, confirm eligibility, and enroll through the RxWL platform. Prescription codes are also documented at each RIPCPC practice, which allows the research team to link the patient with their EMR.

  • Basic Weekly Email Status Report. Clinicians receive a weekly email report summarizing study-wide progress on overall patient enrollment goals, including movement toward recruiting ≥ 30% men and racial/ethnic minorities. Emails will not include identifiable patient information but will include a link for NCMs to log into RxWL to view the list of their patients who have enrolled.

  • Research Team Contact. The research team holds periodic phone calls (1–2 times per month) withNCMs to review study progress and discuss technical issues that arise with the RxWL platform. These calls are also designed to encourage NCMs and RIPCPC administrative staff to stay informed of enrollment numbers and continue efforts to recruit patients to the study. In addition, NCMs and their patients are provided with remote ad hoc technical support as needed throughout the trial. To promote physician stakeholder buy-in, researchers periodically attend RIPCPC physician staff meetings to inform physicians of the program and answer questions.

Enhanced Implementation Intervention.

The Enhanced Implementation program includes all components of the Basic Implementation, plus additional training and online tools designed to reduce barriers to successful delivery of behavioral weight loss treatment in primary care. These components specifically target the lack of training providers typically receive in addressing barriers to health behavior change [11] and their limited time to manage large patient volumes [9].

  • Enhanced Training. In addition to the RxWL program overview provided in the Basic Training program, the Enhanced Training includes more training time (up to one hour). NCMs receive education on strategies to communicate with patients in a way that may increase patient engagement with and follow-through on behavioral weight loss recommendations (e.g., use of patients’ preferred terms to discuss weight [30], motivational interviewing skills [31, 32], problem solving, addressing barriers to adherence). After the in-person training, clinicians have ongoing access to electronic training resources to allow for review of these topics.

  • Enhanced Patient Monitoring Tools in RxWL. Clinicians in the Enhanced condition have access to a dashboard within the RxWL portal that includes provider-specific information on the enrollment status of referred patients, summaries of patient progress, and a listing of “red flag” patients who may benefit from provider follow-up (i.e., for poor weight loss outcomes, low program engagement, or excessive weight loss that may suggest a potential underlying medical concern requiring further evaluation).

  • Enhanced Email Status Report. Enhanced weekly email reports include all components from the Basic reports, in addition to information on practice-specific enrollment progress relative to average performance of all other practices in the network. NCMs also receive an individualized listing of the number of “red flag” patients on their caseload, with a hyperlink allowing them to log in to RxWL and follow up with these patients directly.

  • Monthly Patient Progress Reports. Patients whose NCMs are in the Enhanced condition receive monthly “progress report” letters by mail, summarizing their level of RxWL program engagement and weight loss to date. To facilitate coordination of care, the patient’s referring physician receives a copy of this letter by mail at the RIPCPC practice. Copies of these progress reports are also available to NCMs via the RxWL platform. NCMs are notified each time a new report is available for view.

  • Telephone Check-Ins. NCMs are asked to call each patient two weeks post-referral and as needed (depending on patient progress) to provide ongoing support for engagement in the RxWL program. Skills provided in the Enhanced Training (e.g., motivational interviewing, problem-solving) are designed to help NCMs address patient barriers effectively during these calls.

Implementation measures and data collection.

To achieve the primary implementation aim of comparing Enhanced versus Basic implementation interventions on patient referral, enrollment, and treatment completion, Basic and Enhanced NCM groups will be compared on the following primary outcomes:

  • Proportion of eligible patients directed to RxWL. For Basic and Enhanced groups, respectively, the proportion of all eligible patients who are successfully referred to the program (whether or not they enroll) will be calculated.

  • Proportion of patients directed to RxWL who enroll. The number of referred patients who ultimately enroll in RxWL is automatically recorded in the RxWL platform. For Basic and Enhanced groups, respectively, the relative proportion of all referrals that lead to successful patient enrollment will be calculated.

  • Proportion of patients completing treatment. Completion of RxWL is defined as accessing the platform +/−2 weeks from the final week of (a) the initial 3-month program and (b) the end of the full 12-month program. These outcomes are recorded automatically by RxWL for each patient. The proportion of all patients enrolled in RxWL who achieve treatment completion according to these two metrics will be calculated for each group.

The following metrics of clinician engagement with the RxWL platform and weight-related patient care will be evaluated as secondary implementation outcomes:

  • Clinician contact with patients referred to RxWL. All RIPCPC patient contacts (including those in-person, by phone, and through secure patient messaging systems) are required to be documented in the EMR. This EMR-based contact record will be used as an indirect measure of intervention fidelity (given that clinicians in the Enhanced Implementation condition are prompted to contact their patients more frequently than in Basic), and to evaluate the level of clinical resources required to obtain the observed patient outcomes.

  • Clinician use of RxWL provider dashboard. The RxWL platform automatically records all aspects of use by each NCM, including total number of logins, patient progress reports and clinician skills training resources viewed, and number of weekly emails read.

  • Clinician satisfaction with RxWL. All NCMs who have been provided with a clinician account on RxWL will be asked to complete user satisfaction questionnaires. After the account has been active for 12 months, the clinician receives an email prompting them to complete the survey via the RxWL platform. The measure includes 26 items assessing clinicians’ beliefs about the effectiveness of the RxWL implementation process in their clinics (e.g., “I have adequate support for using the RxWL program”), as well as the program’s utility in facilitating clinician-guided patient weight loss in primary care (e.g., “RxWL makes management of patients with excess weight more time-efficient”). As with patient satisfaction measures, items are rated on a 5-point Likert scale ranging from “Strongly Agree” to “Strongly Disagree.”

Data Analytic Plan

Effectiveness aims: analysis and hypothesized results.

Patients enrolling in RxWL are individually randomized in a 1:1:1 ratio to the three intervention conditions for the maintenance phase, but group comparisons will be made in the context of a cluster randomized trial. A generalized linear mixed modeling approach that accommodates clustering and non-independence of observations within NCM and practice will be used. Weight loss will be regressed on time (represented in the model as weeks since enrollment), baseline weight, treatment assignment (Refresher Course, Monthly Lessons, or Newsletter condition), and the time by treatment assignment interaction. Unconditional models will be used to determine whether the 12-month trajectory is best modeled as linear or curvilinear (to capture potential regain). NCM, practice, and the slope of time will be treated as random effects. The intercept will be fixed at zero (representing zero weight loss at the time of enrollment). Least squares means will be used to compare estimates of 12-month (primary outcome) and 24-month weight loss (secondary outcome) between conditions. A similar approach will be used to analyze secondary outcomes. Effectiveness outcomes will be analyzed using the intent-to-treat principle. Patients will differ in the number of measures available depending on the number of times the patient was seen at RIPCPC. Analyses will include all weight measurements recorded in the EMR over the 12-month and 24-month periods for primary and secondary weight outcomes, respectively.

Retention challenges are inherent to pragmatic trials, in that researchers must limit their contact with study participants. This is particularly challenging for trials involving eHealth interventions, given greater rates of disengagement compared to treatment delivered in-person [33, 34]. To maximize practical relevance of findings, researchers in this study will limit interaction with patients wherever possible. The decision to collect follow-up weight outcome data purely from the patient EMR is consistent with this goal, however this may also contribute to missing data. For evaluation of our effectiveness aims, we assume that data will be missing at random (MAR). Extreme value stochastic imputation [35] and pattern mixture models [36] will be used to test the degree to which our inferences are dependent upon the MAR assumption and explore the effect of missingness on outcomes if it is suspected that our outcomes are missing not at random.

We hypothesize that mean weight losses at 12-months will be greater among patients enrolled in the Refresher Course relative to the Monthly Lessons condition, and that both will outperform control. We expect a similar pattern of results for the analysis of patients achieving at least 5% total weight loss, as well as all secondary outcomes assessed.

Implementation aims: analysis and hypothesized results.

The test of implementation-related aims will focus on comparing the proportion of patients directed to, enrolled in, and completing RxWL in the Basic versus Enhanced Implementation conditions. Generalized linear mixed models will be used to accommodate clustering and non-independence of observations within NCM and practice. Outcomes will be modeled using the binomial distribution and logit link function and regressed on a dummy coded indicator for implementation condition. NCM and practice will be treated as random effects. Analysis of secondary outcomes will also follow this general approach, with appropriate adjustments to account for differences in unit of analysis and distribution of the outcome. Given that all referrals are documented in the EMR, and that patients’ program enrollment and completion are automatically documented in the RxWL platform, no data will be missing in the primary outcomes analysis for the implementation aim.

We hypothesize that the Enhanced Implementation intervention will lead to better implementation outcomes than Basic Implementation on each of the three primary outcomes assessed. Similarly, we hypothesize that clinicians in the Enhanced condition will report greater satisfaction with RxWL and will demonstrate greater engagement with the RxWL platform as a part of weight-related patient care.

Power and sample size.

Power was calculated first for the effectiveness aim, based on the minimum number of participants required to detect a clinically significant mean difference in weight change of 2 kg (SD=5.0) among RxWL maintenance conditions. To achieve a power of .80 to detect this difference with a type I error rate of 5%, 100 participants per patient group would be required, totaling to N=300 for three groups. Accounting for clustering of observations within NCMs and practices (anticipated ICC = .05) increases the total minimum sample to 567, which we rounded to a goal of N=600. This sample size will also yield a power of at least .80 to detect a small to medium effect size of at least φ=.41 [37] when comparing an expected Basic Implementation NCM referral rate of .35 to .5 for the Enhanced group (implementation aim).

Discussion

This paper summarizes the design and methods for a pragmatic trial to identify optimal methods for dissemination of an empirically validated online behavioral obesity treatment in primary care. The approach is informed by the effectiveness-implementation hybrid design framework, which emphasizes simultaneous evaluation of effectiveness and implementation outcomes [22]. The trial’s aims are informed by an emerging focus of the National Institutes of Health on improving routine care outcomes for obesity and diabetes care through pragmatic and disseminable research trials. Despite high rates of obesity and comorbid cardiometabolic disease [38, 39] and a large body of evidence supporting effective management of these conditions [2], few evidence-based interventions are integrated into routine community clinical practice [40]. Research regarding the best ways to implement evidence-based behavioral health interventions in front-line treatment settings, like primary care, are crucial in tackling these highly prevalent and costly medical concerns.

A key premise of this study is that in order to maximize the reach and impact of lifestyle interventions, they should be carefully integrated into the primary care setting. To this end, the trial aims to address key barriers that limit the reach and impact of empirically validated behavioral weight management treatments, namely limited provider training and resources to deliver obesity treatment in primary care. The Enhanced Implementation summarized here is designed to address barriers to successful delivery of behavioral weight loss treatment in primary care by fully integrating Rx Weight Loss into routine practice. We hypothesize that providers who receive the Enhanced Implementation will have higher patient referral, enrollment, and treatment completion rates, relative to providers who receive the less comprehensive Basic Implementation.

This study innovates upon existing evidence for the RxWL program, and primary care-based obesity treatment more broadly, in that it focuses on effective methods for weight loss maintenance in addition to active weight loss. Health outcomes are maximized when losses are maintained long-term [41], but post-treatment weight regain remains a major challenge in the field [42]. Refresher Courses and standard Monthly Lessons approaches to providing on-going contact and increasing self-regulatory behaviors have both shown efficacy in previous trials [18, 19, 43, 44]. Regarding treatment effectiveness, we hypothesize that patients will have greater success at weight loss maintenance when the RxWL maintenance intervention is delivered in short, concentrated bursts (weekly for one month) versus less-concentrated monthly lessons.

Strengths and pragmatic relevance

This study is strengthened by its practical relevance and potential to extend the reach and long-term impact of Web-based behavioral obesity treatment. It is the first study to compare two approaches to weight loss maintenance directly in the context of online obesity treatment. Identifying the optimal approach to weight loss maintenance in primary care is key to ensuring maximal health benefit and improving the public health impact of the RxWL program.

This project is also strengthened by its pragmatic design. It capitalizes on resources within the contemporary model of healthcare delivery and eHealth technology. This enables low-cost implementation of empirically validated behavioral obesity treatment with direct integration in primary care. By nature of the pragmatic design, sustained implementation of the RxWL program will continue within RIPCPC practices well after the trial is complete. Moreover, involvement of research staff in the implementation is limited, and patient inclusion criteria are broad. These factors maximize generalizability of results and confer direct relevance to other routine care settings.

Historically, behavioral health innovations have taken an average of 17 years to become translated into routine clinical practice [45, 46]. The hybrid effectiveness-implementation trial design accelerates this process by allowing for direct testing of the RxWL program (and methods for implementation) within the settings to which it will ultimately be disseminated. As a result, uptake in other community clinical practices (e.g., in other local clinics or across the United States) may be more rapid and the translation timeline shortened [45].

Limitations and challenges

Although this study is innovative in its aim to improve obesity treatment disseminability, we anticipate some limitations and challenges. Providers and patients may vary in their familiarity with and readiness to use technology in treatment [47, 48]. This is particularly relevant in primary care, given that older adults are a fast-growing segment of the patient population and are typically less likely to have experience with Web-based and mobile health applications [49]. Anticipating a wide range of proficiency with Web-based applications, our research team includes support staff to provide remote technical assistance. In addition, internet access is growing faster among older adults than the general population [50]. We expect that RxWL will become increasingly relevant to care for this patient group with time.

Another challenge is that many groups that are disproportionately affected by obesity (e.g., African-Americans and lower-income individuals) have high rates of “smartphone dependence” (i.e., access to internet only via a smartphone versus by personal computer or broadband in the home) [25]. These individuals may be reluctant to use a Web-based program versus a self-contained smartphone app. However, for these patient groups, automated, online programs that can be delivered directly via smartphone, and at very low cost and burden to the user, they may be particularly advantageous. To increase access for smartphone-dependent patients, the RxWL portal has been streamlined for ease of use across a variety of desktop, laptop, tablet, and smartphone devices.

Given the pragmatic nature of the trial and in an effort to minimize researchers’ direct involvement in the implementation, outcome data are drawn directly from the EMR. Data may be differentially available depending on frequency of follow-up care. Patients who move or transfer care to another provider may also have incomplete data. In addition, consistent with results from other studies of online obesity treatment, we may encounter high levels of attrition and difficulty maintaining patient engagement with the program over several months [51, 52]. These challenges are inherent in the nature of an implementation-effectiveness trial design, and the disadvantage of missing data is outweighed by the high disseminability of results. Minimizing researcher involvement in retention allows these numbers to approximate those observed in usual care as closely as possible. Further, the program’s utility in facilitating patient weight loss is also reflected in the implementation outcomes. For these measures, level of engagement and/or treatment completion is documented for all patients with no missingness. Treatment completion rates observed in this study will therefore provide important information about the potential public health impact of online obesity treatment if made freely available to the public in the future.

Conclusion

In sum, this study will provide essential information about the feasibility and utility of implementing RxWL in primary care, and it will simultaneously provide novel information on the most effective approaches to weight loss maintenance for patients in this setting. This project is likely to fill key gaps in evidence regarding best practices for implementation of online behavioral obesity treatment in primary care and the even more limited evidence regarding promotion of weight loss maintenance. Given the high prevalence of obesity and its connection to multiple health concerns that are typically managed in primary care, there is high potential for public health impact.

Acknowledgments

Funding

This trial is supported by a grant from the National Institute of Diabetes and Digestive and Kidney Diseases [R18: DK114715].

Footnotes

Competing Interests Statement

The authors have no competing or conflicts of interest to disclose.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • 1.Fitzpatrick SL, Wischenka D, Appelhans BM, Pbert L, Wang M, Wilson DK and Pagoto SL, An evidence-based guide for obesity treatment in primary care. The American Journal of Medicine, 2016. 129(1): pp. 115.e1–115.e7. doi: 10.1016/j.amjmed.2015.07.015 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Jensen MD, Ryan DH, Apovian CM, Ard JD, Comuzzie AG, Donato KA, … Kushner RF, 2013 AHA/ACC/TOS guideline for the management of overweight and obesity in adults: A report of the American College of Cardiology/American Heart Association task Force on practice guidelines and The Obesity Society. Circulation, 2014. 129(suppl 2): pp. S102–S138. doi: 10.1161/01.cir.0000437739.71477.ee [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Eden KB, Orleans CT, Mulrow CD, Pender NJ and Teutsch SM, Does counseling by clinicians improve physical activity? A summary of the evidence for the U.S. Preventive Services Task Force. Ann. Intern. Med, 2002. 137(3): pp. 208–15. doi: 10.7326/0003-4819-137-3-200208060-00015 [DOI] [PubMed] [Google Scholar]
  • 4.Centers for Disease Control and Prevention. Ambulatory care use and physician office visits 2016. [cited 2018 December 13]; Available from: http://www.cdc.gov/nchs/fastats/physician-visits.htm. [Google Scholar]
  • 5.Wadden TA, Butryn ML, Hong PS and Tsai AG, Behavioral treatment of obesity in patients encountered in primary care settings: A systematic review. JAMA, 2014. 312(17): pp. 1779–1791. doi: 10.1001/jama.2014.14173 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Galuska DA, Will JC, Serdula MK and Ford ES, Are health care professionals advising obese patients to lose weight? JAMA Intern. Med, 1999. 282(16): pp. 1576–1578. doi: 10.1001/jama.282.16.1576 [DOI] [PubMed] [Google Scholar]
  • 7.Sciamanna CN, Tate DF, Lang W and Wing RR, Who reports receiving advice to lose weight? Results from a multi-state survey. Arch. Intern. Med, 2000. 160: pp. 2334–2339. doi: 10.1001/archinte.160.15.2334 [DOI] [PubMed] [Google Scholar]
  • 8.Huang J, Yu H, Marin E, Brock S, Carden D and Davis T, Physicians’ weight loss counseling in two public hospital primary care clinics. Acad. Med, 2004. 79(2): pp. 156–161. doi: 10.1097/00001888-200402000-00012 [DOI] [PubMed] [Google Scholar]
  • 9.Yarnall KS, Pollak KI, Østbye T, Krause KM and Michener JL, Primary care: Is there enough time for prevention? Am. J. Public Health, 2003. 93(4): pp. 635–641. doi: 10.2105/AJPH.93.4.635 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Smith S, Seeholzer EL, Gullett H, Jackson B, Antognoli E, Krejci SA and Flocke SA, Primary care residents’ knowledge, attitudes, self-efficacy, and perceived professional norms regarding obesity, nutrition, and physical activity counseling. J. Grad. Med. Educ, 2015. 7(3): pp. 388–394. doi: 10.4300/JGME-D-14-00710.1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Antognoli EL, Seeholzer EL, Gullett H, Jackson B, Smith S and Flocke SA, Primary care resident training for obesity, nutrition, and physical activity counseling: A mixed-methods study. Health Promot. Pract, 2017. 18(5): pp. 672–680. doi: 10.1177/1524839916658025 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Raaijmakers LC, Pouwels S, Berghuis KA and Nienhuijs SW, Technology-based interventions in the treatment of overweight and obesity: A systematic review. Appetite, 2015. 95: pp. 138–151. doi: 10.1016/j.appet.2015.07.008 [DOI] [PubMed] [Google Scholar]
  • 13.Thomas JG, Leahey TM and Wing RR, An automated internet behavioral weight-loss program by physician referral: A randomized controlled trial. Diabetes Care, 2015. 38(1): pp. 9–15. doi: 10.2337/dc14-1474 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Martin PD, Dutton GR, Rhode PC, Horswell RL, Ryan DH and Brantley PJ, Weight loss maintenance following a primary care intervention for low- income minority women. Obesity, 2008. 16(11): pp. 2462–2467. doi: 10.1038/oby.2008.399 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Wing RR, Tate DF, Gorin AA, Raynor HA and Fava JL, A self-regulation program for maintenance of weight loss. N. Engl. J. Med, 2006. 355(15): pp. 1563–1571. doi: 10.1056/nejmoa061883 [DOI] [PubMed] [Google Scholar]
  • 16.Ross Middleton K, Patidar S and Perri M, The impact of extended care on the long- term maintenance of weight loss: a systematic review and meta- analysis. Obes. Rev, 2012. 13(6): pp. 509–517. doi: 10.1111/j.1467-789X.2011.00972.x [DOI] [PubMed] [Google Scholar]
  • 17.Wing RR, Tate DF, Gorin AA, Raynor HA, Fava JL and Machan J, “STOP regain”: Are there negative effects of daily weighing? J. Consult. Clin. Psychol, 2007. 75(4): pp. 652–656. doi: 10.1037/0022-006X.75.4.652 [DOI] [PubMed] [Google Scholar]
  • 18.Jeffery RW, Levy RL, Langer SL, Welsh EM, Flood AP, Jaeb MA, … Finch EA, A comparison of Maintenance-Tailored Therapy (MTT) and Standard Behavior Therapy (SBT) for the treatment of obesity. Prev. Med, 2009. 49(5): pp. 384–389. doi: 10.1016/j.ypmed.2009.08.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Levy RL, Jeffery RW, Langer SL, Graham DJ, Welsh EM, Flood AP, … Hotop AM, Maintenance-Tailored Therapy vs. Standard Behavior Therapy for 30-month maintenance of weight loss. Prev. Med, 2010. 51(6): pp. 457–459. doi: 10.1016/j.ypmed.2010.09.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Perri MG, Limacher MC, Castel‐ Roberts K, Daniels MJ, Durning PE, Janicke DM, …Kim C, Comparative effectiveness of three doses of weight‐ loss counseling: Two‐ year findings from the rural LITE trial. Obesity, 2014. 22(11): pp. 2293–2300. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Dutton GR, Gowey MA, Tan F, Zhou D, Ard J, Perri MG and Lewis CE, Comparison of an alternative schedule of extended care contacts to a self-directed control: A randomized trial of weight loss maintenance. Int. J. Behav. Nutr. Phys. Act, 2017. 14(1): pp. 107. doi: 10.1186/s12966-017-0564-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Curran GM, Bauer M, Mittman B, Pyne JM and Stetler C, Effectiveness-implementation hybrid designs: Combining elements of clinical effectiveness and implementation research to enhance public health impact. Med. Care, 2012. 50(3): pp. 217–226. doi: 10.1097/MLR.0b013e3182408812 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Diabetes Prevention Program Research Group, The Diabetes Prevention Program (DPP): Description of lifestyle intervention. Diabetes Care, 2002. 25(12): pp. 2165–71. doi: 10.2337/diacare.25.12.2165 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Look AHEAD Research Group, The Look AHEAD study: A description of the lifestyle intervention and the evidence supporting it. Obesity, 2006. 14(5): pp. 737–52. doi: 10.1038/oby.2006.84 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Pew Research Center. Mobile Fact Sheet 2018. February 5 [cited 2018 October 23]; Available from: http://www.pewinternet.org/fact-sheet/mobile/. [Google Scholar]
  • 26.Diabetes Prevention Program Research Group, 10-year follow-up of diabetes incidence and weight loss in the Diabetes Prevention Program Outcomes Study. Lancet, 2009. 374(9702): pp. 1677–1686. doi: 10.1016/S0140-6736(09)61457-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Poirier P, Giles TD, Bray GA, Hong Y, Stern JS, Pi-Sunyer FX and Eckel RH, Obesity and cardiovascular disease: Pathophysiology, evaluation, and effect of weight loss an update of the 1997 American Heart Association Scientific statement on obesity and heart disease from the obesity committee of the council on nutrition, physical activity, and metabolism. Circulation, 2006. 113(6): pp. 898–918. doi: 10.1161/CIRCULATIONAHA.106.171016 [DOI] [PubMed] [Google Scholar]
  • 28.Magkos F, Fraterrigo G, Yoshino J, Luecking C, Kirbach K, Kelly SC, … Klein S, Effects of Moderate and Subsequent Progressive Weight Loss on Metabolic Function and Adipose Tissue Biology in Humans with Obesity. Cell Metab, 2016. doi: 10.1016/j.cmet.2016.02.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Lipson D, Rich E, Libersky J and Parchman M, Coordinating care for adults with complex needs in the patient-centered medical home: Challenges and solutions 2012, Agency for Healthcare Research and Quality: Rockville, MD. [Google Scholar]
  • 30.Volger S, Vetter ML, Dougherty M, Panigrahi E, Egner R, Webb V, … Wadden TA, Patients’ preferred terms for describing their excess weight: Discussing obesity in clinical practice. Obesity, 2012. 20(1): pp. 147–150. doi: 10.1038/oby.2011.217 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Rollnick S, Miller WR, Butler CC and Aloia MS, Motivational interviewing in health care: Helping patients change behavior 2008, Taylor & Francis. doi: 10.1080/15412550802093108 [DOI] [Google Scholar]
  • 32.Carels RA, Darby L, Cacciapaglia HM, Konrad K, Coit C, Harper J, … Versland A, Using motivational interviewing as a supplement to obesity treatment: A stepped-care approach. Health Psychol, 2007. 26(3): pp. 369. doi: 10.1037/0278-6133.26.3.369 [DOI] [PubMed] [Google Scholar]
  • 33.Krebs P and Duncan DT, Health app use among US mobile phone owners: A national survey. JMIR mHealth and uHealth, 2015. 3(4): pp. e101–e101. doi: 10.2196/mhealth.4924 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Glasgow RE, Strycker LA, Kurz D, Faber A, Bell H, Dickman JM, … Osuna D, Recruitment for an internet-based diabetes self-management program: Scientific and ethical implications. Ann. Behav. Med, 2010. 40(1): pp. 40–48. doi: 10.1007/s12160-010-9189-1 [DOI] [PubMed] [Google Scholar]
  • 35.Freedman VA and Wolf DA, A case study on the use of multiple imputation. Demography, 1995. 32(3): pp. 459–470. [PubMed] [Google Scholar]
  • 36.Little RJ and Wang Y, Pattern-mixture models for multivariate incomplete data with covariates. Biometrics, 1996: pp. 98–111. [PubMed] [Google Scholar]
  • 37.Cohen J, Statistical power analysis for the behavioral sciences 1988, Hillsdale, New Jersey: Lawrence Erlbaum Associates. [Google Scholar]
  • 38.Hales CM, Carroll MD, Fryar CD and Ogden CL, Prevalence of obesity among adults and youth: United States, 2015–2016, in NCHS data brief 2017, National Center for Health Statistics: Hyattsville, MD. [PubMed] [Google Scholar]
  • 39.Nguyen NT, Nguyen X-MT, Lane J and Wang P, Relationship between obesity and diabetes in a US adult population: Findings from the National Health and Nutrition Examination Survey, 1999–2006. Obes. Surg, 2011. 21(3): pp. 351–355. doi: 10.1007/s11695-010-0335-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Scamuffa RF, Prevalence and determinants of engagement with obesity care in the United States. Obesity, 2018. 26(5): pp. 814–818. doi: 10.1002/oby.22173 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Neiberg RH, Wing RR, Bray GA, Reboussin DM, Rickman AD, Johnson KC, … Espeland MA, Patterns of weight change associated with long- term weight change and cardiovascular disease risk factors in the Look AHEAD Study. Obesity, 2012. 20(10): pp. 2048–2056. doi: 10.1038/oby.2012.33 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Look AHEAD Research Group, Eight-year weight losses with an intensive lifestyle intervention: The Look AHEAD study. Obesity, 2014. 22(1): pp. 5–13. doi: 10.1002/oby.20662 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Look AHEAD Research Group, Long-term effects of a lifestyle intervention on weight and cardiovascular risk factors in individuals with type 2 diabetes mellitus: Four-year results of the Look AHEAD trial. Arch. Intern. Med, 2010. 170(17): pp. 1566–75. doi: 10.1001/archinternmed.2010.334 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Perri MG, Limacher MC, von Castel- Roberts K, Daniels MJ, Durning PE, Janicke DM, … Kim C, Comparative effectiveness of three doses of weight‐ loss counseling: Two‐ year findings from the rural LITE trial. Obesity, 2014. 22(11): pp. 2293–2300. doi: 10.1002/oby.20832 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Westfall JM, Mold J and Fagnan L, Practice-based research—”Blue Highways” on the NIH roadmap. JAMA, 2007. 297(4): pp. 403–406. doi: 10.1001/jama.297.4.403 [DOI] [PubMed] [Google Scholar]
  • 46.Morris ZS, Wooding S and Grant J, The answer is 17 years, what is the question: Understanding time lags in translational research. J. R. Soc. Med, 2011. 104(12): pp. 510–520. doi: 10.1258/jrsm.2011.110180 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Sadasivam RS, Hogan TP, Volkman JE, Smith BM, Coley HL, Williams JH, … Ford DE, Implementing point of care “e-referrals” in 137 clinics to increase access to a quit smoking internet system: The Quit-Primo and National Dental PBRN HI-QUIT studies. Transl. Behav. Med, 2013. 3(4): pp. 370–378. doi: 10.1007/s13142-013-0230-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Horrigan JB Digital readiness gaps. [Report] 2016. September 20 [cited 2018 November 1]; Available from: http://www.pewinternet.org/2016/09/20/digital-readiness-gaps/.
  • 49.Levine DM, Lipsitz SR and Linder JA, Trends in seniors’ use of digital health technology in the United States, 2011–2014. JAMA, 2016. 316(5): pp. 538–540. doi: 10.1001/jama.2016.9124 [DOI] [PubMed] [Google Scholar]
  • 50.Pew Research Center. Internet/Broadband Fact Sheet 2018. February 5 [cited 2018 December 10]; Available from: http://www.pewinternet.org/fact-sheet/mobile/. [Google Scholar]
  • 51.Levine DM, Savarimuthu S, Squires A, Nicholson J and Jay M, Technology-assisted weight loss interventions in primary care: A systematic review. J. Gen. Intern. Med, 2015. 30(1): pp. 107–117. doi: 10.1007/s11606-014-2987-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.van Beurden SB, Simmons SI, Tang JC, Mewse AJ, Abraham C and Greaves CJ, Informing the development of online weight management interventions: A qualitative investigation of primary care patient perceptions. BMC Obesity, 2018. 5(1): pp. 7. doi: 10.1186/s40608-018-0184-6 [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES