The SMARTER Trial: Design of a trial testing tailored mHealth feedback to impact self-monitoring of diet, physical activity, and weight

Abstract

Background:

Self-monitoring food intake and physical activity (PA) is positively related to weight loss and the addition of feedback (FB) messages has been shown to reinforce behavior change. Moreover, the more immediate the delivery of reinforcing FB messages, the more likely they will promote the desired behaviors.

Purpose:

Describe design and rationale of SMARTER, a National Institute of Heart, Lung, and Blood (NHLBI)sponsored randomized, controlled trial, which compares the differential efficacy of two weight loss treatments among 530 adults, ages 18 and older.

Methods:

Single-site, 2-group design trial with subjects randomized 1:1 to either: 1) self-monitoring (SM), where participants self-monitor diet, PA, and weight using a commercial smartphone application (app); or 2) SM + FB, where participants self-monitor and receive real-time, tailored feedback (FB) as pop-up messages up to 3 times/day for 12 months. Daily FB messages address diet and PA behaviors and a weekly FB message addresses self-weighing. We hypothesize that subjects assigned to SM + FB will show greater weight loss at 6 and 12 months and greater sustained engagement in the program than the SM group, measured by adherence to the study’s lifestyle and SM protocol. We will explore temporal relationships of the frequency, timing, and type of FB delivered and subsequent lifestyle behaviors through examination of serially collected real-time SM (diet, PA, weight) data over 12 months.

Conclusions:

If efficacious, this fully scalable intervention could be efficiently translated and disseminated to reach large numbers of individuals through commercial apps at lower cost than existing in-person weight loss programs.

1. Introduction

Obesity is a chronic condition characterized by weight loss and regain [1] and a well-established risk factor for cardiovascular disease (CVD) and diabetes [2]. At the core of weight management is standard behavioral treatment (SBT) [3]; however, traditional SBT has limitations (e.g., limited sustainability and high cost) [4]. Also, individualized feedback (FB) on dietary and physical activity (PA) self-monitoring is provided after the SBT interventionist reviews data, which may be days or weeks after behavior has occurred, despite extensive evidence that immediate reinforcement increases occurrence of desired behavior [5,6]. Most importantly, adults with obesity do not typically have access to SBT. By using an innovative approach capitalizing on mobile technology and providing real-time tailored FB to self-monitoring, we may be able to offer access to weight management support to those who do not have access to SBT or wish to participate in an in-person program. Moreover, a remotely guided FB system is sustainable and continuously available, unlike SBT programs.

Through a series of trials, our team has documented that consistency and timing of self-monitoring in relation to eating is significantly related to better weight loss outcomes [7–9]. Our SMART Trial tested the use of one of three tools for self-monitoring (SM) dietary intake and physical activity (PA): paper diary, personal digital assistant (PDA), or PDA with a daily, tailored FB message delivered in real time (PDA + FB). Those who received the daily FB messages improved dietary intake and were more likely to achieve PA goals. The results led us to further develop and expand the algorithm reading the SM data and providing FB on weight. We tested it in the SMARTER pilot RCT study [10], the results of which suggested the need for a larger efficacy trial and informed the design of the current study, the SMARTER trial.

The SMARTER trial addresses two major gaps in the mHealth field – the absence of efficacy data on the use of commercially-available self-monitoring smartphone applications (apps) for weight loss and the paucity of data on sustainability of engagement with these apps. We are answering these critical questions by conducting a full-scale, two-group design randomized clinical trial (RCT) and randomizing 530 adults to one of two groups: (1) Self-Monitoring (SM) using a commercially available Fitbit app and weight using a Bluetooth-enabled scale, and (2) SM + FB, where participants self-monitor as described for the SM group but also receive three tailored FB messages per day. We hypothesize that participants assigned to the SM + FB group will show greater weight loss at 6 and 12 months than those assigned to the SM group. We also hypothesize that participants assigned to SM + FB will show greater sustained engagement in the self-directed weight loss program, as measured by adherence to components of study protocol at 3, 6, 9 and 12 months, than those assigned to the SM group. Additionally, we will examine temporal relationships of the frequency, timing and type of FB delivered and subsequent lifestyle behavior. The purpose of this paper is to describe the study design, methods, and the conceptual framework guiding the SMARTER trial.

2. Background

Programs that target behavior change and weight loss maintenance are built on strategies that support the individual’s ability to self-regulate behavior, of which SM is a necessary component. Kanfer’s theory of self-regulation provides the theoretical basis for SM [11,12]. Kanfer describes self-regulation as a process having three distinct stages: SM, self-evaluation, and self-reinforcement, and suggests that changing habits requires developed self-regulatory skills [12]. SM is central to this process, and includes deliberate attention to some aspect of behavior as well as recording details of that behavior. To change behaviors, individuals need to pay adequate attention to their own actions, the conditions under which they occur and their immediate and long-term effects [13]. Thus, successful self-regulation depends on the fidelity, consistency and temporal proximity of SM [13]. Our qualitative study findings on the experience of SM supported the importance of temporal proximity of recording, demonstrating that not only SM but also timeliness is important for improved outcomes [9,14], these findings were confirmed in our PDA study (SMART Trial) [15].

There is strong theoretical support and increasing empirical support for dietary SM and its central role in promoting self-regulatory skills and progress toward goal attainment (Fig. 1) [16–21]. Consistent with these findings, additional behaviors have been added to the SM protocol [22], e.g., PA and self-weighing [23]. It is striking what little progress has been made in the scientific arena of developing and testing evidence-based tools to support SM and reduce the burden to individuals [24]. The paper diary (PD) continues to be used in SBT, although time-consuming and tedious [14,25], and prohibitive of immediate, real-time FB for support and motivation. The technology available today can facilitate tailored, personalized FB that can be delivered remotely and in real-time in close proximity to the behavior. Theoretical underpinnings for optimizing the timing of the FB messages are drawn from behavioral theory that demonstrates that positive reinforcement for an emitted, desired behavior provided immediately following the behavior leads to increases in the occurrences of the desired behavior; the more proximal the reinforcer to the desired behavior, the more likely the desired behavior will be increased [1,26]. Applied to the important strategy of SM, reinforcing FB has the potential to increase SM and lead to improvements in the behaviors being tracked, specifically diet and PA. Also, current mobile technology will enable us to examine the efficacy of delivering tailored FB as pop-up messages on a smartphone, messages that are driven by an algorithm that reads the SM data in real time [10]. Ample evidence supports the role of FB as a reinforcing motivational tool for behavior change, especially when delivered more proximally to progress toward goal attainment [26,27]. Based on the energy and fat intake improvements we observed in response to the daily FB that targeted these behaviors in our earlier trial, this RCT is the logical next step in this line of investigation [28].

Fig. 1.

Conceptual model for the SMARTER intervention.

In the SMARTER trial, baseline screening will document the participant’s need and interest in improving lifestyle habits. Self-regulation guides the process of self-monitoring diet, PA, and weight and has the participant engage in self-review and self-evaluation of his/her progress toward goals. A robust reinforcement will be delivered through the tailored FB messages that are responsive to the reported behaviors, i.e., the algorithm reviews the incoming data, selects an appropriate message from the library, and delivers a message that reinforces progress but also provides guidance for behavior that is within the goals. The intervention is dynamic, interactive and adaptive, drawing upon the reciprocal determinism concept that is part of Social Cognitive Theory [29]. Reciprocal determinism posits that there is a continuous bi-directional influence between the environment, behaviors and personal factors such as cognitions, emotions and physiological cues. Riley et al. noted that the dynamic nature of reciprocal determinism makes it fitting to guide behavioral interventions delivered via mobile technologies [30]. The FB intervention plays a major role in self-efficacy enhancement by establishing specific, proximal goals with the individual at baseline, e.g., parts of diet to change, facilitating success and mastery performance and then using FB to attribute the success to the individual’s efforts. In sum, adherence and self-efficacy provide the pathway through which the intervention will impact primary goals: healthier lifestyle habits and lower weight.

3. Study design/methods

3.1. Setting

The recruitment site for the SMARTER study is the University of Pittsburgh with Institutional Review Board approval obtained from the same institution. The study is registered with clinicaltrials.gov (NCT03367936). Recruitment commenced in August 2018.

3.2. Participants and recruitment

To obtain our target of 530 subjects, we will use all the strategies that have been successful in our previous trials, i.e., Pitt+Me (a research registry hosted by the Clinical Translational Science Institute (CTSI)), university email announcements, electronic and postal mailings, study announcements on various social media sites, and fliers posted in the community. We will also recruit from primary care practices with whom the Co-PI has collaborated for previous lifestyle studies [31]. Table 1 outlines the inclusion and exclusion criteria. We aim for a minimum 30% male representation and a black representation that reflects that of Allegheny County, PA (13.4%). See Fig. 2 for outline of screening, randomization and assessment procedures.

Table 1.

Inclusion and exclusion criteria.

Inclusion

Exclusion

Age ≥ 18 years Currently own and regularly use a smartphone with a data plan (iPhone or Android) BMI ≥ 27 and ≤ 43 Successfully complete a 5-day electronic food diary during the run-in period

Have an unstable condition requiring physician-supervised diet and exercise (e.g. recent myocardial infarction) Physical limitations that preclude moderate-intensity PA Pregnant or intending to become pregnant during the study period Serious mental illness (e.g. schizophrenia) Alcohol intake > 4 drinks/day Currently in a formal weight loss program or use of weight loss medication History of bariatric surgery Score > 32 on the Eating Habits Checklist There are no restrictions on current ownership/use of a Fitbit or other device. Participants need to be willing to use the Fitbit device provided (or their own Fitbit device)

Fig. 2.

Study recruitment, screening, randomization, and assessments.

3.3. Screening

In Phase 1, respondents to recruitment solicitations will be provided a link to complete a basic screening survey [32] to determine if they meet eligibility criteria (i.e., age, height and weight to calculate BMI); those who meet these criteria will be sent a link to complete questionnaires. After completing the online consent to be screened in Phase 2, they will provide personal information that is requested in the Sociodemographic and Lifestyle Questionnaire and health and medical conditions in the General Health History Questionnaire. They also will be asked to complete the Eating Habits Checklist (EHC) and the Center for Epidemiologic Studies Scale (CES-D). Completion of these surveys takes approximately 15 min. Table 2 outlines the measures in the study.

Table 2.

Study measures.

Measure	No. of items	Collection time point
		PB	B	6mo	12mo
Screening measures
Medical and weight history form	43 items	✓
- weight history, medical history, and lifestyle habits
Sociodemographic questionnaire	22 items	✓
- gender, race, income, education, neighborhood, work location/environment, and social setting
Eating habits checklist [33]	16 items (cut-off score:	✓
-screens for eating disorder, higher indicates disordered eating	> 32)
Center for Epidemiologic Studies-Depression (CES-D) [34]	20 items (cut-off score:	✓
- identifies risk for clinical depression, higher score = higher risk	≥16)
Quick WAVE screener [35]	17 items	✓
- assesses current diet and PA habits
Cardiometabolic risk factors
Omron blood pressure monitor cuff	NA		✓	✓	✓
- blood pressure taken while the subject is seated after at least 5-min rest, mean of 3 readings used
Gullick II measuring tape	NA		✓	✓	✓
- waist circumference measured twice following established protocol If the two values are within 2 cm of each other, a mean is calculated; if not, measurements are taken until they are within 2 cm.
Tanita scale (Model BF-350)	NA		✓	✓	✓
- measures weight and body composition
Stadiometer	NA		✓		✓
- measures height
Psychosocial measures (potential confounders)
Self-regulation of eating behavior questionnaire [36]	5 items		✓	✓	✓
- assesses eating self-regulatory capacity
Barriers to healthy eating [37–39]	22 items		✓	✓	✓
- assesses potential barriers to healthy eating
Weight efficacy lifestyle (WEL) [40,41]	20 items		✓	✓	✓
- assesses level of confidence to resist eating in varied situations or emotional states.
Self-efficacy and eating habits survey [42,43]	20 items		✓	✓	✓
- assesses individual’s confidence in motivating themselves to make dietary changes for at least 6 mo.
Self-efficacy and exercise habits survey [43]	11 items		✓	✓	✓
- assesses individual’s confidence in motivating themselves to increase or continue regular exercise for at least 6 mo.
Effort-reward imbalance questionnaire [44]	22 items			✓	✓
- quantify perceived imbalance between effort and reward in the work environment
Self-report measure of dietary goal adherence
Automated self-administered 24HR (ASA24) [45]			✓*	✓*	✓*
- 24-h dietary recall
Intervention satisfaction questionnaire	SM 17 items, SM + FB 32				✓
- assesses participant’s perception of intervention, explores acceptance and usability of SM devices, and most importantly, frequency, content, and timing of t FB messages (for those in SM + FB group).	items

Note: PB – pre-baseline; B – Baseline;

^*on two separate days at each assessment.

Individuals who remain eligible will be consented for further screening over the phone to complete a 5-day SM diary using the Fitbit SM app on their phone or Fitbit website (Phase 3) to minimize attrition and ensure that potential participants are appropriate for the intervention. We have found that this screening SM diary helps individuals evaluate their willingness and ability to self-monitor. Study staff create a Fitbit account for those who consent to participate in the 5-day food diary. Individuals are sent an email which details Fitbit username and password, and how to download the Fitbit app or log into the Fitbit website. Instructions for how to log foods online or in the Fitbit app are also included. Individuals need to record at least 700 kcal/day of food intake to be eligible for the study. Diaries are assessed by a master’s prepared registered dietitian. Food diaries may be referred to during the 1:1 intervention session with the dietitian. No specific feedback regarding the food diary entries are provided at the time of the diary review.

3.4. Baseline

After completing their 5-day SM diary, individuals who have recorded at least 700 cal per day will be invited to a baseline assessment where their height/weight are measured to verify BMI. An explanation of the aims of the study and the 2 group conditions is provided. If still interested in participating, individuals will be asked to sign a consent form to participate in a randomized controlled trial. At baseline, blood pressure and waist circumference are measured. The participants then complete 5 questionnaires on a center laptop. Also, the participants complete two 24-h dietary recalls using the Automated Self-Administered 24HR (ASA24) on the National Cancer Institute (NCI) site, one of which is completed at the Clinical Research Suite (CRS), so the research staff can assist the participant the first time [45]. The baseline visit requires approximately 60 to 90 min. Within two weeks of the baseline visit, participants return for a 90-min, 1:1 session with a registered dietitian. To summarize the screening consent process that precedes written consent, there is a consent to sign for phase 1 and phase 2 screening. When completing the surveys and clicking “next”, there is implied consent to complete each survey. For phase 3, we verbally consent participants to participate in a randomized intervention study.

3.5. Randomization

Randomization is performed at the beginning of the 1:1 session. The research staff use a computer software program to determine to which group the individual will be randomized with equal allocation. Randomization assignments will be generated using minimization, an adaptive randomization method that achieves balance between the treatment groups based on the marginal treatment totals within levels of each of the stratification factors of gender and race (black, non-black).

3.6. 1:1 Session

A registered dietitian covers the core principles of behavior change for weight loss to ensure that all participants understand the basic re commendations for safe and effective weight loss.

During the 1:1 session with a registered dietitian, a research assistant orients the participants on how to use the wireless weighing scale and Fitbit Charge 2™—including the multiple ways in which to record physical activity. Participants are shown how to track activity on their own Fitbit device, how to manually log into the Fitbit app, and the automatic tracking of some exercises by the Fitbit device is also described. In addition to going over physical activity tracking in person, a handout is provided for participants to refer to later. Participants randomized to the SM + FB group are then assisted in downloading the SMARTER app and shown how to view the feedback messages.

3.7. Intervention description

3.7.1. Dietary intake

The components of the intervention are summarized in Table 3. All participants will have an account created for the Fitbit app by study staff. The account can be accessed via their smartphone or any computer. The Fitbit app software permits SM of diet and does not include extensive feedback or messaging to the user, which we liked when we tested this app during a preliminary phase of the current study. Through access to the database we are able to view participants’ food entries as well as total amount of calories, fat grams, and other nutrition components such as sodium. For physical activity, the database shows the total number of steps and number of minutes in the sedentary, lightly active, fairly active, and very active categories. These values are provided per participant-logged activities as well as heart rate. Participants view the nutrient value of each food and the calculated sub totals and daily summaries of intake. Participants are also provided a set of instructions outlining how to search for and record foods in the Fitbit food log. Calorie goal is determined from the person’s baseline body weight (women: 1200 kcal for < 200 lbs. or 1500 kcal for > 200 lbs.; men: 1500 kcal for < 200 lbs. or 1800 kcal for > 200 lbs) [46]. Fat gram goals approximate 25% of the calorie goal, e.g., 33 or 42 g per day for females. Calorie goals can be individualized to achieve weight loss for participants struggling to meet them or for maintenance.

Table 3.

Intervention protocol summary.

Intervention component	SM + FB	SM only
1. Fitbit app for self-monitoring of dietary intake	✓	✓
2. Fitbit Charge 2™ for self-monitoring physical activity (PA)	✓	✓
3. Wireless scale for weight self-monitoring	✓	✓
4. Personalized fat, calorie, and PA goals for weight loss	✓	✓
5. Diabetes Prevention Program (DPP) online	✓	✓
6. Feedback messages through the SMARTER app	✓

3.7.2. Physical activity

We provide all participants a wrist-worn Fitbit Charge 2™ to self-monitor their PA, which syncs with their phone. Participants are told to increase their PA gradually, primarily by walking, and aim for 150 min/week by 12 weeks, the goal established by the American Heart Association (AHA) and the American College of Sports Medicine (ACSM) for American adults. Once at goal, they will be encouraged to add 10 min/wk. with an ultimate goal of reaching 300 min/week, the goal established by the Institute of Medicine (IOM) [47–49]. All aerobic activities, e.g., bicycling, will count toward these goals. We will ask them to disregard Fitbit energy expenditure to “allow” higher calorie intake. They will be encouraged to increase PA throughout the day [50].

3.7.3. Weight

We provide all participants a wireless scale that will transmit weight data to their smartphone. Since the use of the scale and PA tracker is now standard part of SM programs, participants in both groups will be provided these tools so that we can focus on the effect of the FB algorithm on outcomes.

3.7.4. Feedback messages

The FB algorithm that is programmed on the server in our center will use real-time synced SM data to send messages that are responsive to the participants’ SM entries. The server will deliver a prompt that is displayed on the phone home screen that there is a new SMARTER message. The participant needs to click on this message to go to the SMARTER app and view the message, which is delivered 3 times daily over the 12-month intervention. The FB messages address one target behavior at a time – physical activity (e.g., minutes of PA) or a dietary intake (e.g., calories, fat, or sugar). The algorithm reads the data in blocks of time (e.g., 10 am to 1 pm) and looks for conditions that match parameters set for that time of day in the algorithm (see examples in Table 4). This results in a variable FB schedule. In our pilot study, the number of messages sent varied from 2 to 4 times on a given day and was acceptable to the participants [10]. We also send a feedback message once every 6–8 days that addresses weight. If no weights or PA are recorded for 7 days, the dashboard notes with a red dot; the goal is to send a message to the participant by day 14 to determine the basis for no entries or no data from the Fitbit or scale. Pilot participants requested that they be able to save the messages for review later; this feature has been added to the FB program in the current trial. Physical activity feedback messages are sent based on the number of steps taken and the number of minutes of activity in the previous week, for example, Being more active may help you feel more energetic and sleep better. A double win! Feedback on weight is based on whether or not self-weighing occurred and based on the amount and rate of weight loss, maintenance, or gain, for example, It takes time to form healthy habits for weight loss. Stay on track and you will see the results! Table 4 provides examples of dietary feedback messages and the calorie, fat, and sugar criteria that are used to determine an appropriate message.

Table 4.

Algorithm conditions for percentages of participant goals of calories, fat, and sugar at breakfast (10 AM-1 PM) and examples of possible FB messages.

Condition number	Calories (% of goal)	Fat (% of goal)	Sugar (g)	Examples of possible FB messages
1	0			Did you eat breakfast today? Don’t forget to record it!
2	> 0 - < 25%			Your calories seem low for this time of day. Are you logging every food you are consuming?
3	25–30%	25–30%		Way to log your food and stay on track with calories and fat so far today!
4	> 30%	> 30%		You started your day with some higher fat choices. Control your fat intake for the rest of the day by selecting non-fat or low-fat foods. You can do it!
5	25–30%	< 25%		Way to monitor! You are keeping calories on target and have room for fat at your next meal. Enjoy!
6	> 30%	25–30%		Way to keep your fat intake at a reasonable level so far today. Keep an eye on portion sizes to manage calories.
7	25–30%	> 30%		Choose lower fat foods as the day goes on to stay on track with your calorie and fat goal.
8			< 35	Nice effort recording your foods. Sugar intake is on target so far today, way to be!
9			≥35	Added sugars can sneak into your meals and snacks! Many processed foods have sugar. Look at the food labels to choose foods with less sugar content.

As with any intervention that requires regular messaging, there is a concern about habituation or that the participant will become de-sensitized to the FB. We are monitoring participants’ responses to the FB. If there is a change in the inverse to the desired target behaviors, we will consider the possibility that habituation is occurring. Based on the desensitization that we observed in the SMART Trial [28], we are changing the library of messages at least monthly as we did in the pilot study without problems. At the completion of the proposed study, we will ask participants for feedback on the content of the messages they received and suggestions for improvements in the FB system.

3.8. Adherence to dietary and PA goals, and self-weighing

Adherence to dietary SM is defined as reporting 50% or more of the energy goal (e.g., participants prescribed a 1200 kcal/day diet will be adherent if they record at least 600 kcal/day). Adherence to SM behavior (e.g., wearing PA monitor, using scale) will be differentiated from adherence to PA goal by the presence or absence of data, (e.g., did not exercise will be confirmed by Fitbit data, and absence of data indicates device not worn or scale not used). No recording is defined as non-adherence to SM. Adherence to PA is defined as achieving ≥150 min of PA per week, thus % adherence can be calculated as reported active minutes divided by 150 min (PA goal) × 100%, e.g., (100/150) × 100% = 66%. Adherence to self-weighing is defined as weighing once daily; percent adherence is calculated as the number of times weight data are provided for each week divided by 7 × 100%, which includes weighing more than once daily.

3.8.1. Temporal relations between feedback and subsequent behavior

We will be able to examine the effect of a feedback message on a subsequent behavior. Message delivery is date-and time-stamped; the PA data are also date-and time-stamped (each activity and each time the Fitbit is synced with phone). Weight data are date-and time-stamped, so the effect of the weekly summary message can be examined on weight subsequent to the message delivery. A behavioral science expert will work with the research team in the analyses of these data.

3.9. Data management

Based on the study requirements, we developed an integrated online data management system called Awesome Data Acquisition Method (ADAM). As the study data management system, ADAM stores the study data from multiple sources, including: 1) intake questionnaires, 2) participants’ Fitbit data, 3) participants’ weight data from their self-weighing scale, and 4) backup data from the study data sources that do not support a real-time data connection or an Application Programmable Interface (API). The study intake questionnaires include screening, baseline, and 6-and 12-month questionnaires. ADAM supports the creation of templates and online links for those questionnaires, the automatic calculation for the scoring of each questionnaire, displaying the questionnaires to the participants, and storing the answers in the ADAM database.

ADAM allows real-time data synchronization from the Fitbit device and self-weighing scale using their respective APIs. In order to establish this API connection, at the start of the study, appropriate consents on the type of data that are being shared with the study must be completed through Fitbit. After this consenting process, the ADAM system will be able to pull the Fitbit data to the ADAM database through the system’s daily synchronization or in real-time as needed (e.g., when the study coordinator presses the manual data sync button in the ADAM portal). The same concept also applies to weight data synchronization from the scale. Two study data sources do not support or provide an API connection, ASA24 for the dietary recalls and assessment weight data from the Tanita scale. Data from these sources are uploaded every month to the ADAM system manually by the study coordinators.

3.9.1. Dashboard

To improve the efficiency of the research team in reviewing the study data and participant progress, we developed a dashboard in ADAM that includes the recruitment tracking, study status, participant flow, alert and flags. Using this approach, ADAM can provide a full picture of the status of each participant and allows the study coordinators to make the next decision based on the provided information (Fig. 3). The ADAM system also generates a real-time adherence report in the dashboard for all currently enrolled participants (see Fig. 4 for the dashboard image above). This report helps to notify the study coordinators of potential adherence problems and address the problem as early as possible. For example, the alert sign in Fig. 4 indicates that there are no diet data for the first participant and no weight data for the second participant in the last seven days. The number under “Adherence” also indicates the current condition of the participants. “Weighing” shows the percentage of days the participants are weighing themselves out of the total number of days in the study since randomization. “Monitoring” shows the percentage of the days the participants’ calorie input was ≥50% of their calorie goal. “Calorie” and “Fat” show the percentage of calories and fat grams the participant enters based on his/her personal daily goal. Participants in the intervention group are sent a push notification to their phone if they have not opened the study app within the last 10 days. Participants in both the intervention and control groups receive an email message if data on diet, weight, and physical activity have not been received in over 7 days. Troubleshooting instructions are provided as well as reminders about the importance of self-monitoring. Participants continue to receive emails once every 30 days if adherence does not improve.

Fig. 3.

SMARTER study infrastructure.

Fig. 4.

ADAM dashboard.

In addition to providing a general adherence report on all participants enrolled in the study, ADAM also allows a detailed data-visualized adherence report per participant (Fig. 5). This individual detailed report provides a real-time visualization tool for observing the relationship between weight change and adherence to the three types of self-monitoring. “Self-weighing” shows the percentage of days the participants weighed themselves. “Steps” shows the percentage of days the participants’ step count was ≥500. “Diet” shows the percentage of days the participants’ reported calorie input ≥50% of their calorie goal. Also, we have the timestamp data for when the messages were scheduled and if the participant opened the message within the one-hour window. We also have the timestamp data on when the messages were opened or read.

Fig. 5.

Participant adherence report.

The SMARTER app (Fig. 6) is used by the participants to receive notifications of FB messages three times per day, to trigger the wearable (Fitbit) data synchronization, and, most importantly, to receive the FB messages. Since the study adopts the Bring-Your-Own-Device (BYOD) approach, the app is designed to be a cross-platform smartphone app, meaning the app works on Apple iOS devices (iPhones, iPads) and Google Android devices. Hence, the participants can download the app directly from the App Store or the Google Play store into their own smartphone after they are enrolled in the study and provided the code. The app is connected in real-time to the ADAM server through an encrypted connection over Transport Layer Security (TLS).

Fig. 6.

Screenshot of the SMARTER app with sample feedback messages.

Since the app’s main purpose is to push and provide FB messages to the participants in the SM + FB group, the SMARTER app has a straightforward user interface (UI). The main page of the app (Fig. 6) shows all FB messages received by the participants ordered chronologically, with the newest message displayed at the very top. In terms of user input, because the participants will use the Fitbit app to input their calorie intake and record their activity and weight through the Fitbit device and wireless scale, respectively, the app has only a simple “Love it” button to indicate that the participants like or agree with the provided SM. To improve and to maintain adherence, the SMARTER app also supports a push notification system to remind the participants to open the app when prompted.

3.9.2. Sample size and power

The total sample size of 530 (or 265 per group) was determined focusing on the hypothesis testing associated with the primary study aims that target between-group differences (SM vs. SM + FB) in changes in weight at 6 and 12 months (Aim 1) and adherence variables at 3, 6, 9 and 12 months (Aim 2), adjusting the significance level for the multiple testing at key time points, and considering the expected rate of attrition by 12 months. The results of our 12-week pilot study exploring the effects of SM, SM + FB, and SM + FB + SBT on weight loss demonstrated mean (± SD) percent weight losses of −2.47 ± 4.54 and − 3.14 ± 4.36 for the SM and SM + FB groups, respectively, at 12 weeks [10]. Given these early 12-week findings and the expected differences in mean percent weight loss at 6 and 12 months based on the results of our SMART trial [51] (mean percent weight loss for PDA + FB group of −6.58 ± 6.77 and − 6.02 ± 7.73 at 6 and 12 months, respectively), mean differences are expected to increase at 6 and 12 months to at least twice that observed at 12 weeks. With this in mind, we would have 0.80 power to detect small to medium effect sizes on the mean percent weight changes at 6 and 12 months between the SM and SM + FB groups (Aim 1, Hypothesis 1) as small as d = 0.301 with linear contrasts from a linear mixed model at an adjusted significance level of 0.025 with a group sample size of 211 (or 422 total). Regarding the sustainability of engagement (Aim 2), adherence to lifestyle goals and self-monitoring in our SMART study tended to be the greatest in the PDA + FB group, where self-monitoring was accomplished electronically and feedback was given in real-time [18]. When examining between-group differences in adherence outcomes at 3, 6, 9, and 12 months (Aim 2, Hypothesis 2), we would have 0.80 power to detect small to medium effects as small as d = 0.326 with linear contrasts from a linear mixed model at an adjusted significance level of0.0125 with 211 per group (or 422 total). In longitudinal studies, attrition is likely to occur. In our most recent 12-month EMA study we experienced 12% attrition) [52]. To ensure at least 422 participants with complete information through 12-month follow-up time point, we plan to enroll 265 participants per group (530 total) to conservatively allow up to 20% attrition at 12 months.

3.9.3. Data analysis

Data will be analyzed using SAS (version 9.4, SAS Institute, Inc., Cary, NC) to conduct exploratory data analyses and linear mixed modeling. Although directional hypotheses have been posited, hypothesis testing will be two-sided. Significance levels will be set at0.025 and 0.0125 to limit type 1 error inflation for two-tailed testing of planned comparisons for weight change at 6 and 12 months (Aim 1) and adherence at 3, 6, 9, and 12 months (Aim 2), respectively. Confidence intervals (two-sided) for estimates from the linear contrasts will be computed in light of this adjustment. Fixed and/or time-dependent covariates (including baseline values) may be included in models to adjust for group imbalances, variables related to outcome variables or based on the literature. Mplus (version 8.4, Muthén & Muthén, Los Angeles, CA) will be used to explore the effect of feedback on subsequent adherence as a within-subject causal process (Aim 3).

3.9.3.1. Preliminary analyses.

Exploratory data analyses will first be performed for data description and screening for anomalies (e.g., outliers, missing data, nonnormality). The results from these preliminary analyses will be used to: 1) describe univariate and bivariate distributions; 2) identify group imbalances and associations between outcome variables and suspected covariates/confounders; 3) evaluate amount and patterns of missing data; and 4) check for violations of the statistical assumptions of the planned analyses. If statistical assumptions are violated, data transformations or more robust statistical methods will be considered. The randomness of missing data will be investigated using available information on subject characteristics to help identify possible missing data mechanisms and inform the choice of strategies to be applied to handle missing data. If data are ignorably missing, the likelihood estimation procedures to be used will produce unbiased estimates while retaining observations with missing values on the outcome variables. If needed, multiple imputation will be used to impute missing values on covariates when ignorable missingness can be assumed. If data are not ignorably missing (i.e., missing not at random), we will use selection or pattern mixture modeling to investigate the sensitivity of results assuming different patterns of missingness.

3.9.3.2. Analysis strategy for aim 1.

An “intent-to-treat” (ITT) approach will be used for efficacy analyses, where all participants will be included in the treatment groups to which they were randomly assigned, regardless of protocol adherence, treatment received, and withdrawal or protocol deviations. Although recommended for efficacy analyses in RCTs, the sensitivity of the results assuming ITT will be explored using information regarding participant adherence to the intervention.

The primary outcome variable for this aim is weight objectively assessed at baseline and 6 and 12-months. Linear mixed modeling with linear contrasts will be used to examine the effect of randomized treatment assignment (SM vs. SM + FB) on weight (or percent weight change relative to baseline) over time. When fitting models, treatment assignment will be the between-subjects factor and time will be treated as a repeated within-subjects factor with interaction between time and treatment assignment and random effects for participants. Standard fit indices (e.g., AIC, BIC) will also be used to identify the best fitting covariance structure for the repeated measures. F-tests will be used to test main and interaction effects included in the model. Individual regression parameters will be computed and reported with standard errors to yield confidence intervals. Following model fitting, residual analysis will be conducted to identify sources of model misspecification, outliers and influential observations. Sensitivity analyses will be performed to discern the impact of influential observations on results.

To test the hypothesis for Aim 1, linear contrasts will be specified and estimated when fitting the repeated measures model to compare percentage weight changes between the SM and SM + FB groups at 6-and 12-months follow-up (Hypothesis 1). Marginal modeling with generalized estimating equations will also be used to analyze percentage weight change as it tends to be more robust to misspecification of the covariance structure of the repeated assessments and violations in normality assumptions. Sensitivity analyses will be used to compare the results yielded from each modeling approach.

3.9.3.3. Analysis strategy for aim 2.

Linear mixed modeling (or generalized mixed regression modeling [GMRM] if a normal error distribution cannot be assumed) will be first used to examine adherence to dietary and PA goals, self-monitoring and self-weighing over the 12-month period to examine sustainability of engagement. To assess differences in adherence variables between the SM and SM + FB groups at 3, 6, 9, and 12 months (Aim 2, Hypothesis 2), adherence data will be aggregated over the 3 months prior to each time point to yield the percentage of days for calorie and fat goals (or weeks for PA goals) that the participant met goals, self-monitored dietary intake and PA, and self-weighed. A mixed modeling approach, similar to that for Aim 1, will be used to analyze each adherence variable. As the data permit, each adherence variable will also be modeled daily or on a weekly basis as a binary variable using generalized mixed regression modeling assuming a binomial error distribution. For these more intensive longitudinal data, adherence will be modeled as a continuous growth process, with random effects for subject-level intercepts and slopes (i.e., multi-level modeling) [53]. As appropriate, predictor variables will be centered, or rescaled, to enhance interpretation.

3.9.3.4. Analysis strategy for aim 3.

To explore the temporal relationships of the frequency, timing and type of FB messages on subsequent adherence measures, GMRM will first be used to model each adherence component as a continuous growth process over the 12-month period for participants receiving SM + FB. When exploring features of feedback messages (frequency, timing, type, etc.) as possible underlying causal processes of each adherence component over the 12-month period, GMRM will again be employed [53]. In these analyses, we will consider the relative timing of the feedback and the targeted behavior of the feedback (e.g., self-monitoring, self-weighing) to predict future adherence variables. In these analyses, we will consider the precision of the assessment of the timing of the adherence behavior in the analysis, with self-weighing and PA assessed in real-time whereas adherence to dietary goals and self-monitoring of diet assessed on a daily basis. For example, when modeling adherence to calorie and fat gram goals where dietary variables are tracked on a daily basis, we will consider the impact of the number and type of dietary feedback messages for the same day but also those from previous days, in particular the day prior to assessment of dietary goals. As adherence variables may impact one another, models for a given adherence variable may include other adherence variables as predictors.

3.10. Participant safety

There are minimal risks involved in participation in the study. Participants will receive recommendations and guidelines for a nutritionally balanced diet with adequate calories and fat, as well as adequate but moderate physical-activity goals. We realize that some participants may attempt to lose weight too quickly by severely restricting calories or rapidly increasing their exercise. The interventionists will review the database for weights to assess for too rapid weight loss. In the event that a participant loses > five pounds in a week, the ADAM database triggers a flag. Staff will contact participants and counsel them and adjust dietary goals as needed.

Self-monitoring data will be securely downloaded from the Fitbit server to our server with no identifying information via a secure Application Program Interface (API). All participants’ data are accessible only through the ADAM portal, and only authorized research staff will be able to access this portal. Data gathered by the smartphone app are transferred to a secure server using a secure HTTP (HTTPS) connection from the participant’s phone and will be encrypted and fully protected. We do not store participant data locally in the participant’s smartphone.

4. Discussion

Most US adults exceed a healthy weight [54] and relapse and regain following intentional weight loss is exceedingly high [55]. Moreover, inadequate PA levels and poor diet quality are highly prevalent among adults. In the SMARTER Trial, we are targeting two of the major contributors to ideal health-healthful diet and higher PA levels as a means to reduce obesity. By testing an app that can provide ongoing support to an individual trying to manage weight loss and maintenance, we also are addressing the major issue in weight loss treatment today: the challenge of preventing the occurrence of relapse and regaining intentionally lost weight, instead of maintaining weight loss for the long-term [56]. If the SMARTER intervention is successful, it is one that can be readily disseminated at a low cost for longer-term use.

While the weight loss and technology industries have recognized the thirst for individuals to monitor their behaviors with mobile or wea-able devices, the science supporting their use is in its infancy. Despite the fact that by a 2016 estimate over 28,000 weight management apps have been developed, very few (0.05%) have been developed with identifiable professional input [57]. In addition, apps that are evaluated in clinical trials are often either not commercially available or widely used by the public. Two systematic reviews about the role of mobile technology in weight management, one including a meta-analysis, have been published recently [58,59]. These reviews highlighted that while many existing studies presented design concerns (e.g., lack of comparator group), there is a suggestion that mobile apps and wearables can be used for self-regulation and thereby assist with weight management [59]. The meta-analysis showed a decrease in body weight of −2.35 kg (weighted mean difference) that was greatest at 6 months [59]. The authors concluded that while mobile health interventions were associated with weight loss, effects tended to be modest and short term. While the literature in this field continues to grow, there is clearly a need for further well-designed studies, such as SMARTER, that include mobile apps widely available to the public and examine not only weight outcomes but also patterns of adherence to self-monitoring and long term engagement with the use of the devices and apps.

One study included in both systematic reviews was a large clinical trial targeting weight loss among young adults involved three groups over 24 months and compared an interactive smartphone app that delivered behavioral weight loss strategies to personal coaching enhanced by SM and a control group. There was no significant weight loss at 24 months but an end of study survey found that 30% of the interactive smartphone group, 50% of the personal coaching group and 54% of the control group used a commercially available weight loss app during the study [60]. The report of these behaviors in the context of a clinical trial that itself used an app are surprising; however, they may provide insight into individuals’ needs during weight loss treatment and reveal that the use of smartphone apps that reduce the burden of SM can provide some level of feedback by permitting a review of their food entries and provide some needed assistance. It also could be that because the most widely used commercial apps are fairly sophisticated and designed by industry-sponsored internet technology (IT) teams that they are more appealing (and more familiar) to participants than those developed by research teams. As such, in study exit interviews we will ask about the use of other strategies, including other apps, which may have been used by participants during the study.

A recent study not included in the systematic reviews employed an intervention that used a popular commercially available app (MyFitnessPal) and examined a strategy to prevent the frequently observed decline in dietary self-monitoring over the course of a behavioral weight loss intervention [61]. Investigators examined whether a simultaneous (weight and diet monitoring) vs sequential (weight then diet monitoring) approach was superior to diet monitoring alone. The results showed similar weight loss and self-monitoring engagement in both arms. In SMARTER, we have added the dimension of tailored feedback in one of the groups and will be performing detailed analyses to see how feedback timing may influence weight and self-monitoring adherence.

Many of the commercially available apps do not include behavioral theory-based strategies [62]. Considering the extensive theoretical and empirical support for the role of SM in behavior change, the scientific community has been slow in developing and testing evidence-based tools to support SM [62]. To the best of our knowledge, the SMARTER trial is unique in its real-time delivery of the FB messages developed through an extensive algorithm. In our trial, the FB is delivered through pop-up messages on the smartphone that can be saved by the participant, per their request in previous trials – another innovative feature of our intervention.

SMARTER is innovative and different from other mobile phone studies in that we are taking advantage of the wealth of daily longitudinal data and exploring the temporal relationships of the frequency, timing and content of FB delivered and the subsequent lifestyle behaviors through an examination of serially collected real-time SM (diet, PA, weight) data over 12 months so that we can determine the effect size of receiving FB messages. Also, we are testing the efficacy of the integration of mobile technology and a theoretically based system that generates personalized FB using an extensive algorithm that reads the individual’s incoming SM data and selects a message from a large, rotating library. Our supportive messages reinforce healthful behaviors and redirect less healthful behaviors when the market is bombarded with commercial programs that lack a theoretical and evidence-base and efficacy testing [62,63]. Furthermore, the FB system will address a concern that many of our past participants have expressed-a lack of available support outside of structured treatment programs. The auto-mated, tailored and remotely delivered FB message system has the potential to serve as a sustainable, stand-alone support mechanism for individuals who do not have access to or do not wish to participate in a structured or an in-person treatment program and also, could be available on an ongoing basis to anyone wishing to manage his or her weight, regardless of their location.

There are a few limitations of SMARTER that should be discussed. First, it is a single site study conducted in Pittsburgh, Pennsylvania, which may limit the generalizability of our results to other populations and communities. In particular, one of the potential strengths of mobile technology is that it is widely available, including in diverse and traditionally underserved groups. While we are targeting under-represented minorities in our recruitment, the demographics of our recruitment catchment area may limit the extent to which we can recruit a truly diverse sample. In addition, while we will specifically look at how the timing of our feedback affects subsequent behavior, there are many other factors that could affect diet and physical activity behavior that we will not be able to measure as one might in a study that uses ecological momentary assessment, in other words we will not be querying participants as feedback is delivered and received.

Despite these potential limitations, the SMARTER trial is significant in multiple dimensions. We anticipate that use of the smartphone with a personalized FB message system will be sufficient to help an individual develop more healthful habits, manage a lapse and restore self-regulatory skills, potentially contributing to improved and longer-term weight management and a more healthful lifestyle.