Abstract
Background
Digital health interventions may offer a scalable way to prevent type 2 diabetes (T2D) with minimal burden on health care systems by providing early support for healthy behaviors among adults at increased risk for T2D. However, ensuring continued engagement with digital solutions is a challenge impacting the expected effectiveness.
Objective
We aimed to investigate the longitudinal usage patterns of a digital healthy habit formation intervention, BitHabit, and the associations with changes in T2D risk factors.
Methods
This is a secondary analysis of the StopDia (Stop Diabetes) study, an unblinded parallel 1-year randomized controlled trial evaluating the effectiveness of the BitHabit app alone or together with face-to-face group coaching in comparison with routine care in Finland in 2017-2019 among community-dwelling adults (aged 18 to 74 years) at an increased risk of T2D. We used longitudinal data on usage from 1926 participants randomized to the digital intervention arms. Latent class growth models were applied to identify user engagement trajectories with the app during the study. Predictors for trajectory membership were examined with multinomial logistic regression models. Analysis of covariance was used to investigate the association between trajectories and 12-month changes in T2D risk factors.
Results
More than half (1022/1926, 53.1%) of the participants continued to use the app throughout the 12-month intervention. The following 4 user engagement trajectories were identified: terminated usage (904/1926, 46.9%), weekly usage (731/1926, 38.0%), twice weekly usage (208/1926, 10.8%), and daily usage (83/1926, 4.3%). Active app use during the first month, higher net promoter score after the first 1 to 2 months of use, older age, and better quality of diet at baseline increased the odds of belonging to the continued usage trajectories. Compared with other trajectories, daily usage was associated with a higher increase in diet quality and a more pronounced decrease in BMI and waist circumference at 12 months.
Conclusions
Distinct long-term usage trajectories of the BitHabit app were identified, and individual predictors for belonging to different trajectory groups were found. These findings highlight the need for being able to identify individuals likely to disengage from interventions early on, and could be used to inform the development of future adaptive interventions.
Trial Registration
ClinicalTrials.gov NCT03156478; https://clinicaltrials.gov/ct2/show/NCT03156478
International Registered Report Identifier (IRRID)
RR2-10.1186/s12889-019-6574-y
Keywords: type 2 diabetes, user engagement, digital behavior change intervention, trajectories, habit formation, mobile health
Introduction
Type 2 diabetes (T2D) is currently one of the most prevalent noncommunicable diseases burdening public health globally [1]. It is a progressing disease that compromises health-related quality of life [2,3], introduces severe comorbidities, and increases premature mortality [4,5]. It also has a significant economic impact on individuals, health systems, and societies [6]. However, research shows that T2D could be prevented efficiently by lifestyle improvement [7-9]. This highlights the need for early diabetes prevention. In addition, plausible effects of lifestyle changes are reported to sustain for several years [10,11]. Still, after 20 years of evidence of effectiveness, health care systems struggle to find and implement scalable individual-level lifestyle change support in routine practice.
Digital health interventions may offer a scalable way to prevent T2D by providing early support for healthy behavior improvement among individuals at increased risk of T2D with minimal burden to health systems. A recent review supported findings on the effectiveness of technology-driven T2D prevention interventions in producing short-term (≤6 months) and long-term (≥12 months) weight loss, with the number of behavior change techniques applied positively associated with effectiveness [12]. However, continuous user engagement in digital solutions remains a challenge [13-15], impacting the expected effectiveness. For this, finding out who will actively use and potentially benefit from digital solutions is essential for providing the right care to the right person at the right time, which, in turn, will potentially improve the personalization and optimization of the care.
The BitHabit app was developed in the Finnish Stop Diabetes (StopDia) project to provide adults at increased risk of T2D an easy way to form healthy lifestyle habits [16]. The applicability of the BitHabit app was further evaluated in the Finnish health care system in a 12-month randomized controlled trial (RCT) [16]. The aims of this study were to (1) identify long-term user engagement trajectories of the BitHabit app among the participants, (2) examine predictors of the trajectories, and (3) investigate associations between the trajectories and changes in T2D risk factors during the trial.
Methods
Study Design and Participants
This study is a secondary analysis of the StopDia trial, which was an unblinded parallel RCT (ClinicalTrials, NCT03156478). The participants of the trial were randomly allocated to a digital intervention group (DIGI; n=967), a group combining the digital intervention and face-to-face group coaching (DIGI+GROUP; n=971), or a control group (n=969). Randomization was done using a computerized randomization system with 1:1:1 allocation after baseline examinations. In this study, only data from the digital intervention arms (ie, DIGI and DIGI+GROUP) were used. The RCT study protocol, including the intervention lifestyle objectives and outcome measures, has been described in more detail in earlier work [16].
The participants were recruited from several communication channels, such as social media, newspapers, radio, television, websites, health care and social service units, and community pharmacies, by encouraging people to visit the website of the project to fill in the StopDia Digital Screening Tool [17]. The participants were adults aged 18 to 74 years living in 3 provinces of Finland and being at increased risk of T2D as identified with the StopDia Digital Screening Tool including the Finnish Diabetes Risk Score (FINDRISC) developed to estimate the 10-year risk of developing drug-treated T2D [18]. Adults at increased risk of T2D, who scored at least 12 points in the FINDRISC, had repeatedly measured impaired fasting glucose or glucose tolerance, or had a history of gestational diabetes, were invited to participate in the study. Other inclusion criteria were a possibility to use a computer, smartphone, or tablet with internet access; having a mobile phone number; and having adequate Finnish language skills. The exclusion criteria were current type 1 or 2 diabetes, current pregnancy or breastfeeding, having an active cancer, and being less than 6 months from active cancer treatment.
Intervention
Participants in the DIGI and DIGI+GROUP received access to the BitHabit app for the 1-year intervention period in 2017-2019. The BitHabit app was designed in the StopDia project based on principles derived from habit formation theories [19,20] and the Self-Determination Theory [21]. The BitHabit app has been described in more detail earlier [22]. Briefly, the main goal of the app was to help the participants form lifestyle habits that would help prevent T2D. The app provided the user with a broad selection of behaviors in 13 different categories related to diet, physical activity, sleep, positive mood, stress management, smoking, and alcohol consumption. App use included browsing and selecting context-specific minimum-effort behaviors, reporting performances, and monitoring progress. Automatization of the behaviors into frequently repeated habits was facilitated by the nature of the behaviors (simple, with contextual triggers) as well as reminders for reporting performances.
In addition to using the BitHabit app, DIGI+GROUP members also participated in group coaching consisting of 6 meetings organized in local health care centers, as explained in detail previously [22].
Study Visits
At the first examination visit, after the participants had signed informed consent, clinical measurements, including body weight and height, and waist circumference, were taken by a study nurse in a local health care center. Thereafter, participants received a link to a StopDia Digital Questionnaire with standardized and validated questions in 3 sets to assess diet quality, eating behavior, physical activity, sedentary time, smoking, quality of life, stress, sleep, and several other factors related to T2D risk. The second examination consisted of laboratory measures, including blood glucose, taken in the local health care center. All baseline assessments were repeated at the 12-month follow-up.
After using the app for 1 to 2 months, the DIGI and DIGI+GROUP members were also asked to answer questions on app user experience. They were also asked if they would recommend the BitHabit app to their friends, with possible responses ranging from 0 (not at all likely to recommend) to 10 (very likely to recommend).
Assessments
Baseline Characteristics
At baseline, the participants reported on the following clinical factors: age (categorized as <50, 50-59, and ≥60 years), sex, BMI (kg/m2), waist circumference (cm), glycated hemoglobin A1c (HbA1c; mmol/mol), FINDRISC, diet quality, and physical activity. Obesity was defined as BMI ≥30 kg/m2, and abdominal obesity was defined as a waist circumference ≥88 cm for women and ≥102 cm for men [23]. Diet quality was assessed using the Healthy Diet Index that ranges between 0 and 100, and is calculated as the sum of scores for 7 main domains, including meal pattern (0-10 points), grains (0-20 points), fruits and vegetables (0-20 points), fats (0-15 points), fish and meat (0-10 points), dairy (0-10 points), and snacks and treats (0-15 points), with higher scores indicating a healthier diet [24]. Perceived self-efficacy related to nutrition was measured with the Nutrition Emotional Barriers Self-Efficacy Score, with higher scores indicating better emotional barrier self-efficacy [25]. Perceived stress was assessed with the Perceived Stress Scale consisting of 10 questions, with higher scores indicating a higher level of experienced stress [26]. Total physical activity (hours/week) was measured as the sum of leisure time total physical activity, combined total physical activity during work trips, and total conditioning and functional physical activity.
Socioeconomic factors considered were education (categorized as elementary school, high or vocational school, and college or academic degree), household size (categorized as single, and two or more members), and household gross income (categorized as <€25,000/year, €25,000-64,999/year, and ≥€65,000/year). A currency exchange rate of €1=US $1.14 is applicable.
Factors describing digital abilities were prior use of healthy lifestyle digital apps and internet use several times per day. The factor describing the intervention was the use of the BitHabit app together with face-to-face group coaching.
Early User Experience and Use Engagement
User experience was measured after using the app for 1 to 2 months as the rating of the likelihood to recommend the app to others (0-10), which was categorized according to the net promoter score definition [27] into detractors (0-6), passives (7-8), and promoters (9-10).
Furthermore, short-term user engagement was measured as the number of app usage days during the first month of use.
User Engagement
Automatically collected log files of BitHabit app use included time-stamped user interactions from which usage metrics, such as usage days, usage sessions and their durations, selected habits, and habit performances, were derived.
Outcomes
In this study, 12-month changes in selected T2D risk factors (ie, diet quality measured with the Healthy Diet Index, waist circumference, BMI, and HbA1c) were used as outcome measures, which were measured as differences (in absolute scale or in percentage) between baseline and the end of the intervention at 12 months.
Statistical Analyses
Latent class growth models can be utilized to identify homogeneous subpopulations when studying developmental trajectories from a heterogeneous population [28-30]. This data-driven modeling technique was applied to cluster participants into distinct trajectories of app user engagement. App user engagement was measured as monthly usage days during 2 to 12 months after initiation of app use. This provided 11 repeated measurements on the number of app usage days within 1 month (0-31 days/month). First month usage days were not included in the trajectory analyses but were used as a predictor for trajectory membership in later analyses, as early use is a known predictor of sustained use [31-33]. The number of classes (ie, trajectories) is a priori unknown and must be estimated from the data by iteratively fitting alternative models. Models with 1 to 5 classes and varying shapes for the trajectories (linear, quadratic, and cubic) were fitted. Models were estimated with full-information maximum likelihood, and missing data were not imputed, but all available data were used. The method assumes no variation within the trajectories. Unadjusted analyses were conducted. To find the best model, we utilized information from fit indices and the classification performance of the model, as well as clinical interpretation of the trajectories. The Bayesian information criterion was used as a measure of model adequacy, with a lower value indicating a better model with optimal balance between complexity and good fit. The Low-Mendel-Rubin likelihood ratio test with a P value <.05 was considered to indicate better fit for an n-class model than for an n−1 class model. Entropy was used to guide in the classification accuracy of the model, with higher values indicating better classification. Small classes with less than 4% of the total population were not accepted, and to ensure at least 70 participants per group, latent class growth modeling was performed using Mplus Version 8 [34].
Differences in baseline characteristics between the participant groups defined by the identified trajectories were examined with the chi-square test for categorical variables and the Kruskal-Wallis test for continuous variables. Thereafter, a multivariable multinomial logistic regression model was applied to examine the factors associated with the trajectory membership. In a sensitivity analysis, the model was also adjusted for belonging to the DIGI+GROUP. User experience was introduced in the main analysis by conducting a subgroup analysis among the participants who responded to the user experience questionnaire after the first 1 to 2 months of use. Finally, analysis of covariance was used to study associations between the trajectories and short-term changes in T2D risk factors. Models were adjusted for age, sex, and the baseline value of the specific T2D risk factor to account for possible differences in baseline values. Analyses were conducted with IBM SPSS Version 25.0 (IBM Corp) and R Version 4.0.4 (R Core Team). Statistical significance was set at P<.05.
Ethics Statement
The StopDia study was approved by the Research Ethics Committee of the Hospital District of Northern Savo (number: 467/2016). Written informed consent for participation in the study and the use of data from national health care registers was obtained from all study participants.
Results
Population Characteristics
The study cohort included 1926 participants in the intervention arms with at least one login to the BitHabit app during the study period (Figure 1). Among them, the mean age was 55.2 (SD 10.0) years, and 79.7% (1535/1926) of the participants were women (Multimedia Appendix 1).
Figure 1.
Flow chart of the study population. DIGI, digital intervention group; DIGI+GROUP, group combining the digital intervention and face-to-face group coaching.
User Engagement Trajectories
Based on the Low-Mendel-Rubin likelihood ratio test and the clinical interpretation, a 4-class cubic latent class growth model was identified as the best fitting model with good interpretability (Figure 2; Multimedia Appendix 2). Almost half of the participants (904/1926, 46.9%) had very few usage days in the first months of the study, and they dropped close to zero after 6 to 7 months. Another large group (731/1926, 38.0%) started with approximately 6 usage days per month, which decreased gradually to 3 days per month by the 12th month. A further 10.8% (208/1926) of participants had approximately 14 usage days during the second month with a decrease to 8 days during the 12th month. The remaining 4.3% (83/1926) of participants had almost 23 days of use during the second month, and the usage was sustained with a gradual decrease to 18 days of use during the 12th month.
Figure 2.
Estimated BitHabit app user engagement trajectories with their 95% CIs.
The identified trajectories were differentiated well when examined with other usage activity metrics (Table 1). Based on the trajectories and all usage metrics, the groups were named as (1) terminated usage, (2) weekly usage, (3) twice weekly usage, and (4) daily usage (Multimedia Appendix 1).
Table 1.
Distribution of other use activity metrics by trajectories.
| Variable | Terminated usage (n=904), median (IQR) | Weekly usage (n=731), median (IQR) | Twice weekly usage (n=208), median (IQR) | Daily usage (n=83), median (IQR) |
| Number of usage days | 12 (5-23) | 54 (44-69) | 131 (113-157) | 262 (228.25-310) |
| Number of sessions | 14 (6-27) | 60 (48-77.75) | 153 (128-183.50) | 310 (257.25-367.25) |
| Total duration of use (min) | 42.73 (19.55-87.18) | 153.63 (89.45-263.72) | 302.32 (196.24-509.12) | 431.15 (288.45-705.86) |
| Average session duration (min) | 3.23 (1.98-5.23) | 2.50 (1.42-4.18) | 1.97 (1.21-3.44) | 1.32 (0.96-2.02) |
| Usage period from the first day to the last | 124 (51.50-245) | 356 (343-360) | 360 (354-363) | 362 (355.25-364) |
| Weeks in use | 9 (4-17) | 40 (35-45) | 50 (47.5-51) | 51 (50-52) |
| Months in use | 4 (2-7) | 12 (12-12) | 12 (12-12) | 12 (12-12) |
| Number of selected habits | 15 (7-26) | 32 (20-53) | 43.5 (24-61) | 50 (33.5-73) |
| Number of new habits | 7 (2-15) | 15 (8-28.75) | 26 (13-43) | 23 (13-46) |
| Total number of performances | 72 (15-269.5) | 720 (352-1470.5) | 1433 (804-2562.5) | 4516 (2405.5-6281) |
| Number of habits tracked | 13 (6-23.5) | 28 (17-46) | 39 (22-56.5) | 49 (31.5-71.75) |
Predictors of User Engagement Trajectories
Active app use during the first month increased the odds of belonging to the more engaged trajectories (odds ratio [OR] 1.65, 95% CI 1.57-1.75 for daily usage; OR 1.45, 95% CI 1.40-1.51 for twice weekly usage; and OR 1.19, 95% CI 1.15-1.22 for weekly usage) than the terminated usage trajectory (Table 2). In addition, older age was associated with increased user engagement compared with the terminated usage category (OR 0.07, 95% CI 0.02-0.19 in those aged <50 years and OR 0.44, 95% CI 0.22-0.87 in those aged 50-59 years for daily usage; OR 0.19, 95% CI 0.11-0.33 in those aged <50 years and OR 0.40, 95% 0.26-0.64 in those aged 50-59 years for twice weekly usage; and OR 0.45, 95% CI 0.34-0.61 in those aged <50 years and OR 0.61, 95% CI 0.47-0.79 in those aged 50-59 years for weekly usage compared with those aged ≥60 years). Women were less likely to be daily users (OR 0.43, 95% CI 0.19-0.98) than terminated users. Better diet quality at baseline, measured with the Healthy Diet Index, increased the odds of belonging to the more engaged trajectories (OR 1.04, 95% CI 1.01-1.08 for daily usage and OR 1.03, 95% CI 1.01-1.05 for twice weekly usage) than the terminated usage trajectory. Prior internet use of several times per day (OR 0.41, 95% CI 0.21-0.79 for daily usage and OR 0.73, 95% CI 0.57-0.95 for weekly usage) decreased the odds of belonging to the more engaged trajectories than the terminated usage trajectory. According to the sensitivity analysis, face-to-face group coaching decreased the odds of belonging to the more engaged trajectories (OR 0.28, 95% CI 0.15-0.52 for daily usage; OR 0.49, 95% CI 0.34-0.73 for twice weekly usage; and OR 0.64, 95% CI 0.52-0.80 for weekly usage) than the terminated usage trajectory (Multimedia Appendix 3). However, there were no more differences between the sexes in trajectory membership.
Table 2.
Results of multinomial logistic regression analysis.
| Variable | Weekly usagea (n=715), aORb (95% CI) | Twice weekly usagea (n=204), aOR (95% CI) | Daily usagea (n=82), aOR (95% CI) | ||||
| Age |
|
|
|
||||
|
|
<50 years | 0.45 (0.34-0.61)c | 0.19 (0.11-0.33)c | 0.07 (0.02-0.19)c | |||
|
|
50-59 years | 0.61 (0.47-0.79)c | 0.40 (0.26-0.64)c | 0.44 (0.22-0.87)c | |||
|
|
≥60 years (reference) | 1 | 1 | 1 | |||
| Women | 0.94 (0.72-1.24) | 0.73 (0.44-1.21) | 0.43 (0.19-0.98)c | ||||
| Obesity | 0.87 (0.70-1.08) | 0.63 (0.43-0.92)c | 0.59 (0.32-1.07) | ||||
| Healthy Diet Index | 1.01 (1.00-1.02)c | 1.03 (1.01-1.05)c | 1.04 (1.01-1.08)c | ||||
| Education |
|
|
|
||||
|
|
Elementary school | 0.77 (0.50-1.18) | 0.96 (0.48-1.95) | 0.68 (0.22-2.07) | |||
|
|
High or vocational school | 1.02 (0.79-1.31) | 1.32 (0.85-2.06) | 1.33 (0.67-2.63) | |||
|
|
College or academic degree (reference) | 1 | 1 | 1 | |||
| Household size |
|
|
|
||||
|
|
Single | 0.92 (0.69-1.23) | 0.98 (0.57-1.68) | 1.53 (0.69-3.42) | |||
|
|
≥2 members (reference) | 1 | 1 | 1 | |||
| Household income per year |
|
|
|
||||
|
|
≤€24,999 | 1.45 (0.97-2.16) | 1.03 (0.48-2.22) | 2.79 (0.86-9.03) | |||
|
|
€25,000-64,999 | 1.57 (1.21-2.03)c | 1.66 (1.05-2.63)c | 2.87 (1.31-6.32)c | |||
|
|
≥€65,000 (reference) | 1 | 1 | 1 | |||
| Prior use of health lifestyle digital apps | 1.04 (0.83-1.30) | 1.18 (0.79-1.75) | 0.95 (0.50-1.80) | ||||
| Internet use several times per day | 0.73 (0.57-0.95)c | 0.69 (0.44-1.08) | 0.41 (0.21-0.79)c | ||||
| App usage days during the first month | 1.19 (1.15-1.22)c | 1.45 (1.40-1.51)c | 1.65 (1.57-1.75)c | ||||
aTerminated usage (n=895) as a reference.
baOR: adjusted odds ratio.
cP<.05.
In a subgroup analysis among 1314 participants who responded to the questionnaire after the first 1 to 2 months of use and had complete data on predictors, app usage days during the first month remained a predictor for trajectory membership (OR 1.56, 95% CI 1.47-1.66 for daily usage; OR 1.37, 95% CI 1.31-1.43 for twice weekly usage; and OR 1.13, 95% CI 1.10-1.17 for weekly usage compared with terminated usage; Multimedia Appendix 4). Furthermore, the detractors classified by the net promoter score had significantly lower odds of belonging to the more engaged trajectories than the terminated usage trajectory (OR 0.23, 95% CI 0.10-0.55 for daily usage; OR 0.19, 95% CI 0.11-0.35 for twice weekly usage; and OR 0.37, 95% CI 0.25-0.55 for weekly usage) compared with promoters of the app.
Trajectories and Changes in T2D Risk Factors Over 12 Months
High user engagement was associated with 12-month changes in T2D risk factor levels (Figure 3; Multimedia Appendix 5). Compared with the other trajectories, daily usage was associated with a higher increase in the Healthy Diet Index (P=.02) and a more pronounced decrease in BMI (P=.01) and waist circumference (P=.02) at 12 months.
Figure 3.
Changes (%) in risk factor levels over 12 months by user engagement trajectories. The error bars represent 95% CIs for means. P values are obtained from the analysis of covariance for the main effect of the user engagement trajectory on the change score adjusted for age, sex, and baseline value. HbA1c: glycated hemoglobin A1c.
Discussion
Principal Findings
Four different trajectories of digital app use were identified with a data-driven technique. We observed that initial user engagement during the first month of the intervention, the net promoter score after the first 1 to 2 months of use, older age, and better diet quality at baseline predicted long-term user engagement. While those who had sustained use once to a couple of times per week showed small beneficial changes in risk factor levels in comparison to those who terminated use, those who used the app almost on a daily basis throughout the study showed the most beneficial changes.
Comparison With Prior Research
According to our previously reported results, 99% of participants who were allocated to use the BitHabit app logged into the app at least once and selected at least one habit, and 95% reported at least one habit performance during a 6-month follow-up [22]. No significant differences were observed in cumulative usage metrics between the DIGI and DIGI+GROUP, and more than 50% of the participants in both intervention groups continued to use the app at least on a weekly basis in the first 6 months [22]. This study showed that these active users continued to use the app for the entire 12-month intervention. However, according to the sensitivity analysis, receiving face-to-face group coaching in addition to the digital intervention slightly decreased engagement with the app. Participating in the face-to-face coaching may have decreased the need for later digital support. It should be noted that membership in the DIGI+GROUP arm may have mediated the effects in the sensitivity analysis, and therefore, it was omitted from the main analysis.
Overall, BitHabit app use remained remarkably high in comparison with the use of other digital health apps [22]. For example, Goh et al identified 3 short-term 8-week trajectories of weekly use activity for a caloric-monitoring mobile app among T2D patients [35]. A large proportion of the “minimal user” participants (79%) stopped use within the first 2 weeks. Although women seemed to be more active users, we found no gender differences in app use. Older participants were more likely to show frequent use. These background characteristics have been previously reported to be associated with more active use by others as well [36-38]. This result is promising as it demonstrates that digital interventions can be provided to people of all ages. One can only speculate on the reasons for this finding, but a plausible explanation is higher relevance of T2D prevention. Another explanation could be having less work- and family-related daily activities and hence more time for app use. Other than age and household income, no background factors were found to be associated with app use trajectories. However, prior internet use several times per day predicted lower user engagement with the app. Intensive internet use has been associated with lower physical activity in adults [39], which may partly explain the observed association in this study.
App use during the first month and the net promoter score were predictors for user engagement trajectories. Previous studies have shown that short-term treatment adherence predicts future adherence outcomes better than baseline characteristics alone, for example, in the case of adherence to self-management for chronic diseases [31,32]. In a recent study, usage during the first 24 hours after login predicted long-term engagement with a weight loss app [33]. We tested the early adherence hypothesis with a 4-week period, considering it a sufficiently long window of opportunity for monitoring and supporting user engagement in real-world prevention programs.
Theoretically, terminating app use does not necessarily imply disengagement from using the behavior change techniques of the intervention [40]. For example, the habit formation approach in the BitHabit app could have led to termination of app use if participants felt that they had already formed sufficiently strong habits or had learnt to follow the habit formation process without the app. Based on the literature, habit formation takes 10 weeks on average, so even for forming multiple habits, 12 months is a rather long usage time [41]. However, our empirical findings suggest that this was not the case in our study, as those who terminated app use had selected less habits than the other user groups and showed no benefits of the intervention in terms of risk factor levels. Moreover, the benefits increased significantly when the frequency of use increased from weekly to daily. Our results are in line with the findings of previous studies reporting that the effectiveness of digital interventions is dependent on user engagement with the intervention [42-44]. Future studies are needed on whether plausible effects of lifestyle interventions sustain the longest among those with the most active engagement during the intervention or whether effective engagement should be determined through other metrics, such as the content of use (eg, type and quality of habits selected and tracked by the users).
The results indicated that only a minor proportion of the participants, belonging to the trajectory of most active users, achieved marked lifestyle changes. Furthermore, the most active users had better well-being at baseline (ie, better diet, more physical activity, and lower stress levels), which is consistent with previous studies [37,38]. Reaching and engaging those with the highest risks and needs are known challenges in most interventions. Adaptive interventions, that is, monitoring for the early indicators of nonresponse and adapting interventions (eg, augmenting or switching interventions), have been proposed as a solution [45]. This seems especially promising in digital interventions, which inherently enable real-time monitoring of user engagement and detection of waning usage as an early indicator of nonresponse. As this study showed, use activity and user experience after the first month of use predicted long-term user engagement. Thus, early use may be regarded as a window of opportunity where user engagement should be monitored and supported especially in real-world prevention programs. If strategies, such as personalized messages and reminders, fail to re-engage participants with the app, they should be offered alternative interventions, such as face-to-face interventions, or other types of digital solutions for weight management and lifestyle changes [46]. Especially in health care that faces cost containment pressures, this kind of approach would help in providing appropriate types of interventions to different individuals.
Strengths and Limitations
The strengths of our study include a real-life setting together with a larger sample size and a longer follow-up than in previous studies on the use of digital health apps in persons with prediabetes or T2D [31,43,47]. Additionally, the studied app could be accessed easily with any device as indicated by an almost 100% first login rate, which reduced selection bias and further strengthened our sample.
However, our study also has some limitations. Women were overrepresented in the sample. The proportion of participants who dropped out during the study and, therefore, the proportion of missing data in outcome variables differed between the identified trajectories. We utilized monthly usage days as a measure for user engagement. However, the sensitivity of this measure in detecting changes in risk factors compared with other user activity metrics remains unknown. The composition of selected habits may also be an important factor for risk factor changes and should be examined in future studies.
Conclusions
Data-driven analysis of digital app user activity is necessary because it reveals true usage patterns. In this study, short-term user engagement, defined as user activity during the first month after the initiation of app use, was found to predict long-term user engagement. Metrics of user experience, based on the net promoter score and measured after early use, was another predictor of long-term user engagement and is a measure that can be easily implemented. These findings could be used in further developments aimed at predicting responses to digital interventions, adapting digital interventions for persons at risk of disengagement, and identifying persons who would benefit the most from using the app.
Acknowledgments
We would like to thank the Strategic Research Council at the Academy of Finland for funding our “Stop Diabetes—from knowledge to solutions (StopDia)” project in 2016-2019 (diary number 303537), the Novo Nordisk Foundation for providing funding in 2018-2020 (diary numbers 33980 and 63753), the Finnish Diabetes Research foundation for providing funding, and the Academy of Finland for funding the project “T2D-DATA” in 2020-2023 (diary numbers 332464 and 332465). We would also like to acknowledge the valuable contributions of the research consortium, our national and international collaborators, the primary health care providers involved in the study, the citizens who participated in the feasibility testing, and the citizens who enrolled in the study. The funders had no role in designing the study or collecting, managing, or analyzing the data; interpreting the results; writing the manuscript; or deciding to submit the manuscript for publication.
Abbreviations
- DIGI
digital intervention group
- DIGI+GROUP
group combining the digital intervention and face-to-face group coaching
- FINDRISC
Finnish Diabetes Risk Score
- HbA1c
glycated hemoglobin A1c
- OR
odds ratio
- RCT
randomized controlled trial
- T2D
type 2 diabetes
Baseline characteristics of the study participants in total and by trajectories.
Unadjusted latent class growth analyses for app user engagement (monthly usage days during 2-12 months, n=1926).
Results of a sensitivity analysis conducted with multivariable multinomial logistic regression.
Results of a multivariable multinomial logistic regression analysis among participants in the intervention arms who responded to the user experience questionnaire after the first 1-2 months of use (n=1314).
Mean changes in diabetes risk factor levels between baseline and the 12-month follow-up.
Footnotes
Conflicts of Interest: JM is a founding partner of ESiOR Oy and a board member of Siltana Oy. These companies were not involved in carrying out this research. PL, EM, PA, MH, JL, EJR, KA, RM, NL, TTT, TL, MVG, and JP declare no conflicts of interest.
References
- 1.IDF Diabetes Atlas. International Diabetes Federation. [2022-02-05]. https://diabetesatlas.org/
- 2.Väätäinen S, Keinänen-Kiukaanniemi S, Saramies J, Uusitalo H, Tuomilehto J, Martikainen J. Quality of life along the diabetes continuum: a cross-sectional view of health-related quality of life and general health status in middle-aged and older Finns. Qual Life Res. 2014 Sep 8;23(7):1935–44. doi: 10.1007/s11136-014-0638-3. [DOI] [PubMed] [Google Scholar]
- 3.Jalkanen K, Aarnio E, Lavikainen P, Jauhonen H, Enlund H, Martikainen J. Impact of type 2 diabetes treated with non-insulin medication and number of diabetes-coexisting diseases on EQ-5D-5 L index scores in the Finnish population. Health Qual Life Outcomes. 2019 Jul 08;17(1):117. doi: 10.1186/s12955-019-1187-9. https://hqlo.biomedcentral.com/articles/10.1186/s12955-019-1187-9 .10.1186/s12955-019-1187-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Huang Y, Cai X, Mai W, Li M, Hu Y. Association between prediabetes and risk of cardiovascular disease and all cause mortality: systematic review and meta-analysis. BMJ. 2016 Nov 23;355:i5953. doi: 10.1136/bmj.i5953. http://www.bmj.com/lookup/pmidlookup?view=long&pmid=27881363 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Nwaneri C, Cooper H, Bowen-Jones D. Mortality in type 2 diabetes mellitus: magnitude of the evidence from a systematic review and meta-analysis. Diabetes & Vascular Disease. 2013 Jul 15;13(4):192–207. doi: 10.1177/1474651413495703. [DOI] [Google Scholar]
- 6.Williams R, Karuranga S, Malanda B, Saeedi P, Basit A, Besançon S, Bommer C, Esteghamati A, Ogurtsova K, Zhang P, Colagiuri S. Global and regional estimates and projections of diabetes-related health expenditure: Results from the International Diabetes Federation Diabetes Atlas, 9th edition. Diabetes Res Clin Pract. 2020 Apr;162:108072. doi: 10.1016/j.diabres.2020.108072.S0168-8227(20)30138-8 [DOI] [PubMed] [Google Scholar]
- 7.Tuomilehto J, Lindström J, Eriksson JG, Valle TT, Hämäläinen H, Ilanne-Parikka P, Keinänen-Kiukaanniemi S, Laakso M, Louheranta A, Rastas M, Salminen V, Uusitupa M, Finnish Diabetes Prevention Study Group Prevention of type 2 diabetes mellitus by changes in lifestyle among subjects with impaired glucose tolerance. N Engl J Med. 2001 May 03;344(18):1343–50. doi: 10.1056/NEJM200105033441801. [DOI] [PubMed] [Google Scholar]
- 8.Knowler WC, Barrett-Connor E, Fowler SE, Hamman RF, Lachin JM, Walker EA, Nathan DM, Diabetes Prevention Program Research Group Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. N Engl J Med. 2002 Feb 07;346(6):393–403. doi: 10.1056/NEJMoa012512. http://europepmc.org/abstract/MED/11832527 .346/6/393 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Lindström J, Louheranta A, Mannelin M, Rastas M, Salminen V, Eriksson J, Uusitupa M, Tuomilehto J, Finnish Diabetes Prevention Study Group The Finnish Diabetes Prevention Study (DPS): Lifestyle intervention and 3-year results on diet and physical activity. Diabetes Care. 2003 Dec 21;26(12):3230–6. doi: 10.2337/diacare.26.12.3230. [DOI] [PubMed] [Google Scholar]
- 10.Lindström J, Peltonen M, Eriksson JG, Ilanne-Parikka P, Aunola S, Keinänen-Kiukaanniemi S, Uusitupa M, Tuomilehto J, Finnish Diabetes Prevention Study (DPS) Improved lifestyle and decreased diabetes risk over 13 years: long-term follow-up of the randomised Finnish Diabetes Prevention Study (DPS) Diabetologia. 2013 Feb 24;56(2):284–93. doi: 10.1007/s00125-012-2752-5. [DOI] [PubMed] [Google Scholar]
- 11.Diabetes Prevention Program Research Group Long-term effects of lifestyle intervention or metformin on diabetes development and microvascular complications over 15-year follow-up: the Diabetes Prevention Program Outcomes Study. The Lancet Diabetes & Endocrinology. 2015 Nov;3(11):866–875. doi: 10.1016/S2213-8587(15)00291-0. http://paperpile.com/b/ozcYxv/WKSa . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Van Rhoon L, Byrne M, Morrissey E, Murphy J, McSharry J. A systematic review of the behaviour change techniques and digital features in technology-driven type 2 diabetes prevention interventions. Digit Health. 2020 Mar 24;6:2055207620914427. doi: 10.1177/2055207620914427. https://journals.sagepub.com/doi/10.1177/2055207620914427?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub%3dpubmed .10.1177_2055207620914427 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res. 2011 Aug 05;13(3):e52. doi: 10.2196/jmir.1772. https://www.jmir.org/2011/3/e52/ v13i3e52 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Free C, Phillips G, Galli L, Watson L, Felix L, Edwards P, Patel V, Haines A. The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: a systematic review. PLoS Med. 2013 Jan;10(1):e1001362. doi: 10.1371/journal.pmed.1001362. https://dx.plos.org/10.1371/journal.pmed.1001362 .PMEDICINE-D-12-00520 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Kohl LFM, Crutzen R, de Vries NK. Online prevention aimed at lifestyle behaviors: a systematic review of reviews. J Med Internet Res. 2013 Jul 16;15(7):e146. doi: 10.2196/jmir.2665. https://www.jmir.org/2013/7/e146/ v15i7e146 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Pihlajamäki J, Männikkö R, Tilles-Tirkkonen T, Karhunen L, Kolehmainen M, Schwab U, Lintu N, Paananen J, Järvenpää R, Harjumaa M, Martikainen J, Kohl J, Poutanen K, Ermes M, Absetz P, Lindström J, Lakka TA. Digitally supported program for type 2 diabetes risk identification and risk reduction in real-world setting: protocol for the StopDia model and randomized controlled trial. BMC Public Health. 2019 Mar 1;19(1):255. doi: 10.1186/s12889-019-6574-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Jalkanen K, Järvenpää R, Tilles-Tirkkonen T, Martikainen J, Aarnio E, Männikkö R, Rantala E, Karhunen L, Kolehmainen M, Harjumaa M, Poutanen K, Ermes M, Absetz P, Schwab U, Lakka T, Pihlajamäki J, Lindström J, StopDia Study Group Comparison of Communication Channels for Large-Scale Type 2 Diabetes Risk Screening and Intervention Recruitment: Empirical Study. JMIR Diabetes. 2021 Sep 09;6(3):e21356. doi: 10.2196/21356. https://diabetes.jmir.org/2021/3/e21356/ v6i3e21356 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Lindström J, Tuomilehto J. The diabetes risk score: a practical tool to predict type 2 diabetes risk. Diabetes Care. 2003 Mar 01;26(3):725–31. doi: 10.2337/diacare.26.3.725. [DOI] [PubMed] [Google Scholar]
- 19.Wood W, Neal DT. Behavioral Science & Policy. 2016;2(1):71–83. doi: 10.1353/bsp.2016.0008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Lally P, van Jaarsveld CHM, Potts HWW, Wardle J. How are habits formed: Modelling habit formation in the real world. Eur. J. Soc. Psychol. 2009 Jul 16;40(6):998–1009. doi: 10.1002/ejsp.674. [DOI] [Google Scholar]
- 21.Ryan R, Deci E. Self-determination theory. Basic psychological needs in motivation, development and wellness. New York, NY: Guilford Press; 2017. [Google Scholar]
- 22.Harjumaa M, Absetz P, Ermes M, Mattila E, Männikkö R, Tilles-Tirkkonen T, Lintu N, Schwab U, Umer A, Leppänen J, Pihlajamäki J. Internet-Based Lifestyle Intervention to Prevent Type 2 Diabetes Through Healthy Habits: Design and 6-Month Usage Results of Randomized Controlled Trial. JMIR Diabetes. 2020 Aug 11;5(3):e15219. doi: 10.2196/15219. https://diabetes.jmir.org/2020/3/e15219/ v5i3e15219 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.National Cholesterol Education Program (NCEP) Expert Panel on Detection‚ Evaluation‚Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III) Third Report of the National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III) final report. Circulation. 2002 Dec 17;106(25):3143–421. [PubMed] [Google Scholar]
- 24.Lindström J, Aittola K, Pölönen A, Hemiö K, Ahonen K, Karhunen L, Männikkö R, Siljamäki-Ojansuu U, Tilles-Tirkkonen T, Virtanen E, Pihlajamäki J, Schwab U. Formation and Validation of the Healthy Diet Index (HDI) for Evaluation of Diet Quality in Healthcare. Int J Environ Res Public Health. 2021 Feb 28;18(5):2362. doi: 10.3390/ijerph18052362. https://www.mdpi.com/resolver?pii=ijerph18052362 .ijerph18052362 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Schwarzer R, Renner B. Social-cognitive predictors of health behavior: Action self-efficacy and coping self-efficacy. Health Psychology. 2000;19(5):487–495. doi: 10.1037/0278-6133.19.5.487. [DOI] [PubMed] [Google Scholar]
- 26.Cohen S. Perceived stress in a probability sample of the United States. In: Spacapan S, Oskamp S, editors. The social psychology of health. Thousand Oaks, CA: Sage Publications; 1988. pp. 31–67. [Google Scholar]
- 27.Reichheld FF. The one number you need to grow. Harv Bus Rev. 2003 Dec;81(12):46–54, 124. [PubMed] [Google Scholar]
- 28.Laird NM, Ware JH. Random-Effects Models for Longitudinal Data. Biometrics. 1982 Dec;38(4):963. doi: 10.2307/2529876. [DOI] [PubMed] [Google Scholar]
- 29.Muthén B, Shedden K. Finite mixture modeling with mixture outcomes using the EM algorithm. Biometrics. 1999 Jun;55(2):463–9. doi: 10.1111/j.0006-341x.1999.00463.x. [DOI] [PubMed] [Google Scholar]
- 30.Nagin DS, Odgers CL. Group-based trajectory modeling in clinical research. Annu Rev Clin Psychol. 2010 Mar;6:109–38. doi: 10.1146/annurev.clinpsy.121208.131413. [DOI] [PubMed] [Google Scholar]
- 31.Franklin JM, Krumme AA, Shrank WH, Matlin OS, Brennan TA, Choudhry NK. Predicting adherence trajectory using initial patterns of medication filling. Am J Manag Care. 2015 Sep 01;21(9):e537–44. https://www.ajmc.com/pubMed.php?pii=86318 .86318 [PubMed] [Google Scholar]
- 32.Kozma CM, Phillips AL, Meletiche DM. Use of an early disease-modifying drug adherence measure to predict future adherence in patients with multiple sclerosis. J Manag Care Spec Pharm. 2014 Aug;20(8):800–7. doi: 10.18553/jmcp.2014.20.8.800.2014(20)8: 800-807 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Andrade AQ, Beleigoli A, Diniz MDF, Ribeiro AL. Influence of Baseline User Characteristics and Early Use Patterns (24-Hour) on Long-Term Adherence and Effectiveness of a Web-Based Weight Loss Randomized Controlled Trial: Latent Profile Analysis. J Med Internet Res. 2021 Jun 03;23(6):e26421. doi: 10.2196/26421. https://www.jmir.org/2021/6/e26421/ v23i6e26421 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Muthén L, Muthén BO. Mplus User's Guide. Eighth Edition. Los Angeles, CA: Muthén & Muthén; 2017. [Google Scholar]
- 35.Goh G, Tan NC, Malhotra R, Padmanabhan U, Barbier S, Allen JC, Østbye T. Short-term trajectories of use of a caloric-monitoring mobile phone app among patients with type 2 diabetes mellitus in a primary care setting. J Med Internet Res. 2015 Feb 03;17(2):e33. doi: 10.2196/jmir.3938. https://www.jmir.org/2015/2/e33/ v17i2e33 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Mattila E, Lappalainen R, Välkkynen P, Sairanen E, Lappalainen P, Karhunen L, Peuhkuri K, Korpela R, Kolehmainen M, Ermes M. Usage and Dose Response of a Mobile Acceptance and Commitment Therapy App: Secondary Analysis of the Intervention Arm of a Randomized Controlled Trial. JMIR Mhealth Uhealth. 2016 Jul 28;4(3):e90. doi: 10.2196/mhealth.5241. https://mhealth.jmir.org/2016/3/e90/ v4i3e90 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Perski O, Blandford A, West R, Michie S. Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis. Transl Behav Med. 2017 Jun 13;7(2):254–267. doi: 10.1007/s13142-016-0453-1. http://europepmc.org/abstract/MED/27966189 .10.1007/s13142-016-0453-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Edney S, Ryan JC, Olds T, Monroe C, Fraysse F, Vandelanotte C, Plotnikoff R, Curtis R, Maher C. User Engagement and Attrition in an App-Based Physical Activity Intervention: Secondary Analysis of a Randomized Controlled Trial. J Med Internet Res. 2019 Nov 27;21(11):e14645. doi: 10.2196/14645. https://www.jmir.org/2019/11/e14645/ v21i11e14645 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Sahin M, Lok S. Relationship between Physical Activity Levels and Internet Addiction of Adults. J Depress Anxiety. 2018;07(02):310. doi: 10.4172/2167-1044.1000310. [DOI] [Google Scholar]
- 40.Yardley L, Spring BJ, Riper H, Morrison LG, Crane DH, Curtis K, Merchant GC, Naughton F, Blandford A. Understanding and Promoting Effective Engagement With Digital Behavior Change Interventions. Am J Prev Med. 2016 Nov;51(5):833–842. doi: 10.1016/j.amepre.2016.06.015.S0749-3797(16)30243-4 [DOI] [PubMed] [Google Scholar]
- 41.Lally P, Gardner B. Promoting habit formation. Health Psychology Review. 2013 May;7(sup1):S137–S158. doi: 10.1080/17437199.2011.603640. [DOI] [Google Scholar]
- 42.Short C, Rebar A, Plotnikoff R, Vandelanotte C. Designing engaging online behaviour change interventions: a proposed model of user engagement. The European Health Psychologist. 2013;17(1):32–38. https://www.ehps.net/ehp/index.php/contents/article/view/763/pdf_39 . [Google Scholar]
- 43.Sepah SC, Jiang L, Ellis RJ, McDermott K, Peters AL. Engagement and outcomes in a digital Diabetes Prevention Program: 3-year update. BMJ Open Diabetes Res Care. 2017 Sep 07;5(1):e000422. doi: 10.1136/bmjdrc-2017-000422. https://drc.bmj.com/lookup/pmidlookup?view=long&pmid=28948027 .bmjdrc-2017-000422 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Painter SL, Lu W, Schneider J, James R, Shah B. Drivers of weight loss in a CDC-recognized digital diabetes prevention program. BMJ Open Diabetes Res Care. 2020 Jul 13;8(1):e019171. doi: 10.1136/bmjdrc-2019-001132. https://drc.bmj.com/lookup/pmidlookup?view=long&pmid=32624481 .8/1/e001132 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Almirall D, Nahum-Shani I, Sherwood NE, Murphy SA. Introduction to SMART designs for the development of adaptive interventions: with application to weight loss research. Transl Behav Med. 2014 Sep 6;4(3):260–74. doi: 10.1007/s13142-014-0265-0. http://europepmc.org/abstract/MED/25264466 .265 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Teeriniemi A, Salonurmi T, Jokelainen T, Vähänikkilä H, Alahäivälä T, Karppinen P, Enwald H, Huotari M, Laitinen J, Oinas-Kukkonen H, Savolainen MJ. A randomized clinical trial of the effectiveness of a Web-based health behaviour change support system and group lifestyle counselling on body weight loss in overweight and obese subjects: 2-year outcomes. J Intern Med. 2018 Nov 04;284(5):534–545. doi: 10.1111/joim.12802. doi: 10.1111/joim.12802. [DOI] [PubMed] [Google Scholar]
- 47.Alwashmi MF, Mugford G, Abu-Ashour W, Nuccio M. A Digital Diabetes Prevention Program (Transform) for Adults With Prediabetes: Secondary Analysis. JMIR Diabetes. 2019 Jul 26;4(3):e13904. doi: 10.2196/13904. https://diabetes.jmir.org/2019/3/e13904/ v4i3e13904 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Baseline characteristics of the study participants in total and by trajectories.
Unadjusted latent class growth analyses for app user engagement (monthly usage days during 2-12 months, n=1926).
Results of a sensitivity analysis conducted with multivariable multinomial logistic regression.
Results of a multivariable multinomial logistic regression analysis among participants in the intervention arms who responded to the user experience questionnaire after the first 1-2 months of use (n=1314).
Mean changes in diabetes risk factor levels between baseline and the 12-month follow-up.