Predictors of users’ adherence to a fully automated digital intervention to manage insomnia complaints

Abstract

Objective

Fully automated digital interventions show promise for disseminating evidence-based strategies to manage insomnia complaints. However, an important concept often overlooked concerns the extent to which users adopt the recommendations provided in these programs into their daily lives. Our objectives were evaluating users’ adherence to the behavioral recommendations provided by an app, and exploring whether users’ perceptions of the app had an impact on their adherence behavior.

Material and methods

Case series study of individuals completing a fully automated insomnia management program, conducted by a virtual agent, during December 2020 to September 2022. Primary outcome was self-reported adherence to the behavioral recommendations provided. Perceptions of the app and of the virtual agent were measured with the Acceptability E-Scale and ECA-Trust Questionnaire. Insomnia was evaluated with the Insomnia Severity Index at baseline (phase 1), after 7 days of sleep monitoring (phase 2) and post-intervention (phase 3).

Results

A total of 824 users were included, 62.7% female, mean age 51.85 (±12.55) years. Of them, 32.7% reported having followed at least one recommendation. Users’ trust in the virtual agent and acceptance of the app were related to a pre-intervention effect in insomnia severity (phase 2). In turn, larger pre-intervention improvements predicted better adherence. Mediational analyses showed that higher levels of trust in the virtual agent and better acceptance of the app exerted statistically significant positive effects on adherence (β = 0.007, 95% CI, 0.001-0.017 and β = 0.003, 95% CI 0.0004-0.008, respectively).

Discussion

Users’ adherence is motivated by positive perceptions of the app’s features and pre-intervention improvements.

Conclusions

Determinants of adherence should be assessed, and targeted, to increase the impact of fully automated digital interventions.

Background and significance

Insomnia constitutes a major public health concern due to its prevalence and its negative impact on mental and physical health.¹ Cognitive behavioral therapy for insomnia (CBT-I) is an evidence-based psychotherapy that has been recommended as the treatment of choice for insomnia by the American College of Physicians.² However, the implementation of CBT-I faces substantial challenges. The most salient one is the difficulty to access CBT-I, as the population that suffers from insomnia largely surpasses the number of available trained providers.³ The introduction of digital CBT-I has been viewed as part of the solution to this challenge of disseminating this treatment more widely.⁴ Digital CBT-I (dCBT-I) refers to the administration of CBT-I using technology, such as a mobile application or computer. Several studies on dCBT-I, with or without human support, have been published in the last decade, and reviews and meta-analyses suggest that this type of intervention show promising results.^5–8 However, to increase its impact, important issues still need to be addressed.⁹ It is a common problem in digital mental health care in general that treatment dropout is high,¹⁰ and this is especially true for interventions that are offered without any human support.¹¹ But not only that, an often-stated issue in traditional face-to-face CBT-I, that could be translated to dCBT-I, is patients’ poor adherence to the behavioral changes proposed during therapy (ie, difficulties including the treatment recommendations into one’s daily routines).^12–15 CBT-I recommendations, such as limiting the amount of time spent in bed to match total sleep time or to get out of bed when unable to sleep, are frequently viewed by those with insomnia as counterintuitive and difficult to implement. Arguably, for dCBT-I, and especially for those versions with no human support, users’ adherence to the recommendations is expected to be, at least, equally problematic. Although research in the field of user/patient adherence to technology-mediated insomnia treatment is more limited, some studies suggest that adherence levels are far from adequate.¹⁶ Research on factors associated with adherence to dCBT-I is also scarce, and most analyses to date have not established yet any reliable predictors of this behavior.¹⁴

Objectives

In this study, using data from a real-world sample of persons who completed a fully automated smartphone-based virtual agent (VA) program, named Kanopee, we aimed to test users’ adherence to the recommendations provided by the app. Kanopee monitors and treats insomnia complaints employing an abbreviated version of CBT-I that emphasizes the behavioral component of the therapy known as stimulus control.¹⁷

Additionally, we aimed to explore variables that could have an impact on the adherence behavior. Based in our previous research showing that patients’ perceptions of a virtual medical agent conducting a clinical interview were related to patients’ willingness to stay engaged with the VA during repetitive use,¹⁸ we aimed to investigate whether users’ perceptions of the app and of the VA would have an impact on their adherence to the behavioral recommendations provided. However, instead of assuming a direct effect of these variables on adherence, we developed a mediational model, based on our ongoing observations about the effects of Kanopee and borrowing elements from the Expectation-Confirmation Theory (ECT),¹⁹ which is widely used in the consumer behavior literature. In our studies on the effectiveness of Kanopee,^20,21 we noted that insomnia severity improved significantly when individuals were evaluated after 7 days of sleep monitoring, before any recommendation to manage insomnia symptoms was provided. This observation raised the question as to whether high expectations about the treatment would lead to an early pre-intervention effect, similar to the concept of the “self-fulfilling prophecy”.²² Therefore, we hypothesized that subjects who accept and trust the treatment to a higher degree (high expectations) were more likely to focus on positive aspects as the intervention progresses, searching to confirm their pre-existing expectations. Thus, if after an initial use of the app, the user finds early/preliminary evidence that the treatment is working as expected (confirmation of expectations), this would affect the final decision of adopting the behavioral recommendations provided by the VA, just as ECT posits that the confirmation of expectations after using a product for a while increases the likelihood of continuing to use the product or repurchasing it. Therefore, in this study, we tested the hypothesis that positive perceptions of the app’s features, coupled with experiencing a pre-intervention positive effect, would in turn facilitate subsequent adherence to the recommendations provided.

Material and methods

Study design and participants

Case series study of a self-selected sample of individuals aged 18 years or older, completing a fully automated insomnia management program named Kanopee, freely available to the general population in France, during the period December 2020 to September 2022. Informed consent was obtained directly on the app by all users before any data collection. The approval of the ethics committee of the University of Bordeaux was obtained, as well as agreement with respect to the General Data Protection Regulations (GDPR) by the French authorities (CNIL).

Kanopee is a smartphone app available on Google Play store and Apple Store in France that was launched during the peak of the COVID-19 pandemic in 2020. The app includes an animated character able to engage in face-to-face dialogue through verbal and nonverbal behavior, known in the literature as a VA. During the screening interview (Phase 1), the VA, named Louise, introduces herself and administers the Insomnia Severity Index (ISI),²³ as the main outcome of the intervention, along with other questionnaires (the Patient Health Questionnaire-4 [PHQ-4]²⁴ and the Fatigue Severity Scale²⁵). Then, visual feedback using a colored line including the traffic lights’ 3 colors (red, orange, and green) is provided. The user’s score on the ISI is located in one point along the colored line (if it is on the red area, it connotes severe insomnia, if it is on the orange area, it connotes moderate insomnia, and if it is in the green area, it connotes mild or no insomnia symptoms) and the user is asked to begin a personalized program. The first step includes completing a sleep diary. The VA explains why completing a sleep diary is important and starts collecting information from the previous night. Once this step is completed, another screen informs the users that they need to complete the same sleep diary for 7 days, every morning, and that, a week later, the VA will re-evaluate their sleep in another interview, and will analyze the information provided in their sleep diaries. An icon with the name “Sleep diary” appears then at the bottom of the screen that the user can access every day to answer the sleep-related questions (ie, what time did you go to bed last night?). After users complete the sleep diary for 7 days, Louise conducts another interview (phase 2). Users learn about their sleep patterns from the previous week and complete the ISI for the second time. Next, Louise delivers the intervention. She provides sleep recommendations, highlighting the specific ones most useful to the user, based on their sleep diary data and on their answers to the ISI questions on phase 2. Apart from general good sleep hygiene practices, Louise proposes evidence-based behavioral recommendations that have been shown to improve insomnia symptoms. These instructions are part of the stimulus control component of CBT-I. The goals of stimulus control are to: (1) extinguish the association between the bed/bedroom and wakefulness to restore the association of bed/bedroom with sleep; and (2) establish a consistent wake-time. Stimulus control instructions include: go to bed only when sleepy, get out of bed when unable to sleep, use the bed/bedroom for sleep (no reading, eating, watching TV, etc. in bed) and wake up the same time every morning.¹⁷ Once the recommendations are provided, individuals are asked to complete the sleep diary for 10 more days. After this, users are interviewed by Louise again (Phase 3—Post-intervention) and complete the ISI. Finally, users are asked if they have followed or not each one of the recommendations provided by Louise. After this final interview, users can choose to continue using the app, if they wish (ie, completing sleep diaries for a longer period of time). If they consider that their sleep problems are persisting, they are prompted to consult a sleep specialist. Further details about the design of the VA and the design and implementation of the intervention program are described elsewhere. Figure 1 summarizes the flow of the intervention.

Figure 1.

Flow of the intervention with examples of interfaces of Kanopee. (A) Screenshot of Louise questioning the Insomnia Severity Index (ISI); (B) screenshot of sleep diary; (C) screenshot of a sleep recommendation given by Louise during Phase 2; and (D) screenshot of visual feedback provided by the app on the completion of each day of sleep diary.

Measures

Adherence

Treatment adherence to the behavioral recommendations provided by the VA was evaluated via self-report questions at phase 3. Users were asked with yes/no questions if they had followed each one of these recommendations. The total number of “yes” answers was taken as a quantitative indicator of adherence. Values ranged from 0 to 3.

Users’ perceptions of app and VA

These variables were measured with 2 questionnaires. Acceptance of the app was quantified with the Acceptability E-Scale (AES).¹⁸ It comprises 6 items and provides a total score as well as 2 subscores regarding usability (ie, the perceived ease of using the app) and satisfaction (ie, experiencing pleasure using the app and realizing its value). Items are answered on a 5-point Likert-type scale ranging from 1: Very unsatisfied, to 5: Very satisfied. To assess Trust in the VA, we used the ECA-Trust Questionnaire (ETQ).¹⁸ It includes 6 items that are answered on a 4-point Likert-type scale ranging from 0: Not at all, to 3: Totally agree. It provides a total score as well as subscores on 2 separate dimensions, named perceived credibility (ie, whether the VA seemed convincing and believable) and perceived benevolence (ie, whether the VA seemed to be of help). At the end of interview in phase 1, users were asked to complete these measures, but their completion was presented as optional.

Health-related variables

Insomnia was evaluated with the ISI. It includes 7 self-report questions and provides a global measure of insomnia severity ranging from 0 to 28. The score is interpreted as follows: absence of insomnia (0–7); subthreshold insomnia (8–14); moderate insomnia (15–21); and severe insomnia (22–28). According to research on insomnia therapies, the treatment endpoint, insomnia improvement, was computed as the change in ISI scores from phase 1 (baseline score) to phase 3 (post-intervention score).^26,27 A change in ISI score from phase 1 to phase 2 was considered as a pre-intervention effect. The PHQ-4²⁴ was used to assess anxiety and depression symptoms. It includes 4 items answered on a 4-point Likert-type scale. Global score >8 suggests possible psychiatric comorbidity/severe psychosocial distress. For description of the sample, ISI scores and PHQ-4 scores at phase 1 were used. To explore whether insomnia severity and psychological symptoms were associated to adherence in the bivariate analyses, ISI scores and PHQ-4 scores at phase 2 were considered, since they were collected right before the VA provided the sleep-related recommendations.

Sociodemographic variables

At phase 1, information on age, sex, and educational level was collected.

Statistical analysis

Data were analyzed using IBM SPSS version 28. Descriptive statistics (frequency, percentage, mean, and SD) were used to report summary outcomes. The assumption of normality was assessed for the main study continuous variables: adherence scores, ETQ and AES scores, and insomnia change scores from phase 1 to phase 3. As a result, nonparametric Spearman correlation coefficients were calculated for bivariate analyses including 2 continuous variables.²⁸ However, despite violation of the assumption of normality, parametric tests such as independent sample t-tests, 1-way ANOVA with Games-Howell post hoc tests, and linear regression were used due to their robustness with sizeable sample sizes.^29,30 Binomial data were analyzed using a chi-square test. We conducted a series of mediation analyses with change in ISI scores from phase 1 to phase 2 (pre-intervention effect) as a mediator, adherence score as a dependent variable, and users’ scores on the AES and ETQ measures (only scores which showed a statistically significant correlation with the pre-intervention effect scores in the bivariate analyses), as the independent variables. Mediation analyses were performed using the PROCESS macro v.4.1 (Model 4) by Hayes in SPSS. Indirect effects were deemed significant when the confidence interval did not include zero. When the direct effect became nonsignificant but the indirect effect was significant, then full mediation was established.

Results

Sample characteristics

A total of 824 users were included in the study. The flow of participants is depicted in Figure 2. The percentage of users finalizing the intervention after completing phase 1 was 5.7%.

Figure 2.

Flow of participants.

As shown in Table 1, 517 (62.7%) of the users were female. Mean age was 51.85 (±12.55) years. About 3 quarters of the sample had a university level of education. At phase 1, 502 subjects (60.9%) had moderate to severe insomnia, according to the ISI cut-off points. In addition, 20% reported severe anxiety/depression symptoms.

Table 1.

Participants characteristics at phase 1 (n = 824).

Demographics
Age (years)	51.85 ± 12.55
Sex (female)	517 (62.7)
Education level
Middle school	78 (9.5)
High school	114 (13.8)
Less than 5 years of university	490 (59.5)
More than 5 years of university	142 (17.2)
Health-related
Insomnia severity (ISI scores)
No insomnia (0–7)	47 (5.7)
Subthreshold insomnia (8–14)	275 (33.4)
Mild-to-moderate insomnia (15–21)	440 (53.4)
Severe insomnia (22–28)	62 (7.5)
Psychiatric comorbidity (PHQ-4 scores >8)	166 (20.1)

Data are presented as n (%) unless otherwise indicated. The mean age was reported as the mean ± SD.

Abbreviations: ISI: Insomnia Severity Index; PHQ-4: Patient Health Questionnaire.

Users’ adherence to behavioral recommendations and correlates

For these analyses, we eliminated the group of individuals with ISI scores at phase 1 classified as having no insomnia, according to the ISI cut-off points (ISI score <8), n = 47 (5.7%). Of the remaining individuals (n = 777), 254 users (32.7%) followed at least one of the behavioral recommendations provided by the VA. Mean adherence score was 0.47 (±0.75) (range 0-3). Bivariate analyses showed that adherence scores were lower as ISI scores (phase 2) increased (Spearman’s rho = −0.08; P = .03). Individuals with more severe psychiatric symptoms (PHQ-4 scores >8) (phase 2) showed lower adherence scores than individuals with milder symptoms (mean = 0.36 (±0.65) vs mean = 0.48 (±0.77); T = 1.83, P = .03). Regarding the demographic variables analyzed, age, sex, and educational level, we found a statistically significant association only between educational level and adherence (F = 4.24; P = .006): post hoc analyses indicated that users having the highest educational level showed better adherence scores, mean = 0.67 (±0.87), compared to those having a middle school level, mean = 0.34 (±0.58), P = .008, and compared to users having fewer than 5 years of university studies, mean = 0.43 (±0.73), P = .02.

There was a statistically significant positive correlation between adherence scores and end-of-treatment insomnia improvement, as measured by the ISI score change between phase 1 and phase 3 (Spearman’s rho = 0.08; P = .02). Multiple regression analysis with ISI score change as the dependent variable and adherence score as the predictor variable, controlling for the study variables that were related to adherence in the bivariate analyses (educational level, as well as insomnia severity and psychiatric symptoms [phase 2]), yielded a statistically significant positive effect of adherence on insomnia change scores (unstandardized regression coefficient [β] = 0.48 [SE = 0.20]; P = .01). In this analysis, the ISI score at phase 2 was also a statistically significant predictor (β = −0.19 [SE = 0.03]; P < .001).

Treatment acceptability and trust in VA and correlates

Since completing the AES and ETQ measures was presented as optional, only a subset of users completed these questionnaires. A total of 500 users completed the AES and the ETQ measures (67.7% of the total sample). There were no differences in insomnia severity at phase 1 or in sex distribution between users completing these questionnaires (n = 500) and users who did not (n = 324). However, users completing the AES and the ETQ were older, 53.93 (±11.98) versus 48.65 (±12.05) (P < .001), and had a lower education level than those who did not (P = .025). Acceptance and trust data by subscale scores (n = 500) are presented in Figure 3A–D.

Figure 3.

Distribution of usability, satisfaction, benevolence, and credibility perceptions among users completing the Acceptability E-Scale and the ECA-Trust Questionnaire (n = 500).

We did not find any statistically significant relationship between the demographic variables age and sex and total or subscale scores of the AES and of the ETQ. However, only scores on the Benevolence scale of the ETQ were associated with educational level, F = 2.65, P = .04. Post hoc tests showed that the group of users having the lowest educational level had higher mean scores, mean = 10.80 (±13.1) than the group of users having fewer than 5 years of university studies, mean = 10.31 (±1.35), P = .01.

For the analyses on the relationship between treatment acceptability and trust in the VA and adherence, we excluded individuals with ISI scores at phase 1 classified as having no insomnia, according to the ISI cut-off points (ISI score <8), n = 27.

As displayed in Table 2, we found statistically significant Spearman’s correlation coefficients (rho values ranging from 0.09 to 0.12) between the variable pre-intervention effect and scores on the AES and the ETQ benevolence measures: higher scores were associated with pre-intervention improvements. By contrast, while pre-intervention improvement was associated with better adherence (Spearman rho = 0.09, P = .01, n = 777), we did not find any association between adherence and scores on the AES and ETQ measures.

Table 2.

Spearman correlation coefficients of users’ perceptions of the app and of the virtual agent, pre-intervention effects, and adherence.

	Pre-intervention effects	Adherence score
AES total score (n = 473)	0.10*	0.06
AES usability score (n = 473)	0.12*	0.08
AES Satisfaction score (n = 473)	0.10*	0.03
ETQ total score(n = 473)	0.06	−0.01
ETQ Benevolence score (n = 473)	0.09*	0.00
ETQ Credibility score (n = 473)	0.01	−0.01
Adherence score (n = 777)	0.09*

^* P < .05.

Abbreviations: AES: acceptability E-scale; ETQ: ECA trust questionnaire; Pre-intervention effect: ISI score change from phase 1 to phase 2.

To confirm whether the positive impact of AES and ETQ scores on the pre-intervention effects remained consistent, as well as the positive impact of the pre-intervention effects on the adherence scores, adjusting for age, sex, education level, and baseline ISI scores (phase 1), we conducted a series of multiple regression analyses. The results of these analyses are provided in Table S1. In these multiple regression analyses, AES total scores, AES usability scores, AES satisfaction scores, and ETQ benevolence scores remained statistically significant predictors of pre-intervention effects (all Ps<.05). Likewise, pre-intervention effects remained a statistically significant predictor of adherence (P = .007).

Relationship between treatment acceptance and trust in VA, pre-intervention effects, and adherence

Higher acceptability of the app and higher perceived benevolence of the VA showed a positive and statistically significant indirect effect on adherence. These indirect effects were mediated by pre-intervention effects. As depicted in Figure 4A, higher scores on the ETQ benevolence subscale significantly predicted a pre-intervention improvement in ISI scores (path a: β = 0.364, t = 2.79, P = .005), and the pre-intervention improvements predicted higher adherence scores (path b: β = 0.021, t = 2.28, P = .02). The significance of the indirect effect of ETQ benevolence scores on adherence scores by means of pre-intervention improvement in ISI scores was verified by bootstrapping. The results of the analyses showed that perceived benevolence had a significant indirect effect on adherence, as 0 was not included within the 95% CI, β = 0.007, 95% CI, 0.001-0.017. The direct path (c′) between ETQ benevolence scores and adherence was not significant, thus showing a full mediation.

Figure 4.

Path diagram of relationship of users’ perceptions, pre-intervention improvement, and adherence (n = 473).

Again, as seen in Figures 4B–D, a full mediation was found, where total scores on the AES, as well as scores on the AES Satisfaction and AES usability subscales, had an impact on adherence through pre-intervention insomnia improvements. For the 3 independent variables, the indirect effect was statistically significant, as 0 was not included within the 95% CI.

Discussion

Data from this case series study shows that users’ perceptions of the app and of the VA have an impact on their adherence behavior. Higher scores on the AES scale and higher scores on the ETQ benevolence scale were related to a pre-intervention improvement in insomnia, prior to receiving any sleep recommendations. Subsequently, this pre-intervention improvement was associated with better adherence to the instructions provided by the VA.

These findings represent a starting point to understand the complex, dynamic interrelationships between the users’ perceptions of a given technology (ie, in our study, perceived ease of use, perceived value of using the app, and perceiving the VA as convincing and helpful), which could predict the initial use or rejection of technology, and the subsequent adoption of more complex behaviors, such as adherence to the recommendations, as the intervention progresses. Whereas it seems clear that, during the first usage phase, the app should stimulate users’ interest to keep them engaged (ie, being highly acceptable and trustworthy),³¹ it appears that, in order to motivate users thereafter to follow the behavioral recommendations provided, it would be helpful to get some preliminary evidence of the benefits/gains of engagement with the app.³² We found that those users considering that the app could possibly help them (ie, higher scores on the acceptability and trust questionnaires) experienced a pre-intervention positive effect on insomnia severity after monitoring their sleep for 1 week. At this point, we can only speculate about the mechanisms underlying this improvement. Positive effects on health outcomes in the absence of an active treatment have been reported widely in the literature (the placebo effect or the self-fulfilling prophecy). Furthermore, it has been suggested that, in mental health, prospective self-monitoring of symptoms enhances the sense of control over health, since it facilitates the discovery of patterns of symptoms.³² Of importance, individuals in this study were in a real world situation, in which they were active decision-makers, and choose this specific treatment. Studies have suggested that the fact of “making a choice” elicits perceptions of control over an outcome.^33,34 Perceptions of control may lower anxiety, stress, and make people interpret a health threat as more surmountable.³⁵ Hence, this pre-intervention effect could be the result of this enhanced perception of control over their insomnia symptoms. Subsequently, the confirmation of their expectations after using the app for a period of time (ie, the app seemed to help them) was related to the fact that the users finally enacted the recommendations provided in the app.

These findings have practical implications for the designing and refining of behavioral treatment apps like ours. Firstly, not only it is important to design highly acceptable apps that will engage the users; the introduction of early feedback loops, either highlighting the user’s progress at identifying a pattern to their health problem, or enhancing their perception of control over their health concern, could possibly serve as an additional motivating factor to ultimately enact the treatment recommendations.

Admittedly, the percentage of individuals reporting at the end of treatment that they followed the recommendations provided by the VA is somewhat low, even though all of them completed all the intervention assessment phases (high experimental compliance/no attrition).³⁶ This percentage seems consistent with findings reported in the meta-analysis by Horsch et al¹⁶ on adherence to technology-mediated insomnia treatment. Furthermore, we should keep in mind that adherence in nonexperimental real-life settings is usually lower than adherence in experimental settings. In all, this figure suggests there is still room for improvement in this area. Additionally, variables previously described in the literature as negatively affecting patient adherence to traditional face-to-face therapy, such as a poorer mental health status and poorer pre-treatment sleep, also seem to be related to adherence to this fully automated abbreviated digital version of CBT-I.^14,37 Finally, and of importance, although adherence and intervention effects were evaluated only 10 days after the provision of the sleep recommendations (in other abbreviated CBT-I studies treatment effects are typically measured 14 days after the completion of the intervention),^8,38 our results show that adherence plays a role in insomnia improvement. To our knowledge, this is one of the few studies that show that adherence to treatment in fully automated digital therapy for insomnia is a predictor of treatment efficacy. However, we acknowledge that greater long-term follow-up will be necessary to document sustained intervention effects.

This study has further limitations. The adherence measure employed does not reflect the dose of the recommendation the user receives. Users only reported if they had followed or not a given recommendation, but did not indicate how often they followed the recommendation during the intervention period. Nevertheless, our findings show that adherent users tend to exhibit greater insomnia improvements than nonadherent users at the end of the treatment, which adds validity to the adherence measure used herein. Of note, Kanopee features an abbreviated version of CBT-I, highlighting the behavioral component of therapy known as stimulus control. For a behavioral treatment to be relevant in general population settings, it must be brief, acceptable to patients, and efficacious over a short time interval, conditions all met by the stimulus control intervention. Typically, abbreviated behavioral therapy for insomnia also includes sleep restriction.³⁹ However, sleep restriction recommendations were not included in Kanopee due partly to safety concerns (ie, it may be contraindicated in certain populations such as those working in high risk occupations, such as heavy machinery operators or drivers, or those predisposed to mania/hypomania, poorly controlled seizure disorders or excessive daytime sleepiness).¹⁷ In addition, the most recent guidelines for the use of behavioral and psychological treatments for chronic insomnia recommends that individuals using sleep restriction therapy should be monitored by a clinician during treatment.¹⁷ Furthermore, the distribution of scores on the AES and ETQ measures in our sample is strongly left skewed. This is an inevitable bias of our self-selected sample. Indeed, in a real world situation, initial positive perceptions of the app are deemed necessary to stay engaged using the app. Nonetheless, despite the restricted variability on the scores, we did find significant associations between these variables and adherence. Moreover, inasmuch as participants in this study were self-identified, the generalizability of the results is restricted. The female preponderance (62.7%) in our sample may reflect the higher prevalence of insomnia in women, and women also tend to seek treatment for insomnia more often than men. However, the percentage of participants with university level education is overrepresented compared to the percentage found in the general population. Finally, 5.7% of users who initiated treatment completed the full intervention. This issue is not exclusive to Kanopee: high attrition/dropout rates are ubiquitous problems in fully automated digital mental health interventions.¹¹ In a 2020 meta-analysis of 18 randomized trials of mobile apps for treating depression, the reported dropout rate was 47.8%.⁴⁰ In real-world settings, dropout rates are more dramatic: a study of 93 apps targeting mental wellbeing found that the 15- and 30-day retention rate were very low, 3.9% and 3.3%, respectively.⁴¹ Therefore, further research is needed to help understand this phenomenon and improve the retention rates.

In conclusion, mobile and web technologies that can deliver fully automated behavioral treatment are presented as a solution to extend psychological treatments to more people. However, not only these interventions require the individuals to use their mobile phones or computers to read, watch videos, or complete questionnaires in an easy and engaging way. The users also need to enact the behavioral recommendations provided. This study suggests that adding features to the program that enhance or highlight sense of control over the health outcome or self-efficacy in the early stages of the intervention could possibly increase the impact of these innovative clinical solutions. Additionally, these results stress the importance of a close interdisciplinary collaboration between clinicians, researchers, ergonomists, engineers, and computer specialists for the design and refinement of treatment apps.

Author contributions

M.M.S.-O., F.P., J.C., and P.P. conceptualized the study. E.d.S., Y.L., and P.S. contributed to data acquisition and management. M.M.S.-O. and F.P. conducted the statistical analyses and constructed the figures. J.C. and P.P. reviewed the analysis. P.S., N.S., J.A.M.-F., F.S., and M.A. contributed to data interpretation. M.M.S.-O. and F.P. wrote the first draft and all other authors reviewed and commented on the report. All authors had full access to all the data in the study, had final responsibility for the decision to submit for publication, and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Supplementary material

Supplementary material is available at Journal of the American Medical Informatics Association online.

Funding

This work was supported by the National Grants GPR BRAIN 2030 and Neurocampus (University of Bordeaux). M.M.S.-O. was supported by an award from the European Union, Next Generation EU, for the retraining of university professors. F.P. was supported by the French National Research Agency—ANR (Junior Professor Chair).

Conflicts of interest

None declared.

Data availability

The data underlying this article will be shared on reasonable request to the corresponding author.