Fidelity in the context of adapting a digital intervention for depression from an evidence-based in-person format in Vietnam

Abstract

Background

Digital interventions have emerged as a promising way to better meet growing population mental health needs. Our team developed a digital depression intervention (VMood; smartphone app) in Vietnam. VMood is adapted from an evidence-based in-person intervention (SSM) developed in Canada and uses cognitive behaviour therapy (CBT) principles with remote coaching by non-specialist providers. Fidelity-adaptation is a major tension in implementation science. Fidelity is the degree an intervention is designed and delivered as intended. Conversely, adaptations are sometimes made for specific contexts. This paper aims to identify key elements of fidelity-adaptation – the degree VMood is consistent theoretically with the SSM intervention and practically with implementing digitally in the Vietnamese setting.

Methods

This study uses Perez et al.’s modified version of Carroll et al.’s Implementation Fidelity Framework, focusing on Objective 1: Conceptualizing what intervention fidelity means in this specific context (across modes and cultures) and Objective 2: Conducting fidelity testing to identify key elements along the fidelity-adaptation continuum. Ethnographic data from team meetings explored essential components that must remain intact and necessary adaptations. Non-specialist providers and app users from Vietnam tested VMood. Experts familiar with CBT provided theoretical feedback. Interviews or focus groups were conducted with all participants to gain insights into the adaptive intervention. Qualitative data were analyzed using thematic content analysis.

Results

Participants agreed that VMood captures the essential theoretical components from SSM, noting certain elements of SSM (e.g., change in human contact to online) could not be replicated digitally. Participants also presented adaptation suggestions unique for the digital format to strengthen VMood’s acceptability, including keeping the app simple by reducing the amount of text; incorporating more dynamic content (e.g., animations) to increase engagement; and including more culturally appropriate scenarios. Finally, key potential moderators to fidelity reported included quality of program delivery and participant responsiveness.

Conclusions

Findings identified intervention specific elements of fidelity-adaptation and showed that VMood retained essential components of SSM while incorporating adaptations to support implementation within the Vietnamese context. With the global increase in digital health services adapted from in-person delivery, understanding how to balance fidelity with necessary adaptations is important both theoretically and practically.

Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1186/s12913-026-15031-x.

Background

Introduction

Worldwide, one in two people will develop a mental disorder by the age of 75 [1]. Improving access to mental health care is thus a critical global priority. This is especially pertinent in low- and middle-income countries (LMICs) like Vietnam, where care is unavailable for up to 90% of the population who need it [2]. The urgency to reduce the treatment gap has been illuminated within the context of the COVID-19 pandemic, which has led to increases in mental health conditions, including 28% for major depressive disorders and 26% for anxiety disorders [3], the two most common mental disorders [4]. Digital mental health interventions (DMHIs), including smartphone apps, have emerged as a promising way to better meet population needs given the persistent severe shortage of mental health resources [5]. The evidence for many of these DMHIs is, however, limited, especially in the context of those adapted from in-person evidence-based formats [6].

One of the major debates in implementation science is the tension between fidelity and adaptation of evidence-based interventions for delivery in routine practice [7, 8]. Fidelity is the degree an intervention is designed and delivered as intended [9]. Conversely, adaptation is where implementers or users make changes to the initial design or delivery of an intervention [10]. Pro-fidelity adherents (often program developers) advocate for close adherence to the original intervention, while pro-adaptation adherents (often program providers) argue adaptations are necessary to address specific contexts, with particular regards to ensuring cultural appropriateness [7, 11]. Developers understand the essential components that must be kept intact, whereas providers are met with practical real-world concerns of failure to implement without adaptations to fit the local context and culture [11]. Influencing factors include time constraints, availability of resources, and community norms [7].

There are various challenges to defining and conceptualizing fidelity and associated adaptations in general, including the use of inconsistent terminology [12, 13], which contributes to inconsistencies in methods, reporting standards, and operationalization of the concepts [8, 13, 14]. There has also been limited attention to cultural adaptation methods for adaptations across contexts to ensure cultural appropriateness for relevance, applicability, and ultimately their effectiveness [15, 16]. Conceptualizing fidelity is further complicated in the context of modifying in-person interventions to digital formats, where there is limited evidence on the unique considerations and practical guidance for these adaptations as many of the available adaptation frameworks were developed for face-to-face interventions [16]. In this context, there is a need to evaluate if, and how, the technical and theoretical intervention components are applied in the new mode, or enhanced by drawing on the unique features of a digital format to increase engagement with the digital format (e.g., through videos, animations), without straying from its theoretical principles [17].

Research shows that the shortage of mental health professionals and other related mental health services will persist into at least the next decade [18]. DMHIs are thus one of the most promising ways to provide accessible mental health services to those who need it, especially for those who encounter additional challenges to face-to-face care, including geographic barriers and challenges imposed by stigma [19, 20]. Another advantage is their potential to support delivery of interventions with fidelity when it is used as an adjunct to in-person care [21]. Finally, DMHIs adapted from evidence-based in-person interventions show potential to provide scalable [5, 21] and effective mental health care [20]. As treatment fidelity inconsistencies have been identified as a key contributor to the mixed evidence on the effectiveness of many of the DMHIs [22], there is a need to conceptualize and evaluate fidelity in the context of adapting DMHIs from in-person formats.

This study

Despite the recent proliferation of digital health interventions [23], there is a paucity of evidence on the unique considerations and practical guidance for the process of adapting interventions with fidelity from evidence-based in-person interventions to digital formats. In this context, there is a need to evaluate if and how the technical and theoretical intervention components are applied in the new mode, while maintaining or enhancing its effectiveness [10].

Goal of this study

This paper aims to contribute to the field of implementation science by conceptualizing and identifying key elements of fidelity during the process of adapting an evidence-based in-person depression intervention to a digital format in Vietnam. This paper is based on a feasibility study that included the evaluation of a smartphone depression app (VMood) that was developed from an evidence-based in-person intervention, in Vietnam [24].

Specific interconnected objectives are:

1. Conceptualize what intervention fidelity means in this specific context (across modes and cultures);

2. Conduct fidelity testing to identify key elements advocated by participants along the fidelity-adaptation continuum.

Methods

Prior and ongoing work

VMood is developed from an in-person supported self-management (SSM) intervention initially developed in Canada [25]. SSM is a modified approach to psychotherapy, where elements of Cognitive Behavioural Therapy (CBT) are delivered through an Antidepressant Skills Workbook (ASW) and supportive coaching by a non-specialist, non-physician provider [24, 25]. SSM was culturally adapted to the Vietnamese context and subsequently shown to be effective through a randomized controlled trial (RCT) [24]. Due to the restrictions on in-person health services imposed by the COVID-19 pandemic and the Government of Vietnam’s budgetary changes, SSM has since been adapted to delivery via a smartphone app. VMood is comprised of 3-months’ engagement with the VMood program, which includes modules from the ASW introducing depression and antidepressant skills (reactivating your life, thinking realistically, and solving problems) delivered in an interactive format with multiple choice options. Videos with a text option introduce each module that participants can navigate through. The VMood program also includes integrated support on app navigation and basic information on depression provided remotely by a MOLISA Social Worker. Social Workers provide support as part of their regular job responsibilities and have received training on basic depression, providing coaching via the app, and emergency and suicide management through an online training program developed by the Institute of Population, Health and Development (PHAD). PHAD is a leading non-profit technological and scientific organization in Vietnam and the team’s implementing partner.

The results of the feasibility study have informed the foundation of further work, conducted via an RCT in eight provinces of Vietnam to test the effectiveness and cost-effectiveness of VMood [26] by establishing key elements of fidelity. Fidelity is critical to establishing internal validity to ensure effectiveness and integrity of the intervention and supports replicability and generalizability. Dunst et al. [12], have further differentiated between theoretical fidelity, which focuses on the intervention, and practical fidelity, which focuses on the implementation process [12]. Though there has been an increase in literature in recent years capturing theoretical fidelity of the intervention itself, most of the research has focused on an examination of enhancing fidelity (practical fidelity) [27].

Overview

Our fidelity testing involved two objectives: (1) conceptualizing fidelity and (2) fidelity testing. Fidelity testing, where essential components are identified, is an important first stage for the evaluation of fidelity [8, 28]. Adaptive [28] components reflecting local cultural contexts are crucial to successful implementation and should be captured carefully. Led by our implementing partners in Vietnam and working with a local software development company, we began development of the VMood app in 2021. The prototype was shared with team members for their review of fidelity and suggestions for adaptations.

All procedures were approved by the Research Ethics British Columbia (BC) Board in Canada [H21-02938] and the Institutional Review Board at PHAD in Vietnam [PHAD-2022/VMOOD-01].

Conceptual framework

Fidelity was examined according to Perez et al.’s [14] modified version of Carroll et al.’s Implementation Fidelity Framework for Interventions, focusing on specific elements as detailed in Table 1 [29].

Table 1.

Implementation fidelity evaluation framework for interventions [14]

Elements	Definition	Data Sources
E1: Essential components	Identification of the elements that are essential for an intervention through component analysis	Identifying essential components of the SSM intervention (workbook + supportive coaching) Ethnographic data Intervention logs by providers Interviews and focus groups
E2: Adaptive intervention	Interventions for which stakeholders are allowed, or even encouraged, to bring changes to the original design	Identifying adaptive elements of the SSM intervention (workbook + supportive coaching) Ethnographic data Intervention logs by providers Interviews and focus groups
E3: Potential moderators	Factors that can influence the level of fidelity: 1) Comprehensiveness of policy description 2) Quality of delivery 3) Participant responsiveness 4) Setting or situation specific moderators	Interviews and focus groups

Objective 1: Conceptualizing fidelity

An embedded ethnography method was used for this component, where research team members’ participation as “embedded researchers” [30] in discussions around fidelity helped to inform its conceptualization within the context of adapting interventions from an in-person to a digital format. The process involved first defining what fidelity means in this specific context, which included a review of the literature on fidelity in general and in this context. The results from this review are provided in the background section. The tension between fidelity and adaptation served as the analytical framework for conceptualizing fidelity [7].

Next, discussions amongst team members around fidelity in the context of adapting the VMood app from SSM took place through monthly team meetings and various small-group meetings (in-person and online), along with personal communications (emails). The main sources of data were the meeting minutes and field notes from the in-person team meetings held in Hanoi, Vietnam from August 22–25, 2022. Most of the team members (from Canada, Australia, and Vietnam) attended. The main purpose of the meetings was to officially launch the project in Vietnam, with a substantial portion of the discussion focusing on a workshop examining the app development process, including identifying critically important components of SSM that needed to be transferred over intact to the app format (essential components) and elements that could be adapted (adaptive components). The team systematically reviewed a fidelity checklist of essential components identified prior to the meetings (App Adaptation Fidelity Checklist – see Appendix A). Items were rated as definitely present, somewhat present, or absent. Team members reached consensus about the rating categories and then proceeded to rate the essential components of the workbook and discuss the best ways to incorporate them. Meetings were also held with the local app development company to communicate decisions.

All ethnographic data were either transcribed or captured through detailed meeting minutes and field notes summarizing and interpreting the discussions. A main output in embedded ethnography is that the embedded researcher provides formative recommendations arising from the analytical framework [31].

Objective 2: Fidelity testing

Fidelity testing took place across six communes (municipal subdivisions) in one district of one province (Thanh Hoa) in Vietnam. Names of the communes and district have been omitted for confidentialitypurposes. The communes were selected in collaboration with MOLISA to represent diversity in terms of economic status (e.g., urban/rural; poorer/richer) and population composition (e.g., men/women;elderly/young). Participants included app users identified with depression caseness as measured by a Patient Health Questionnaire (PHQ-9) score > = 5 (n= 154), experts familiar with CBT principles (n= 2), andproviders involved in implementation of the VMood intervention (n= 15). Providers included two Social Workers based at the provincial Social Protection Centre (SPC)who provided administrative support, aPolicy Stakeholder at the SPC, and 13 Community Women’s Union staff (i.e., lay workers who are primarily volunteers active in the community supporting social and health initiatives) reviewing the app fortheoretical and practical considerations.

App users were recruited from the community through engagement meetings held by the PHAD team and recruitment flyers posted at local Commune Health Centres (CHC). Staff at the CHCs were trained byPHAD staff on study objectives and helped to facilitate recruitment. Recruitment took place in May and April 2023. App users were invited to participate and test the beta version of the VMood app. The PHQ-9was selected because it has been validated for use in Vietnam [32,33] and has demonstated diagnostic effectiveness on mobile devices [29,34]. The experts and providers were recruited through officialGovernment channels. Fidelity testing also involved experts (n= 3) from one city (Vancouver) of one province (BC) in Canada. Two of the experts in Canada were recruited through a local community-basedcounselling and support services organization and the third expert was invited to participate given their familiarity with SSM.

Experts and providers were asked to review the ASW and the previous RCT (MAC-FI Study) protocol [24] to familiarize themselves with SSM. Informed by a typology of adaptations summarized by Perez et al. [10] participants were subsequently asked to review the app for fidelity with SSM, documenting their thoughts on: (1) additions or new components; (2) deletions or radical modifications to the interventioncomponents that make the app significantly different to the original in-person intervention; (3) minor or major modifications to an existing component. Additionally, all experts were asked questions assessingcultural fidelity. App users were invited to download the app onto their mobile phones to test for a one-month period and share their experiences engaging with the app.

Qualitative exit interviews with the experts (n= 5), Policy Stakeholder, and Social Workers based at the SPC (n= 2), along with focus group discussions (n= 2) with the remaining providers (n= 13) wereconducted to gain insights into the degree to which interactive activities focused attention on required essential elements of the app. Questions examined whether participants noticed any key changes in the digitalformat and their opinions on these changes (what they liked, did not like, etc.,) and explored participant logs on adaptive components. In addition, qualitative interviews were conducted with app users (n= 6),based on their availability, to examine the extent participants felt they were engaged by and involved in the activities of the app, along with their level of satisfaction and suggestions for improvement.

Analysis

Ethnographic data were recorded and transcribed where applicable. All interviews and focus groups were conducted by first author LC, with the assistance of an interpreter (Vietnamese team member trained in medicine and public health and familiar with the aims and implementation of the study). All interviews and focus groups were recorded, transcribed, and translated to English using forward and backward translation. Translations were completed by a professional translator in Vietnam. Detailed field notes during and after the interviews were recorded by LC to capture thoughts and observations about the interactions that could help inform the data analysis process [35]. All qualitative data were analyzed by LC using thematic content analysis as outlined by Braun & Clarke [36] to identify and analyze common themes across transcripts that capture important meanings and patterns in the data. A coding framework was developed deductively, informed by Perez et al.’s modified version of Carroll et al.’s implementation fidelity framework [14] and inductively, through analysis of the qualitative data. Coding was a multi-stage process: (1) open coding, where preliminary summary statements were provided for elements discussed in the transcript and grouped according to Perez et al.’s framework; [14] (2) gathering all statements from open coding to collapse into main categories according to elements from the framework; (3) refinement of categories into themes; and (4) verification through validation of the coding by another research team member involved with the project (senior author JON) in the form of independently reviewing 20% of the transcripts [37]. Data were coded using NVivo 14 software [38].

Data triangulation and reflexivity were used to help ensure the trustworthiness of the data analysis and research findings [39]. Triangulation was used to explore congruence and divergence between data sources, thus increasing the rigor of the findings [36]. Reflexivity, aided by the detailed field notes and memos made during the qualitative coding process, helped to ensure trustworthiness of the research findings [36]. The audit trail also includes detailed records of the study’s methods and procedures. NVivo 14 software [38] was used for all qualitative analysis.

Results

Objective 1: Conceptualizing fidelity

For this component, the tension identified in the literature review between essential components for fidelity and necessary adaptations to fit the local context served as the analytical framework [7]. Key elements are described below.

Essential components

From the initial discussions during app content development process in 2021, the team agreed the “priority is to ensure fidelity; make sure the app is as close to the workbook [ASW] as possible.” As the supported in-person self-management intervention is comprised of the ASW and support provided by a lay health worker, the app’s modules reflect the different sections of the ASW, and the support function is embedded within the app through a chat feature with Social Workers. Therefore, when developing the app, fidelity of the ASW focused on the modules and a separate but related evaluation focused on the support component. Findings from the app development workshop component of the in-person August 2022 meetings, guided by the App Adaptation Fidelity Checklist (see Appendix A), demonstrated that the team members agreed that essential components (E1) from the workbook were present in the VMood app and delivered with fidelity. For example, team members indicated that all workbook modules were present and delivered appropriately (i.e., in chunks on separate pages). They also agreed that “modules should be administered in sequential order,” similar to the workbook,” but should include “an option to skip forwards or backwards” along with a “note in introduction recommending participants to go through sequentially.” And similar to navigating through the paper-based workbook, “participants will also have the option of leaving a module mid-way and coming back to where they left off.”

Adaptations

Themes pertaining to the adaptive intervention (E2) were: (1) format (gamification, videos), (2) limitations to a digital format, and (3) culture and language. For the first theme around app format, the team provided a number of considerations, including that the “workbook needs to be simplified for the app”, “more structure is required to address individuals’ limited attention spans, and “need to make sure the app is engaging for participant retention and participation.” Suggestions included increasing gamification to help participants “keep track of progress” through a progress tracker on the main page using different icons (e.g., logos, stars) indicating progress, and providing “motivation about setting goals and making progress towards those [goals]” via positive feedback, such as “pieces that participants have to do in order to unlock new components of the app.” The team agreed that these suggested adaptations retain the original theoretical content while potentially enhancing VMood’s efficacy by providing additional Behavioural Activation. This could be particularly beneficial for app users who are more likely to experience difficulties with motivation and engaging as a result of their depression. In line with these suggestions, minor revisions to revise the app module content for simplicity were proposed, including further dividing the word-heavy content for easier access and providing additional examples for realistic thinking exercises, and using open-source catchy graphics to replace those from the ASW (developed nearly two decades ago in 2005).

The team also agreed that there were important aspects of SSM that could not be replicated in the digital format. The theme of “limitations to a digital format” included discussions around the Social Worker support function, providing incentives for app users, and screening challenges. A key strength of SSM was the role that non-specialist providers (who were mostly volunteers) played in not only delivering the intervention and providing coaching support but also human contact to the participants in their homes. The format and scope of this support would need to be adapted as the team noted that, importantly, there is no method of digital delivery that can achieve equivalence with the in-person intervention. VMood will involve MOLISA Social Workers, who will provide virtual support remotely through the app as part of their normal responsibilities, so their time will be limited. Their training should also be scalable and sustainable and perhaps be delivered online through a succinct training program that is administered centrally by MOLISA. Some of these losses resulting from the replacement of a real person as coach could, at least partially, be mitigated by the Behavioural Activation enhancements (e.g., instructional videos) mentioned above. Further, the in-person SSM intervention was already an adaptation and step-down of traditional fully in-person psychotherapy services, but had still shown effectiveness.

Challenges resulting from the loss of a real person as coach also included additional retention concerns, where team members indicated that up to 90% of participants may be lost over the course of the larger RCT. However, as contact information is captured in the app, follow-ups can be done by telephone as necessary and a suggestion was made to perhaps include financial incentives for participants in the RCT to conduct the various required assessments administered in the form of a survey at baseline, and again at 3 and 6 months. Screening of inclusion criterion depression caseness and exclusion criterion psychosis in SSM was conducted in-person, whereas for VMood the PHQ-9 is self-administered via the app, with no comprehensive method for screening those with possible psychosis.

Lastly, “culture and language” was an important theme arising from the discussions. The team unanimously agreed at the August workshop that more culturally appropriate scenarios for the realistic-thinking and problem-solving skills should be included to capture different contexts and diversity in occupations, such as including more rural examples and occupations such as factory workers and farmers. Team members also indicated that for scale-up to different regions, attention should be given to important language differences as Vietnam is a multi-ethnic country with language variations across provinces and regions within provinces.

Objective 2: Fidelity testing

Similar to Objective 1 (conceptualizing fidelity), results from Objective 2 (fidelity testing) demonstrated the overall fidelity of the app to the SSM. This is described below. See Table 2 for a summary of the data sources.

Table 2.

Summary of data sources

Code	Source	Notes
IEX*-01	Interview Expert 1	Vietnam
IEX-02	Interview Expert 2	Vietnam
IEX-03	Interview Expert 3	Canada
IEX-04	Interview Expert 4	Canada
IEX-05	Interview Expert 5	Canada
IPS^-01	Interview Policy Stakeholder 1	Provincial Social Protection Centre (SPC)
IPR+-01	Interview Provider 1	Provincial-level Social Workers based at the SPC
IPR-02	Interview Provider 2	Provincial-level Social Workers based at the SPC
FGPR~-01	Focus Group Provider 1	8 commune-level Community Women’s Union Staff
FGPR-02	Focus Group Provider 2	5 commune-level Community Women’s Union Staff
IAU**-01	Interview App User 1	With depression caseness, as measured by a PHQ-9*(43) score > = 5; Vietnam
IAU-02	Interview App User 2
IAU-03	Interview App User 3
IAU-04	Interview App User 4
IAU-05	Interview App User 5
IAU-06	Interview App User 6

*IEX = Interview with Expert; ^IPS = Interview with Policy Stakeholder; ⁺IPR = Interview with Provider; ~FGPR = Focus Group with Provider; **IAU = Interview with App User

Essential components

This dimension explored the consistency of the theoretical concepts in the adapted VMood digital intervention compared to SSM. Overall, experts agreed that VMood captures the important theoretical content. As one expert reported:

In terms of content, I don’t see anything… It’s just a different experience. One is reading the workbook and doing the workbook. And the other one is experiencing app. And the feeling is like something is talking to you and guiding you. That is the only thing that I can see the differences here. (IEX-05)

Another expert echoed this, indicating “all the information in there, I think that its really good.” (IE-04).

Lastly, one expert concluded that “And I think that this is, this is an appropriate tool…. I don’t recommend any changes at all”. Experts were unanimous in their feedback on the theoretical fidelity of VMood and did not express any concerns on missing or inconsistent content compared to the in-person intervention.

Adaptive intervention

There was consensus amongst the experts on the fidelity of essential theoretical VMood components to SSM. However, participants presented a variety of suggestions for adaptations which they felt would strengthen VMood and increase its acceptability without compromising the essential components critical to theoretical fidelity. These suggestions included: gamification, the importance of keeping the app simple, and technical suggestions to improve app.

Gamification

The first suggestion for program differentiation made by the participants was to incorporate more gamification. This included suggestions on increasing the app’s interactiveness through additions such as reminders, progress bars, and encouragement. One provider from a focus group mentioned that:

In many cases, users only use this app when we collaborators [community women’s union staff] demonstrate how to install and use it; after that, despite having this software on their phone, they rarely open it out and use it. As a result, I think that for future efforts, there should be something in the app that catches people’s attention and draws them into using it. (FGPR-01)

Other participants suggested unique and advantageous features for a digital format, including “pop-up notification or message once a month or every other month, similar to a Viettel [phone company] message, or a screen notification” (IAU-03), “variety of games for users to enjoy” (IPS-01), “more pictures” (IEX-04)” or as a provider noted:

…a form of record to track a user’s app usage. It could be in the form of money/compensation, points, or stickers… Users, for example, will receive a sticker every time they log into the app, or once a week. This can also be displayed as a point. Furthermore, the total number of points can be turned into any other type of value. This, I believe, will encourage more people to download and use the app. (IPS-01)

From the app user’s perspective, a participant responded to a question on whether they feel motivated to use the software, “Motivation exists only when there is a purpose. Currently, the software has barely any reminders (notifications), so I sometimes forget about it.” (IAU-04) Therefore, a critical element identified by participants is to gamify the app more to increase usability and foster long-term engagement, which is crucial, especially as individuals with depression may be less motivated and require external encouragement. This importance was highlighted by a provider who emphasized that first and foremost:

If we want people to be interested in using the app, we must include something of interest for them to try and experience, correct? …. As they use the app more frequently, they are able to benefit from it more, which ultimately supports them in overcoming their problems. I think that should be the objective of the app, and not whether or not it is capable of conveying all the content and information in the book. (IPS-01)

The importance of keeping the app simple

Participants spoke also of the importance of keeping the app simple, largely by reducing the amount of content and text, particularly by avoiding complicated terminology, to increase engagement and understanding. One participant mentioned that while the app “is easier to grasp now [compared to SSM]… it can still be condensed slightly more, only providing the important principles.” (IEX-01) Although participants emphasized that they enjoyed the videos, which provided them with “useful information, and also learned more about [their] mental health” (IAU-03), a participant also suggested that:

The duration of each video should be kept to a minimum, as it is often excessively lengthy, causing viewers to become bored. The video should be concise and to the point so that viewers are less likely to become fatigued. (IAU-03)

Another participant explained that:

We must keep it (the software) as basic as possible. the language must be simple to grasp; avoid using slang or difficult-to-understand language to avoid people getting tired and not using it (the app). Phrases like delusions, hallucinations, and perceptual abnormalities are exceedingly difficult to understand as they are medical and psychiatric terminology. (IEX-01)

Unlike SSM, where ongoing coaching support was provided by a lay health worker who visited the participants in their homes on a frequent basis, VMood requires simplification to ensure participants navigating the app mostly on their own (with only basic support provided remotely through the app by Social Workers) are able to engage with and digest the content.

Technical suggestions to improve the app

Lastly, there were technical suggestions made by participants to improve the utility of the app. These included “some resource to connect and to realign on [when] there are some errors with the internet or some errors with the app… There should be some, I mean like a second resource.” (IEX-05) Many other participants indicated that “the font size is a little small,” (IAU-02) which is problematic in general as people are usually viewing from their small smartphone screen, but particularly problematic for elderly participants or other participants with vision challenges. Lastly, participants indicated that for those who identified as not having depression caseness based on the PHQ-9, the current wait time for an opportunity to retake the PHQ-9 of 14 days is “excessively long… because a person’s psychology as of today and tomorrow may have changed.” (IAU-04) This suggestion from the app user, however, may not necessarily be in alignment with mental health expert advice.

Potential moderators

Quality of program delivery

Effectiveness of the intervention at delivering program content

When participants were asked their thoughts on the ability of VMood at delivering program content, many participants spoke to the strengths of the app in delivering the theoretical content more effectively. For example, one participant mentioned:

In my opinion, if we compare the book with the app, I find that using the app is much easier and more effective than reading the book… This one has the advantage of being easier to understand than the book, which is overly long… Depressed individuals often don't want to do anything, and they lose interest in the things they used to enjoy, particularly reading books… As a result, using the software will be more advantageous. (IEX-01)

Another participant spoke more specifically about the strengths of some of the content being delivered by video format in the app, “Regarding the two contents, they are mostly the same; however, one of them the app) is not solely reading but also has videos to go along with it, so people find it more suited.” (IEX-02) They shared that “after watching videos inside the app several times, I really enjoyed it and shared it with others at my hospital, because many of my co-workers are psychologists or specialize in psychology.” Another participant also spoke of the strengths of the instructional videos at delivering the intervention content, indicating:

We will need someone, such as a professional, to interpret the book for us. However, the VMood app has detailed instructional videos, which I thoroughly enjoy watching it. I watched several of these and found them to be quite simple to understand. (IPS-01)

Participants enjoyed the format the content was presented in and were optimistic about VMood’s acceptability in the community. They appreciated the accessibility of the app and its effectiveness at delivering content that in other contexts could be complicated for app users to understand and apply. However, within the app format participants shared that the essential theoretical content was both present and delivered effectively.

Participant responsiveness

The extent to which participants are engaged by and involved with the program.

This dimension was captured mainly by app users sharing their experiences on downloading and navigating the app. One app user spoke about how they engaged with the app:

Many times in life, you can get furious over something. But now whenever I’m angry and need to calm myself down but I don’t know how, then I’d open this (app) and read the question; sometimes I skimmed through it and read it, 5 or 6 phrases or so, and occasionally, when I got to the third sentence, it (app) delved into the specific situation and advised me how to manage it. (IAU-01)

Another app user wished to:

…thank the software, which, as you can see, is highly functional, supports the issues at hand, and not only reflects on your emotions but also on the others in the community and your specific needs, and it is also close to life. (IAU-01)

This app user mentioned that “On average, I use it [app] between once per day and twice per day, or in general, the more we use it, the better.” App users were receptive to the app personally as they reported it being useful and shared their support for its use more broadly in the community.

Discussion

Principal findings

Findings from Objective 1: Conceptualizing Fidelity provided a clear framework for conceptualizing fidelity and identifying essential components of the intervention that needed to remain intact along with those that could be adapted in this context. Through team members systematically reviewing and discussing how to translate the SSM to a digital format, consensus was reached on what constituted practical and theoretical fidelity, along with limitations and potential moderators. Results from Objective 2: Fidelity Testing helped to contextualize fidelity in the process of developing a digital intervention in Vietnam from an in-person intervention developed originally in Canada and field tested in Vietnam. Findings also demonstrated both theoretical fidelity to the original psychological CBT principles, including around the psychoeducation presented in the ASW to enhance VMood’s efficacy in the larger RCT, along with additional necessary adaptations to fit local culture and support implementation.

Fidelity testing similarly showed participants were confident that theoretical content from the workbook integral to the in-person intervention’s effectiveness were intact in the digital format, albeit in a different form. In fact, many participants expressed their preference for the digital format with the workbook content being delivered via an app as it supported an accessible interface to encourage uptake and engagement, particularly for those who may experience low motivation and willingness to engage, symptoms common in those who have depression [40]. Findings may contribute further to the evidence on practical fidelity by providing guidance informing the implementation process, and more importantly findings help to fill a critical gap in the limited literature examining theoretical fidelity by focusing on the intervention itself [12]. In addition, while prior studies have typically focused only on adherence, which does not provide a comprehensive understanding of fidelity [7, 9], this study measured several additional aspects of fidelity (quality of program delivery, participant responsiveness). Most importantly, this study provided evidence that these fidelity concepts are applicable to the process of creating an adaptive intervention to ensure fit with the local context [18, 41].

Participants also shared adaptation considerations to strengthen the app, including increasing gamification, or the “use of game design elements in non-game contexts,” emphasizing the importance of keeping the app simple, and providing technical suggestions to improve the app. Gamification in the context of VMood has potential to increase intervention effectiveness by adding an additional Behavioural Activation. Similar to the literature, findings demonstrated that team members and research participants agreed and discussed in considerable detail how gamification of the app, including through the use of progress trackers and rewards, would help increase engagement, and there was no discussion of using the app for social comparison or competition. Participants also emphasized keeping the app simple to increase user engagement, or actual use of the app, and is consistent with past research showing that less complex interventions are less difficult to implement with high fidelity compared to complex interventions; [42] i.e., intervention complexity influences implementation fidelity.

Participants emphasized the importance of adaptations taking into account the local context and culture. Suggestions were made to increase culturally appropriate examples for specific modules such as realistic thinking involving more scenarios with farmers so participants, especially those in rural Vietnam, where services are severely limited, could better relate. Cultural adaptations to interventions to have more relevance with individuals from ethnic minorities have been shown to have greater efficacy [43] and more favourable health outcomes [44]. Vietnam is a diverse multi-ethnic country with 54 ethnic groups, with the minority of the population (14.6%) from the 53 ethnic minorities [45]. Most of the ethnic minorities are concentrated in mountainous and rural areas [45], where 65.6% of the Vietnamese population live [40]. Developing an intervention that will cater to the entire population is challenging, but an intervention that first considers the culture and context of the diverse groups is critical to increasing relevance and appropriateness to those who often have the greatest need.

Participants were unanimous that VMood has theoretical fidelity with the CBT program from SSM; however, they also recognized a key limitation is that the coaching support component cannot be replicated. The importance of human contact in psychotherapy has been discussed extensively in the literature [46–48]. Some mitigation strategies were shared, including gamification and rewards to help offset the losses with replacement of a real person as coach, to support Behavioural Activation and increase engagement and efficacy. Further, Newman et al. [41] demonstrated that while therapist-assisted treatments are optimal, interventions with reduced clinician contact have also been shown to be clinically effective (although with reduced effectiveness), with increased cost-effectiveness and reduced intensity. Another study comparing guided self-help with face-to-face treatments for depression and anxiety disorders showed comparable clinical effects [49]. VMood seeks to balance effectiveness with feasibility and scability to better meet growing population mental health needs.

The larger project uses an integrated knowledge translation approach, where knowledge users are engaged throughout the entire research process, to support navigation through some of these challenges identified early on in the full research program [50]. As an example, using an iterative process, data collected from the fidelity testing was continuously fed back to the project upon completion of the different fidelity components, including this conceptualizing fidelity component. This was done to ensure any necessary modifications to the app and overall study design are addressed in a timely manner.

Limitations

While fidelity testing involved both experts and Social Workers who engaged with both the workbook and the VMood app, the study was not able to recruit participants from the previous MAC-FI study to provide their experience on the actual process of utilizing the intervention. Due to health and safety restrictions imposed by the COVID-19 pandemic, the feasibility study was delayed, resulting in a substantial amount of time elapsing between MAC-FI and the current project. As such, fidelity testing was primarily conceptual and did not involve the experience of participants familiar with SSM who could share similarities and perceived differences based on their actual experiences. However, psychological skills are conceptual and with the involvement of key experts intimately familiar with CBT and depression in general, combined with practical feedback from the providers who were involved in introducing the intervention to app users, we are confident of VMood’s fidelity with SSM.

Second, all interviews and focus groups were conducted by an English speaker (LC), with the assistance of an interpreter. This introduces a potential limitation due to possible misinterpretations, interpreter and researcher bias, and inconsistencies in translation. However, the interpreter was a Vietnamese team member at PHAD trained in medicine and public health and familiar with the aims of the study, and trained by senior research team members familiar with interviewing and interpreting to practice neutrality. The interviewer was also heavily involved with the project. Transcripts were translated by a a professional translator and checked for equivalency in meaning by Vietnamese staff at PHAD. Transcripts also included the original quotes in Vietnamese alongside the translated English text to provide context and enhance trustworthiness of the data.

In addition, the interviews with the app users involved a small convenience sample of participants who were available and willing to participate, which may have introduced selection bias. Participation was largely determined by availability and may therefore reflect the perspectives of individuals who were more able to participate at the time of recruitment or more engaged with the intervention. The perspectives captured may thus not fully reflect the experiences of all app users, particularly those who disengaged earlier or were less available to participate. However, consistent with the exploratory objectives of feasibility studies, the interviews were intended to provide in-depth insights into user experiences to inform intervention refinement rather than a comprehensive representation of all user perspectives.

Lastly, there are related ongoing practical issues with balancing fidelity and adaptations beyond the development of the VMood intervention. Implementation in real-world settings can be vastly different from controlled settings [51] and factors found to impact fidelity-adaptation in this study may not fully apply in the different context. Our research team is currently conducting an RCT that seeks to thoroughly examine the implementation of VMood in real-world settings in Vietnam [26].

Conclusions

Fidelity and adaptations are crucial to the development and implementation of digital health interventions. Findings from this study identified intervention specific descriptors of fidelity and adaptation and essential components that are required to remain intact along with those that could be adapted in this context. Fidelity testing results showed that the adapted VMood app retained essential components of the original in-person SSM intervention, resulting in theoretical fidelity. Importantly, VMood also contains necessary adaptive components for cultural appropriateness to support implementation within the Vietnamese context. Finally, modifications were made to the final version of the VMood app based on adaptation suggestions from the participants involved in the fidelity testing to further improve the app. Establishing fidelity and understanding how to balance fidelity to enhance efficacy with necessary adaptations to fit the local context is important not just for the VMood intervention, but also applies globally with the widespread increase in health interventions adapted from in-person to digital delivery.

Supplementary Information

Below is the link to the electronic supplementary material.

Abbreviations

LMICs

Low- and middle-income countries

CIHR

Canadian Institutes of Health Research

GCC

Grand Challenges Canada

MOLISA

Ministry of Labour, Invalids and Social Affairs

SSM

Supported Self-Management

CBT

Cognitive Behavioural Therapy

ASW

Antidepressant Skills Workbook

RCT

Randomized controlled trial

MAC-FI

Mental Health in Adults and Children – Frugal Innovations Project

PHAD

Institute of Population, Health and Development

BC

British Columbia

SPC

Social Protection Centre

CHC

Commune Health Centre

PHQ-9

Patient Health Questionnaire-9

Author contributions

LC: contributed to the design of the study; funding acquisition; conceptualizing; project management; conducted data collection and analysis; led manuscript preparation. JM: contributed to funding acquisition; conceptualizing; writing – review and editing. VCN: contributed to the design of the study; funding acquisition; conceptualizing; project development and implementation; writing – review and editing. HT: contributed to project management and implementation; writing – review and editing. HM: contributed to the design of the study; funding acquisition; conceptualizing; writing – review and editing. RL: contributed to the design of the study; funding acquisition; conceptualizing; writing – review and editing. KH: contributed to the design of the study; funding acquisition; conceptualizing; writing – review and editing. XN: contributed to project management and implementation; writing – review and editing. EK: contributed to conceptualizing; writing – review and editing. JON: contributed to the design of the study; funding acquisition; conceptualizing; supervision; writing - review and editing.

Funding

This work was funded by by the Canadian Institutes of Health Research, Grant ID PJT 178156 and Grand Challenges Canada, Grant ID R-TTS-2205-52454.

Data availability

The datasets used and analysed during this study are available from the corresponding author on reasonable request.

Declarations

Ethics approval and consent to participate

All procedures were approved in Canada by the BC Harmonized Behavioural Ethics Board [H21-02938] and in Vietnam by the PHAD Institutional Research Board [PHAD-2022/VMOOD-01]. Informed consent to participate was obtained from all participants. Research was performed in accordance with the Declaration of Helsinki.

Consent for publication

Not applicable.

Competing interests

RWL has received honoraria for ad hoc speaking or advising/consulting, or received research funds, from: Abbvie, Asia-Pacific Economic Cooperation, Bausch, BC Leading Edge Foundation, Brain Canada, Canadian Institutes of Health Research, Canadian Medical Protective Association, Canadian Network for Mood and Anxiety Treatments, Carnot, CB Solutions, Genome BC, Grand Challenges Canada, Healthy Minds Canada, Janssen, Lundbeck, Michael Smith Foundation for Health Research, MITACS, Neurotorium, Ontario Brain Institute, Otsuka, Shanghai Mental Health Center, Unity Health, Vancouver Coastal Health Research Institute, and VGH-UBCH Foundation. The other authors declare that they have no competing interests.