Skip to main content
NCBI home page
Cambridge Prisms: Global Mental Health logoLink to Cambridge Prisms: Global Mental Health
. 2023 May 23;10:e31. doi: 10.1017/gmh.2023.23

Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya

Noah S Triplett 1, Anne Mbwayo 2, Sharon Kiche 1, Lucy Liu 1, Jacinto Silva 1, Rashed AlRasheed 1, Clara Johnson 1, Cyrilla Amanya 3, Sean Munson 4, Bryan J Weiner 5,6, Pamela Y Collins 6,7, Shannon Dorsey 1
PMCID: PMC10579659  PMID: 37854429

Abstract

Opportunities exist to leverage mobile phones to replace or supplement in-person supervision of lay counselors. However, contextual variables, such as network connectivity and provider preferences, must be considered. Using an iterative and mixed methods approach, we co-developed implementation guidelines to support the implementation of mobile phone supervision with lay counselors and supervisors delivering a culturally adapted trauma-focused cognitive behavioral therapy in Western Kenya. Guidelines were shared and discussed with lay counselors in educational outreach visits led by supervisors. We evaluated the impact of guidelines and outreach on the acceptability, feasibility, and usability of mobile phone supervision. Guidelines were associated with significant improvements in acceptability and usability of mobile phone supervision. There was no evidence of a significant difference in feasibility. Qualitative interviews with lay counselors and supervisors contextualized how guidelines impacted acceptability and feasibility – by setting expectations for mobile phone supervision, emphasizing importance, increasing comfort, and sharing strategies to improve mobile phone supervision. Introducing and discussing co-developed implementation guidelines significantly improved the acceptability and usability of mobile phone supervision. This approach may provide a flexible and scalable model to address challenges with implementing evidence-based practices and implementation strategies in lower-resourced areas.

Keywords: Task-shifting, supervision, global mental health, mobile phones, implementation

Impact statement

Task-shifting, in which lay counselors without formal mental health training are trained and supported to deliver mental health interventions, is an acceptable and effective way of delivering mental health treatment. However, there are challenges with ensuring the feasibility and sustainability of supervision in task-shifting, particularly in areas with fewer numbers of trained supervisors. Mobile phones may present an opportunity to support lay counselors from afar, but special attention must be paid to ensure that lay counselors are being appropriately supported via mobile phones. Additionally, given the vast differences between mobile phone access, cellular network connection, and other contextual variables in many low-resource areas, approaches to support mobile phone supervision must be flexible and encourage tailoring to specific contexts. The current research describes the co-development of a flexible and pragmatic approach to support mobile phone supervision with lay counselors in Western Kenya. Lay counselors and supervisors delivering a culturally adapted trauma-focused cognitive behavioral therapy participated in a co-design process to design an approach to support mobile phone supervision. The co-design process resulted in the development of implementation guidelines, which were to be shared and discussed in a brief educational outreach visit between supervisors and lay counselors. We compared the effects of the guidelines and visit on the acceptability, feasibility, and usability of mobile phone supervision between lay counselors who did and did not receive guidelines. The introduction and discussion of guidelines was associated with improvements in the acceptability and usability of mobile phone supervision. In comparing our approach and benefits to other implementation strategies, we highlight the importance of scalability, flexibility, and simplicity in our approach. We also note the importance of valuing the expertise of lay counselors and supervisors to select and tailor implementation strategies.

Background

Though there are mental health treatment gaps in nearly every country, there are larger treatment gaps in low-to-middle-income countries (LMICs), compared to high income countries, due to fewer trained mental health care providers (Demyttenaere et al., 2004; Kohn et al., 2004; Moitra et al., 2022). Task-shifting has emerged as a potential strategy to address the human resource shortages that, in part, contribute to the mental health treatment gap (Murray et al., 2014; Weiss et al., 2015; Chibanda et al., 2016; Hoeft et al., 2018). As part of task-shifting models, lay counselors (e.g., teachers and health workers) without formal mental health training are trained and supported to deliver interventions (van Ginneken et al., 2013). Evidence supports the effectiveness of task-shifting to deliver evidence-based practices (EBPs) for mental, neurological, and substance use disorders in LMIC (van Ginneken et al., 2013; Murray et al., 2014; Weiss et al., 2015; Chibanda et al., 2016; Hoeft et al., 2018; Dorsey et al., 2020b). As task-shifting continues to expand, more research is needed to understand how to fully embed and sustain task-shifting in LMIC, including how to sustainably supervise and support lay counselors (Padmanathan and De Silva, 2013).

A growing body of literature has examined how technology can be used to build capacity and scale-up task-shifting globally (Naslund et al., 2019), including as a tool to support supervision during in-person meetings (Rahman et al., 2019). In particular, mobile phones may offer a cost-effective and accessible means of replacing or supplementing in-person supervision during task-shifting; however, it is important to consider and address the various challenges and nuances that may accompany mobile phone supervision in LMIC (Triplett et al., 2023). Though less research has examined mobile phone and digital technology use in task-shifting, a large body of research has examined the challenges and opportunities of mobile phone use in LMIC (Kusimba et al., 2015; Murphy and Priebe, 2017; Wyche and Olson, 2018). Research has noted the importance of considering the limitations of mobile phones when designing and implementing digital technologies in LMIC (Wyche and Murphy, 2012), including digital supports to healthcare in LMICs (Henry et al., 2016; Feroz et al., 2020).

Given the challenges with implementing digital technologies in LMIC, flexible and adaptable implementation strategies are needed. Guidelines that contain recommendations to guide mobile phone supervision may be one pragmatic strategy; however, specific attention must be paid to ensure guidelines are flexible, account for contextual differences, and are appropriately introduced to clinicians. Guidelines are frequently used to support implementation of healthcare innovations, including within LMIC (Francke et al., 2008; Nabyonga Orem et al., 2012). Despite this, there are noted challenges with dissemination of guidelines, such as guidelines being too difficult to understand, challenges implementing without support from peers and managers, and insufficient staff and time to implement guidelines (Francke et al., 2008). These challenges may be intensified in lower-resource settings, such as LMIC (Nabyonga Orem et al., 2012). There is evidence to suggest that, with proper planning, stakeholder engagement and dissemination, guideline implementation can result in desired impacts on clinician behavior (Peters et al., 2022). Thus, if guidelines are designed to allow for flexibility and tailoring and are combined with other minimally intensive implementation strategies, such as educational outreach visits to discuss and plan for their implementation, they may be an appropriate and resource-efficient means of supporting mobile phone supervision.

The present article outlines the results of an iterative, mixed methods study that evaluated the effects of sharing co-developed implementation guidelines in a brief educational outreach visit on the acceptability, feasibility, and usability of mobile phone supervision for lay counselors in Kenya. Guided by human-centered design (HCD), we engaged supervisors and lay counselors to co-develop the implementation guidelines and determine the most feasible way of discussing and disseminating the guidelines – educational outreach visits. Leveraging an ongoing stepped wedge trial evaluating the implementation of trauma-focused cognitive behavioral therapy (TF-CBT; see Dorsey et al., 2020a), we compared effects of guidelines and outreach across quantitative measures of acceptability, feasibility, and usability. Results were contextualized with semi-structured interviews to explain the benefits of guidelines and outreach as well as how they might have impacted outcomes.

Methods

Study design

This study used an iterative and mixed methods approach, with qualitative interviews and a HCD workshop informing the development of the implementation guidelines and outreach strategy, which was subsequently tested in a randomized pilot trial. Full trial procedures are described elsewhere (Triplett et al., 2021). The trial was situated within a larger trial that examined the effectiveness and implementation of a locally adapted version of TF-CBT (Cohen et al., 2006) in Bungoma, Kenya (Building and Sustaining Interventions for Children [BASIC]; see Dorsey et al., 2020a for parent trial protocol). BASIC utilizes an eight-session version of TF-CBT (“Pamoja Tunaweza”), which was adapted by longstanding Kenyan partners at Ace Africa (i.e., supervisors and counselors) for cultural relevance and acceptability. Lay counselors work together in groups of three to provide the treatment in a group-based format and are trained and supervised by three Kenya-based supervisors. Lay counselors work within two governmental sectors in Kenya, identified as potentially viable systems for scale-up – Education (via teacher delivery) and Health (via community health volunteer [CHV] delivery). Supervisors are Ace Africa employees who were previously trained and subsequently delivered the treatment in a randomized controlled trial (Dorsey et al., 2020b) that preceded the current trial.

To develop and evaluate the guidelines and outreach, we made use of the parent study’s stepped-wedge cluster-randomized design. BASIC was comprised of seven sequences, in which a total of 40 schools and 40 communities surrounding the schools were randomized to begin TF-CBT implementation at seven different time points throughout the study. Counselors from sequences one through five of the BASIC trial were randomly selected to participate in qualitative interviews regarding their experiences using mobile phones for clinical supervision, which included strategies to improve use (Triplett et al., 2023). All BASIC supervisors also participated in interviews regarding their experiences using mobile phones to provide supervision.

Following interviews and thematic analysis, BASIC trial supervisors presented back results from the semi-structured interviews and led discussion with lay counselors on any additions, edits, or clarifications they felt were needed for the themes. This workshop also included other HCD activities, which aimed to engage participants and determine how interview findings could be translated into actionable solutions for counselors for the implementation guidelines. Following the workshop, the supervisors and principal investigator of the project (NST) reviewed all solutions and compiled them into a one-page document for supervisors and lay counselors to review together (the implementation guidelines). The co-development process occurred while sequence six of the BASIC trial was implementing TF-CBT, and they served as a no-guidelines control group. After the guidelines were developed, they were implemented with all counselors in sequence seven. This process is depicted in Figure 1. The Institutional Review Boards (IRB) at the University of Washington and Kenya Medical Research Institute approved all study procedures.

Figure 1.

Figure 1.

Open in a new tab

Trial sequence and activities.

Implementation guidelines

The guidelines are presented in Supplementary Material S1. One example goal and corresponding strategies are also presented in Table 1. As described above, the guidelines included multiple strategies that were clustered under the four goals: A) Explain to counselors that they will use in-person and mobile phone supervision together to ensure counselors are supported through delivery. B) Ensure counselors get all the information and support that they need through mobile phone supervision. C) Plan for any challenges with network connection. D) Decrease distractions and disruptions during mobile phone supervision. All strategies were presented as optional, and counselors were encouraged to select and attempt strategies that they felt best matched their unique school and community contexts. Counselors were not required to implement any solutions, and there was no expectation of continued follow-up from supervisors.

Table 1.

Sample implementation guidelines for Goal A

Goal A: Explain to counselors that we will use in-person and mobile phone supervision together to ensure they are supported through PT delivery.
A1. We will provide in-person supervision at least four times during PT delivery. This will be at the beginning, middle, and end of PT groups.
  • Supervision will be provided through mobile phones throughout the PT program. Counselors should feel free to use their mobile phones to communicate with their supervisors for support at any point during the program.
A2. If you need additional support, you may request that your supervisor comes for additional in-person supervision visits.
  • Other counselors have requested support when facing challenges with certain topics or clarifying the goals of the PT program.
Open in a new tab

Participants

Participants included lay counselors recently trained in and who subsequently delivered TF-CBT as part of the parent trial (N = 59; 29 teachers; 30 CHVs). All parent trial supervisors (N = 3) also participated in semi-structured interviews. Lay counselors and supervisors were delivering TF-CBT as part of sequences six and seven of the BASIC parent trial. To be eligible to participate in the BASIC trial, lay counselors had to be nominated from leaders at their site (i.e., head teacher and/or deputy teacher, or community health extension worker). Nominations were based on criteria to identify the most appropriate counselors (e.g., desire to be counselors; experience working with children). There were no exclusion criteria. All participants provided informed consent at the time of enrollment.

Procedure

We compared acceptability, feasibility, and usability measures of mobile phone supervision from counselors in sequences six and seven of the parent trial. Lay counselors in sequence six (n = 29) did not receive any additional support or guidelines for mobile phone supervision. Lay counselors in sequence seven (n = 30) received the implementation guidelines document. Supervisors reviewed and discussed the guidelines with sequence seven lay counselors in an educational outreach visit before they began TF-CBT delivery. These visits were integrated into the first in-person supervision meeting for the parent trial, which occurred shortly following training and before lay counselors began TF-CBT groups. The addition of mobile phone supervision outreach added approximately 30 min to the initial in-person supervision meeting. Supervisors traveled to lay counselors’ sites and facilitated separate meetings with each group of lay counselors (i.e., the meetings included the supervisor and the three counselors who co-delivered the treatment in each site). Some phone contact occurred prior to this meeting to plan a meeting time, then phone supervision continued following this meeting.

Per the parent trial protocol, in-person supervision happened at least four times during each TF-CBT group (8 weeks). Through both sequences, supervisors were also available to conduct additional in-person supervision as needed with lay counselors. The manipulation was solely the introduction of guidelines and an educational outreach visit to support successful supervision by mobile phone. Lay counselors in both sequences completed measures of mobile phone supervision acceptability and feasibility, as well as a measure of usability, after delivering two rounds of TF-CBT. All three supervisors and a randomly selected sub-sample of lay counselors from sequence seven (N = 12; six teachers; six CHVs) also participated in qualitative interviews to gather more information on their experience with the guidelines and perspectives on mobile phone supervision. The final interview guide is included in Supplementary Material S2.

Measures

Measures were adapted from existing measures, prioritizing acceptability and feasibility measures already translated and used cross-culturally in the parent trial (i.e., BASIC) and other studies globally. All adaptations to the usability measure were made following established procedures to ensure common understanding of the construct (Dorsey et al., 2020a) and completed in consultation with longstanding Kenyan partners on the trial.

Acceptability

The four-item Acceptability of Intervention measure (Weiner et al., 2017) was adapted and used to assess lay counselor perspectives of mobile phone supervision acceptability (e.g., “I like mobile phone supervision”). Scores range from 1 to 5, with higher scores representing greater acceptability. This brief, pragmatic measure has acceptable internal consistency and test–retest reliability in other samples (Weiner et al., 2017), and had good internal consistency in our sample (α = 0.89).

Feasibility

The four-item Feasibility of Intervention measure (Weiner et al., 2017) was used to assess lay counselor perspectives of mobile phone supervision feasibility (e.g., “mobile phone supervision seems doable in this school/community”). Scores range from 1 to 5, with higher scores representing greater feasibility. This measure has acceptable internal consistency and test–retest reliability in other samples (Weiner et al., 2017), and had excellent internal consistency in our sample (α = 0.93).

Usability

The 10-item Intervention Usability Scale (IUS; Lyon et al., 2020) was used to assess lay counselor perspectives of mobile phone supervision usability (e.g., “mobile phone supervision was easy to use”). Scores range from 0 to 100, with higher scores representing greater usability. The IUS has acceptable internal consistency in other samples (Lyon et al., 2020), and had good internal consistency in our sample (α = 0.87).

Analysis

We present descriptive statistics (mean, standard deviation, and range) to understand counselor and supervisor ratings of acceptability, feasibility, and usability following mobile phone supervision. We also conducted independent samples t-tests to compare average ratings of acceptability, feasibility, and usability across sequences that did and did not receive the guidelines and educational outreach visit. Following best practices for smaller sample sizes (Weissgerber et al., 2016), quantitative data are also visualized to illustrate any outliers and differences between sequences. All quantitative analyses were conducted using R (R Core Team, 2022).

Recordings from interviews were transcribed and identifying information was removed. Transcripts were coded in Dedoose (QSR International Pty Ltd., 2018) by researchers in the US and Kenya. Analysis was informed by thematic analysis (Braun and Clarke, 2006). Kiswahili interviews were translated by native Kiswahili speakers and trained translators. To develop an initial codebook, coders reviewed three transcripts independently, then met to identify potential codes and produce an initial codebook. This codebook was subsequently applied and refined on the remaining interviews. All coding was done independently, and consensus was reached through group dialog (Hill et al., 1997). We followed a QUAN → qual mixed methods approach for data explanation, using the embedded qualitative data to elaborate on or contextualize quantitative results (Palinkas et al., 2011).

Results

Sample demographics and quantitative results

Our sample included 29 teachers and 30 CHVs who delivered TF-CBT in sequences six and seven of the BASIC trial. Demographics are presented in Table 2. Differences in mobile phone supervision acceptability, feasibility, and usability between sequence six and sequence seven are presented in Table 3 and Figure 2. As shown, there were statistically significant differences in acceptability and usability between counselors who did receive guidelines and those who did not. Given potential conceptual overlap between quantitative measures, we also present Pearson correlation coefficients between outcome measures in Supplementary Material S3.

Table 2.

Demographics and baseline characteristics

No guidelines sequence six (n = 29) Guidelines sequence seven (n = 30)
Characteristic No. (%) No. (%)
Sector (counselor type)
   Education (teachers) 14 (48.3) 15 (50.0)
   Health (CHVs) 15 (51.7) 15 (50.0)
Sex
   Male 11 (37.9) 12 (40.0)
   Female 18 (62.1) 18 (60.0)
Highest level of education
   Primary education 7 (24.1) 2 (6.7)
   Secondary education 8 (27.6) 13 (43.3)
   Certificate 2 (6.9) 9 (30.0)
   Diploma certificate 8 (27.6) 4 (13.3)
   Master’s degree 4 (13.8) 2 (6.7)
Received prior training in psychosocial counseling
   No 14 (48.3) 20 (66.7)
   Yes 15 (51.7) 10 (33.3)
Provided prior psychosocial counseling
   No 6 (20.7) 11 (36.7)
   Yes 23 (79.3) 19 (63.3)
Experience working with children/adolescents
   No 7 (24.1) 6 (20.0)
   Yes 22 (75.9) 24 (80.0)
Experience working with parents/guardians
   No 5 (17.2) 10 (33.3)
   Yes 24 (82.8) 20 (66.7)
M (SD) M (SD)
Age (in years) 42.9 (7.3) 40.9 (6.7)
Years of part-time psychosocial counseling experience 6.2 (3.7) a 8.7 (8.1) b
Years of full-time psychosocial counseling experience 0.04 (0.2) a 5.4 (9.3) b
Open in a new tab
a

n = 23.

b

n = 19.

Table 3.

t-Test results for acceptability, feasibility, and usability

No guidelines (sequence six) Guidelines (sequence seven)
M SD M SD t-Value df p
Acceptability 4.08 0.34 4.35 0.18 −2.05 57 0.04
Feasibility 3.78 0.79 4.15 0.29 −1.92 57 0.06
Usability 66.12 18.46 75.58 12.05 −2.34 57 0.02
Open in a new tab

Figure 2.

Figure 2.

Open in a new tab

Differences in outcomes by guidelines condition.

Qualitative results

Lay counselors and supervisors in sequence seven described benefits of the mobile phone supervision guidelines, and specifically the benefits of having a dedicated educational outreach visit to discuss strategies to improve mobile phone supervision prior to its implementation. The qualitative themes on benefits of the guidelines and outreach included: setting expectations for mobile phone supervision; emphasizing the importance of mobile phone supervision; increasing comfort with mobile phone supervision; and, sharing strategies for mobile phone supervision.

Setting expectations for mobile phone supervision

Guided by the guidelines document (see Goal A), supervisors discussed what counselors might expect during mobile phone supervision and worked to collaboratively set expectations for how supervisors and counselors themselves would approach mobile phone supervision, including expectations for carrying phones and ensuring their phones were working. As one counselor described, “this meeting revealed to me… that everyone is required (to have a phone), if you do not have a phone, try your best to have a phone so that any info comes from a supervisor (you won’t miss it)….” Counselors and supervisors also discussed how the educational outreach visit enabled them to set expectations for scheduling and rescheduling mobile phone supervision, which ultimately led to more successful mobile phone supervision meetings: “The other times before we did these meetings of mobile phone supervision, we were just calling (counselors)…. But now due to these meetings, it made communication work better because now we were planning earlier before the calls.”

Emphasizing the importance of mobile phone supervision

As one supervisor described, having educational outreach visits and discussions about mobile phone supervision before beginning treatment delivery “let the counselors know that it’s not all about in-person (supervision). Because initially, (counselors) were thinking that supervision is mainly important if it’s in person, but then they took the phone supervision more seriously.” Counselors also noted how the visit and guidelines influenced their perceptions of the importance of mobile phone supervision, with one counselor explaining that “(they) had never been subjected to telephone supervision (before this program)…. But, this time (their) supervisor was able to teach (them) the importance of having telephone supervision….” Defining the importance of mobile phone supervision was closely related to setting expectations for supervision, as counselors and supervisors both noted that setting expectations for carrying phones and being available for phone supervision meetings reinforced the importance of mobile phone supervision.

Increasing comfort with mobile phone supervision

This was facilitated by a variety of factors, including decreasing counselor anxiety surrounding mobile phone supervision and increasing comfort with specific strategies. Counselors reported feeling some anxiety calling and getting something wrong or inconveniencing their supervisors; however, as one counselor reported, “(the meeting) put me in a good position to talk on the phone, getting used to doing something like a telephone interview without trembling.” This increased comfort was essential for counselors to engage in all the mobile phone supervision strategies, including those that were intended to reduce counselor airtime usage, such as flashing and reverse calling. Flashing required lay counselors to call and quickly hang up such that their supervisor’s airtime was charged as opposed to their own. Reverse calling is a specific way of dialing that charges the recipient’s airtime. As one supervisor noted, “we made them comfortable to flash us initially. Initially, (the counselors) were like, sometimes they didn’t have their time, so I made the counselors comfortable to just flash me.”

Sharing strategies for mobile phone supervision

The experience of receiving mobile phone supervision was new for counselors, who noted the benefits of not only learning to use phones but also strategies to make their use easier: “We were taught on how we were to use the phones. Now these are the strategies on how we were to use (them).” As one supervisor explained, the educational outreach visit was a more comprehensive and efficient way of sharing strategies that might have naturally occurred later: “Without the meeting, we could have done some of the activities, like making reminder phone calls, but it could have not created as much emphasis, and they could have not gotten as much more information as they did during the meetings.”

Goals and strategies

In describing the benefit of sharing strategies for mobile phone supervision, counselors and supervisors referenced strategies that aligned with each guidelines document goal.

Goal A: Explain logistics of mobile phone supervision

Among the most frequently mentioned (relative to the overall goal) were requests for in-person supervision, in which counselors noted that having this option was essential in building their confidence and supporting them in handling clinical challenges: “As I was delivering the lesson, the girls were so emotional…. It was even difficult for me to continue with the lesson. So, I called my supervisor….” This flexibility and additional support were crucial for supporting counselors as they assumed new counseling roles and the associated emotional difficulties.

Goal B: Ensure counselors get all needed information and support

Counselors frequently mentioned the importance of sharing contact information at the educational outreach visit, both with supervisors and co-counselors. They also discussed how they would vary their communications between phone calls, SMS, and WhatsApp messages. One strategy that was stressed to ensure all counselors were receiving information was to communicate messages to the entire group, either via similar yet separate messages, singular messages to the entire group, or even group calls (i.e., conference calls or calling one counselor and having them place their phone on speaker). Counselors discussed how this helped them to not miss important information from their supervisors, “We could not worry because at least the three of us, there is no way we could (all miss messages).” Another counselor noted that group communications also had the benefit of facilitating learning between counselors: “I learn more because in the, that is WhatsApp, we are in a group, then we share. And as you are sharing that with our supervisor, you get more information from other co-teachers.”

A crucial piece of ensuring counselors received all needed information and support was ensuring that their phones were supplied with airtime, charged, and properly working. To address airtime shortages, counselors discussed various strategies to communicate when their airtime was low, such as “reverse calling” (i.e., calling and charging the recipient’s airtime), “flashing” (i.e., calling and hanging up quickly such that the other person returns the call and charges their own airtime), and sending a free “please call me” text message to supervisors. As one counselor described, “So, we were told in the meeting that in case we don’t have airtime, you are just supposed to flash or even use WhatsApp. We have please call, send a please call me. And our supervisor will call us back.” Counselors also referenced strategies to ensure their phones remained charged and minimize battery use.

Goal C: Plan for challenges with network connection

Counselors mentioned the importance of updating their phones and SIM cards to improve their network connection. Counselors also indicated that the educational outreach visit was helpful in facilitating their identifying locations with strong network connection where they could take mobile phone supervision calls and receive messages. One counselor noted, “I also learned that I have to find a good place where the network is stable to talk to the supervisor for things to be better….” For some counselors, having scheduled mobile phone supervision meetings (another strategy) was also important to ensure they could be in a location with network connection: “there’s just a particular point where we have the network…. So, (the guidelines) forced us to make sure that at a certain time, you need to move somewhere at a certain point so that you get some communication.”

Goal D: Decrease distractions and disruptions

Given that counselors frequently completed supervision meetings at the schools in which they delivered treatment, they noted the importance of identifying quiet and secure places to have supervision phone calls: “during the meeting (the supervisor) also told us to make sure that we are confidential… to make sure that when we decide that we have to make a call, we were to look for a quiet place.” Advance notice of when supervision would occur was crucial for counselors to go to these locations on the school ground, prepare themselves, and notify others that they would be busy with mobile phone supervision. One counselor noted, “I can also tell those who are around, who may make noise, that I have (supervision) now, let us not be noisy. So that when we start, we do not get any disturbance….”

Discussion

Co-developed implementation guidelines and educational outreach visits were associated with improved acceptability and usability of mobile phone supervision. Qualitative interviews with lay counselors and supervisors contextualized how the guidelines and visits impacted acceptability and usability – by setting expectations for mobile phone supervision, emphasizing its importance, increasing comfort, and sharing strategies to improve mobile phone supervision. In discussing how specific strategies were utilized across different contexts, interviews showed how the guidelines were implemented across contexts and highlight the flexibility of our approach. Our results indicate that, with some flexibility and minimal support around their introduction and dissemination, guidelines may be a viable tool to implement and sustain some clinical innovations or implementation strategies across areas with varying levels of resources.

Flexible implementation guidelines

Given the documented challenges with implementing and sustaining interventions and their components (e.g., mobile phone supervision), there have been calls for greater research on how to best tailor implementation support (Baker et al., 2015; Powell et al., 2017; Waltz et al., 2019). Research has examined implementation support approaches that identify and match implementation strategies to address specific determinants prior to implementation (Baker et al., 2015) as well as more intensive approaches to provide individualized, on-going implementation support, such as implementation facilitation (Smith et al., 2022) and technical assistance (Katz and Wandersman, 2016). Though effective, there may be challenges with implementing or scaling complex, multi-faceted implementation strategies in lower-resource settings. We expand the existing research by examining the effect of a pragmatic multi-faceted implementation strategy: co-developed implementation guidelines for mobile phone supervision and brief educational outreach visits. Incorporated into our guidelines were multiple suggested strategies for improving mobile phone supervision, thereby providing guidance to counselors while accounting for needed flexibility and tailoring to different contexts (e.g., multiple options for communicating with different levels of network bandwidth).

In comparing our approach and benefits to other implementation strategies, we highlight the importance of scalability, flexibility, and simplicity in our approach. Noting resource constraints and the need for scalability, we were guided by the concept of “minimal intervention needed to produce change” (Glasgow et al., 2014). Though formative work was completed to develop the implementation guidelines (Triplett et al., 2023), the educational outreach visits were fully integrated into existing research and clinical activities. Lay counselors were supplied with the paper guidelines documents, but they were not required to track strategies or evaluate their own fidelity to the strategies. Increasingly, implementation researchers are trying to understand how little support or intervention can be provided to achieve desired impacts (Glasgow et al., 2014; Lyon et al., 2022). Our approach was designed with this in mind, prioritizing pragmatism and scalability with minimal resources.

Our approach also prioritized flexibility and acknowledged the expertise of lay counselors and supervisors. Children’s mental health implementation research has often only consulted with stakeholders to understand challenges following implementation efforts, which limits the ability of researchers to empower stakeholders, preempt challenges to EBP implementation, and ensure implementation success (Triplett et al., 2022). Conversely, our approach enabled lay counselors and stakeholders to develop their own guidelines. It also allowed for flexibility and daily tailoring within guidelines by lay counselors and supervisors. As interview quotations illustrate, this approach was highly acceptable as it increased counselor comfort enabled them to seek and receive more support while implementing the intervention. Much of the existing literature on acceptability has focused on the acceptability of interventions to stakeholders (Lewis et al., 2015). However, it is also important to investigate the acceptability of implementation strategies (Proctor et al., 2013), particularly those that may be critical to implementation and eventual sustainment, such as mobile phone supervision. Particularly given the power dynamics inherent in both global mental health and implementation science research, where researchers often put forward EBPs to implement, it is important to assess and potentially improve the acceptability of interventions and implementation strategies.

Impacts on mobile phone supervision

Our approach was associated with greater acceptability and usability of mobile phone supervision. There is no evidence that the introduction of guidelines and the educational outreach visits were associated with significant changes in mobile phone supervision feasibility. There are significant challenges in implementing digital health solutions in LMIC (Kusimba et al., 2015; Murphy and Priebe, 2017; Wyche and Olson, 2018), including with lay counselors (Triplett et al., 2023). By focusing specifically on these challenges within our guidelines, we may have even increased the salience of barriers to mobile phone supervision. Interestingly, though average feasibility ratings were high, there was great variability in lay counselor reports of feasibility across both sequences, meaning counselors varied more in their perception of mobile phone supervision as compared to acceptability or usability. This again highlights the importance of flexible approaches that allow tailoring to individual contexts, given that initial perceptions of feasibility may be discrepant across contexts.

The guidelines most often included short-term solutions to challenges that could be implemented with minimal resources. Other potentially viable strategies to improve feasibility may have included the provision of additional resources (e.g., phones or airtime) or formal workload adjustment to allow for lay counselors to receive supervision. There was no intervention on structural challenges (e.g., network coverage), which gave rise to the barriers and could ultimately impact feasibility. Instead, as evidenced in the qualitative findings, the guidelines’ focus on individual actions supported counselors through challenges. Though these types of solutions may be necessary in the present, it should be stressed that equitable and sustainable implementation of mental health interventions, particularly when projects are driven by U.S. investment in lower-resource areas, must also aim to address the structural and systemic factors that have created and maintain inequities (e.g., working to understand and address global power structures that dictate resource allocation).

Limitations

Our implementation guidelines were co-developed alongside lay counselors and supervisors from previous sequences in the trial and were very specific mobile phone supervision in Kenya. While we believe the flexibility and rationale underlying the approach would extrapolate well to other projects, further research is needed to evaluate similar approaches across contexts. Similarly, though the cluster-randomized trail design may have protected against some threats to validity in terms of participant selection, our design cannot account for natural improvements in mobile phone supervision that might have occurred due to the progression of time. Additionally, because of the aims of this trial and desire to reduce reporting burden on participating counselors, we did not track the impact of the facilitation on frequency of mobile phone contacts or the frequency with which specific strategies were used. Further research is needed to investigate the impacts of the program on these as well as other implementation outcomes (i.e., what are the effects of mobile phone supervision on intervention fidelity and clinical outcomes). Finally, our coding team consisted of one native Kiswahili speaker who consulted the Kiswahili audio and answered team questions related to translation and coding for Kiswahili interviews. However, it is possible that some nuance in conversations was lost in the translation to English.

Conclusion

Mobile phones may present an opportunity to increase access to lay counselor supervision; however, implementation of mobile phone supervision must acknowledge and address contextual barriers. Our approach significantly impacted the acceptability and usability of mobile phone supervision. Qualitative interviews with lay counselors and supervisors contextualized how the guidelines and educational outreach visits impacted acceptability and usability – by setting expectations for mobile phone supervision, emphasizing importance, increasing comfort, and sharing strategies to improve mobile phone supervision. Together, results highlight the importance of minimally intensive and flexible implementation supports that can be tailored across contexts and empower stakeholders to select and implement their own solutions. Importantly, we argue this must not come at the cost of addressing structural issues that give rise to downstream barriers. Both elements are crucial to ensuring interventions can be implemented and sustained across areas with varying levels of resources.

Acknowledgments

We thank the Ace Africa TF-CBT supervisors, community mobilizer, and interviewers: Elijah Agala, Moses Malaba, Emmanuel Muli, Bernard Nabalia, Micah Nalianya, Michael Nangila, Omariba Nyaboke, Daisy Okoth, Annette Sulungai, Sylvia Wafula, and Nelly Wandera. Their hard work, dedication, and high integrity made this research possible. We are also grateful to the Kenyan Ministry of Health, Ministry of Education, Teachers Service Commission, participating schools, communities, children, and families who are taking part in this trial and the parent trial.

Open peer review

To view the open peer review materials for this article, please visit http://doi.org/10.1017/gmh.2023.23.

Supplementary material

For supplementary material accompanying this paper visit https://doi.org/10.1017/gmh.2023.23.

S2054425123000237sup.zip (73.1KB, zip)

click here to view supplementary material

Data availability statement

The datasets generated and analyzed for this study are available from the corresponding author upon reasonable request.

Author contribution

Conceptualization: N.S.T., S.M., B.W., P.C., S.D.; Data curation: N.S.T.; Formal analysis: N.S.T., S.K., L.L., J.S.; Funding acquisition: N.S.T.; Project administration: C.A.; Supervision: N.S.T., A.M., S.M., B.W., P.C., S.D.; Writing – original draft: N.S.T., S.K., L.L., J.S.; Writing – review and editing: N.S.T., A.M., S.K., L.L., J.S., R.A., C.J., C.A., S.M., B.W., P.C., S.D.

Financial support

Funding for this study has been provided by the National Institute of Mental Health (NIMH) through a National Research Service Award (NRSA) Individual Pre-Doctoral Fellowship (NIMH F31 MH124328; N.S.T., PI) and the larger randomized clinical trial that is providing the sample of lay counselors and supervisors (NIMH R01 MH112633; Whetten and S.D., MPI). Funding was also provided by a dissertation grant from the Health Policy Research Scholars Program, a program of the Robert Wood Johnson Foundation. S.M.’s time was supported through the University of Washington ALACRITY Center which is funded through the National Institutes of Health (P50 MH115837). The funders/sponsor have no role in study design or analysis.

Competing interest

N.S.T. has received honoraria for speaking engagements with the Robert Wood Johnson Foundation, a funder of this work. S.D. has received grants and honorariums for providing training and consultation on the treatment model that was adapted and delivered by the lay counselors (trauma-focused cognitive behavioral therapy). The remaining authors declare that the research was conducted in the absence of any significant commercial or financial relationships that could be construed as a potential conflict of interest.

Ethics standard

All authors declare to adhere to the publishing ethics of Global Mental Health.

References

  1. Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, Robertson N, Wensing M, Fiander M, Eccles MP, Godycki-Cwirko M, van Lieshout J and Jäger C (2015) Tailored interventions to address determinants of practice. Cochrane Database of Systematic Reviews 2015(4), CD005470. 10.1002/14651858.CD005470.PUB3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Braun V and Clarke V (2006) Using thematic analysis in psychology. Qualitative Research in Psychology 3(2), 77–101. 10.1191/1478088706qp063oa. [DOI] [Google Scholar]
  3. Chibanda D, Weiss HA, Verhey R, Simms V, Munjoma R, Rusakaniko S, Chingono A, Munetsi E, Bere T, Manda E, Abas M and Araya R (2016) Effect of a primary care-based psychological intervention on symptoms of common mental disorders in Zimbabwe: A randomized clinical trial. JAMA 316(24), 2618–2626. 10.1001/jama.2016.19102. [DOI] [PubMed] [Google Scholar]
  4. Cohen JA, Mannarino AP and Deblinger E (2006) Treating Trauma and Traumatic Grief in Children and Adolescents. New York: Guilford Press. [Google Scholar]
  5. Demyttenaere K, Bruffaerts R, Posada-Villa J, Gasquet I, Kovess V, Lepine JP, Angermeyer MC, Bernert S, de Girolamo G, Morosini P, Polidori G, Kikkawa T, Kawakami N, Ono Y, Takeshima T, Uda H, Karam EG, Fayyad JA, Karam AN, Mneimneh ZN, Medina-Mora ME, Borges G, Lara C, de Graaf R, Ormel J, Gureje O, Shen Y, Huang Y, Zhang M, Alonso J, Haro JM, Vilagut G, Bromet EJ, Gluzman S, Webb C, Kessler RC, Merikangas KR, Anthony JC, von Korff M, Wang PS, Brugha TS, Aguilar-Gaxiola S, Lee S, Heeringa S, Pennell BE, Zaslavsky AM, Ustun TB, Chatterji S and WHO World Mental Health Survey Consortium (2004) Prevalence, severity, and unmet need for treatment of mental disorders in the World Health Organization world mental health surveys. JAMA 291(21), 2581–2590. 10.1001/JAMA.291.21.2581. [DOI] [PubMed] [Google Scholar]
  6. Dorsey S, Gray CL, Wasonga AI, Amanya C, Weiner BJ, Belden CM, Martin P, Meza RD, Weinhold AK, Soi C, Murray LK, Lucid L, Turner EL, Mildon R and Whetten K (2020. a) Advancing successful implementation of task-shifted mental health care in low-resource settings (BASIC): Protocol for a stepped wedge cluster randomized trial. BMC Psychiatry 20(1), 10. 10.1186/s12888-019-2364-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Dorsey S, Lucid L, Martin P, King KM, O’Donnell K, Murray LK, Wasonga AI, Itemba DK, Cohen JA, Manongi R and Whetten K (2020. b) Effectiveness of task-shifted trauma-focused cognitive behavioral therapy for children who experienced parental death and posttraumatic stress in Kenya and Tanzania: A randomized clinical trial. JAMA Psychiatry 77(5), 464–473. 10.1001/jamapsychiatry.2019.4475. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Feroz A, Jabeen R and Saleem S (2020) Using mobile phones to improve community health workers performance in low-and-middle-income countries. BMC Public Health 20(1), 1–6. 10.1186/S12889-020-8173-3/PEER-REVIEW. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Francke AL, Smit MC, De Veer AJE and Mistiaen P (2008) Factors influencing the implementation of clinical guidelines for health care professionals: A systematic meta-review. BMC Medical Informatics and Decision Making 8, 38. 10.1186/1472-6947-8-38. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Glasgow RE, Fisher L, Strycker LA, Hessler D, Toobert DJ, King DK and Jacobs T (2014) Minimal intervention needed for change: Definition, use, and value for improving health and health research. Translational Behavioral Medicine 4(1), 26. 10.1007/S13142-013-0232-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Henry JV, Winters N, Lakati A, Oliver M, Geniets A, Mbae SM and Wanjiru H (2016) Enhancing the supervision of community health workers with WhatsApp mobile messaging: Qualitative findings from 2 low-resource settings in Kenya. Global Health, Science and Practice 4(2), 311–325. 10.9745/GHSP-D-15-00386. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Hill CE, Thompson BJ and Williams EN (1997) A guide to conducting consensual qualitative research. Counseling Psychologist 25(4), 517–572. 10.1177/0011000097254001. [DOI] [Google Scholar]
  13. Hoeft TJ, Fortney JC, Patel V and Unützer J (2018) Task-sharing approaches to improve mental health care in rural and other low-resource settings: A systematic review. Journal of Rural Health 34(1), 48–62. 10.1111/jrh.12229. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Katz J and Wandersman A (2016) Technical assistance to enhance prevention capacity: A research synthesis of the evidence base. Prevention Science 17(4), 417–428. 10.1007/S11121-016-0636-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Kohn R, Saxena S, Levav I and Saraceno B (2004) The treatment gap in mental health care. Bulletin of the World Health Organization 82(11), 858–866. [PMC free article] [PubMed] [Google Scholar]
  16. Kusimba SB, Yang Y and Chawla NV (2015) Family networks of mobile money in Kenya. Information Technologies & International Development 11(3), 1–21. [Google Scholar]
  17. Lewis CC, Fischer S, Weiner BJ, Stanick C, Kim M and Martinez RG (2015) Outcomes for implementation science: An enhanced systematic review of instruments using evidence-based rating criteria. Implementation Science 10(1), 1–17. 10.1186/s13012-015-0342-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Lyon AR, Liu FF, Connors EH, King KM, Coifman JI, Cook H, McRee E, Ludwig K, Law A, Dorsey S and McCauley E (2022) How low can you go? Examining the effects of brief online training and post-training consultation dose on implementation mechanisms and outcomes for measurement-based care. Implementation Science Communications 3(1), 79. 10.1186/S43058-022-00325-Y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Lyon AR, Pullmann M, Jacobson J, Osterhage K, Achkar MA, Renn B, Munson S and Areán P (2020) Assessing the usability of complex psychosocial interventions: The intervention usability scale. Implementation Research and Practice 2. 10.1177/2633489520987828. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Moitra M, Santomauro D, Collins PY, Vos T, Whiteford H, Saxena S and Ferrari AJ (2022) The global gap in treatment coverage for major depressive disorder in 84 countries from 2000-2019: A systematic review and Bayesian meta-regression analysis. PLoS Medicine 19(2), e1003901. 10.1371/JOURNAL.PMED.1003901. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Murphy LL and Priebe AE (2017) My co-wife can borrow my mobile phone! Gender, Technology and Development 15(1), 1–23. 10.1177/097185241101500101. [DOI] [Google Scholar]
  22. Murray LK, Dorsey S, Haroz E, Lee C, Alsiary MM, Haydary A, Weiss WM and Bolton P (2014) A common elements treatment approach for adult mental health problems in low- and middle-income countries. Cognitive and Behavioral Practice 21(2), 111–123. 10.1016/j.cbpra.2013.06.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Nabyonga Orem J, Bataringaya Wavamunno J, Bakeera SK and Criel B (2012) Do guidelines influence the implementation of health programs? – Uganda’s experience. Implementation Science 7(1), 1–16. 10.1186/1748-5908-7-98/FIGURES/4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Naslund JA, Shidhaye R and Patel V (2019) Digital technology for building capacity of nonspecialist health workers for task sharing and scaling up mental health care globally. Harvard Review of Psychiatry 27(3), 181–192. 10.1097/HRP.0000000000000217. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Padmanathan P and De Silva MJ (2013) The acceptability and feasibility of task-sharing for mental healthcare in low and middle income countries: A systematic review. Social Science & Medicine 97, 82–86. 10.1016/j.socscimed.2013.08.004. [DOI] [PubMed] [Google Scholar]
  26. Palinkas LA, Aarons GA, Horwitz S, Chamberlain P, Hurlburt M and Landsverk J (2011) Mixed method designs in implementation research. Administration and Policy in Mental Health 38(1), 44–53. 10.1007/s10488-010-0314-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Peters S, Sukumar K, Blanchard S, Ramasamy A, Malinowski J, Ginex P, Senerth E, Corremans M, Munn Z, Kredo T, Remon LP, Ngeh E, Kalman L, Alhabib S, Amer YS and Gagliardi A (2022) Trends in guideline implementation: An updated scoping review. Implementation Science 17(1), 1–17. 10.1186/S13012-022-01223-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK and Mandell DS (2017) Methods to improve the selection and tailoring of implementation strategies. Journal of Behavioral Health Services & Research 44(2), 177. 10.1007/S11414-015-9475-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Proctor E, Powell BJ and McMillen JC (2013) Implementation strategies: Recommendations for specifying and reporting. Implementation Science 8(1), 1–11. 10.1186/1748-5908-8-139. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. QSR International Pty Ltd (2018) NVivo qualitative data analysis software (No. 12). https://support.qsrinternational.com/nvivo/s/article/How-do-I-cite-QSR-software-in-my-work.
  31. R Core Team. (2022) R: A Language and Environment for Statistical Computing. (3.3.0). Vienna, Austria: R Foundation for Statistical Computing. [Google Scholar]
  32. Rahman A, Akhtar P, Hamdani SU, Atif N, Nazir H, Uddin I, Nisar A, Huma Z, Maselko J, Sikander S and Zafar S (2019) Using technology to scale-up training and supervision of community health workers in the psychosocial management of perinatal depression: A non-inferiority, randomized controlled trial. Global Mental Health 6, 1–5. 10.1017/gmh.2019.7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Smith JL, Ritchie MJ, Kim B, Miller CJ, Chinman M, Kelly PA, Landes S and Kirchner JE (2022) Getting to fidelity: Scoping review and expert panel process to identify core activities of implementation facilitation strategies. SocArXiv. 10.31235/OSF.IO/6XFVJ. [DOI] [PMC free article] [PubMed]
  34. Triplett NS, Johnson C, Kiche S, Dastrup K, Nguyen J, Daniels A, Mbwayo A, Amanya C, Munson S, Collins PY, Weiner BJ and Dorsey S (2023) Understanding lay counselor perspectives on Mobile phone supervision in Kenya: Qualitative study. JMIR Formative Research 7(1), e38822. 10.2196/38822. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Triplett NS, Munson S, Mbwayo A, Mutavi T, Weiner BJ, Collins P, Amanya C and Dorsey S (2021) Applying human-centered design to maximize acceptability, feasibility, and usability of mobile technology supervision in Kenya: A mixed methods pilot study protocol. Implementation Science Communications 2(1), 2. 10.1186/S43058-020-00102-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Triplett NS, Woodard GS, Johnson C, Nguyen JK, AlRasheed R, Song F, Stoddard S, Mugisha JC, Sievert K and Dorsey S (2022) Stakeholder engagement to inform evidence-based treatment implementation for children’s mental health: A scoping review. Implementation Science Communications 3(1), 1–16. 10.1186/S43058-022-00327-W. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. van Ginneken N, Tharyan P, Lewin S, Rao GN, Meera SM, Pian J, Chandrashekar S and Patel V (2013) Non-specialist health worker interventions for the care of mental, neurological and substance-abuse disorders in low- and middle-income countries. Cochrane Database of Systematic Reviews 11, CD009149. 10.1002/14651858.CD009149.pub2. [DOI] [PubMed] [Google Scholar]
  38. Waltz TJ, Powell BJ, Fernández ME, Abadie B and Damschroder LJ (2019) Choosing implementation strategies to address contextual barriers: Diversity in recommendations and future directions. Implementation Science 14(1), 42. 10.1186/S13012-019-0892-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, Boynton MH and Halko H (2017) Psychometric assessment of three newly developed implementation outcome measures. Implementation Science 12(1), 1–12. 10.1186/s13012-017-0635-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Weiss WM, Murray LK, Zangana GAS, Mahmooth Z, Kaysen D, Dorsey S, Lindgren K, Gross A, Murray SMI, Bass JK and Bolton P (2015) Community-based mental health treatments for survivors of torture and militant attacks in southern Iraq: A randomized control trial. BMC Psychiatry 15(1), 249. 10.1186/s12888-015-0622-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Weissgerber TL, Garovic VD, Savic M, Winham SJ and Milic NM (2016) From static to interactive: Transforming data visualization to improve transparency. PLoS Biology 14(6), e1002484. 10.1371/JOURNAL.PBIO.1002484. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Wyche SP and Murphy LL (2012) “Dead China-make” phones off the grid: Investigating and designing for mobile phone use in rural Africa. In Proceedings of the Designing Interactive Systems Conference, DIS ‘12, 186–195. 10.1145/2317956.2317985. [DOI] [Google Scholar]
  43. Wyche SP and Olson J (2018) Gender, mobile, and mobile internet| Kenyan women’s rural realities, mobile internet access, and “Africa rising”. Information Technologies & International Development 14, 15. [Google Scholar]
Glob Ment Health (Camb). doi: 10.1017/gmh.2023.23.pr1

Author comment: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya — R0/PR1

Noah Triplett 1

To Drs. Bass and Chibanda,

Editors-in-Chief

Cambridge Prisms: Global Mental Health

My co-authors and I are submitting the attached article titled, Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya, for your consideration for publication in Cambridge Prisms: Global Mental Health. Little scholarship has focused how to scale up and sustain task-shifting of evidence-based practices (EBPs) for mental health disorders in lower-resourced settings, including how to sustainably supervise lay counselors. We describe methods and results from an iterative, mixed methods trial that engaged lay counselors and supervisors providing trauma-focused cognitive behavioral therapy (TF-CBT) in Kenya to redesign supervision to most effectively leverage mobile technology.

We briefly outline the importance of developing scalable and sustainable approaches to supervision—an essential yet resource-intensive implementation strategy for ensuring fidelity to EBPs. The rise in global internet access, coupled with COVID, has highlighted opportunities to leverage technology to increase access to care; however, contextual variables, such as network connectivity and user preferences, must be examined and considered when designing and implementing mobile technology interventions and implementation strategies. We describe our approach, which engaged lay counselors and supervisors to design and test a method of optimizing mobile phone supervision. We honored their contextual expertise and engaged them throughout the research process to identify potential barriers and generate solutions to using mobile technology to provide supervision. Ultimately, we co-developed implementation guidelines that were shared and discussed with newly implementing counselors in a brief outreach visit. Results showed significant improvements in mobile phone supervision acceptability and usability. Qualitative results contextualize how the program was effective. Our findings and approach highlight the importance of co-developing flexible and pragmatic implementation strategies to support task shifting and global mental health approaches.

The manuscript is 4,917 words in length (excluding title pages, abstract, tables, and figures), 3 tables, 3 figures, and 2 supplemental files. This manuscript is not in submission to any other journals. All authors have contributed to and approved this submission. Thank you for your full consideration of this manuscript. I look forward to your review. If you have questions or would like further information, please feel free to contact me at (336) 596-8132 or by email at nst7@uw.edu.

Sincerely,

Noah S. Triplett, MS

PhD Candidate in Clinical Psychology

University of Washington

Glob Ment Health (Camb). doi: 10.1017/gmh.2023.23.pr2

Review: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya — R0/PR2

Reviewed by: Alexandra Klein1

Comments to Author: Manuscript Review

Title: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya

These authors present findings on the iterative development and implementation of guidelines focused on facilitating mobile supervision among lay providers administering Trauma-Focused Cognitive Behavioral Therapy (TF-CBT) in Kenya. The framework presented is both innovative and pragmatic and has the potential to optimize supervision-focused implementation in other contexts and with various technologies/modalities. Importantly, the guidelines were developed collaboratively, centering on community participatory principles, emphasizing the expertise of local collaborators. Additionally, the manuscript is well-written, and the methodology and design are very clear. In its current state, this manuscript makes a meaningful contribution. There are a few limitations and points of clarification below that if address may further improve the impact of these findings.

Major comments/revisions

Method- Implementation Guidelines

• It may be useful for the reader if the authors could expand a bit more on how the educational outreach sessions were conducted. Were these held in person with all supervisors and sequence seven counselors in attendance? Or were sessions conducted via one-on-one contact with supervisors being assigned to hold sessions with a certain number of counselors? Was there a gap between when these sessions were held and when phone supervision began (if this gap was meaningful, it might be helpful to report this)?

Method- Measures

• It seems plausible that the acceptability, feasibility, and usability constructs may have some conceptual overlap. Would the authors be able to provide correlations between these measures to confirm that these are divergent constructs?

Method- Analysis

• Would the authors consider adding some type of effect size for their independent-samples t-test? From the results/ visualization, it appears that the effect of the guidelines may vary by construct.

• For the qualitative analysis, it would be helpful if the authors included information about the number and qualifications of the coders. Further, it would be informative to know the extent to which the coders were involved in the study and whether/how they were trained on coding procedures.

Results

• The authors present the means/SDs of acceptability, feasibility, and usability across sequences. However, Figure 3 demonstrates these findings across both sequences and sector. This makes it somewhat difficult to visualize the findings collapsed across sectors. Would the authors be able to either collapse across sectors or present an additional figure that highlights the findings as they are presented in Table 3?

Discussion

• A few questions came to mind while reading this manuscript regarding the impact of the guidelines/ outreach sessions: 1) Did these contribute to an increase in number of telephone supervision contacts?; 2) Were there specific strategies that lay counselors/supervisors used more than others?; 3) Did guidelines improve the effectiveness of supervision conducted via phone?; and 4) Did the any of this related to fidelity with implementing the intervention/clinical outcomes. I realize that answering many of these questions is beyond the scope of this manuscript. If possible to address questions 1 and 2, I think those would be interesting to the reader and emphasize the utility of the guidelines/ sessions (above and beyond the utility that is already highlighted!). If you are unable to include these in the manuscript, I may consider a limitation/future direction is that it is unclear how these sessions impact strategies chosen and frequency of these sessions. Questions 3 and 4 may be worth mentioning as future directions if the authors have adequate space. If not, something to ponder on!

Minor comments/revisions

Introduction

• On page 5, the authors briefly mention noted challenges of disseminating guidelines. It may be helpful for the reader to briefly expand on what these challenges are (if the authors have sufficient space).

Method- Participants

• The authors note that there were no exclusion criteria. Was there any inclusion criteria to be a lay counselor for the parent study? If so, would they be able to add that in?

Figures

• Figure 3: The x-axis on these figures is difficult to read. The words of the categories listed on the x-axis are overlapping. It’s possible that this is not an issue for others and may be a result of the way the file downloaded on my computer, but I thought it was worth mentioning if the formatting was somehow not converted correctly during the submission process.

Glob Ment Health (Camb). doi: 10.1017/gmh.2023.23.pr3

Review: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya — R0/PR3

Reviewed by: Eugene Augusterfer

Comments to Author: This paper represents an important move forward in providing the use of mobile phone technology supervision of mental health field workers and ultimately better mental health care to those in need. The paper also addresses the need to provide mental health care to formerly underserved and difficult to reach populations in LMIC.

Glob Ment Health (Camb). doi: 10.1017/gmh.2023.23.pr4

Recommendation: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya — R0/PR4

Editor: Ikenna Ebuenyi1

Comments to Author: The study explores a very important subject on co-development of implementation guidelines on mobile phone supervision in Kenya. However, the reviewers have made very relevant observations and recommendations that may improve the quality and content of the manuscript. I invite the authors to respond to comments of the reviewer 1 and resubmit the manuscript.

Glob Ment Health (Camb). doi: 10.1017/gmh.2023.23.pr5

Decision: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya — R0/PR5

Editor: Dixon Chibanda1

No accompanying comment.

Glob Ment Health (Camb). doi: 10.1017/gmh.2023.23.pr6

Author comment: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya — R1/PR6

Noah Triplett 1

April 10, 2023

To Drs. Bass and Chibanda,

Editors-in-Chief

Cambridge Prisms: Global Mental Health

My co-authors and I are re-submitting the attached article titled, Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya, for your consideration for publication in Cambridge Prisms: Global Mental Health. Little scholarship has focused how to scale up and sustain task-shifting of evidence-based practices (EBPs) for mental health disorders in lower-resourced settings, including how to sustainably supervise lay counselors. We describe methods and results from an iterative, mixed methods trial that engaged lay counselors and supervisors providing trauma-focused cognitive behavioral therapy (TF-CBT) in Kenya to redesign supervision to most effectively leverage mobile technology.

We briefly outline the importance of developing scalable and sustainable approaches to supervision—an essential yet resource-intensive implementation strategy for ensuring fidelity to EBPs. The rise in global internet access, coupled with COVID, has highlighted opportunities to leverage technology to increase access to care; however, contextual variables, such as network connectivity and user preferences, must be examined and considered when designing and implementing mobile technology interventions and implementation strategies. We describe our approach, which engaged lay counselors and supervisors to design and test a method of optimizing mobile phone supervision. We honored their contextual expertise and engaged them throughout the research process to identify potential barriers and generate solutions to using mobile technology to provide supervision. Ultimately, we co-developed implementation guidelines that were shared and discussed with newly implementing counselors in a brief outreach visit. Results showed significant improvements in mobile phone supervision acceptability and usability. Qualitative results contextualize how the program was effective. Our findings and approach highlight the importance of co-developing flexible and pragmatic implementation strategies to support task shifting and global mental health approaches.

The manuscript is 4,991 words in length (excluding title pages, abstract, tables, and figures), 3 tables, 2 figures, and 3 supplemental files. This manuscript is not in submission to any other journals. All authors have contributed to and approved this submission. Thank you for your full consideration of this manuscript. I look forward to your review. If you have questions or would like further information, please feel free to contact me at (336) 596-8132 or by email at nst7@uw.edu.

Sincerely,

Noah S. Triplett, MS

Glob Ment Health (Camb). doi: 10.1017/gmh.2023.23.pr8

Review: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya — R1/PR8

Reviewed by: Eugene Augusterfer

Comments to Author: This is an excellent paper focused on the urgent need for clinical supervision in low-resource settings. The creative use of mobile phone technology to reach these often overlooked settings is a major movement to level the “playing field” in low-resource settings.

Glob Ment Health (Camb). doi: 10.1017/gmh.2023.23.pr7

Review: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya — R1/PR7

Reviewed by: Alexandra Klein1

Comments to Author: I appreciate the authors' response to my comments. They have addressed all of my concerns, and I’m happy to recommend this article for publication. Thank you for the opportunity to review this exciting and innovative work!

Glob Ment Health (Camb). doi: 10.1017/gmh.2023.23.pr9

Recommendation: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya — R1/PR9

Editor: Ikenna Ebuenyi1

Comments to Author: Thank you for taking into account the feedback provided by the reviewers and incorporating the necessary changes in the revised manuscript. We are pleased to inform you that your manuscript has been accepted for publication.

Glob Ment Health (Camb). doi: 10.1017/gmh.2023.23.pr10

Decision: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya — R1/PR10

Editor: Dixon Chibanda1

No accompanying comment.

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    For supplementary material accompanying this paper visit https://doi.org/10.1017/gmh.2023.23.

    S2054425123000237sup.zip (73.1KB, zip)

    click here to view supplementary material

    Data Availability Statement

    The datasets generated and analyzed for this study are available from the corresponding author upon reasonable request.

    Articles from Cambridge Prisms: Global Mental Health are provided here courtesy of Cambridge University Press

    RESOURCES