Adaptation of the trauma group intervention ‘Teaching Recovery Techniques’ for online delivery: A participatory design and usability study

Abstract

Background

Video-telehealth delivery of trauma-based care is promising and may help address structural and perceptual barriers to receiving support. However, existing evidence relies heavily on samples from adult populations. There is potential to transfer existing child and adolescent trauma interventions to a video-telehealth delivery format; but, this requires careful consideration. The aim of this project was to adapt a group-based intervention called Teaching Recovery Techniques for online delivery and investigate the usability of the new intervention format.

Methods

A qualitative needs assessment was performed (n = 3 intervention leaders, 4 youth), followed by participatory workshops and advisory panel consultation to generate adaptation recommendations. Usability testing was performed in two cycles; the first tested the adapted manual with intervention leaders (n = 5), and the second tested newly developed digital resources with youth (n = 5).

Results

The needs assessment uncovered a number of issues that, when generating recommendations, were distilled into three topics: safety, participation and learning. Recommendations included safety rules, an emergency response protocol, communication strategies, and guidance on group composition and intervention delivery. Usability testing indicated acceptability but highlighted the need for more detailed and explicit guidance, particularly on safety processes.

Discussion

The present study demonstrates the potential for delivery format to affect intervention feasibility and acceptability, and provides recommendations that can be used to guide the transfer of other group-based mental health interventions to an online format. The young people, parents and professionals involved in the project provided rich and varied perspectives, which illustrated the value of broad stakeholder engagement.

Highlights

• •Transferal of trauma group support to Internet-based delivery requires adaptations.
• •Safety, participation and learning should be given particular attention.
• •Findings endorsed broad stakeholder engagement via participatory research.

1. Introduction

The development of affordable video-conferencing technologies has resulted in a surge of video-telehealth delivery of mental health interventions. Such technological approaches to delivery of trauma-based care may help address structural and perceptual barriers, such as geographic locality of services and social stigma of being seen attending clinical settings, by offering access to interventions at home and preserving privacy (Weiss and Marsac, 2019). For post-traumatic stress disorder (PTSD) in particular, because it is unlike other mental health conditions as a clear cause is known, those who have been exposed to a trauma (e.g., armed conflict, sexual assault, major accident) could be provided scalable secondary prevention through video-conferencing technology (Kuhn and Owen, 2020). Randomised controlled trials of PTSD intervention delivered by video-telehealth demonstrate it is as effective as when delivered in-person (Kuhn and Owen, 2020). This appears to be the case for both individual and group treatment. Moreover, the benefits on co-morbid symptoms like depression appear to transfer to video-telehealth delivery. However, the evidence relies on findings from (mostly male military/veteran) adult samples. There are a number of in-person interventions available to children and adolescents (Rafieifar and Macgowan, 2021); but, transferring them to an online-delivery format requires careful consideration.

One example of an in-person group intervention is Teaching Recovery Techniques (TRT) (Yule et al., 2013). TRT is predominantly based on trauma-focused cognitive behavioural therapy (CBT). Children are introduced to various techniques to manage PTSD symptoms related to intrusion, arousal and avoidance. Examples of techniques to address intrusion include a visual imagery technique to allow children to create a safe place in their imagination, imagery techniques to control intrusive images, and a dual attention task to change the way traumatic memories are stored in the brain. Arousal techniques include muscle relaxation to help children to relax when feeling anxious and physiologically aroused, a breath control technique to counter over-breathing that can occur when anxious, and positive self-talk whereby children learn to substitute anxiety-provoking thoughts with more adaptive, positive thoughts. Avoidance techniques include graded exposure whereby children make a plan to gradually face a traumatic reminder they are avoiding, and drawing of traumatic memories. Both the in-person format and the adapted online format reported in this paper comprise 7 sessions, including a get-to-know-you meeting and one follow-up. The recommended number of participants is up to 15 for in-person TRT groups, which has been amended to 4–7 for online TRT groups. Guidelines for the length of the sessions are 1.5–2 h for in-person groups and 1–1.5 h for online groups. In-person TRT has been evaluated in several studies across war-affected areas, including Palestine (Barron et al., 2013; Barron et al., 2016) and Gaza (Punamäki et al., 2014), as well as countries hosting refugee children, including Australia (Ooi, 2013), Norway (Oppedal et al., 2019) and Sweden (Sarkadi et al., 2018). Positive effects on mental health have been reported (Barron et al., 2013; Barron et al., 2016; Ooi, 2013; Oppedal et al., 2019; Sarkadi et al., 2018), and qualitative research indicates the intervention is well accepted (Ooi, 2013; Sarkadi et al., 2018).

Uncovering usability issues at an early stage before implementing a new intervention can inform re-design that enhances the quality of the intervention, thereby saving costs and resources and increasing the benefits for the end-user (Anderson et al., 2012). Thus, usability testing has an important role to play in the development of health care services (Anderson et al., 2012) and, arguably, in the case of service adaptation. Both professionals and the public/patients “using” the service, or device, should be included in testing in order to uncover relevant usability issues (Anderson et al., 2012). Adopting a participatory and iterative approach to usability testing increases the possibilities to uncover usability issues and designing an intervention that is usable, useful and enjoyable (Lin et al., 2010).

1.1. Objective

In this paper, we present an example of an intervention adaptation project. The project took place within the context of the COVID-19 pandemic. Given the associated societal restrictions, it was decided in March 2020 that ongoing TRT groups in Sweden would move to online delivery using video-conferencing technology. The aims of the project were to: (i) adapt TRT for online delivery and (ii) investigate the usability of the new intervention format.

2. Method

The process consisted of three steps (Fig. 1): (1) needs assessment; (2) adaptation, consisting of participatory workshops and advisory panel consultation; and (3) usability testing of the online manual and resources with TRT facilitators and youth. Key aspects within each step are described below. The needs assessment took place August to September 2020, workshops and consultations August 2020 to February 2021, and usability testing February to April 2021. Participants were unique to each phase of the study, i.e. no participant took part in more than one phase.

Fig. 1.

Overview of process.

2.1. Needs assessment phase

Three TRT facilitators and four young males between 18 and 20 years were interviewed about their experiences of participating in in-person TRT groups that swiftly moved to online delivery due to the COVID-19 pandemic. The young people all had experience of migrating to Sweden, from Afghanistan, Syria and Eritrea. The semi-structured interview guide (see Supplementary material) directed them to reflect on attendance, safety, and perceived effectiveness of intervention components, to identify areas that service providers/users perceived as important for online TRT delivery. The first author (APA), who conducted the interviews, listened to the recordings repeatedly to identify key insights following an analysis approach of direct coding from audiotapes described by Gravois et al. (1992). The themes derived from this process were taken forward into the adaptation phase described below, as a supplement to the workshop methodology.

2.2. Adaptation phase

The adaptation phase consisted of workshops and feedback on emerging ideas from a panel of professionals and parents. A workshop team was formed of three youths (i.e. service users), one non-researcher TRT facilitator (i.e. service provider) and two TRT-trained researchers (SGL and APA). The youths all had experience of migrating to Sweden, from Syria, Afghanistan and Somalia. They were selected from former TRT participants, two of whom received partial online delivery due to COVID-19. Selection was informed by demographic characteristics, such as gender and arrival status (unaccompanied or accompanied), to promote a variety of perspectives. The non-researcher TRT facilitator was chosen based on his experiences of having co-held TRT groups that were moved online due to COVID-19. In line with INVOLVE guidelines (invo.org.uk), the youths and the facilitator were reimbursed for their work.

An initial ‘planning workshop’ was held, in which the team decided upon the format and structure of the process. The Design and Evaluation of Digital Health Interventions (DEDHI) framework (Kowatsch et al., 2019) was selected to support the online adaptation process by providing a ‘topic guide’ for the sessions. Developed for researchers and practitioners alike, the framework outlines evaluation criteria and implementation barriers to be considered at each phase of a typical digital health intervention life cycle. Ahead of introducing the framework components (adherence, personalisation; perceived benefit; content quality; ethics; service quality; safety; privacy and security; and accountability) as the topic guide, the workshop team were given an opportunity to generate their own ideas on important aspects to be considered in the process. The second workshop focused on online delivery idea generation, which was followed by professional and parent review. The final workshop focused on consolidation, bringing together the various opinions and recommendations. Throughout the process, active consideration of the key insights from the needs assessment interviews was taken, in order to bring the views expressed by those additional participants/facilitators into the workshops.

An international professional panel was formed of: TRT developers (UK); child trauma psychiatrist (USA) and psychologist (UK); and personnel from a children's rights organisation experienced in delivering TRT and other online youth interventions (Sweden). A parallel parent panel was engaged of three mothers and one father, with experience of seeking refuge but unrelated to the interview/workshop participants. The parent panel had been advising on TRT research for two years prior to this project. Online delivery ideas generated at the second workshop were shared in written format with the professionals and via online meetings with the parents. The panel members were asked to draw upon their professional and personal experiences to consider the feasibility and appropriateness of the online adaptation ideas, including alignment with the original TRT logic model, participant safety, and practicalities. The parents were also asked to consider the adaptation of the caregiver modules within TRT.

2.3. Usability phase

The usability testing consisted of two cycles (Fig. 1). Usability testing sessions were held using a videoconferencing system and recorded as digital videos. A purposive sample of TRT facilitators (i.e. service providers) (n = 5; 1 male, 4 female; 3 school counsellors, 1 trainee social worker, 1 psychologist/researcher) and young people (i.e. service users) (n = 5; 15–22 years; 3 male, 2 female; first languages Somali (1), Arabic (2), Dari (2)) were recruited. Five participants per target group have been shown to be sufficient for usability testing and can identify 80 % of usability issues (Lewis, 1994; Virzi, 1992). Selection of the participants was informed by demographic characteristics such as age, gender, arrival status (unaccompanied or accompanied), profession and experience of delivering TRT to different ages. Experimental research indicates that gender is important when usability testing with children, with girls tending to report more problems than boys (Baauw and Markopoulous, 2004). The youth were selected among former in-person TRT participants known to the research group.

Cycle 1 testing focused on service providers, i.e. TRT facilitators, who were asked to read the online TRT manual and instructed to think-aloud to verbalise their thoughts during reading. A usability observer was present during the sessions but did not guide the participants or provide answers to their questions on the manual once the task had begun. Cycle 2 focused on service users i.e. youth with experience of trauma and of seeking refuge. A member of the research group facilitated individual usability sessions with service users, which focused on online TRT resources. The participants were instructed to think-aloud as they were demonstrated prototypes of online TRT resources: visual aids for imagery techniques, techniques for gradually facing a traumatic reminder, instructional movies for relaxation techniques, and audio files for a positive visual imagery task called “A safe inner place”.

The usability testing analysis procedure described by Hertzum (2020), in which each identified usability issue is given a severity rating and proposed solution(s) are generated, was followed. Two TRT-trained researchers (Cycle 1: APA & EL; Cycle 2: MT & AT) watched all the video recordings and analysed both verbalisations and behaviour to identify usability issues. Independently, both created a session-by-session list of the identified usability issues (both positive and negative). The research group then discussed their usability findings and merged the findings into a final usability issue list. Based on the list, the researchers agreed on modifications of the online TRT-manual. Some modifications were suggested directly by the users during the think-aloud sessions.

2.4. Assessment of participatory activities

The participatory activities in the project were evaluated using behavioural observation and a questionnaire. Two researchers (EL, AT) attended the workshops and the parent panel meeting to passively observe and rate the interactions in the group using the Active Involvement of Users in Research Observation Schedule (Warner et al., 2019), a semi-structured observation protocol developed to objectively assess aspects of group dynamics in the context of research meetings involving public contributors. The workshop team members and parent panel members were also asked to independently and anonymously respond to the Active Involvement of Users in Research Questionnaire (Warner et al., 2019), containing eleven Likert-rated items that correspond to those on the observation schedule.

In the first assessment domain ‘interpersonal relations between researchers and advisors’, the observations, as well researcher and contributor responses to the questionnaire, indicated positive social relations, including researchers viewing contributors as experts and the contributors showing engagement in the meetings. Although, some language difficulties were identified, which were more present in online meetings. Regarding ‘nature of advisor contributions’, the contributors were both invited into the discussion and actively took the initiative to speak. On some occasions, the researchers perceived that the contributors gave less relevant feedback, but mostly their input was seen as valuable to the project. Concerning ‘how advisors guide research development’, there were mixed reports on whether the contributors challenged the research ideas, but otherwise both observations, as well as researcher and contributor reports, showed that contributors were active in guiding the research development.

2.5. Ethical considerations

According to Swedish legislation, the research activity conducted with TRT facilitators is exempt from the law on research ethics since they participated in a professional capacity. The interviews with youth exploring the online delivery of the standard TRT manual and the usability testing with youth were approved by the Swedish Ethical Review Authority (Ref. 2020-03126; Ref. 2020-06693). Participants received information about the research and informed consent was documented. For the adaptation phase, both the researchers and the public contributors occupied the role of experts, not study participants. Imposing a consent procedure on one party, but not the other disrupts the power balance and can also cause potential damage, where procedures relating to standard ethics requirement can be perceived as oppressive rather than protective of integrity (Liabo et al., 2018). Thus, a consent process was not performed for this part of the work. However, the observation of the workshops to assess the involvement of public contributors was assessed and approved by the Swedish Ethical Review Authority (Ref. 2020-03911) and a consent process was performed ahead of observations.

3. Results

3.1. Needs assessment phase

Non-verbal communication was perceived to have been more challenging online, due to some participants with no or limited camera display, or Wi-Fi issues. This had consequences for both learning, safety and relationship building. Facilitators said it had made it more difficult for them to assess how the participants were feeling. The young people described having felt less comfortable to share, as the lack of non-verbal communication created feelings of unease and loneliness, enhanced by the lack of attention when other participants appeared to be preoccupied during the sessions. However, the youth did highlight that some young people might experience lower barriers to share in an online group – this was regarded to depend on personality. Both facilitators and young people said there was less ‘small talk’ in the online sessions. Some facilitators saw advantages in this, as they thought it could increase efficiency and maintain the focus on practicing techniques. However, the young people considered becoming ‘part of a group’ a valuable feature of TRT and had missed the opportunities to talk.

Learning was regarded to have been negatively affected by online delivery, due to decreased opportunities for feedback, restricted views of each other, and increased risk of being distracted e.g. by cellphones or a busy environment. The youth emphasised that perceived learning was important for attendance, and the negative impact online delivery had on learning and socialising could increase dropout. On the other hand, the increased accessibility provided by online delivery was seen as an advantage, in particular for young people who had long-distance commuting or were both attending school and working. The facilitators described that some young people, who were feeling very stressed and exhausted and might not have had the energy to attend in-person groups, had chosen to attend online sessions albeit with their camera off. This was considered good for those who would otherwise not attend, but it conflicts the young people's expression of discomfort when other participants appear to be distracted. Both the young people and the facilitators pointed out that digital literacy could affect attendance; some of the participants had very little digital experience and their facilitators had to give time to the process of joining the online platform. In general, platforms that worked well on mobile phones were perceived as more accessible than platforms that needed to be accessed via computers.

Sense of privacy could both improve and decrease by online delivery. If a participant joined the online groups from a public space the young people were concerned that others might overhear the sessions. However, some privacy issues related to onsite groups were said to improve with online delivery, e.g. the risk of being seen by other students when attending groups delivered at school locations.

3.2. Adaptation phase

Fig. 2 gives an overview of the final set of adaptation recommendations. The workshops focused on three prominent issues: safety, participation and learning. As illustrated by the figure, certain recommendations were considered to respond to more than one of the main issues, e.g. taking regular short breaks could reduce the risk of dropout as well as promote learning. One of the main adaptations the team agreed upon was to add a section to the manual that focused on building relationships online and how to handle safety aspects such as what to do if a participant has an adverse reaction during a session. The professionals and parents on the panel were largely in agreement with the workshop recommendations; however, there were points of disagreement. For example, workshop ideas around intervention personalisation, which largely came from the youth perspective, were somewhat challenged. Inclusion of various calming images to choose from in the starting kit and co-scheduling of sessions with participants were agreed upon. However, suggestions for a flexible number of sessions and the option for participants to stay online after sessions were challenged, with the professional panel conveying the importance of consistency in manualised interventions.

Fig. 2.

Online adaptation recommendations.

3.3. Usability phase

A summary of the identified usability issues is presented in Table 1, together with the related modifications. TRT facilitators found it valuable that the manual contained a section on building relationships online. However, they expressed uncertainty on how to use this guidance in practice. Similarly, they agreed that safety aspects are important for online TRT, yet showed confusion and repeatedly asked for clarifications and more detailed guidance on the safety routines. These important usability issues were addressed by adding more details and explanations of the safety procedures in the manual as well as more specific guidance and practical advice on facilitating positive group dynamics online. The testing uncovered uncertainties among the facilitators about how to identify if children are suitable to participate in online support groups or not, and how to refer them on if needed. After discussion with the panel of professionals, this was addressed by adding content to the manual based on the family centred approach, whereby participants are provided with information to empower them to make their own decision about participating. This included a list of ‘pros’ and ‘cons’ of taking part and a pyramid image showing the different levels of care and where online-delivered TRT was located within the care structure. The TRT facilitators approved of sending a starting kit, but expressed hesitation regarding the suggestion to send exercises and diplomas beforehand as this would go against stepwise introduction of intervention content. This led to a suggestion to send exercises in sealed envelopes, one for each session; however, the panel of professionals and refugee parents raised concerns about whether children could be expected to wait with opening envelopes. Instead, it was decided that the exercises would be shared electronically between the sessions.

Table 1.

Usability issues and corresponding modifications.

Usability issues	Modifications to manual and resources
Cycle 1
• The main manual lacked details from and information about the safety protocol and the checklists	• More information about the safety routines was added, including how to handle common scenarios
• Adding a section on “online relationships” to the manual was valuable, but the users missed specific guidance and practical advice e.g. on how to facilitate discussions and how to use non-verbal communication online	• The section on online relationships was revised to include more clarifications and practical recommendations, including putting some of the recommendations as point-by-point advice
• There were uncertainties about how to identify which children are suitable to participate in online support groups and how to refer them onto other services if needed	• A section based on the family-centred approach, whereby participants are provided with information to empower them to make their own decision whether to participate or not, was added. This included a list of ‘pros’ and ‘cons’ of taking part and a pyramid image showing the different levels of care and where online-delivered TRT was located within the care structure
• The language was at times difficult to read and some terminology was confusing	• A language and readability check was performed. Words that had caused confusion were changed
• The layout was overall appealing but users were sometimes unsure about what the different boxes meant and in which order they should be read	• Instructions on how to use the manual were added. The use of boxes with practical tips were increased, since these were identified as helpful in the usability test
• The manual lacked instructions on how to work with interpreters online	• Guidance on working with interpreters online was added
• Holding individual meetings before the group sessions started was regarded as a good suggestion, but users were not sure when these meetings should be held	• The instructions on when to hold individual meetings were clarified
• Some facilitators might need more technical guidance on videoconferencing platforms/software	• Technical guidance was added on a separate page, so it could be easily skipped if not required. A section with guidance on how to choose a suitable platform and what to consider was included, based on existing guidelines for telehealth
• Sending a starting kit was expressed to be a good idea, but there were hesitations regarded some of the content, e.g. sending exercises and/or a diploma beforehand, as this would go against the idea of stepwise introduction of intervention content	• The content of the starting kit was revised and session-specific resources shared stepwise between sessions
• It was not clear to the users when to use the various online games suggested, and how to choose games for particular moments or groups	• A brief introduction was added, explaining when to use games and how to choose which games (e.g. depending on content of session and different groups). The layout was changed to “post-it notes”, with some left blank to enable facilitators to add their own game ideas
• The manual lacked recognition throughout that the content of the sessions needed to be adapted for different groups, depending on e.g. type of traumatic experiences and ages	• It was clarified throughout the manual that the content needs to be adapted to the group (age/type of experiences/cultural background), including which examples to use and which games to choose
Cycle 2
• It was not clear to users how and where to send the scoring of their wellbeing prior to each session	• Instructions were added to the manual on how to administer the wellbeing check. This involved the facilitator sending a message to the participants (by SMS, messaging app, or email), with a picture of the Cantril Ladder and a reminder of how to rate, sometime between 1 day to 1 h before. They were advised it can be helpful to agree upon a specific time with participants. The participants are asked to respond directly to the message with the number that corresponds to their rating
• Due to low volume and high speed, users sometimes found it challenging to follow the instructional audio and video recordings intended for use in between group sessions	• The video and audio files were re-recorded and/or edited, with improvements made in terms of volume and speed
• For the instructional videos intended for use in between group sessions, users called for clearer demonstrations of the techniques, e.g. breathing from the stomach	• The instructional videos were re-recorded with a young instructor.
	• Clearer instructions and demonstrations of the techniques were added to the manual, along with a prompt to use the videos in between sessions
• Users experienced technological disruptions, e.g. glitching in audio and video recordings and users not being able to see the full content of the screen when the facilitator used screen sharing	• Technical guidance for facilitators was added to the manual
• Interpreters were not able to log in, adjust camera and/or microphone and repeatedly asked for clarification of the techniques, which resulted in delays	• Guidance on working with interpreters online was added to the manual
• Users were disturbed by background noise	• It was clarified in letters to the participants' adult contact person, as well as to the participating children and young people, that participants should consider a place where they can sit without being disturbed before the session begins

Although the layout of the online manual appeared to be appealing, a need was detected to add instructions on how to use the manual. A page with guidance ‘How to use this manual’ was added, for example, explanation that text sections presented in speech bubbles throughout the manual are suggestions of phrasings to use with the participants in an online TRT group. At times, the language was difficult to read, and certain terminology seemed to confuse the facilitators. In light of this, the language in the manual was revised and large parts rewritten to be more approachable, more inclusive and less academic. For example, the section on identifying which children might not be suited for online delivery originally contained the term “screen out”. This gave the group leaders the impression they should follow a specific screening protocol. As a result, the section was changed to advice on how to help children and their parents make the decision about participation in online groups, which included a visual representation of the levels of mental health care. A further example is technical terminology regarding videoconferencing tools, which was amended to more accessible terms.

Usability issues related to technical issues emerged in Cycle 2, for example that the instructional video- and audio recordings did not work well with low-speed or unstable Internet. The participants also experienced issues with the sound in the recordings, as well as issues with document-based resources when the facilitator was not using a PDF-reader in full-screen mode. A page with technical advice was added to the manual. Further, the instructions for some of the document-based resources, such as an aid for daily activity planning, seemed unclear to the participants who expressed uncertainty of what they were supposed to do. For the resources in question, clear instructions were added to the actual sheets.

For the instructional video- and audio recordings, intended for use at home in between group sessions, a need for greater clarity was detected. This was especially relevant for imagery techniques where the participants are asked to imagine a picture on a TV-screen and on the palm of their hand. A sketch of a TV and a sketch of a hand, with a picture inside each, were added to the video to clarify. Similarly, the instructional video for breathing techniques raised some confusion among the participants. In the original video, the person demonstrating how to breathe using the belly, rather than the chest, was facing the camera. The video was re-made, with the instructor first facing the camera, then changing position and showing the technique in profile for clarity. Moreover, a young person was chosen as the instructor in the new videos, instead of an adult, following a recommendation by the youth in the workshop team. For some of the resources where participants seemed unsure of the instructions, additional prompts for the TRT facilitators were added to the manual.

4. Discussion

The intervention adaptation process described in this paper illustrates the need to consider how shifting the delivery of a group-based mental health support from in-person to online may require alterations to the manual guidance and the intervention itself. A number of pragmatic changes relating to safety, participation and learning were generated. This demonstrates the potential for delivery format to affect intervention feasibility and acceptability. Although, it should be noted that the original manual did not undergo usability testing, and so some issues could be remnant from the first iteration of the intervention design and description. It is not surprising that safety was a prominent theme in the adaptation process, as concerns regarding the appropriateness of video-telehealth interventions for people experiencing mental health difficulties have been raised before (Fletcher et al., 2018) including, for example, how to access emergency services to the participants' locations if needed. However, reviews of the literature indicate it is a safe and effective option for increasing access to mental health care (Fletcher et al., 2018). Looking to the extant literature on group-based telehealth treatments, the emphasis on participation is well-founded as difficulties developing a connection to the facilitator and other group members have been reported (Batastini and Morgan, 2016; Morland et al., 2010; Zhou et al., 2016). Similarly, in relation to the learning theme, education research has previously highlighted some challenges with using web-based video conferencing systems including technical glitches, students often unintentionally interrupting each other and teaching techniques requiring constant modification (Al-Samarraie, 2019).

When working with manualised interventions it is important to keep the original logic model in mind. Intervention fidelity, i.e. delivering the intervention as intended, has been shown to affect outcomes (Allen et al., 2012). If adaptations stray too far from the original manual content there is a risk the mechanisms of change are affected. It is for this reason the original developers and professionals experienced in delivering the intervention in-person were involved in the adaptation of TRT. Intervention ‘dose’, i.e. the amount and duration of sessions, is one aspect of fidelity that was discussed in the adaptation process for which youth and professionals had differing opinions. A further point of contention between stakeholders was the high value placed on the social nature of the group by the youth, whom put forward suggestions to not to reduce the group size too much and for participants to be able to stay online after sessions. This was not surprising as previous qualitative evaluation of TRT in Sweden uncovered the importance of the social aspect of the intervention to young people (Sarkadi et al., 2018). Yet, the professional stance point that a lower number of participants would aid efficiency and session closure was important to set clear boundaries regarding responsibility of facilitators counteracted the youth perspective. Both viewpoints are valid and speak to the importance of broad stakeholder engagement when conducting participatory research, to enable all relevant voices to be heard. Overall, the assessment of participatory activities showed that the researchers and public contributors had different input and were not always in agreement, which are signs of meaningful participatory activities with variation in stakeholder contributions. Additionally, the assessment showed that the activities were experienced in a positive way by both researchers and public contributors, and in most regards align with guidance for public involvement (Kaisler and Missbach, 2019).

4.1. Strengths and limitations

The thorough usability testing performed in this study can be considered a strength. Each of the usability sessions were independently analysed by two TRT-trained researchers (Cycle 1: APA & EL; Cycle 2: MT & AT). Analysis was first performed session-by-session, then across sessions to allow for more complete identification of usability issues (Hertzum, 2020). The participants in Cycle 1 were quite homogenous; however, this reflects the broader population of TRT facilitators in Sweden. The Cycle 1 analysts (APA & EL) shared similar demographics to the participants, which is believed to support the identification of ‘true’ usability issues (Hertzum, 2020). The study would have been further strengthened by extending youth involvement beyond workshop participation to involvement in Cycle 2 usability analysis; however, this was not possible due to capacity issues, as the youth were attending school and about to become involved in a concurrent project. A limitation was the use of interpreters during the Cycle 2 sessions. Whilst this enabled participation from the target population, whose first language was not spoken by the research team, as well as highlighted some usability issues when working with interpreters online, the use of interpreters in research can affect validity (Kapborg and Berterö, 2002). The findings from the assessment of participatory activities were fed back to the team after the online adaptation process was concluded, making it a summative rather than formative evaluation. For future use of the evaluation method, continuously feeding back the findings to the research team, including contributors, has the potential to lead to improved participatory activities.

4.2. Implications and future research

A practical output from the current study is the production of the online TRT manual. The resultant recommendations for online adaptation (Fig. 1) can be a valuable resource to guide the transfer of other group-based mental health interventions to an online format. A number of the practical solutions generated during the usability testing could also benefit in-person delivery of TRT. For instance, applying a family-centred approach to identifying participants i.e. providing them with the knowledge to decide for themselves whether they would benefit from taking part, could be used when forming in-person groups. This aligns with the extant literature, which posits family-centred care as increasingly important given the rise in mental health services being delivered in the community (MacKean et al., 2012). Further to this, the digital resources developed for online delivery could be shared among participants of in-person groups. On their own initiative, the youth involved in the workshops drafted a set of recommendations for involving youth, which covered aspects such as attitude, preparation, physical environment, and the size of the advisory group. These recommendations will be published in full in a concurrent paper focused on the participatory processes in and impact on the study.

With regard to future research, it will be important to explore youth perceptions of receiving TRT online and whether an impact on mental health outcomes can be achieved via the online format. At the time of submitting this manuscript, a pilot trial of online TRT was underway. A further research direction, suggested by youth in the workshops, is to explore the utility of app-based support. This idea prompted the research team to raise funds to initiate this project, and a co-design process is currently underway to develop a mobile application based on TRT.

5. Conclusions

With growing evidence for video-telehealth interventions to support the mental health of children and adolescents, existing manualised interventions initially intended for in-person delivery can be considered for online delivery. However, certain adaptations may be required. Specifically, attention should be given to safety processes and how to promote participation and learning when utilising video-telehealth delivery. Recommendations should be as specific as possible and presented in a clear and accessible way. Whilst involvement of young people is advocated, professionals familiar with the intervention should also be involved to ensure the adaptations do not stray too far from the original intervention logic.

The following is the supplementary data related to this article.

Supplementary material

. Online delivery of standard TRT interview guide

CRediT authorship contribution statement

APA conducted the qualitative interviews, co-led the workshops, collected feedback from the panel of professional and parents, performed part of the usability testing, analysed usability findings and wrote sections of the manuscript. MT performed part of the usability testing, analysed usability findings and wrote sections of the manuscript. EL observed the workshops and performed the assessment of participatory activities, analysed usability findings and wrote sections of the manuscript. SGL led the workshops. AT observed the workshops, performed part of the usability testing and analysed usability findings. NT, KI and RA participated in the workshops and writing sections of the manuscript. GW conceived the project idea, supervised all project activity, collected feedback from the panel of professionals, discussed usability analysis findings and wrote parts of the manuscript. All authors reviewed the manuscript.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.