A qualitative evaluation of Pesky gNATs in primary care – The experiences of assistant psychologists providing computer-assisted CBT to children experiencing low mood and anxiety

Abstract

Introduction

Technology-assisted cognitive behavioural therapy (CBT) is recognized as an evidence-based and cost-effective way to address psychological difficulties in children. Increasingly, these interventions are provided by staff with different levels of psychological training, such as assistant psychologists (APs). However, there is limited qualitative understanding regarding their experiences of providing technology-assisted CBT.

Method

Semi-structured interviews were conducted with APs (n = 14) in primary care settings in Ireland. This qualitative evaluation is part of an ongoing RCT for the CBT game Pesky gNATs (O'Reilly and Coyle, 2015). All data were inductively analysed using qualitative thematic analysis by Braun and Clarke (2006).

Results

Five overarching themes were identified: 1) positive experiences, 2) integrating Pesky gNATs with traditional CBT, 3) managing parental ‘buy-in’, 4) implementation complexities, and 5) future perspectives. Two subthemes were reported for each overarching theme: benefits for children, AP professional growth, facilitating engagement, different child characteristics, unclear role for parents, child autonomy, managing waitlists, external factors, design considerations and advice to future APs.

Conclusions

Pesky gNATs is well-received by APs in primary care, and is mostly experienced as helpful by both APs and children. However, a number of factors may be impacting the ability of APs to effectively provide the intervention.

Highlights

• •Recent evidence supports the use of technology-assisted cognitive behavioural therapy for children with low mood or anxiety.
• •Few studies have examined the experiences of different types of staff using these interventions for the first time.
• •This is the first qualitative study to analyse the experiences of assistant psychologists using a CBT game in primary care.
• •Key themes were: positive experiences, integrating technology with CBT, managing parents, implementation, and future outlook.
• •Participants emphasised the benefits of a CBT game, and provided insight into how to maximise implementation effectiveness.

1. Introduction

1.1. Background

Globally, there are insufficient primary care psychology resources to meet the needs brought about by the growing prevalence of low mood and anxiety in children (McGorry et al., 2013). One of the most effective interventions for such psychological difficulties is cognitive behavioural therapy (CBT) (Ebert et al., 2015). CBT examines the often-complex relationship between one's thoughts, feelings, and behaviours that are theorized to be related to maintaining psychological difficulties (Beck, 1963). Increasingly, CBT has been adapted using a range of technology-assisted formats. These include: internet-delivered or computerised CBT (cCBT) (Spek et al., 2007; Gilbody et al., 2015), serious games (Fleming et al., 2016, Fleming et al., 2017); and more recently, blended CBT (Titzler et al., 2018). Technology-assisted CBT has been recognized as both an evidence-based and cost- efficient approach to address the unmet needs across different settings for children (Pennant et al., 2015; Stasiak et al., 2016).

1.2. The challenges of applying technology-assisted CBT in real-world settings for children

Despite the supporting evidence base and cost- effectiveness of technology-assisted CBT, significant challenges have emerged. Firstly, there are considerable research-to-practice gaps such that evidence-based interventions often struggle to achieve effectiveness when transferred from clinical trials to service settings (Grimshaw et al., 2012). Secondly, high attrition rates are reported across the literature (Vigerland et al., 2016) – for reasons that are poorly understood. Thirdly, CBT requires careful adaptation for children due to different developmental needs (Carr, 2008); and without effective tailoring, CBT may be inappropriate for some children (Stallard, 2019; Grave and Blissett, 2004). Indeed, while the body of evidence for technology-assisted CBT is strong for many populations (Hofmann et al., 2012), the research on its effectiveness for children has less weight due to lower numbers of studies (Arnberg et al., 2014; Fleming et al., 2014). Fourthly, to achieve effective implementation, technology-assisted interventions require complex collaborations between psychologists, politicians, industry-based technologists, and service provider management – these cross-disciplinary collaborations have often produced sub-standard intervention outcomes (Blandford et al., 2018). A final challenge relates to a lack of understanding regarding the potentially.

novel factors that may impact children's (digital) therapeutic alliance (Henson et al., 2019; Torous and Hsin, 2018; Knowles et al., 2014) – how, and to what extent, do new technologies help or hinder the therapeutic process?

To address the above challenges, a variety of both interdisciplinary and adaptive methods have been proposed to ensure interventions are implementable (Mohr et al., 2017). This includes understanding the journey of clinical stakeholders (Mohr et al., 2015); and a recognition of the role of the deployment environment of interventions (Ben-Zeev et al., 2015; van der Meulen et al., 2019). Additionally, alongside quantitative outcomes, the use of qualitative research is recognized as an integral part of understanding how an intervention works and why (Noyes et al., 2018). When considering technology-assisted CBT for children, it is important to factor in the unique characteristics of this intervention. At present, a variety of approaches are commonly utilized for the successful adaptation of CBT for children, including: computer games (Linehan et al., 2015), and the use of child-friendly narratives, metaphors and characters therein (O'Reilly, 2018; Coyle et al., 2011). Moreover, the use of staff with differing levels of psychology qualifications – namely graduate students, assistant psychologists (APs), or primary care mental health workers – is increasingly common in the provision of low-intensity CBT interventions (Ekers et al., 2011; Bower, 2002).

1.3. Prior research on technology-assisted CBT for children – the role of assistant psychologists

Despite the abundance of quantitative research on technology-assisted CBT, the field is lacking in comparatively high-quality qualitative evidence concerning effectiveness and implementation. This is particularly the case for clinical stakeholders in primary care services, where there is limited qualitative understanding on the experiences of staff providing such interventions. Due to unmet needs in primary care, stepped models of care continue to explore the provision of cost-effective interventions using staff with various levels of psychology qualifications (Kakuma et al., 2011; Kohn et al., 2004). This approach proposes that staff without professional psychology qualifications can be efficiently trained in structured CBT programs, for which there is some evidence of effectiveness (Richards et al., 2016; Mead et al., 2005). Nonetheless, variable definitions as to what constitutes this type of staff (such as APs) have limited the generalizability of research heretofore. Additionally, despite the high job satisfaction and competency development reported by APs, there are potential risks for systemic exploitation and elitism within a competitive field (Byrne and Twomey, 2011).

As more service providers turn to APs to provide technology-assisted CBT, concerns have been highlighted regarding the aforementioned risks; but also the proliferation of unpaid positions leading to financial hardship in some settings (such as Ireland, Hughes et al., 2015). Notably, this is occurring in already resource-depleted environments; where some staff believe that face-to- face therapy is superior to technology-assisted CBT (Stallard et al., 2010; Perle et al., 2013; Vigerland et al., 2014). Moreover, it has been found that there is often a lack of staff knowledge about technology- assisted interventions and their underpinning research (Donovan et al., 2015; Du et al., 2013).

However, within the available qualitative research, much of the focus has been on service users. A recent qualitative synthesis of children's experiences of technology-assisted CBT found that it: was mostly helpful, assisted the therapeutic process, transferred into everyday life, provided a gameplay experience; but had some limitations for a minority (McCashin et al., 2019). Of the perceived advantages and disadvantages of blending technology with CBT from the staff perspective, there have been mixed results reported. In relation to advantages, many studies have found that: most mental health workers acknowledged the usefulness of technology-assisted CBT for mild to moderate psychological difficulties, but not for more severe problems (Stallard et al., 2010). Furthermore, many are optimistic and positive about the increased use of technology as an adjunct to traditional approaches (Sinclair et al., 2013). However, there were perceived disadvantages related to: displacing human contact, reducing the therapeutic relationship, variable beliefs in technology, mixed stakeholder buy-in, time restraints and external factors (Fleming and Merry, 2013; Stallard et al., 2010; Du et al., 2013).

There are three key limitations to the extant qualitative literature. Firstly, the studies are typically restricted by small sample sizes. Secondly, there is disproportionate coverage from UK, New Zealand and Australian systems. Thirdly, there is very limited research that focuses exclusively on APs that work directly with children aged 12 and under. Thus, the applicability to other settings is unknown. These limitations are especially relevant due to the importance of understanding staff experiences with interventions, as they are associated with effective uptake and sustainability over time (Du et al., 2013).

1.4. The current study

This study addresses the lack of qualitative research on AP experiences of providing technology-assisted CBT to children. Notably, the context for this study is primary care psychology in Ireland – itself a country where substantial challenges have been documented regarding long waitlists for children with psychological difficulties (McGorry et al., 2013; Malla et al., 2016). Notably for Ireland, in 2017, an unprecedented 114 APs were recruited with the remit of providing technology- assisted CBT for children – a first for primary care services (National Service Plan – Health Service Executive, 2016, Health Service Executive, 2017). This service development provides an opportune time to examine the experiences of APs from training through to routine practice. Primary care in Ireland is defined as all of the health and social care services one finds in their local community (outside of hospital settings). Therefore, the aim of this study is: to use qualitative methodology to examine the overall experiences of APs in Ireland providing technology-assisted CBT.

2. Method

2.1. Participants

This study is part of an ongoing randomized controlled trial (RCT) of a CBT computer game called Pesky gNATs (O'Reilly and Coyle, 2015) which is currently being used in primary care psychology services in Ireland (RCT number: ISRCTN60159987). APs were hired by the Health Service Executive (HSE) to assist with the provision of low intensity interventions using a stepped model of care, within which the Pesky gNATs RCT was situated. Of the 114 APs recruited by the HSE in Ireland during 2017, a total of 89 were trained in Pesky gNATs between June 2018 and March 2019. From this, 23 (female n = 21) became active in the RCT and were thus eligible to participate in the qualitative evaluation. In primary care in Ireland, APs must have a primary degree in psychology that confers eligibility for graduate membership of the Psychological Society of Ireland; and some clinically-relevant experience. AP positions were established to support service provision on a fixed-term basis of no more than two years to allow career development opportunities (such as acceptance onto professional clinical training). Between September and December 2019, 14 Irish self-selected APs consented to participate in this study (female n = 12, on full-time 2-year contracts). From this, 10 APs provided their date of birth – the mean age was 30.7 (sd = 8.3).

2.2. Intervention

The APs in this study were applying Pesky gNATs in primary care locations across Ireland. Pesky gNATs is 3D CBT computer game (with accompanying mobile app) for children aged 7 and over who present with clinically significant levels of low mood and/or anxiety. For this study, APs were using the intervention with children between the ages of 8 to 12 (inclusive). Children play the 7-level game in-session with their AP; and use the app and workbook to consolidate their learning in school and home settings. Pesky gNATs was developed on a not-for-profit basis by O'Reilly and Coyle (2015), merging both evidence-based clinical psychology models and game design. Further details on the underpinning theoretical model, game features, mobile app, and workbook are available at www.peskygnats.com and in a chapter by O'Reilly (2018). A level-by-level breakdown of the game and its CBT content is provided in supplementary materials. In the game, the guiding metaphor is one of gNATs - they are little flies that can sting us, referring to the idea of negative automatic thoughts in CBT (NATs).

All APs in the RCT fully completed a 3-day mandatory training course in Pesky gNATs that covered CBT theory, core therapeutic skills, intervention roleplay, and a full 7-level gameplay practice of Pesky gNATs under supervision. Also, all APs had access to a suite of online training videos to support their training. All APs using Pesky gNATs were clinically supervised by clinical psychologists in the HSE.

2.3. Procedure

Between August and December 2019, all eligible APs were provided with an email invitation to the study. APs were eligible to participate if they were actively providing the intervention within the RCT. Participants were invited to provide their completed informed consent document; and face-to- face (n = 2) and Skype interviews (n = 12) were subsequently arranged. Qualitative data was gathered using semi-structured interviews to allow for flexible but in-depth data collection (Forrester and Sullivan, 2018). A semi-structured interview guide was developed to allow APs to provide a wide range of insights regarding their overall experiences, but also their implementation and future-oriented perspectives (fully detailed in Table 1). The average interview time was 38 min; with a duration range of 26 min to 59 min across all interviews. All interviews were conducted by the first author, with all data anonymized during transcription. Interviews were recorded using the voice recorder function on a research-only laptop (Dell XPS 13) – and stored using password- protected encrypted files thereafter.

Table 1.

Semi-structured interview guide for APs.

General review

1.How would you describe your approach to clinical practice? 2.What was your view about using technology in your AP role? How did your approach to practice and the use of technology match your eventual experience of using Pesky gNATs? 3.Can you speak about your overall experience of providing Pesky gNATs please? 4.When thinking about any clinical changes that you observed in your clients throughout their intervention, can you discuss the role Pesky gNATs played within this process? 5.From your perspective, what was most useful about the Pesky gNATs game? Similarly, what was most useful about Pesky gNATs from the child's perspective? 6.How did the use of either the Pesky gNATs workbook or mobile app(s) assist your clinical practice?

Implementation-oriented questions

7.Can you discuss any external or organizational factors that may have assisted or impeded your ability to effectively provide Pesky gNATs to young people? 8.As an AP providing CBT to children experiencing low mood/anxiety, could you reflect on your involvement throughout Pesky gNATs, did you feel appropriately involved and supported by the technology? 9.In what way, if any, did the integration of Pesky gNATs assist your own professional growth as an AP tasked with providing CBT to children? 10.If you could make one major design change to Pesky gNATs, what would it be and why? 11.Do you think primary care services should continue to use Pesky gNATs? Please explain your answer? 12.Would you recommend the use of Pesky gNATs in other clinical child settings?

Future-oriented questions

13.If a future AP asked you about how to use Pesky gNATs to maximise positive clinical outcomes for children experiencing low mood/anxiety, what would you say? 14.If you were to undergo Pesky gNATs training, and implement Pesky gNATs in practice again, would you do anything differently and why? 15.Is there anything else that we have not covered that you would like to discuss?

2.4. Coding and analysis strategy

To allow for an inductive, rigorous and flexible approach to interview data processing, qualitative thematic analysis was chosen (Braun and Clarke, 2006, Braun and Clarke, 2019). All data was managed using the qualitative software package Nvivo (version 12 for Windows – QSR International, 2020). The six-phase coding procedure for qualitative thematic analysis was then followed. Firstly, data were fully transcribed, read and re-read to establish familiarity. Secondly, initial coding of keywords occurred throughout the dataset. Thirdly, all codes were organized within provisional themes and sub-themes. Fourthly, a thorough review and revision of themes was implemented. A portion of the overall dataset (10%) was coded separately by the third author, and an independent coder (a psychology PhD student). Any coding disagreements or lack of clarity was reviewed until consensus was reached. For the fifth phase, all themes and sub-themes were appropriately defined and named; before the sixth and final phase: compiling the final analysis with the relevant supporting quotation. Finally, to ensure transparent and comprehensive reporting, the Consolidated criteria for reporting (COREQ) were applied (see Supplementary Table 2).

2.5. Ethics

Both the RCT evaluation of Pesky gNATs and the present qualitative study have received ethical approval by the Research Ethics Committee in University College Dublin (ref. HS-18-76-McCashin-O'Reilly).

3. Results

Overall, five themes were identified from the dataset – each with two key subthemes: 1) positive experiences, 2) integrating Pesky gNATs with traditional CBT, 3) managing parental ‘buy-in’, 4) implementation complexities, and 5) future perspectives. A full summary of each theme and the aligning subthemes is provided in Table 2. The following subsections expand on each of the 5 themes, interspersed with illustrative quotation from APs.

Table 2.

Thematic summary.

Theme	Subthemes
1) Positive experiences	a)Positive clinical change for children b)AP professional growth
2) Integrating Pesky gNATs with traditional CBT	a)Facilitating engagement b)Different child characteristics
3) Managing parental ‘buy-in’	a)Unclear role for parents b)Child autonomy
4) Implementation complexities	a)Managing waitlists b)External factors
5) Future perspectives	a)Design considerations b)Advice to future APs

3.1. Positive experiences

The vast majority of APs spoke about the positive experience of their time using Pesky gNATs in primary care. This was related to both their own professional growth as APs, and their supportive supervision; but also the positive clinical change seen within their clients.

3.1.1. Positive clinical change for children

Many APs remarked on the communication of enjoyment, learning, improved functioning and positivity from their clients – a dynamic echoed in an observed process of positive clinical change throughout the Pesky gNATs intervention:

Yes, I think it was very clear to see, again, from the psychometrics, which was involved in the Pesky gNATs program, and seeing the change, seeing the levels decreasing, which was great in one case. Actually, then seeing the change in them coming in, they affect and being able to engage more and the more they got used to the game and had the structure with the game, as well, and looking at all the different kinds of names, the characters really helped for the client, as well. To have the different elements, as well. I think they really enjoyed that, from the breathing techniques and visually being able to see that to the thought and the negative automatic thoughts. I think having all of those kinds of strategies helped, as well. You were catering to a broader audience in that sense, some clicked more with others. I thought that was effective for clients, for sure.

(AP 2)

I would, yes, I would say in both cases, I would have observed those changes. Again, both within the room, but also, as I said, with parents and the child themselves offering information, that there was improvement being made outside of our session with me each week. They were seeing improvements at home, they were seeing improvements at school, they would have provided that information for me pretty much on a weekly basis across both clients and then within the room itself, both cases the clients I was working with, it was more anxiety related difficulties and low mood, but I think in both cases, we will have seen a positive change both within the room and also the home and the school environment, as well. I think Pesky gNATs had a big part to play in that.

(AP 7)

Indeed, many APs stated that children found the basic concepts in Pesky gNATs to be helpful, even when the deeper CBT lessons were more challenging:

I think that they became more aware of their thoughts, so I think that something, like thoughts and feelings is a hard thing to grasp even as an adult, so as a child, that can be really confusing, you're talking about thoughts. They think you're talking about feelings. I think with the Pesky gNATs and the different types of gNATs and figuring out what kinds of thoughts they had, it allowed them to realise that and catch them. I think that was really helpful.

(AP 4)

3.1.2. AP professional growth

In addition to APs noticing how the game benefitted their clients, it was evident that Pesky gNATs also facilitated AP growth in several ways. This included: enhanced understanding and confidence with the theory-to-practice of CBT, allowing for experience-building with technology in CBT, and recognizing their role within the game:

I think that is really good and it was quite significant for me. Whether I would have gotten that individual work without it, as the program exists up here in [removed], I don't think I would have. I think it's really important that way. I think if the AP system continues to really – whether Pesky gNATs in itself fully resolves the child thing in one go, as a short-term intervention. I think it helps, it definitely helps and begins a move toward the shift. I think it adds a lot of value to the AP experience to have this tool. I think it adds value in using the APs and giving them experience. That experience to work with clients, child clients. Whereas, I don't see any other program that works that well, do you know what I mean?

(AP 12)

I think it has provided such an excellent experience of being able to work individually with young people but using a very structured program. In a sense, I feel like I'm working within my competencies, like with support and supervision. I think if there's been a huge learning in it, in terms of working with children, also, in terms of, like I said before, identifying with adaptions are needed, and being able to spot that in order I suppose to meet the needs of young people and children. I think it's been a huge learning and I feel like I've learned so much from it. It's really built on my CBT skills, as well, which I will definitely bring forward with me, hopefully, in my career into the future.

(AP 14)

3.2. Integrating Pesky gNATs with traditional CBT

This second overall theme relates with how every AP balanced the use of a computer game alongside the traditional challenges of face-to-face CBT with children. Two consistent subthemes illuminate this overall theme: facilitating engagement and different child characteristics.

3.2.1. Facilitating engagement

A large portion of APs noted that Pesky gNATs, unlike traditional CBT, was immediately more likeable, accessible, interesting and therefore appealing to children in the early stages of their time in primary care. APs felt that this was related to positive engagement throughout the intervention. Moreover, many aspects of the game were continually highlighted as facilitative of engagement. Although considerable screening and referral needs to occur, APs emphasized how Pesky gNATs provided a notably different experience for children compared to face-to-face CBT:

I think that it has such merit and I know it's obviously in the early stages, but I think from the clients that I've seen, like any intervention, when it's the right fit, when it's the right timing, when CBT is what they need, I think it can just enhance their experience so much, especially in that age range, where it's just so different from the traditional way of having a CBT intervention. I think it's really engaging for kids.

(AP 10)

It's been interesting to see clients going through the process from start to finish. I think, broadly speaking, there's been really good engagement.

(AP 2)

3.2.2. Different child characteristics

This next subtheme describes AP observations of a variety of child characteristics that were brought to Pesky gNATs – in ways that both helped or hindered the CBT process. Many APs highlighted the fact that some children identified as gamers or as being very familiar with new technologies. Consequently, this prompted the child to rush through the game:

Some of the boys who were very used to computer games found it a bit slow, so they would try and go through it really fast. A lot of the time I would say, slow down, don't keep pressing the button. They didn't want to listen to the explanations, if you know what I mean, I think because their buzz is a faster buzz, they were banging the button and saying, “Do I have to listen to that?” I was saying, yes. Because it's not fast enough if they are used to computer games I suppose.

(AP 12)

On the other hand, some APs linked the specific features of the Pesky gNATs in-game narrative and characters as being helpful for applying the CBT model, but for specific age- groups and genders:

Do you know what? It actually depended very much on the children that came in. I had one little girl who was using it, I now have another one probably in and around the same age group, in the ten-year-old bracket. They loved it. It was very user-friendly, the social stories that were being taught, they got the humour, they got the use of the characters and were able to very much relate to it, it was helpful and is helpful with the child that I'm doing it with now, in terms of getting them to talk about their thoughts, feelings, and behaviours, and the different thinking patterns. It makes it easy to work with the child and give them that context because it's put in a very child-friendly way, there are great benefits in that, and for working very much on a CBT basis with those children.

(AP 4)

3.3. Managing parental ‘buy-in’

All APs organically identified the management of parents as a crucial factor in the ultimate effectiveness of Pesky gNATs, and indeed the child's engagement. To unpack this overall theme, two interrelated subthemes were generated: the unclear role for parents, and child autonomy.

3.3.1. The unclear role for parents

Throughout every AP interview, the level of parental involvement in Pesky gNATs was heavily discussed. APs reflected over the role and resources parents could - or should have - with respect to: joining some sessions, having structured feedback loops with the AP and child, understanding CBT, and managing their own anxiety or low mood as it relates to their child's difficulties. While APs acknowledged that there is no effective formula for establishing parental inclusion, there were mixed experiences regarding those parents that were involved to differing extents:

I did, as I said, have one parent or one child where his parents did sit in for the whole program. I don't know how useful that was for him either because it didn't allow him that space to just let out what he was thinking and feeling, but at the same time, he was extremely anxious and really needed that supported and couldn't separate from parents. It is a fine line between how much you let them in and how much you let them be their own person too, you know?

(AP 5)

Yes, the transfer of learning, I do think because the parent wasn't involved, I don't know how well it transferred into daily life, especially after it finished. I found it really difficult to get them to complete the home activities. Like, they might do it one week and not do it the other week. Then I was like, okay, they're learning all of these skills, which is great, but are they integrating it within their life? When they finish here, will this continue? That's something that really played on my mind and just like, I kind of felt like if the parents were more involved, not fully involved, but were involved in the last ten minutes of every session or something, that it might encourage that a bit more because it would have been such a shame for them to lose all of those skills once the session is finished.

(AP 1)

3.3.2. Child autonomy

Related to the unclear role of parents, some APs also commented on the distinct autonomy established by the child. APs found that some children enjoyed being-in-control, taking the lead and discovering their agency when playing Pesky gNATs. As some children had found rapport, trust, and psychoeducation in both Pesky gNATs and the AP, there was apparent reluctance to integrate their parents:

Yes, well, I think for some kids they actually didn't want to share with their parents, their examples, day-in-day-out, it might have been because they were about having a fight with their parent or something, or something like that. You have to respect that.

(AP 3)

However, in the opposite way, some children also utilized their autonomy to involve their parents to a specified degree:

To let the child take the lead maybe. I know for the child I had, she was so anxious that she didn't want to sit in the session without the mom, so for the first two sessions, I was doing Pesky gNATs with the mom in the room, then the mom could sit int eh waiting area. Then after that, the mom could sit outside. Even for the mom sitting in on those sessions, it was probably good for her to just get an idea of what was going on for the child and what the Pesky gNATs actually, what material was covered and then maybe the mom was a bit curious about what was in the homework book or whatever.

(AP 8)

3.4. Implementation complexities

This fourth theme underscores the many external, organizational, and environmental complexities that were associated with AP experiences of implementing Pesky gNATs in primary care. The two supporting subthemes are: managing waitlists and external factors.

3.4.1. Managing waitlists

A substantial number of APs connected their experiences with issues relating to long-standing waitlists for children and families seeking psychology services. Due to the long wait-time for some children, their needs may have grown in complexity by the time they meet the AP. This meant that many children had presented with needs other than low mood and anxiety, which then brought about further screening and assessment needs:

I suppose maybe a more neutral point would be the identification of relevant clients from the wait list and I suppose working within that wait list structure and I suppose that might come down to how clients are categorised when they're initially referred in and they are in the waitlist because Pesky gNATs and a lot of CBT approaches might go for that, that standard anxiety or low-mood case. It's sometimes very hard to identify that case from a wait list. I'd say there is a pretty detailed screening that has to be done to make sure that the client is getting a benefit from the approach being used.

(AP 6)

Of the children that were deemed eligible, APs were clear on the functional role of Pesky gNATs within primary care. Indeed, despite the waitlist challenges, all APs supported the continued use of Pesky gNATs in primary care in some form, in addition to suggesting that it may be applicable in other settings (such as specialist ASD services, or schools for the prevention of psychological difficulties):

There's a lot of talk around the step-care model - I think that was the main driving force behind the recruitment of all of the APs that the HSE took on. I think Pesky gNATs is a good way, as I said, it's a really good way for APs to access clients, but also for clients to access a psychology service. I think it can accelerate the process of clients actually being seen. I'm not sure. At the moment, my feeling is that it's probably something additional rather than standalone, but I think it will accelerate the process and allow children to be seen by primary care psychology services a lot quicker than if they're – in our service at the moment, the main way that clients are seen is through one-to-one interventions and they can wait, at the moment, probably for about two years to get that.

(AP 7)

3.4.2. External factors

Although a minor subtheme, there were a variety of other external factors were highlighted by some APs as potentially impacting the overall implementation of Pesky gNATs in primary care. These factors were: training-to-practice delays, local socio-economic difficulties, local internet or technology inefficiencies, geographical challenges impeding client accessibility, and uncertainty over government funding for APs. Another factor that was cited as helpful to implementation was the training and Pesky gNATs website that could consolidate APs' training and help prepare for upcoming sessions, in addition to informal AP peer- support.

3.5. Future perspectives

Based on the AP experiences, this final theme elaborates on what they felt was significant into for Pesky gNATs into the future. This theme subsumes the following subthemes: design considerations, and advice to future APs.

3.5.1. Design considerations

As per earlier themes, most APs were especially positive about their experiences of Pesky gNATs. However, this subtheme demonstrates that many APs were nonetheless exact in their preferences for reconsidering some of the game design elements. Specifically, based on their playing of the game, APs suggested better game design for the following: tracking take-home tasks within the laptop, making relaxation exercises more playful, diversifying blog- input options, pausing and restarting levels, skipping avatar dance features, enhanced graphics, character navigation through the island, elongating levels (particularly core beliefs level), and simplifying the language further. The following quote provides insight into one AP's design suggestion:

I think all of the kids that I had, or most of them, asked about climbing the ladders. I don't even know if that can be implemented. But there are bits like that, like, there's a basketball hoop and there are ladders and things that they want to explore and that element of it too because I think a lot of them were really excited by, it's an island and I can explore. Just that one added piece of a game. The games, they loved that actually. Yes, we would have, depending on the kid obviously, we would have done different things at the end, some like to explore, some like to play those games. Yes, I think design-wise, maybe a little bit more interactive in that sense.

(AP 10)

3.5.2. Advice to future APs

All APs provided a vast array of advice for future colleagues who may use Pesky gNATs in primary care. As per other themes, APs stressed the importance of securing ‘buy-in’ from parents and other colleagues, ensuring time is made for appropriate rapport-building, and maintaining familiarity and mastery of the game:

Get the parents on board, make sure the child is hitting the criteria that Pesky gNATs was designed for. Also, be aware that the common factors like warmth and therapeutic alliance seem to be a strong element of it. Yes, maybe getting the parents on board and making sure they understand the programs, as well, and do the homework.

(AP 12)

Continuously, some APs offered very nuanced advice regarding the strategies to check-in with children during sessions:

Try to anticipate just how much a client may be able to take in session-to-session and find ways to check in with them around how much they're understanding the material. Just so you can develop that sense of it as they go through the sessions. Because even the client who at the very start doesn't seem to be taking on a lot of material, by the end, can really be taking it on. Even if they're not using the gNAT terminology, they're using a lot of insight to their own thoughts. That development is really useful to be able to observe.

(AP 6)

4. Discussion

This study aimed to gain a qualitative understanding of APs experiences of using Pesky gNATs in primary care psychology services in Ireland. Five overarching themes (with two subthemes) were interpreted from the data: 1) positive experiences, 2) integrating Pesky gNATs with traditional CBT, 3) managing parental ‘buy-in’, 4) implementation complexities, and 5) future perspectives.

In keeping with prior research (Sinclair et al., 2013; Byrne and Twomey, 2011), this study also observed predominately positive experiences with, and attitudes to, technology-assisted CBT – in this case, the CBT game Pesky gNATs. Importantly, from the perspective of APs, there was insight regarding positive clinical change seen in children – complementing the positive trends seen in both the quantitative and qualitative literature (Stasiak et al., 2016; McCashin et al., 2019). There was also consistent evidence that Pesky gNATs was assisting APs to professionally develop, enhance their CBT knowledge, and learn about the balancing of technology in real-world therapeutic settings for children. This somewhat allays earlier concerns in the literature regarding the perceived risk that technology-assisted CBT could adversely impact the human connection or therapeutic alliance (Stallard et al., 2010; Fleming and Merry, 2013).

Indeed, the second theme outlined how APs were skillfully integrating Pesky gNATs with traditional approaches to providing CBT for children. There was a recognition from APs that Pesky gNATs offered unique appeal to children that is distinct from traditional CBT (…it's just so different from the traditional way of having a CBT intervention. I think it's really engaging for kids – AP 10). Moreover, APs also detailed their observations of different child characteristics that were amenable to Pesky gNATs. This understanding is especially relevant for APs who need to effectively tailor CBT for younger populations (Carr, 2008; Stallard, 2019).

A notable theme that arose with every AP was the management of parents; and the need to secure their ‘buy-in’ to Pesky gNATs, despite the need for a level of child autonomy during the intervention at the same time. Given the known clinical association between parental and child psychological difficulties (McLeod et al., 2007), it is prudent to explore what these AP experiences convey. It appears that there will likely be case-by- case needs for parents, counterbalanced with the preferences emerging from the given child's autonomy during the intervention. Many APs suggested resources for parents. The creation of specific parent resources that systematically mirrored both the structure and content of Pesky gNATs without interfering with child autonomy may therefore be a useful exploration. Indeed, this theme of managing parental involvement is extendable to the wider literature. It is apparent that the field of technology-assisted CBT for children does not provide a consistent evidence-based pathway to specify how to involve parents within intervention timelines. It remains contested the extent to which parental involvement impacts clinical outcomes (Reynolds et al., 2012; Bodden et al., 2008); but this study highlights the mixed experiences from clinical stakeholders. This knowledge gap is especially relevant given the potential intergenerational digital divides; and differing parent perceptions of, and interactions with, evolving technologies (Livingstone and Blum-Ross, 2020).

The fourth theme – implementation complexities – underscored the difficulties faced by APs in frontline primary care settings, as highlighted by earlier research (Malla et al., 2016; McGorry et al., 2013; Stallard et al., 2010). APs explained how the mechanics of waitlists and external factors impacted the suitability of children for interventions such as Pesky gNATs. However, despite variable funding and a strong but limited evidence base for technology-assisted CBT for children, all APs supported its continuance in primary care. Moreover, APs acknowledged its role within Ireland's sustained adaption to a stepped model of care (National Service Plan – Health Service Executive, 2016, Health Service Executive, 2017). These are noteworthy insights given that prior research has emphasized the importance of staff acceptance of interventions to ensure sustainable uptake and effective implementation (Du et al., 2013).

Finally, the fifth theme addressed AP future perspectives; and provided helpful design suggestions for Pesky gNATs. Additionally, specific advice was offered to future APs and provided a useful linkage between many of the other themes (Get the parents on board, make sure the child is hitting the criteria that Pesky gNATs was designed for. Also, be aware that the common factors like warmth and therapeutic alliance seem to be a strong element of it. Yes, maybe getting the parents on board and making sure they understand the programs, as well, and do the homework – AP 12). These contributions are also in keeping with recent calls for the increased input of clinical stakeholder throughout the (re)design stages of interventions (Mohr et al., 2015, Mohr et al., 2017).

4.1. Strengths and limitations

This study has a number of strengths and limitations. Importantly, the overall contextual relevance of these findings will be significantly complemented by the later RCT results for Pesky gNATs (due 2020). Together, both the quantitative and qualitative findings may assist practitioners and policy-makers alike with the integration of technology-assisted CBT in primary care. To our knowledge, this is the first qualitative study of AP experiences with a CBT game in primary care settings for children in Ireland.

However, a number of limitations should be noted: the majority of interviews were not held face- to-face which may have impacted the depth of responses, and female APs vastly outnumbered males. In addition, the sample was vulnerable to self-selection biases. Regarding limitations, there was a risk of researcher bias given that the first author conducted all interviews. This study is also limited by the absence of in-depth demographic and professional backgrounds of APs. Lastly, the role of socially desirable responding must also be considered a potential risk given the nature of AP positions (Byrne and Twomey, 2011).

4.2. Clinical implications

Taken together, the themes from this study indicate that the majority of clinical stakeholders experience technology-assisted CBT as a helpful and positive intervention. Nonetheless, insights regarding real-world challenges of managing parental involvement, transferring technology-assisted CBT into everyday life, and implementation complexities hold several clinical implications. Clinicians should be mindful of exploring a priori parental inclusion strategies to offset any potential negative effects during or after interventions. For example, this could take the form of parent-specific materials, or establishing a clinician-child-parent contract to include all parties in a portion of sessions. Relatedly, such strategies may then support the transferring and maintenance of CBT content in the everyday life of the family. Turning to implementation complexities, clinicians should consider predetermining: the implementation capacity of their service, technology infrastructure appropriateness, waitlist challenges and opportunities regarding the suitability of technology-assisted CBT, and devise service strategies to measure and maintain intervention fidelity. Prearranging these strategies could have positive impacts on overall clinical implications of technology-assisted CBT.

4.3. Conclusions and future research

Technology-assisted CBT interventions such as Pesky gNATs are well-received by both APs and children in primary care. Despite the many challenges faced by service providers trying to address the unmet needs of children experiencing psychological difficulties, technology-assisted CBT has now demonstrated both quantitative and qualitative evidence of the role it can play. However, there remains a lack of mixed methods research with large samples – particularly of children aged 12 and under. Future research should aim to address these gaps to further ascertain if technology-assisted CBT is resource-efficient and effective for this age group across different real-world settings, in addition to building-in evidence-based implementation strategies.

Funding

This research was funded by Technology Enabled Mental Health for Young People (TEAM). TEAM has received funding from the European Union's Horizon 2020 - Research and Innovation Framework Programme under the H2020 Marie Skłodowska-Curie Actionsgrant agreement number 722561.

Declaration of competing interest

Both the second and third authors are founders of the nonprofit company Handaxe that created Pesky gNATs.

Appendix A. Supplementary data

Supplementary material