Implementing telepsychiatry in an early psychosis service during COVID‐19: Experiences of young people and clinicians and changes in service utilization

Abstract

Aim

In response to the COVID‐19 pandemic, our early psychosis program rapidly transitioned to telepsychiatry. This study examined the change in health service utilization and experiences of young people and clinicians in response to the implementation of telepsychiatry.

Methods

Mixed methodology and triangulation of evidence drawn from health service databases and survey data. Using a retrospective observational design, health service data from pre‐ (Time 1) and post‐(Time 2) telepsychiatry periods were compared. Surveys were also conducted with representation from clinicians and young people.

Results

The number of appointments increased between Time 1 and 2, although this was accompanied by a near‐doubling in missed appointments (8% to 13%). Young people had mixed views about telepsychiatry. While convenience was a frequently cited benefit, clients reported technological issues, isolation and lack of human connection. A preference for face‐to‐face appointments was linked to younger age and anxiety when using telepsychiatry. Clinicians reported improved workplace satisfaction and efficiency but noted some limitations in the use of telepsychiatry including difficulty interviewing and managing unwell clients remotely and called for greater skill development.

Conclusions

The introduction of telepsychiatry in response to COVID‐19 was associated with an increase in service activity; however, there was an increase missed appointments by young people. Although clinicians and clients reported positive experiences, telepsychiatry was not completely endorsed as a replacement for face‐to‐face interactions.

1. INTRODUCTION

Telepsychiatry, the use of technology to provide psychiatric care at a distance, is becoming increasingly relevant given the global COVID‐19 pandemic (American Psychiatric Association, 2020; Hilty et al., 2002; Lal et al., 2020). In response to COVID‐19, many mental health clinics rapidly introduced telepsychiatry to deliver multidisciplinary services such as assessment, therapy, education and medication management (Hilty et al., 2002; Kasckow et al., 2014). There is evidence that use of telepsychiatry is reliable and effective in adults (Hilty et al., 2002; Kasckow et al., 2014; Kruse et al., 2017) and recent studies have shown success in targeted psychological interventions in young people (Sankar et al., 2021). Evidence, however, is lacking as to whether telepsychiatry can be effectively used to deliver multidisciplinary services to young people with complex mental health needs (Bell & Alvarez‐Jimenez, 2019; Dixon et al., 2016; Lal et al., 2020).

Some studies suggest that telepsychiatry may increase attendance rates by improving accessibility to services and removing barriers to travel (Chaudhry et al., 2021; Leigh et al., 2009). Several challenges to implementation however have been raised, including technological challenges and maintaining privacy and confidentiality, while carrying out clinical assessments and treatment (Zhang et al., 2021).

Patient satisfaction is pivotal to success. A pre‐COVID‐19 feasibility study explored perspectives of young people with early psychosis towards the use of telepsychiatry (Lal et al., 2020). Almost half of all participants were highly favourable towards telepsychiatry but concerns were raised about loss of human contact impacting on engagement with clinicians. Loss of therapeutic engagement is associated with cancellation of appointments and service drop‐out (Sezgin et al., 2021). Given engagement is vital to treatment success (Dixon et al., 2016) a better understanding of the young person's experience with telepsychiatry is needed.

The clinician experience in delivering telepsychiatry is equally important. Concerns have been raised about privacy and confidentiality and whether telepsychiatry can effectively deliver psychiatric care and treatment (Uscher‐Pines, Raja, et al., 2020; Uscher‐Pines, Sousa, et al., 2020). However, few studies have examined telepsychiatry from the clinician viewpoint and there is paucity of data on this topic.

This study examined health service utilization in an early psychosis outpatient clinic, before and after transition to telepsychiatry in response to the COVID‐19 pandemic, and the experiences of telepsychiatry from the perspectives of both young people and clinicians.

2. METHODS

2.1. Participants

Participants included young people with a first episode of psychosis (FEP) or at high risk of psychosis engaged in the Black Swan Health Headspace Early Psychosis program in Perth, Western Australia. Clinic referral criteria included a diagnosis of FEP or high risk of psychosis, maximum 25 years of age, and no more than 12 months of treatment by another service.

2.2. Procedures

We used a mixed methodology, which is a naturalistic study approach to health service research (Johnstone, 2004), and includes the use of both quantitative and qualitative methods. Using a triangulation of evidence from service databases and survey data from young people and clinicians, we analysed changes in service utilization using quantitative methods, and reasons provided for preferences regarding mode of service delivery using quantitative and qualitative analyses.

2.2.1. Health service data

Using a retrospective observational study design, an audit of health service databases was conducted. Data from 2 months before (Time 1, January–February) and after (Time 2, April–May) the introduction of telepsychiatry was compared, excluding March to allow the transition in service model. Service utilization data and demographic information was extracted (including age, gender, diagnosis, medication use, appointment date, type, duration, clinician position, mode of delivery, and failure to attend scheduled appointments (missed appointments)).

2.2.2. Client and clinician surveys

Six weeks post‐implementation of telepsychiatry (May 2020) separate surveys were conducted with young people and clinicians. The survey design was informed by previous studies and adapted after consultation with clinicians and consumers.

The client survey was offered to all young people over age 16 during appointments or via email. It asked about demographics and service contact and included one question about preferences for mode of appointment (telepsychiatry only, face‐to‐face only, a combination, or undecided), and another six questions about experience of telepsychiatry (attitude, accessibility, clinical engagement, technology use and privacy/confidentiality), rated on a 5‐point Likert scale (1 = strongly disagree to 5 = strongly agree). Additional questions invited open responses to explain what they liked and disliked about telepsychiatry.

The clinician surveys were offered to all clinicians via email. It included nine questions with themes focusing on the clinical and therapeutic impact of telepsychiatry (engagement quality, ability to conduct assessments and provide psychiatric care) and practicalities (accessibility, time management and technological issues) rated on a Likert scale, and open‐ended questions inviting further comments.

2.3. Data analysis

Analysis of quantitative data included descriptive analyses and statistical comparisons of data for clients who were active during both T1 and T2 using chi‐square or analyses of variance (ANOVA) for discrete and continuous variables respectively, along with Cohen d’ for effect size. A linear regression analysis was conducted to identify quantitative survey factors which might contribute to a preference for telepsychiatry or face‐to‐face appointments. Demographic variables were entered in the Regression Step 1. At Step 2, survey item predictors were added as determined by significant t‐tests.

Qualitative analyses of survey answers in response to open‐ended questions were analysed using guidance from framework methods and involving content‐driven thematic analysis. Answers provided by clinicians and young people were transcribed into Microsoft Excel. Reflexive thematic coding (Braun & Clarke, 2006) involved data immersion, iterative reviews and open coding of keywords of repetitive concepts. Themes were generated and progressively refined until saturation, and themes were cross‐validated iteratively after discussion amongst co‐authors.

2.4. Approvals

All participants provided informed consent. This project was conducted as part of a quality improvement activity with governance, approval, and oversight by the Black Swan Health Institutional Quality Assurance Review Board following approved processes (Rickwood et al., 2015).

3. RESULTS

3.1. Audit of health service data

There were 140 participants with data at both time points (n = 141 at T1 and 157 at T2), with a mean age of 21 years (69.1% males). Table 1 shows that the most common diagnoses were schizophrenia spectrum disorders (35.7%) and undifferentiated psychosis (35.7%). Length of service averaged 16.4 months (range 1–56 months).

TABLE 1.

Client characteristics and service utilization for T1 (Jan–Feb 2020) and T2 (Apr–May 2020)

Demographic and clinical information, baseline (n = 140)
Age, M ± SD	21.8 ± 2.8
Gender, n (%)
Males	96 (69.1%)
Females	43 (30.9%)
Diagnosis, n (%)
Schizophrenia	50 (35.7%)
Undifferentiated psychosis	46 (35.7%)
Mood disorder with psychosis	25 (17.9%)
Other non‐psychotic disorders	19 (13.6%)
Medications, n (%)
Antipsychotics	101 (72.1%)
Mood stabilizers	23 (16.4%)
Anxiolytics	7 (5.0%)
Antidepressants	49 (35.0%)
Length of service (days), M ± SD	492 ± 458

Service events (n = 140)	T1	T2	Statistical test
Service events total, N (% of total)	993 (47.3%)	1106 (52.7%)	Two‐proportion z = −2.24, p = .01
Average per client, M ± SD	7.0 ± 4.3	7.9 ± 5.5	t(139) = 1.53, p = .11, d′ = 0.12
Mode of contact for booked appointments, M ± SD per client (% of total events)
Face‐to‐face	7.0 ± 4.3 (99.7%)	1.4 ± 2.2 (16.2%)	t(139) = 15.7, p < .001, d′ = 1.33
Phone	0.0 ± 0.1 (0.03%)	1.5 ± 2.3 (19.4%)	t(139) = 8.28, p < .001, d′ = 0.70
Telepsychiatry	0.0 ± 0.0 (0.0%)	4.9 ± 4.5 (64.4%)	t(139) = 13.0, p < .001, d′ = 1.09
Appointment duration by mode of contact (minutes), M ± SD 2			F(2,1869) = 28.2, p < .001, d′ = 0.05
Face‐to‐face	50.6 (24.5)	43.9 (21.7)
Phone	40.0 (17.3)	33.8 (9.9)
Telepsychiatry	‐	45.5 (12.8)
Missed appointments, M ± SD (%)	0.5 ± 0.9 (8.0%)	1.0 ± 1.3 (13.3%)	t(139) = 4.09, p < .001, d′ = 0.34
Missed appointments by mode of contact, n (% of all service events)
Face‐to‐face	78 (7.8%)	8 (0.7%)	T2 X 2(2) = 37.8, p < .001
Phone	0	13 (1.1%)
Telepsychiatry	0	126 (11.3%)
Missed appointments by diagnosis, n (%)			X 2(3) = 6.7, p = .08
Schizophrenia	35 (45.5%)	39 (26.5%)
Mood disorder with psychosis	13 (16.9%)	27 (18.4%)
Undifferentiated psychosis	18 (23.4%)	51 (34.7%)
Other non‐psychotic disorders	11 (14.3%)	30 (20.4%)
Type of appointment, n (%)
Case management	425 (47.1%)	441 (39.9%)	X 2(5) = 12.9, p < .001
Psychiatric reviews	226 (25.0%)	306 (27.7%)
Medication (depot, clozapine)	56 (6.2%)	68 (6.1%)
Family work	35 (3.9%)	75 (6.8%)
Functional recovery	126 (14.0%)	186 (16.8%)
Crisis intervention	35 (3.9%)	30 (2.7%)

Table 1 shows that significantly more appointments were booked during T2 than T1 (z = −2.24, p = .01), although they tended to be shorter in duration by an average of 7 min (from 50.5 to 42.6 min overall), F(1,7215) = 7215, p < .001, d′ = .01. The number of appointments per client increased from an average of 7.0 (T1) to 7.9 (T2) but this was not statistically significant, t(139) = 1.53, p = .11. As expected, the percentage of appointments using face‐to‐face contact reduced from 99.7% at T1 to 16.2% at T2, corresponding to a mean difference of 5.6 appointments per person, t(139) = 42.3, p < .001, d′ = .83. At T2, telepsychiatry was the most common mode of contact (64.4% of all appointments), followed by telephone appointments (19.4%). Unexpectedly, there was a significantly greater percentage of missed appointments at T2 than T1 (13.3% of all appointments at T2, compared 8% at T1, t(139) = 4.09, p < .001, Cohen's d = 0.34, with a mean difference of 53%). At T2, missed appointments were more common for telepsychiatry than any other contact modes, averaging 11.3% of all booked appointments at T2, compared to 0.7% for face to face or 1.1% for telephone appointments. Finally, appointments at T2 were characterized by a decrease in case management and crisis intervention, but an increase in psychiatric reviews, family work and functional recovery appointments (X ²[5] = 12.9, p < .001).

3.2. Client surveys

A total of 71 clients (50.7%) completed the surveys. Clients attended a range of 1 to 10 telepsychiatry sessions (1–4 sessions = 81.7%, 5–10 sessions = 18.3%). Figure 1 shows client responses to questions about their experience of telepsychiatry. Most (76%) agreed that it was easier to attend appointments, with the majority (72.2%) having no difficulty in accessing the online platform or concerns about confidentiality (69%). However, over one third (38%) found it difficult to find a private space for appointments and approximately 60% were undecided or disagreed that they were less anxious or found it easier to communicate with clinicians using telepsychiatry.

FIGURE 1.

Young people survey responses (% agreed, undecided, or disagreed)

Similar issues were raised in the open‐ended questions which produced 167 responses that were analysed thematically (Table 2). More benefits were listed than drawbacks (70.7% vs. 29.3%). The most common reported benefit referred to improved service access (72 responses) such as the convenience and removal of transport and scheduling barriers. One client stated “I like that I can do it from any location and don't have to worry about travelling to the clinic”. Some clients (34 responses) reported telepsychiatry improved their clinical experience, for example “I feel I can express myself with less pressure”, and that it increased their sense of control over the clinical setting and autonomy over their care (12 responses). The most common drawbacks (20 responses) referred to technological difficulties including internet access and lack of technical support. Ten percent (16 responses) reported a negative service experience, with comments highlighting isolation, lack of human connection, or that “it doesn't feel real”. Some clients raised privacy and confidentiality issues and communication difficulties.

TABLE 2.

Themes from client surveys (N = 167), n (%)

Benefits (n = 118)
1. Improved service access	72 (43.1%)	2. Improved service experience	34 (20.0%)	3. Client control over clinical settings	12 (7.2%)
Ease of service access & convenience Removes transport/scheduling issues Pandemic appropriate	46 22 4	Positive experience Efficient & flexible Provides additional engagement option Decreases anxiety Technology benefits Easier to express self	8 7 7 6 3 3	Improves comfort Can choose own environment Seen in own surroundings	8 3 1

Drawbacks (n = 49)
1. Technological difficulties	20 (12.0%)	2. Negative service experience	16 (9.6%)	3. Confidentiality & communication challenges	13 (7.8%)
Internet issues Difficulties with use of “Zoom” Lack of technical support	12 6 2	Increases isolation/detached Preference for face to face Negative experience Feeling self‐conscious	5 5 5 1	Privacy & confidentiality concerns Communication difficulties	7 6

A greater percentage of clients indicated preference for face‐to‐face appointments (n = 30, or 42.2%) compared to telepsychiatry‐only appointments (n = 14, 19.7%), although 36.6% (n = 26) preferred a combination of face‐to‐face and telepsychiatry and one person was undecided. Analyses were conducted into factors which contributed to differences between groups. Those who preferred face‐to‐face appointments were significantly younger than other clients (mean = 20.6 vs. 22.1 years, t[69] = 2.5, p < .01). Linear regressions showed that age was a significant predictor and explained 8% of the total variance (Table 3). Negative responses to the following questions—“it is easier to talk about myself, my worries and my emotions using videochat” and “I am less anxious because I can videochat from home or another safe place”, contributed to a further 41% of the variance, suggesting that younger clients found it harder to talk about themselves and were more anxious when using telepsychiatry. After controlling for the effects of age, these still predicted an additional 33% of the variance.

TABLE 3.

Regression analysis: Predictors of preference for face‐to‐face interactions

	Predictors	Beta (95% CI)	R‐square change before and after predictor added
Demographic	Age	−0.29 (0.01–0.10) μ	0.08 α	n/a
Survey questions	It is easier to talk about myself, my worries and my emotions when using videochat	−0.34 (0.03–0.25) α	0.41 μ	0.33 μ
	I am less anxious because I can videochat from home or another safe place	−0.31 (0.02–0.25) α

^μ p < .001. Significance values from the final model.

^α p < .01.

3.3. Clinician survey

A total of 27 clinicians completed the surveys (81% females, 53% aged 21–40). Figure 2 shows clinician responses to questions about their experience of telepsychiatry. Most (85.2%) agreed telepsychiatry improved time management and increased accessibility to clients. However, almost half (48.1%) had difficulty assessing mental state and felt the content of reviews were negatively impacted. Despite these challenges, two thirds (66.7%) disagreed they felt uncomfortable assessing risk and 44.4% disagreed that technological issues impacted on interactions with clients.

FIGURE 2.

Clinician survey responses (% agreed, undecided, or disagreed)

A total of 304 responses were provided in the open‐ended questions about clinician experience of telepsychiatry and were analysed thematically (Table 4). More benefits were listed than drawbacks (60.5% vs. 39.5%). Similar to the client survey, the most common benefit cited by clinicians was improved service access (78 responses). One clinician commented “Clients who struggle to get to the centre and work or study full time, are better able to attend appointments”. Some clinicians (67 responses) found that telepsychiatry improved service delivery and created a more flexible and contemporary service for young people. Another benefit reported was improved workplace satisfaction (39 responses) mainly due to increased efficiency and time management. The most common drawback (37 responses) referred to was decreased quality of therapeutic engagement, for example one clinician commented “some young people do not take it seriously as regard it as a chat” and another “it's difficult to build a relationship with a new client”. Some clinicians (31 responses) reported challenges using clinical and therapeutic skills including conducting mental state examinations, managing acutely unwell clients and assessing risk. Other clinicians (25 responses) raised concerns over a lack of specific telepsychiatry skills including boundary setting, for example “there is an assumption that clinicians are able to navigate the system of delivering telehealth” and “one young person continued to play playstation throughout the appointment”. Some clinicians raised technological difficulties including access inequality for clients and a small proportion reported occupational health and safety issues.

TABLE 4.

Themes from clinician survey (N = 304), n (%)

Benefits (n = 184)
1. Improved service access	78 (25.7%)	2. Improved service delivery	67 (22.0%)	3. Improved workplace satisfaction	39 (12.8%)
Ease of service access & convenience Removes travel/scheduling issues Facilitates contact/geographical reach Pandemic appropriate	39 22 12 5	Additional tool of engagement Flexible service delivery Modernizes service	52 10 5	Increases efficiency & time management Creates flexible work environment Less commuting/environmental benefits Increased morale & work/life balance	21 6 8 4

Drawbacks (n = 120)
1. Decreased quality of engagement	37 (12.2%)	2. Restricted clinical/therapeutic skills	31 (10.2%)	3. Lack of telepsychiatry skills	25 (8.2%)
Affects depth/flow of interaction Difficulties with rapport Increases anxiety for some clients Higher non‐attendance rate	18 8 7 4	Mental state/nonverbal cues challenges Challenges managing acute clients Difficulties conducting therapy Risk assessment/management issues Physical health screening challenges	15 5 5 4 2	Lack of specific clinical skills Difficulties in boundary setting Challenges assessing client suitability Issues managing privacy & confidentiality	10 6 6 3

4. Technological Difficulties	17 (5.6%)	5. Occupational Health & Safety Issues	10 (3.3%)
Internet connection issues Access inequality Lack of technical support/training	7 6 4	Increased Fatigue/sedentary/screen time Workplace environment limitations	7 3

Overall clinicians preferred a combination of face‐to‐face, telepsychiatry and telephone interactions (n = 20, 74.1%), while 25.9% (n = 7) preferred face‐to‐face appointments only and there was no preference for telepsychiatry alone.

4. DISCUSSION

Our study showed an increase in service activity associated with telepsychiatry use, however there was an increase in missed appointments by young people. The surveys of clients and clinicians highlighted many positive experiences with telepsychiatry however also identified challenges and limitations impacting on its use. Our study also demonstrated the need for flexible implementation including switching to face‐to‐face appointments for some clients and clinical situations.

As expected, improved service access was the main benefit of telepsychiatry reported by both clinicians and young people, and this is in keeping with established literature (Kruse et al., 2017; Uscher‐Pines, Raja, et al., 2020). Telepsychiatry overcomes known barriers to service access including difficulties with travel and transport, time constraints and competing priorities (Lal et al., 2020), and therefore has the potential to enhance engagement. While our study showed an increase in service activity which may in part be explained by improved service access, there was a higher percentage of missed appointments. This differs from previous studies which suggest that with telepsychiatry, clients are more likely to keep appointments and it may increase attendance rates by improving accessibility to services (Chaudhry et al., 2021; Leigh et al., 2009).

Sezgin et al. (2021) suggests that loss of engagement is a leading cause for cancellation of telehealth appointments. Our survey responses show that telepsychiatry still appears to be outside the narrative and comfort level of young people, with 10% reporting a negative experience with feelings of isolation and lack of human connection. In particular, younger clients were more anxious when using telepsychiatry and reported it was harder to talk about themselves. These negative experiences may have contributed to the higher number of missed appointments in our study. In addition, clinicians reported that telepsychiatry was not taken as seriously by young people and there were difficulties establishing rapport and maintaining depth and flow of the interaction. Clinicians found these challenges impacted on the quality of therapeutic engagement. It is important to note that during our study, young people were experiencing limited social connections due to a global pandemic, and this may have impacted on their experience with telepsychiatry.

Interestingly, young people described technological difficulties as the main drawback to telepsychiatry, which was unexpected given this population are exposed to technology from a young age (Lal et al., 2015), and are accustomed to making connections through online platforms (Khan & Ramtekkar, 2019). Young people described internet issues, difficulty using the online platform and lack of technical support. Clinicians agreed with these difficulties but also raised concerns regarding access equality. This suggests that services must consider addressing technology challenges in order to support telepsychiatry delivery.

Past studies have raised privacy and confidentiality issues with telepsychiatry (Lal et al., 2020; Uscher‐Pines, Sousa, et al., 2020), however in our study this was not identified as a major limitation to its use.

Importantly, telepsychiatry was not associated with increased crisis presentations, which suggests it can be incorporated into early psychosis service models and maintain the delivery of multidisciplinary services without observing an increase in crisis interventions. However, telepsychiatry was not able to completely replace face‐to‐face interactions. One in six appointments still required face‐to‐face assessment, predominantly when managing acutely unwell clients or when there were risk concerns. Two thirds of clinicians indicated in the survey they were comfortable assessing risk using telepsychiatry and only some expressed uncertainty managing acutely unwell clients remotely. Interestingly, this did not completely translate into practice. Almost half of all clinicians reported difficulty in assessing mental state, behavioural cues and non‐verbal body language which is consistent with previous literature (Norman, 2006; Uscher‐Pines, Sousa, et al., 2020). Our findings suggest that flexibility when implementing telepsychiatry is important including the use of face‐to‐face appointments as clinically indicated.

Studies indicate that clinician acceptance and attitudes are crucial to telepsychiatry success (Lal et al., 2020; Wade et al., 2014). In our study, while clinicians described improved workplace satisfaction with flexibility, efficiency and better time management, two thirds reported feeling uncomfortable or undecided using telepsychiatry. Most clinicians indicated they would incorporate telepsychiatry as another tool, but none indicated they would solely use it in practice. Clinicians felt they lacked training in specific telepsychiatry skills such as assessing client suitability, boundary setting, carrying out online assessments and managing privacy and confidentiality issues. Our study highlighted that for successful introduction of telepsychiatry, it is vital to build clinician confidence by provision of specific training in telepsychiatry.

This study demonstrates that it is possible to incorporate a telepsychiatry model within an early psychosis service. Telepsychiatry resulted in increased service activity however due to increased missed appointments and difficulties with therapeutic interactions, there was no observable increase in engagement. Contrary to expected findings, clinicians and young people did not wholly endorse the use of telepsychiatry despite both groups reporting positive experiences.

There were limitations to this study. The pandemic and the introduction of telepsychiatry were intrinsically linked and could not be independently assessed, so the above results must be regarded with caution. However, the survey questions were designed for participants to reflect specifically on telepsychiatry as a platform, and our findings help to build knowledge regarding implementation of telepsychiatry into youth mental health services.

While telepsychiatry has the potential to change the way we deliver mental health services in the future, more research is required regarding long‐term outcomes and developing specific telepsychiatry training and skills.

FUNDING INFORMATION

The authors received no financial support for the research, authorship and/or publication of this article.

CONFLICT OF INTEREST

The authors declare no conflicts of interest.

Randall, L.‐A. , Raisin, C. , Waters, F. , Williams, C. , Shymko, G. , & Davis, D. (2022). Implementing telepsychiatry in an early psychosis service during COVID‐19: Experiences of young people and clinicians and changes in service utilization. Early Intervention in Psychiatry, 1–8. 10.1111/eip.13342

DATA AVAILABILITY STATEMENT

The storage, retention and disposal of research data complies with relevant privacy, ethical and publication requirements. Data is not available for public sharing in line with clinical governance policies.