Patient satisfaction and acceptability with telehealth at specialist medical outpatient clinics during the COVID‐19 pandemic in Australia

Abstract

Background

Outpatient clinics were shifted rapidly to telehealth in Australia during the Coronavirus disease 19 (COVID‐19) pandemic, drastically altering patient care and experience.

Aims

To investigate patient satisfaction and acceptability of telehealth consultations during the COVID‐19 pandemic.

Methods

Prospective observation study conducted in two hospital rheumatology outpatient departments (OPD) undertaking telehealth consultations during COVID‐19. A modified version of a validated telehealth evaluation survey was posted to all patients attending the telehealth OPD rheumatology clinics, including balanced 5‐point Likert scales and free‐text responses. Cluster analysis was applied to the Likert‐scale questions, alongside thematic analysis of free‐text responses.

Results

There were 128 respondents (29% response rate), of which 69.5% were women and the majority (87.5%) was aged 50 years or older. All telehealth consultations were conducted by telephone. Nearly one‐fifth of patients indicated consistent dissatisfaction with telehealth across the range of questions. These patients were older, reported lower educational qualifications and lower health literacy scores and lacked access to the Internet. While many patients found this mode of consultation to be convenient, patients expressed concerns regarding absence of physical examination. A recurrent theme was a desire for a mixed‐model clinic in the future, with flexibility of having both telehealth and face‐to‐face consultations.

Conclusions

This study offers unique insights into patients' experiences with telehealth, which until the current global pandemic, has been an uncommon mode of consultation delivery in urban areas. This study suggests when defining the place of telehealth in future healthcare delivery, patient perspective and careful patient selection will be key. Disease progression, language and cognitive ability, health literacy, technology access and patient and clinician preference are important considerations when deciding how effectively to embed and integrate telehealth into consultations.

Introduction

Australian outpatient clinics were moved rapidly to a telehealth model due to the current Coronavirus disease 19 (COVID‐19) pandemic, as a harm‐reduction method for both patients and clinicians. Telehealth is defined as the use of telecommunication techniques for providing telemedicine; this includes voice calls, images and videoconferencing to provide health.¹ Globally, there has been increased interest in telehealth in response to the COVID‐19 pandemic.², ³ Telehealth has been deployed as an emergency strategy⁴ and it appears that many outpatient clinics,³, ⁵ specialist or in general practice, have taken a similar approach.⁶

Telehealth has previously been largely inaccessible in urban populations in Australia for a range of reasons, including lack of appropriate technology, lack of consumer take‐up, low clinician interest and lack of Medicare reimbursement, many of which were quickly resolved in the pandemic. Telehealth has many advantages: greater accessibility for rural and regional patients;⁷ greater accessibility for those who work or have difficulty accessing appropriate and affordable transport;⁸ and cost‐reduction.⁹ However, studies¹⁰, ¹¹, ¹², ¹³ have cautioned that this mode of delivery may be inappropriate or unacceptable for certain patients, or certain aspects of care. The rapid and unexpected move to telehealth clinics, which would traditionally involve a long‐planned change, is a previously unstudied phenomenon. Effectively all appointments were moved involuntarily into the virtual space, meaning patients who may traditionally reject telehealth, underwent this experience, which the study aimed to capture.

Telehealth is being increasingly advocated as a method of equalising healthcare access and as a cost reduction strategy.⁹ However, evaluation of its acceptability and effectiveness is needed prior to its further implementation outside the context of this current pandemic. Healthcare delivery is only appropriate if it is acceptable, accessible and affordable for its consumers, the patients.

The aim of this study was to investigate patient satisfaction and acceptability of telehealth consultations at the rheumatology outpatient departments of two hospitals.

Methods

Participants

The Queen Elizabeth Hospital (TQEH) and the Royal Adelaide Hospital (RAH) are tertiary hospitals in metropolitan Adelaide, South Australia. They predominantly service urban communities in the central and western areas of Adelaide. In response to the COVID‐19 pandemic, rheumatology outpatient consultations at both hospitals were required to move to delivery by telehealth in early March 2020 for the majority of patients as a harm‐reduction method within a predefined health network pandemic response plan. All patients (n = 438) who attended a telehealth clinic between 26 March 2020 and 27 April 2020 (n = 235) at TQEH rheumatology outpatient department, and between 7 April 2020 and 17 April 2020 (n = 203) at the RAH rheumatology outpatient department, were mailed paper‐based surveys. This utilised the existing general patient satisfaction survey system, which accompanies prescriptions and appointment cards in the postal system. Completed surveys were mailed back using a postage‐paid envelope to the rheumatology units and data were then entered into a database (SurveyMonkey software) by research assistants. Identifying details were not included in the survey (Table 1).

Table 1.

Patient demographics

	Percentage	Frequency
Age group (years)
18–49	11.9	15
50–59	23.0	29
60–69	30.2	38
70–79	23.0	29
80 and over	11.9	15
Gender, female	69.8	88
Highest level of education
Bachelor degree or higher	16.7	21
Diploma or Trade certificate	15.9	20
TAFE/apprenticeship	6.3	8
Completed high school	15.9	20
Some high school	34.1	43
Finished primary school	6.3	8
Location of appointment
Royal Adelaide Hospital	56.3	71
The Queen Elizabeth Hospital	37.3	47
Missing	6.3	8
Requires interpreter at appointment, yes	4.8	6
Rheumatic disease
Rheumatoid arthritis	42.9	54
Connective tissue diseases	21.1	27
Osteoarthritis	17.5	22
Non‐articular rheumatism	13.5	17
Other	11.9	15
Psoriatic arthritis	11.9	15
Gout	9.5	12
Vasculitis	4.0	5
Ankylosing spondylitis	3.2	4
Enteropathic arthritis	1.6	2
Currently receiving biologics, yes	16.7	21
How often do you search the Internet for health information?
Always	4.0	5
Sometimes	19.8	25
Usually	4.8	6
Rarely/Never	69.0	87
Missing	2.4	3
How often do you need to have someone help you when you read instructions, pamphlets or other written material from your doctor or pharmacy?
Always	7.9	10
Sometimes	7.1	9
Often	4.8	6
Rarely	12.7	16
Never	65.1	82
Missing	2.4	3

Survey

The patient satisfaction survey was a modified version of the Telemedicine Satisfaction and Usefulness Questionnaire,¹⁴ a validated survey for the evaluation of telehealth clinics (Supporting Information Appendix S1). Patient satisfaction with telehealth was assessed using 18 multiple‐choice questions and four free‐text questions. Balanced 5‐point Likert scales were used to explore themes of confidence, health status, communication, confidentiality, physical examination and convenience. Health literacy was assessed using the Single Item Literacy Screener (SILS), a single‐item question intended to identify adults in need of help with printed health material. The SILS asks, ‘How often do you need to have someone help you when you read instructions, pamphlets, or other written material from your doctor or pharmacy?’. Possible responses were: 1, Never; 2, Rarely; 3, Sometimes; 4, Often; and 5, Always. Scores greater than 2 were considered positive, indicating some difficulty with reading printed health‐related material. In addition, respondent demographics (five questions), details on rheumatological health (two questions) and technology access (five questions) were recorded. Four questions with a free text response area were included to capture further details about patients' experiences of telehealth (Table 2).

Table 2.

Technology access and telehealth experience

	Percent	Frequency
Computer access
Yes, with Internet access	56.3	71
Yes, with no Internet access	4.8	6
No	38.1	48
Missing	0.8	1
Smartphone/tablet access, yes	61.1	77
Previous telehealth experience, yes	23.8	30
If experienced with what service(s)
GP	15.9	20
Other specialist	7.1	9
Other rheumatologist	1.6	2
Number of telehealth consultations with this rheumatology service
1	73.0	92
2–5	15.9	20
Over 5	5.6	7
Missing	5.6	7

Statistical analysis

Using spss (version 9.6.0.0), numerical data were described using mean and standard deviation (SD) when approximately normally distributed and using median and interquartile range when skewed. Categorical variables were described using absolute and relative frequencies. To determine if there were distinct groups of respondents with different responses to the patient satisfaction survey, we undertook a cluster analysis. Partitioning of the participants was performed using a Gower distance matrix for ordinal data and a partitioning around medoid (PAM) iterative clustering procedure. Silhouette width, which is an aggregated measure of how similar an observation is to its own cluster compared its closest neighbouring cluster, was used to define the optimal number of clusters. Analysis was performed using R version 3.6.3¹⁵ and the ‘cluster’ package.¹⁶

Qualitative analysis of free text data

The open‐ended questions were designed to collect further details about patient experiences of the new service model and expand data tied to the closed questions in the survey. These data were analysed using the principles of thematic analysis stated by Terry et al.,¹⁷ which promote an inductive focus on patients' comments. Analysis began with familiarisation of the data individually by both clinical and non‐clinical researchers. Open coding of the transcripts was used to account for multiple perspectives and themes (related to emotions, values and experiences). Nvivo software (version 12.6.0) was used to support this thematic analysis, and the 11 themes identified were further interpreted using a theoretically informed and empirically tested framework recently developed by Greenhalgh et al.¹⁸ This framework is designed to support the design and delivery of technological innovations in healthcare by identifying seven key domains that interact to influence the non‐adoption, or abandonment, or non‐sustainability (NASSS) of these innovations.

The survey was registered as a Quality Improvement Project and then approved by Central Adelaide Local Health Network Human Research Ethics Committee (Reference Number 13525).

Results

In total, 128 patients completed the satisfaction survey, with a response rate of 29.2% (35.0% RAH, 21.3% TQEH; Table 1). All consultations were conducted via the telephone (including mobiles) with no videolink consultations. Therefore, this study only investigates telephone consultations

Most (69.5%) respondents were women (Table 1), and the majority (87.5%) was aged 50 years or older; 60–69 years was the most commonly reported age group. Rheumatoid arthritis was the most commonly reported condition (43.8%). Most (53.9%) patients had either completed high school and/or received higher education. Overall, 19.8% reported requiring help with reading instructions, pamphlets or other written material from their doctor or pharmacy either sometimes, usually or always, indicating some difficulty with reading printed health related material. Searching the Internet for health information was uncommon, with 69.5% reporting never or rarely doing this. More than half (56.3%) of the respondents had access to a computer with Internet access (Table 2), but 38.3% did not have access to a computer. A larger percentage (61.7%) could access a smartphone/tablet. For most (71.9%) patients this was their first experience with telehealth, and if they had used telehealth before it was most commonly at the practice of their general practitioner (15.6%; Table 2).

Overall, 48.3% of respondents would continue to use telemedicine after COVID‐19, and 61.7% agreed or strongly agreed when asked, ‘In general, I am satisfied with the telemedicine system’ (Fig. 1). Patients generally agreed that there was little compromise of communication or the relationship with their clinician, although they did not find telehealth as satisfying as talking in person. In contrast, patients were less likely to agree that that they were involved in their care and less likely to manage healthcare needs and follow doctor's advice. For example, only 28% of respondents agreed or strongly agreed they were more involved in their care than before using telehealth, and only 16% of respondents agreed or strongly agreed that their health was better than before using telehealth.

Figure 1.

Patient responses to patient satisfaction survey. (), Strongly disagree; (), disagree; (), neutral; (), agree; (), strongly agree.

Using the cluster analysis, two distinct clusters were identified, designated Gp 1 (104/127, 82%), and Gp 2 (23/127, 18%). The smaller group, Gp 2, indicated significant dissatisfaction with telehealth across the range of questions asked (Fig. 1). None in Gp 2 would continue to use it after COVID‐19 (Fig. 2), compared with 71% in Gp 1. Further the majority of Gp 2 patients do not have access to a computer with Internet or smart phone, so videolink telehealth is not a viable option to improve their telehealth experience. These patients tend to be older and tend to be more poorly educated with a higher proportion having poor health literacy (Fig. 3).

Figure 2.

Telehealth survey questions: Q14–Q31 by cluster (Gp 1 and Gp 2). (), Strongly disagree; (), disagree; (), neutral; (), agree; (), strongly agree.

Figure 3.

Demographics and other relevant variables by cluster.

In the survey free text comment section, 96% of respondents left at least one comment; a total of 384 comments was received across the four free text questions. Eleven themes were identified, with 23 sub‐themes related to patient satisfaction and acceptability of telehealth. All themes and corresponding sub‐themes appear in Table 3 and are presented in relation to the domains identified in the NASSS framework. The 11 main themes are illustrated by the following quotes from the free text questions in Table 4.

Table 3.

How the qualitative themes and subthemes fit with Greenhalgh's non‐adoption, or abandonment, or non‐sustainability (NASSS) domains

NASSS Domains	Themes	Sub‐themes	Number of times domain identified in analysis
Medical condition	1. Disease progression 2. Physical examination requirements	1. New diagnosis 2. Flare‐up 3. New medication 4. Fear aspect of condition will be missed due to lack of physical examination	51
Technology	3. Method of telehealth delivery 4. Privacy	5. Sound quality 6. Lack of access to Internet or video‐conferencing device 7. No caller ID 8. Concerned about being overhead 9. Background noise	11
Value of service to certain sub‐populations of patients (value proposition)	5. Vulnerable populations 6. Workers 7. Availability of services during COVID‐19 pandemic	10. Homeless population 11. Patients living with a disability 12. Patients who are carers or receiving services from carers 13. Patients from rural or regional areas 14. Patients who require interpreting services 15. Fear of contracting COVID‐19 in healthcare setting	42
Barriers to adoption of services	8. Difficulty communicating 9. Relationship between patient and clinician	16. Cognitive overload 17. Difficulty conveying condition severity, condition's impact, physically and emotionally 18. Rapport with treating clinician	47
Organisation	10. Logistics	19. Timing of consultation 20. Posting of blood forms, prescriptions 21. Convenience	90
Embedding and adaptation over time (organisational resilience)	11. Mixed model clinic structure (telehealth and face‐to‐face clinic simultaneously run)	22. Able to see doctor if condition deteriorates 23. Able to see doctor annually/bi‐annually for examination and ‘check‐in’	40

Table 4.

Illustrative comments

Theme	Illustrative comments
Disease progression	‘I've only recently been diagnosed & am uncertain about my condition, treatment & prognosis & so am very grateful to receive expert advice at a stage where I'm just learning about psoriatic arthritis.’ (Female, 60–69 years)
Physical examination requirements	‘I really needed to show my hands and feet to the doctor.’ (Female, 60–69 years)
	‘I would think from visual clues from being together like how much I'm limping.’ (Female, 40–49 years)
	‘The doctor cannot rule out the problem with my health.’ (Female, 50–59 years)
Privacy	‘Privacy issues, as I was not somewhere private at the time of the call.’ (Female, 30–39 years)
	‘I was at work and being a receptionist, it is difficult to have personal phone calls.’ (Female, 60–69 years)
Vulnerable populations	‘As I'm in a wheelchair, it's easier for me to remain at home when I need accessible toilet facilities. Even disabled person's public toilets can be difficult to use as transferring can be problematic and can increase pain levels.’ (Female, 60–69 years)
	‘must leave house very early for morning appt's (to avoid overcrowded train carriages during peak hour. This means I must cancel my carer so I can get to train on time.’ (Female, 60–69 years)
Availability of services during COVID‐19 pandemic	‘Thank you for conducting the appointment over the phone instead of postponing till after the current crisis.’ (Female, 60–69 years)
	‘During the COVID 19 period the less face to face appointments the better.’ (Male, 60–69 years)
Relationship between patient and clinician	‘every time I leave the clinic my hope that I can keep going is renewed. There is an indefinite quality that I get from seeing my doctor in person which I only get to do 2–3 times per year, that cannot be replaced.’ (Female, 40–49 years)
	‘No rapport or physical examination. Feel pressured to get consult done.’ (Female, 50–59 years)
	‘on the phone basically only determined if I was worse than before then moved on. I need eye contact. I need to know I'm doing ok. I felt worse emotionally after the phone call, then if I hadn't had it at all.’ (Female, 40–49 years)
	‘What I did not like it was very impersonal.’ (Male, 50–59 years)
Interest in a mixed‐model (telehealth and face‐to‐face) clinic system in the future	‘happy to continue to use telemedicine after COVID‐19, provided I did not have a physical ailment that needed to be looked at.’ (Male, 50–59 years)
	‘I feel that telemedicine could be used in conjunction with in‐person appointments. I feel more comfortable speaking in person with a doctor.’ (Female, 60–69 years)
	‘I would continue to use, but with the option to see doctor at regular times between phone consults.’ (Male, 60–69 years)
	‘Would want a face to face consultation if my condition worsened.’ (Male, 70–79 years)
	‘I believe there is a strong case for a mix of personal visits where the doctor can glean nuances of facial expression and body language expression on occasion support by telemedicine appointments.’ (Female, 60–69 years)

Discussion

This study focussed on patient experience and satisfaction of telehealth (in this case, telephone consultations) in outpatient rheumatology clinics during the COVID‐19 pandemic. It captured patient experiences during a stressful, unplanned and unexpected global time and healthcare system change. The study found telehealth is acceptable and preferred in certain groups of patients, largely driven by convenience. However, a significant number of patients had negative experiences and felt that their health was impacted, with cognitive overload and anxiety about health information being overlooked being of particular concern for some patients. Furthermore, patients advocated for a flexible mixed delivery mode for clinical consultations, where care could be delivered both face‐to‐face, and using a telehealth health clinic, negotiated by the patient and their treating clinician. This would allow the positive aspects to be balanced against negative experiences of some patients and contribute to a more patient‐centred healthcare delivery system.

The mixed level of satisfaction with telehealth in this study differs from other studies in this field. The rapid and all‐inclusive nature of how this programme was rolled out may account for this. Additionally, in Australia the majority of studies surrounding telehealth have been completed in rural areas. Two studies of rheumatology telehealth clinics completed in rural Australia had patient satisfaction rates of 89% and 98%,¹⁹, ²⁰ significantly greater than the 61% in this study, suggesting urbanality has a significant impact on telehealth satisfaction, similar to results seen overseas.²¹, ²² A recent, large study of older Australians found 49% of people surveyed would continue to use telehealth following the COVID‐19 pandemic.²³ This study was a public survey online, potentially representing a different, more‐technologically literate, patient population than this study. The difference may also reflect that Australian rural and urban populations have different attitudes and experiences to telehealth in a rheumatology setting, and COVID‐19 forced many urban patients into telehealth who would never previously have used this system. This could not be confirmed from these data as the responses were de‐identified. Interestingly, there was little criticism of technology or the logistics of the telehealth service, which was anticipated given the rapidity of implementation and the use of a familiar method of communication. A further contributing factor may be that in the majority of these studies, clinic was conducted over videolink rather than by telephone, as in this one. A study from the UK found, patient satisfaction was 56% for a telephone consultation compared with 90% in video and 97% in face‐to‐face consultations.²⁴ This suggests that had video consultations occurred, as was planned, levels of patient confidence and satisfaction may have been higher. However, 27% of patients did not have access to appropriate technology meaning that video consultation is not a viable option for many patients. Lack of patient access to appropriate technology is a major barrier for large‐scale telehealth roll‐out and needs to be strongly considered by policy makers. A further aspect to consider when evaluating the satisfaction with telehealth in this study is that many patients were grateful to receive care during the COVID‐19 pandemic and considered this to be the safest and only option. Therefore, their experience was sufficiently satisfactory during that time, but this may not be applicable post‐COVID‐19. Six‐month follow up of a teledermatology clinic found patients wanting to return to telehealth clinic had reduced 33% from immediate follow up.²⁵ A large contributor to this was lack of examination.

A strong positive theme of this study was increased convenience and time saved. This mirrors other studies¹⁹, ²⁶ that found reduced travel time and reduced time off work were recurring themes. This is reflective of healthcare access being a critical point in overall patient healthcare experience,²⁷ also with potential economic benefits to patients.²⁸ Vulnerable populations, such as those with a disability or who live in rural or remote areas, appear to be more likely to want to return to telehealth clinic. A systematic review found telehealth empowers and improve outcomes for people living with disabilities and their caregivers.¹² However, other vulnerable populations, such as those who require an interpreter, who were not well represented in this study, do not find telehealth offers added benefit, especially interpreting via telephone.¹¹, ²⁹

A worrying finding in this study is that only 16% of respondents agreed or strongly agreed that their health was better than before using telehealth. This is in contrast with other studies³⁰ that found patient‐reported health outcomes were no different between in‐person and telehealth rheumatology outpatient clinics. While this study did not assess an objective means, a study in the UK found diagnostic accuracy increased over 25% to 97% when telerheumatology services shifted to video consultations from telephone consultations.²⁴ Other studies have shown a diagnostic accuracy of 79% when using video consultation.²² Eighty‐one percent of patients in a video‐dermatology clinic believed their overall care to be satisfactory, compared to 90% in the face‐to‐face clinic.³¹ Another study²³ found 38% of patients were concerned by lack of examination. Telehealth did not seem acceptable to patients with complex or new disease, which mirrors findings in other studies.²¹, ³² There is little in the literature regarding patient cognitive overload within a teleconsultation context, nor contributing factors to this, suggesting that this is an area that requires further research.

Future and public health implications

As in previous studies, patients in this study frequently voiced a desire to have a choice between in‐person and telehealth consultations.¹⁹ Many patients expressed a desire for a face‐to‐face consultation annually or bi‐annually, with telehealth clinic between face‐to‐face visits, and the option of a face‐to‐face consultation available when needed, for example when they felt their disease was flaring. Similarly, a systematic review found that telehealth was of most benefit in chronic conditions when used remotely to monitor conditions or provide counselling, not in the diagnosis or treatment phase of care.³³ This suggests that hospitals and clinics need to continue permanent face‐to‐face facilities, and not rely on telehealth as the primary form of health service provision. This finding points to the need for patients and their treating clinicians to negotiate their individual needs related to appropriate use of telehealth consultations. Face‐to‐face consultations should be continued at regular intervals, with telehealth as an optional supplement. For some patients it may not provide optimal delivery of healthcare at any point, for either themselves or their clinician. There are many factors that will play into this decision, including disease progression, location, language and cognitive ability, health literacy, technology access and patient and clinician preference. Furthermore, the deployment of telehealth in our study was an emergency measure and the findings concerning the acceptability of telehealth may not be generalisable to a non‐pandemic context. If telehealth continues, patients should be re‐surveyed once COVID‐19 measures ease to gain a more accurate picture.

Limitations

A limitation of this study is the incomplete response rate; however, the respondents are broadly representative of the patient cohorts attending these two clinics. People with low English literary skills, due to either lower education or non‐English speaking backgrounds, were probably less likely to respond and therefore their experiences could not be considered. A study using culturally accessible methods is needed to ensure these vulnerable patients do not have their experiences overlooked. A further limitation of this study is that we did not appropriately assess accessibility of the clinic for certain patient populations, such as asking specifically about rurality and patients who are living with disability.

Additionally, we were not able to correlate the results with disease activity or status. Therefore, it is unknown whether patients with well controlled disease were more likely to be more satisfied with telehealth than patients with unstable or progressive disease. In addition, we were also not able to correlate the clinician's perception of the efficacy with the telehealth encounter in this study.

A strength of this study is as almost every patient was seen exclusively in telehealth clinic, it is a representative sample of the rheumatology patient population. Every patient had the opportunity to respond and the postal method meant that patients with lack of technology access were not excluded.

Conclusion

Telehealth, delivered as telephone consultations, provided to rheumatology patients during the COVID‐19 pandemic was an acceptable and satisfactory form of healthcare delivery in Australia. For many patients, telehealth was a positive experience and one they want to continue into the future. However, for others, it had negative self‐reported health impacts and caused distress. To deliver better patient centred care moving into the future telehealth should be considered as an adjunct, but not a full replacement, to face‐to‐face rheumatology care.

Supporting information

Appendix S1. CAHLN Rheumatology Outpatients Patient Satisfaction Survey.