Perception and satisfaction of patients after telemedicine urology consultations: A matched analysis with physicians’ perspective

Abstract

Introduction

During the first regional COVID-19 lockdown in March 2020, we conducted a study aimed at evaluating completeness of telemedicine consultation in urology. Of 1679 consultations, 67% were considered completely managed by phone. The aim of the present study was to assess patients’ experience and satisfaction with telemedicine and to compare them with urologists’ perceptions about quality and completeness of the telemedicine consultation.

Methods

We contacted a randomly selected sample of patients (n=356) from our previous study to enquire about their experience. We used a home patient experience questionnaire, inspired by the Patient Experiences Questionnaire for Out-of-Hours Care (PEQOHC) and the Consumer Assessment Health Profile Survey (CAHPS).

Results

Of 356 patients contacted, 315 agreed to complete the questionnaire. Urological consultations were for non-oncological (104), oncological (121), cancer suspicion (41), and pediatric (49) indications. Mean patient satisfaction score after telemedicine consultation was 8.8/10 (median 9/10) and 86.3% of patients rated the quality of the consultation as either excellent (54.6%) or very good (31.7%). Consultations regarding cancer suspicion had the lowest score (8.3/10). Overall, 46.7% of all patients would have preferred an in-person visit outside of the pandemic situation. Among patients whose consultations were rated suboptimal by urologists, almost a third more (31.2%) would have preferred an in-person visit (p=0.03).

Conclusions

Despite high reported patient satisfaction rates with telemedicine, it is noteworthy that nearly half of the patients would have preferred an in-person visit. Post-pandemic, it will be important to incorporate telemedicine as an alternative, while retaining and offering in-person visits.

Introduction

The COVID-19 pandemic has provided a unique opportunity for telemedicine to be deployed in healthcare systems. Studies involving several medical specialties have demonstrated excellent patient and physician satisfaction rates with telemedicine consultations;^1–4 however, there is a paucity of data on satisfaction among urologic subspecialties and the potential association between patients’ and doctors’ perspectives.

We conducted a prospective, multisite study involving all 18 urologists practicing in the region of Quebec City, Canada, asking them after each telephone appointment if it translated into a complete (CCM), incomplete (ICM), or suboptimal case management (SCM, adequate only in the context of the pandemic).⁵ This study was performed during the first four weeks of complete regional confinement (March 23 to April 16, 2020), while only patients with emergency situations were seen in person. We have previously reported health providers’ perception after telemedicine appointments and have shown that 67% of the visits were considered as CCM.⁵

In this second phase of the study, we wanted to determine how care delivered through telemedicine in urology meets patients’ clinical needs and if it offers them a positive experience, recognizing that more positive care delivery experience has been associated with higher compliance and better health outcomes.^6,7 Here, we report patients’ experience and satisfaction regarding telemedicine consultation in the Quebec City urology telemedicine study and report potential discrepancies with physicians’ opinions.⁵

Methods

Objective

Specific objectives of the present study were to evaluate quality of patients’ experience and overall satisfaction with telemedicine and explore if there was an association between patient experience and satisfaction and: 1) the type of urology visit (oncology, non-oncology, suspicion of cancer, or pediatric); 2) the urologist’s opinion about the completeness of consultation; and 3) the patient’s home proximity to the hospital.

Initial cohort from mother study

Between March 23 and April 16, 2020, all 18 urologists from the Quebec City area were required to manage patients by telemedicine when immediate intervention was not needed. Physicians completed a questionnaire after a telephone appointment with their patients. The types of visits included new consultations and followups and covered all urology subspecialties and all practice locations (hospital clinics, cancer center, private clinics). Consultation types were subdivided into non-oncology, uro-oncology, cancer suspicion, and pediatric.

In the first study, we asked urologists after each telemedicine visit to assess the interaction as either: 1) ICM, further necessitating an in-person visit; 2) CCM; or 3) SCM, otherwise adequate during COVID-19 pandemic. In the present study, we contacted a randomly selected sample of 356 patients from the same cohort (N=1679) to enquire about their perspective on their experience and satisfaction levels with their phone consultation using a home questionnaire.

Assessment measure

We used a French adaptation of a questionnaire inspired by two validated instruments: the Patient Experience Questionnaire for Out-of-Hours Care (PEQ-OHC)⁸ and the Consumer Assessment of Healthcare Providers and Systems (CAHPS^®) adult visit questionnaire.^9,10 The questionnaires were completed via phone survey. Additional questions aimed at assessing quality of patients’ experiences and perception of telephone consultations were developed for this study by an expert (LB) from our local patient experience office, in accordance with our organization’s patient experience framework.^11–13 The final version of our questionnaire comprised a total of 16 items aimed to assess:

1. Patient’s preference regarding telephone consultation outside the pandemic (questions 1, 2)

2. Quality of experience (questions 3–7)

3. Logistical characteristics (proximity, transportation, and need for a companion) (questions 8–12)

4. Patient’s overall impression and global satisfaction using a scale grading satisfaction from 1 (lowest satisfaction possible) to 10 (highest satisfaction possible) (questions 13, 14, and 16).

5. Patients’ opinion about video option (question 15)

The questionnaire is available in the online Appendix (at cuaj.ca).

Sampling

To adequately represent each type of case (non-oncological, oncological, cancer suspicion, or pediatric), we calculated (using PASS 13 Power Analysis and Sample Size Software, 2014 NCSS, LLC. Kaysville, UT, U.S.) our total sample size of >240 based on our previous sample, with 67% CCM noted by urologists and a 95% confidence interval width of 12%. So, we needed 76 non-oncological, 75 oncological, 41 cancer suspicion, and 48 pediatric patients. We used systematic randomization to select patients within the four different groups.

After our local ethics committee’s authorization, patients were contacted by phone and verbal consents were obtained to participate in the study.

Statistical analyses

Quantitative variables were described as means with 95% confidence interval (95%CI), median, and interquartile range (Q1, Q3). Descriptive variables were presented as frequencies, percentages, and Clopper-Pearson exact 95% CI of percentage. The Wilcoxon Mann-Whitney or Kruskal-Wallis tests were used for continuous data comparisons; Chi-squared or Fisher exact tests were used for categorical data. In case of multiple comparisons, Bonferroni correction of p-values were applied. Statistical analyses were performed using SAS Statistical Software v.9.4 (SAS Institute, Cary, NC, U.S.) with a two-sided significance level set at p<0.05.

Results

Cohort

From July 2020 to October 2020, 356 patients were contacted by phone and 315 agreed to complete our patient experience questionnaire. Table 1 reports the type of case and completeness of consultation according to the physicians’ impression in the actual subanalysis cohort compared to the whole cohort in the mother study (1679 consultations). Both cohorts were statistically different according to the relative type of cases (p<0.01) but not according to percentage of CCMs (p=0.22). In this cohort, 65.1% of the telephone consultations were considered CCM and 30.8% of consultations were considered SCM but adequately managed in the pandemic context.

Table 1.

Comparison of mother study cohort and sub-study cohort for the type of cases and completeness of consultation according to doctor’s opinion⁵

Type of cases (p<0.01)	Initial cohort*		Actual cohort
Non-oncological	686	40.9%	104	33.0%
Oncological	629	37.5%	121	38.4%
Cancer suspicion	104	6.2%	41	13.0%
Pediatric	142	8.5%	49	15.6%
Not classified	118	7.0%	0	0.0%
Total	1679	100.0%	315	100.0%
Completeness of consultation according to urologists (p=0.22)
Complete case management (CCM)	1135	67.6%	205	65.1%
Suboptimal case management (SCM)	455	27.1%	97	30.8%
Incomplete case management (ICM)	73	4.4%	9	2.9%
Unknown	16	1.0%	4	1.3%
Total	1679	100.0%	315	100.0%

^*Turcotte and al.⁵

More than 80% of patients reported living less than one hour from the facilities and they usually used a car to reach the place of consultation (Table 2). As much as 31.4% of patients usually needed to be driven to the consultation site and if a companion was needed, 34.7% had to miss a day of work to accompany the patient (Table 3).

Table 2.

Patients’ experience and satisfaction about telemedicine

	n	Mean (95% CI)	Median (Q1, Q3)
Out of 10, what is your satisfaction level regarding the phone consultation with your urologist? (Q15)	313	8.8 (8.7, 9.0)	9.0 (8.0, 10.0)
	Excellent	Very good	Good	Fair	Bad	No response
What was the quality of the phone consultation you had with your urologist? (Q14)	172 (54.6%)	100 (31.7%)	30 (9.5%)	10 (3.2%)	1 (0.3%)	2 (0.6%)
	Yes, always	Sometimes	No	No response
If the option would be available in the future, would you be interested to have a telephonic followup instead of an in-clinic appointment with your urologist? (Q13)	120 (38.1)	155 (49.2%)	39 (12.4%)	1 (0.3%)
	Yes	Maybe	No	No response
If there was no COVID-19 pandemic, would you prefer to see your urologist in person? (Q1)	147 (46.7%)		168 (53.3%)
According to you, is phone consultation a good option for medical followup when patients don’t need to be physically examined? (Q2)	265 (84.1%)	39 (12.4%)	4 (1.3%)	7 (2.2%)
During your phone consultation, did you feel equally at ease to ask all your questions, as if you were in person? (Q3)	252 (80.0%)	43 (13.7%)	20 (6.3)
During ...did your urologist take enough time to answer your questions? (Q4)	290 (92.1%)	17 (5.4%)	4 (1.3%)	4 (1.3%)
During…did your urologist give you all the information you needed about your health status or your medication? (Q5)	287 (91.1%)	23 (7.3%)	5 (1.6%)
Was the information concerning your health status or your medications clear and easy to understand? (Q6)	294 (93.3%)	15 (4.8%)	6 (1.9%)
Do you think your experience would have been better with video consultation? (Q7)	100 (31.7%)	209 (66.3%)		6 (1.9%)

Table 3.

Patient’s characteristics regarding logistics

	<1 hour	1–3 hours	>3 hours	Missing
How much time do you spend to come to the clinic from home? (Q8)	257 (81.6%)	43 (13.7%)	14 (4.4%)	1 (0.3%)
	Own car	Taxi	Bus	Adapted transport	Walk/bicycle	Airplane	No response
What means of transportation do you use to get to your appointment? (Q9)	277 (87.9%)	3 (1.0%)	16 (5.1%)	5 (1.6%)	4 (1.3%)	1 (0.3%)	9 (2.9%)
	Yes, always	Sometimes	No	No response
When you come to see your urologist at the clinic, do you need someone to come with you, to drive you, for example? (Q10)	99 (31.4%)		213 (67.6%)	3 (1.0%)
Do you usually need to miss work when you have an appointment with your urologist? (Q11)	106 (33.7%)		197 (62.5%)	12 (3.8%)
Does your companion need to miss work when you have an appointment with your urologist? (If Q 10=yes) (Q12)	39 (39.4%)	18 (18.2%)	42 (42.4%)

Experience and satisfaction

Most patients (86.3%) rated the quality of their consultation as either excellent (54.6%) or very good (31.7%) (Table 2). Also, 92.1% of patients indicated that the urologist had taken enough time to answer their questions. Physician explanations were considered satisfactory 91.1–93.3% of the time, and 80.0% of the patients felt equally at ease to speak on the phone compared to interacting in person (Table 2), whereas 31.7% of patients thought that a video call would have been better. Almost half of the patients (46.7%, 95% CI 41.1, 52.3) would still have preferred an in-person visit for their urology consultation had it occurred outside the pandemic period. Mean and median overall patient satisfaction with telemedicine were 8.8/10 (95% CI 8.7, 9.0) and 9/10 (Q1, Q3 8.0, 10.0), respectively.

Associations

We found an association between patients’ overall satisfaction and the type of consultation (p<0.001) (Table 4). Parents of pediatric patients expressed the highest mean overall satisfaction score (9.3/10) with telemedicine, while patients having a consultation for cancer suspicion expressed the lowest overall mean satisfaction score (8.3/10). Patients traveling 1–3 hours for their appointments expressed a higher overall mean satisfaction score with telemedicine (9.3/10) compared to those traveling less than one hour (8.8/10) or more than three hours (8.8/10) (p=0.04).

Table 4.

Satisfaction scores

	Mean satisfaction/10 (95% CI)		Preference for an in-person visit
Type of cases		p<0.001	n	%	p=0.19
Non-oncological	8.6 (8.3, 8.9)		49	47.1%
Oncological	9.1 (8.9, 9.3)		54	44.6%
Cancer suspicion	8.3 (7.9, 8.7)		25	61.0%
Pediatric	9.3 (8.9, 9.7)		19	38.8%
Travel time (h)		p=0.04			p=0.96
<1	8.7 (8.6, 8.9)			46.3%
1–3	9.3 (8.9, 9.6)			46.5%
>3	8.8 (7.9, 9.6)			50.0%

When patients were asked if they would have preferred an in-person urology visit, patients with cancer suspicion (increased prostate specific antigen [PSA] or hematuria, for example) showed a higher preference to meet their urologist in person (61.0%) than non-oncology (47.1%), oncology (44.6%), or pediatric (38.8%), but this did not reach statistical significance (p=0.19).

Also, almost a third more patients (31.2%) with SCM, according to physicians’ opinion, would have preferred an in-person visit (55.7%) compared to patients with a CCM visit (42.4%, p=0.03).

Discussion

In this study, we present data assessing patients’ satisfaction and perspectives on the quality of their telemedicine consultation experiences. Overall, participants reported a mean satisfaction rate of 8.8/10. Even though no comparison was found in the literature, we consider this a high satisfaction rate, as most participants also rated the quality of the consultation as either excellent (54.6%) or very good (31.7%). Nevertheless, almost half of the patients would have preferred an in-person visit should the latter have been possible. This rate was higher for patients with cancer suspicion (61.0%) and lower for pediatric patients (38.8%). The low parent interest for an in-person visit in the pediatric population was in disagreement with urologists’ opinion about the completeness of consultation management with pediatric visits conducted through telemedicine.⁵ Indeed, parents of pediatric patients seemed to be more satisfied with telemedicine than urologists themselves. This needs further analysis to determine the cause of the discrepancy in perception.

We found an association between urologists’ perception of ICM management and patients’ preference for an in-person visit. Patients whose consultations were rated SCM by urologists were 31.2% more likely to prefer an in-person visit than patients with CCM (p=0.03). In most cases, an inperson visit was probably rescheduled, therefore increasing stress and dissatisfaction. This was especially important if the visit was for cancer suspicion, for which a lower satisfaction score was reported compared to other types of consultations. Additionally, travel time seemed to influence experience and satisfaction. Patients with a travel time of 1–3 hours to the hospital reported higher satisfaction scores with telephone consultation compared to those with <1 hour or >3 hours. Intriguingly, the group with >3 hours of travel time to the hospital reported a lower mean satisfaction score. One possibility is that these patients had distinct, complex pathologies that could not be fully evaluated outside of a tertiary center or the travel reimbursement provided by the government might have biased the desire for an in-person visit.

The observation that 47% of patients would have preferred an in-person visit despite evaluating their overall experience as positive shows that it is important to differentiate patient satisfaction with telemedicine and patient preference for an in-person visit. Based on these findings, we believe that telemedicine should not completely replace in-person visits; however, the option of telemedicine consultation should be offered to patients after the pandemic to respect patient preferences when it is clinically, geographically, or economically appropriate. For example, a third of the patients typically need someone to drive and accompany them to their appointment and a third of these individuals miss work to do so. Alternating between telemedicine and in-person visits could also be a reasonable option. Furthermore, since only 31.7% of the patients thought that seeing their physician through a camera would have benefit their consultation, we do not think that adding video to telemedicine is mandatory but rather should be offered as an option.

Some studies have examined patient experience and satisfaction with telemedicine.^{1–3,6,10,14} These studies have described a wide range of telemedicine services in several health systems. For the most part, the published work describes results from small-scale pilot or retrospective feasibility studies. Moreover, such studies have used simple survey instruments to ascertain patient satisfaction and quality of experience and have generally reported positive results. Even though numerous studies have claimed that experience and satisfaction is acceptable with telemedicine, detailed studies with a focus on urology have only been reported recently.^5,14–17 Our study stands out due to its prospective nature and that it included all types of consultations due to a complete lockdown during the pandemic.

In this study, patients’ level of overall satisfaction was consistent with other studies.^1–4 Pinar et al observed similar patient satisfaction rates in urology, where 83.8% rated their experience with their teleconsultation as being good;¹ however, regarding preferred consultation modality (in-person visit or telephone), Locke et al reported that only 23% of their sample preferred an in-person visit rather than telemedicine. ¹⁵ They used a simple Likert-scale methodology and 45% of their sample reported that they had no preference for in-person visits or telephone. In our study, we did not allow patients to select a neutral answer. Furthermore, their study included five urologists as compared to ours, which included 18 urologists from all areas of urology. These factors may explain some of these differences in the results.

Now that both providers’ and patients’ experience and satisfaction have been explored during the COVID pandemic, it remains of interest to understand the long-term impacts of telemedicine on quality of care, especially on health outcomes or patient compliance. Moreover, it would be relevant to conduct a socioeconomic study including patients, healthcare managers, and professionals’ perspectives. Also, if long-term telemedicine is to be offered as an option, should clinics rethink their facilities? A recent Australian review examined if telehealth could reduce health system costs and concluded that cost reduction does not automatically occur, this depending on the cost of administering and monitoring telehealth systems.¹⁸ Health system costs vary largely across countries and we think local assessments are needed.

Some argue that assessing satisfaction and perception of quality of care based on one’s experience and meeting patients’ needs and expectations are different. A patient’s evaluation of a service may largely be independent of actual care received and satisfaction could be influenced by expectations. For example, if a patient experiences something new, he/she may have “unformed expectations,” implying that the patient may not have any expectation and their satisfaction level may reveal little about the quality of the actual care received.^2,19 Satisfaction levels may have been higher in the context of the pandemic and newness of telemedicine than under other circumstances; however, using a patient experience questionnaire developed for the purpose of this study that was adapted from two validated instruments (PEQ-OHC and CAHPS questionnaires), we explored different aspects of patient experience and not only overall satisfaction, and the findings were in agreement. Also, since there were up to almost six months between the phone survey and their last consultation, the last participants to complete the survey may have experienced recall bias. Alternatively, the delay between the initial consultation and the survey may have revealed a more composed state of mind, as early surveys may have captured more emotional thoughts.²⁰

Among the major strengths of our study is its sample size, which represents all patients eligible for telemedicine, as it included participation of all urologists in the region, covering a population of 750 000 for primary and secondary urological care, and a tertiary center catchment population of 2 million. Our study was performed during a period when all consultations were encouraged to be made by telephone, thus reducing selection bias. Another strength of this study is that we contacted patients by phone instead of using an online survey, which would have excluded patients who did not have access to a computer or smartphone. Online surveys may also have an inherent bias of selecting patients who are already more inclined toward virtual medicine. Additionally, our questionnaire included analyses of different domains and was not limited to a simple Likert scale, which is usually used in satisfaction studies. In addition, phone surveys were performed by members of the research team independently from the medical team, so patients may have been less likely to exaggerate positive responses compared to a process where the surveys were conducted by their treating urologist, thus limiting reporting bias.^21,22

Conclusions

High patient satisfaction scores for telemedicine visits in urology were observed. It is noteworthy, however, that almost half of the patients would have preferred an in-person visit if it would have been possible. While telemedicine is now established as a valid option for healthcare delivery, it will be important to incorporate it principally as an alternative for patient visits, keeping in-person appointments available and ensuring they are offered to our patients.