Virtual Cancer Care During the COVID-19 Pandemic in Alberta: Evidence From a Mixed Methods Evaluation and Key Learnings

PURPOSE

This study reports on a mixed methods evaluation conducted within a provincial cancer program in Alberta, Canada. The purpose was to capture key learnings from a rapid virtual care implementation because of the COVID-19 pandemic and to understand the impact on patient and staff experiences.

METHODS

Administrative data were collected for 21,362 patients who had at least one virtual or in-person visit to any provincial cancer center from April 1, 2020, to June 10, 2020. Patient surveys were conducted with 397 randomly selected patients who had received a virtual visit. Surveys were also conducted with 396 Cancer Care Alberta staff.

RESULTS

14,906 virtual visits took place in this period, and about 40% of weekly visits were virtual. Significant differences were observed in both patient-reported symptom questionnaire completion rates and referrals to supportive care services between patients seen in-person and virtually. Patients receiving active treatments reported significantly lower levels of satisfaction with virtual visits than those seen for follow-up, but overall 90% of patients indicated interest in receiving virtual care in the future. Staff thought virtual visits increased patients' access to care but less than one third (31.5%) felt confident meeting patients' emotional needs and having conversations about disease progression and/or end of life virtually.

CONCLUSION

The COVID-19 pandemic has driven the rapid implementation of virtual visits for cancer care delivery in health care settings. The findings from this mixed methods evaluation provide a concrete set of considerations for organizations looking to develop a large-scale, enduring virtual care strategy.

INTRODUCTION

The COVID-19 pandemic has had a tremendous impact on the oncology community as it has faced unprecedented challenges to provide timely, safe cancer care while balancing an increased risk of exposure to the virus for a vulnerable patient population.¹ As physical distancing is an effective way to mitigate viral transmission, using the telephone, video, and/or mobile apps to deliver care virtually has become common across health care facilities worldwide.² Before the pandemic, virtual care was mainly used to supplement, rather than replace, in-person cancer care³; however, this has shifted with COVID-19. Data on the trends, scale, volume, and distribution of virtual visits during the pandemic are now being published, but few have addressed experiences of patients and care teams in receiving and providing care virtually.

In Canada, oncology care is part of the public health care system and is provided through publicly funded provincial cancer programs. Cancer Care Alberta (CCA) delivers ambulatory oncology services to Albertans through a network of 17 cancer centers. Because of the pandemic, CCA rapidly integrated virtual clinic visits in March 2020. With a lack of secure electronic messaging between clinicians and patients, and the urgency of this practice change, a pragmatic choice to use the telephone was made.⁴ The purpose was to minimize viral spread while maintaining high-quality care. To optimize practice during the pandemic and inform the postpandemic use of virtual care, an evaluation was conducted within CCA using administrative and experience data.⁴

METHODS

A mixed methods evaluation was designed to understand key aspects of virtual care early in the pandemic. First, in-person and virtual visit administrative data were analyzed. Second, a patient experience survey was completed focusing on patients' perceptions of the quality of virtual care received. Finally, a staff survey was conducted to understand the experience of delivering care virtually. Both surveys provided quantitative and qualitative data. After screening with the ethical principles established by A pRoject Ethics Community Consensus Initiative (ARECCI),⁵ full Research Ethics Board review was waived as this project was deemed quality improvement, with no harm anticipated or reported. Quantitative data were exported into SPSS Statistics 25 (IBM Corp) for analysis. Qualitative data were analyzed using NVivo 12 software (QSR International Pty Ltd).

Data Collection and Analysis

Virtual and in-person administrative visit data.

Data were collected from ARIA MO (Medical Oncology), the electronic medical record used in CCA, from April 1, 2020, to June 10, 2020. Patients with at least one virtual or in-person visit to any provincial cancer center during this time were captured.

Descriptive statistics (mean, standard deviations [SDs], interquartile range, and percentages) were conducted to report characteristics including the trend of clinic visit types (in-person or virtual), clinical documentation, patient-reported symptom questionnaires, and referrals to supportive care. Visits were then broken down by patient age group, sex, and tumor group. Chi-square test of independence was used to test the association between supportive care referrals and visit type (virtual v in-person).

Virtual care patient survey.

Telephone surveys were conducted between May 26, 2020, and July 3, 2020 with randomly selected patients who received a virtual visit the previous week. Patients needed at least one virtual visit from May 18, 2020, to June 26, 2020 for inclusion. A random sample of 1,000 patients from this period was pulled. Sample size was calculated to estimate the population proportion of satisfaction, with 5% margin of error (±5%) and 95% confidence. The survey included questions adapted from Your Voice Matters (YVM), an experience survey for adult cancer outpatients undergoing treatment,⁶ and the EuroQol Five-Dimension—Five-Level (EQ-5D-5L)⁷ (Data Supplement, online only). Several virtual experience questions were also included. Surveyors entered data on behalf of patients into an online questionnaire. Mann-Whitney U tests were applied to examine if patients reported different satisfaction levels based on their treatment status.

Virtual care staff survey.

The staff survey was distributed to approximately 1,700 CCA staff via a program-wide e-mail invitation on July 2, 2020. The survey included questions about identifying patients appropriate for a virtual care visit, scheduling these visits, and delivering care virtually. It also included questions aimed at understanding how confident staff felt providing virtual care and addressing patient symptoms and/or concerns virtually (Data Supplement). Staff entered data directly into an online questionnaire.

RESULTS

Virtual and In-Person Administrative Data

Sample characteristics.

A total of 21,632 patients with cancer had at least one clinic visit (virtual or in-person) to a cancer center between April 1, 2020, and June 10, 2020. The mean age was 63.5 years (SD = 14.1) and 54.3% were female. The most common tumor groups were hematology (23.7%) and breast (20.2%; Data Supplement).

In total, the 21,632 patients had 39,961 visits, with 14,906 (37.3%) virtual and 25,055 (62.7%) in-person. Most virtual visits (99.2%) were conducted via telephone. Figure 1 depicts weekly visits in the study period. Virtual visits increased quickly from the start of the pandemic, peaking in week 6 of the study period (May 6-May 12) when 46% occurred virtually. This followed the peak of the first wave of COVID-19 in Alberta, in the last week of April 2020.^8,9 Since then, the percentage of virtual visits has remained stable at approximately 40%, although rates of COVID-19 have fluctuated considerably in the year since this study.¹⁰

FIG 1.

Weekly visits (virtual plus in-person) and COVID-19 cases in Alberta (April 1-June 9, 2020).

From April 1, 2020 to June 10, 2020, of the 14,906 virtual visits, only 2,505 (16.8%) had a completed patient-reported symptom questionnaire. By contrast, questionnaires were completed for 13,955 of the 25,055 in-person visits (55.7%). Completion rates before the pandemic were approximately 75%.

Patients who had virtual visits received significantly fewer supportive care referrals compared with patients who had in-person visits, χ² (7, N = 1,169) = 19.4, P < .01 (cells ≤ 5 were excluded in chi-square). Out of 10,000 virtual visits, 48 social work, 50 nutrition, and 46 psychology referrals were made, whereas these numbers were 219, 209, and 123, respectively, for in-person visits. Figure 2 shows the breakdown of supportive care referrals made for all visits within the study period.

FIG 2.

Supportive care referrals (April 1, 2020-June 10, 2020). CO, community oncology; OT, occupational therapy.

Virtual Care Patient Survey

Sample characteristics.

Patients were randomly selected from the larger random sample of 1,000 and called on the phone. Of the 567 patients who answered the phone, 397 (70%) completed the survey, for a final sample size slightly larger than anticipated. The mean age of those who completed the survey was 63.8 years (SD = 13.8). A slightly higher percentage of the patients were female (51.6%). The most common tumor groups were hematology (25.9%) and breast (15.9%; Data Supplement).

Quantitative findings.

Almost all respondents (97%) received virtual care via telephone. Generally, patients reported high satisfaction with virtual visits, as measured with 12 Likert-scale questions (percentage of positive responses ranged from 69.3% to 94.5%). We further examined differences by treatment status, by comparing patients on active treatment to those on follow-up. The results revealed that patients on treatment reported significantly lower levels of overall satisfaction with their virtual care than those on follow-up (73.2% v 83.6%; P < .05). Other areas of statistically significant differences included feeling like the care team treated you with respect (91.1% v 97.1%; P < .05), explained things in a way you could understand (85.8% v 94.2%; P < .01), and let you ask questions (87.9% v 94.6%; P < .05; Table 1).

TABLE 1.

Percentage and Significance Level of Positive Experiences by Treatment Status

Qualitative findings.

Patients expressed positive experiences with ease of accessing care virtually while saving travel time for themselves, their family, and/or care providers. Overall, 67% of patients indicated interest in receiving some degree of virtual care beyond the pandemic. Only 10% did not want to receive any virtual care in the future. The remaining 23% were either unsure or responded with a variation of yes, but depends, as long as they were feeling well or doing great. Patients also expressed concerns with scheduling (common responses included not enough time and I should have asked more questions), lack of emotional support (with examples like social aspect missing and I did not feel connected), and a need for video calls (with comments like I want to see my doctor's face and instead of a telephone, I prefer in-person or a video option like FaceTime). Only select open-ended responses from the patient and staff surveys are shared in this paper. For additional qualitative results, please contact the corresponding author.

Virtual Care Staff Survey

Sample characteristics.

Of the 1,700 staff who received the survey, a total of 396 completed it (a response rate of 23%). Of those, 233 (59.1%) indicated they were directly involved in delivering care virtually and 161 (40.9%) indicated they were not. Additionally, 45.6% of staff indicated they were employed in their current role with CCA for more than 10 years. The most common professional groups were nurses (25.5%) and clerical and/or Medical Office Assistants (18.9%). More than 72% of respondents indicated they typically worked at one of the two tertiary cancer centers in the province (Data Supplement).

Quantitative findings.

Specific challenges to providing virtual care were highlighted by staff. Only 35.8% agreed or strongly agreed that it was easy to meet patients' needs who attended a virtual appointment. When asked about their confidence in meeting specific patient needs virtually, generally low levels (< 50% confident) were reported (Fig 3). Less than one third of staff (31.5%) felt confident meeting patients' emotional needs virtually.

FIG 3.

Percentage of staff who were confident or extremely confident in addressing the following patient needs virtually.

Qualitative findings.

The open-ended responses supported many of the challenges identified in the questions discussed above. One respondent reflected it is really difficult to comfort someone virtually and some shared they felt less confident addressing topics of death and end-of-life in this setting. There were also concerns regarding impaired nonverbal communication inherent in conducting assessments over the phone. Staff expressed they were unable to pick up on nuanced or nonverbal communication and could not read body language. Staff also expressed concerns about triaging patients appropriately for virtual care, seen in feedback like there are no clear guidelines as to which patients are appropriate for virtual assessment and which are not.

Generally, however, staff who completed the survey believed virtual care increased patients' access to care, especially for those living in rural and remote communities, and was convenient for patients by saving time and money on traveling, parking, and waiting. Furthermore, staff identified areas where virtual care facilitated their ability to deliver cancer care, using terms like efficiency, flexibility, interdisciplinary coordination, and ability to see many patients. Staff believed virtual visits worked well for patients who are stable, not on active treatment, on follow-up, or have little or no complications.

DISCUSSION

In a short time, CCA converted thousands of in-person visits to virtual visits to minimize the risk of COVID-19 for patients with cancer.^4,11 This study examined the results of a three-pronged evaluation during the first three months of the pandemic in Alberta, Canada. While virtual care was implemented as a rapid response to COVID-19, development of a large-scale virtual care strategy is now underway as active cases in Alberta are finally declining, more than a year into the pandemic.¹⁰ The evaluation presented here is one component of a larger, multiphased assessment, which will inform the development and implementation of a comprehensive virtual care strategy.

Virtual care seemed to work best when a patient's health status was considered. Staff survey results showed that virtual visits worked well for stable patients with few symptoms. Congruently, patients were more willing to receive virtual care if they had good health status. Virtual care is seemingly less suitable when discussing difficult topics such as end-of-life or palliative care, as noted in the staff survey. Other parameters for consideration are patient preferences and if they live in a rural or remote area. Additionally, patients on active treatment were significantly less satisfied with their virtual care experience than those on follow-up. Even with these general indicators, staff noted that there is no blueprint for triaging patients to virtual care, suggesting guidelines to assist staff would be useful. At this stage in the pandemic, however, with the risk of COVID-19 declining, patient preference should play a key role, as virtual care is no longer an essential safety measure for all visits. All of this will be considered as development of a comprehensive virtual care strategy progresses.

While virtual care was primarily provided through telephone, evidence suggests that video-based platforms allow for better nonverbal communication¹² and higher levels of patient satisfaction.^12-15 This is supported by feedback from both surveys, with patients indicating a preference for video calls in the future and staff suggesting that the ability to see patients' body language and other nonverbal communicative cues would be helpful. We plan to research this aspect in the near future by surveying patients who receive virtual care through Telehealth, web-based platforms, and asynchronous methods such as e-mail.

During the rapid implementation, it was difficult to account for patient assessments and the collection of patient-reported information virtually. Raghavan et al¹⁶ recommend that, especially in virtual environments, standard prearrival symptom screening should be reinforced for patients with cancer to ensure appropriate care is provided. Current workflows for collecting patient-reported information rely on patients receiving and completing paper forms upon check-in and bringing them to the visit; the information is then entered into the electronic medical record by staff after the visit.¹⁷ This process, although well integrated into in-person settings, was disrupted by virtual care,¹⁸ and the results of this evaluation reflected that there were significant differences between virtual and in-person visits regarding completion of patient-reported symptom questionnaires. There were also generally low numbers of referrals to supportive care services made for virtual patients, likely partially because symptom questionnaires aid staff in determining which patients require referrals. The added fear and stress of the pandemic may make supportive care resources even more crucial for patients in virtual settings, emphasizing the importance of finding alternative symptom screening formats.¹⁹ Previsit phone calls to conduct symptom screenings have since been implemented as a temporary solution in some settings, but this depends on staff capacity. Visibility of symptom burden and supportive care needs of patients seen virtually continues to be a challenge in CCA, but planning is underway for the routine electronic collection of patient-reported information via an online cancer patient portal, launching in 2022. Additionally, collecting this information will aid in identifying patients who are a good fit for virtual appointments in the future.

A compelling finding from this evaluation was the lack of confidence staff felt in addressing patients' needs in the virtual setting. Providing virtual care pulls on existing clinical competencies, but findings point to the reality that new virtual-specific skills are needed for staff to provide high-quality care. As highlighted in Figure 3, focused efforts are needed to develop competencies in discussing emotional and palliative care topics in virtual settings.²⁰ In Alberta, two-day professional development courses called LEAP (Learning Essential Approaches to Palliative and End-of-Life Care) are available for health care staff; promoting courses like this to staff is one strategy to address the lack of confidence in these areas.²¹ Additional courses need to be explored to aid in addressing a variety of other concerns virtually, especially given that two thirds of staff reported some difficulty meeting patients' needs in virtual visits, whereas two thirds of patients wanted to receive care virtually beyond the pandemic. This should serve as a call to action, as virtual care will likely be a viable option for every patient with cancer at some point on their cancer care trajectory.

While CCA was proactive in establishing this evaluation strategy early in the pandemic, since then, Alberta has experienced two subsequent waves of infection, resulting in the redeployment of health care staff and resources to manage the care needs of patients with COVID-19.²² As a result, although small modifications to CCA's virtual practice have occurred along the way, implementing the recommendations drawn from this evaluation is just now being explored. Understanding the initial rapid virtual care response was step 1 of a three-step development plan for a comprehensive virtual care strategy. Step 2 involves understanding other modes of virtual care delivery, and step 3 will focus on creating an implementation plan that effectively uses technology, maximizes workflows, and supports clinical staff to best use virtual care after the pandemic.

This study is not without limitations. The sampling of staff survey respondents was not random, as staff chose whether to complete the survey; thus, the data may not be representative of all CCA staff. Furthermore, the staff survey's low response rate compromises the generalizability of the results. For our purposes, the decision was made to group all staff together; however, analyzing responses from different groups separately may reveal important differences. Additionally, we collected a relatively narrow scope of variables in the administrative data, limiting our interpretation of differences between in-person and virtual visits. Finally, because of the limited completion of patient-reported questionnaires in virtual visits, we were unable to determine whether patients seen virtually had different symptoms than those seen in-person. It is possible that the lower number of supportive care referrals made for virtual patients was because of their lower symptom burden; however, this conclusion cannot be made as we do not have symptom data for most virtual patients.

In conclusion, COVID-19 has been an unprecedented catalyst for the rapid adoption of virtual care in Alberta's pandemic response. Some challenges were encountered, such as lower completion rates of patient-reported symptom questionnaires and a significant difference in referrals to supportive care services for patients seen virtually. Other areas identified for improvement include the necessity for additional staff training to confidently support patient needs in virtual settings. Despite these challenges, CCA is well positioned to learn from this evaluation as work continues on developing an in-depth virtual care strategy. This study can also serve to influence a concrete set of recommendations for other cancer organizations looking to develop a large-scale virtual care strategy, now or in the future.

AUTHOR CONTRIBUTIONS

Conception and design: Linda Watson, Siwei Qi, Andrea Delure, Lindsi Chmielewski, Krista Rawson

Collection and assembly of data: Linda Watson, Siwei Qi, Eclair Photitai, Lindsi Chmielewski

Data analysis and interpretation: Siwei Qi, Claire Link, Eclair Photitai, April Hildebrand, Dean Ruether, Krista Rawson

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

Virtual Cancer Care During the COVID-19 Pandemic in Alberta: Evidence From a Mixed Methods Evaluation and Key Learnings

The following represents disclosure information provided by the authors of this manuscript. All relationships are considered compensated unless otherwise noted. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/op/authors/author-center.

Open Payments is a public database containing information reported by companies about payments made to US-licensed physicians (Open Payments).