Clinician experience with telemedicine at a safety-net hospital network during COVID-19: a cross-sectional survey.

Abstract

Objective:

The COVID-19 pandemic prompted unprecedented expansion of telemedicine services. We sought to describe clinician experiences providing telemedicine to publicly-insured, low-income patients during COVID-19.

Methods:

Online survey of ambulatory clinicians in an urban safety-net hospital system, conducted May 28 2020-July 14 2020.

Results:

Among 311 participants (response rate 48.3%), 34.7% (N=108/311) practiced in primary/urgent care, 37.0% (N=115/311) medical specialty and 7.7% (N=24/311) surgical clinics. 87.8% (273/311) had conducted telephone visits, 26% (81/311) video. Participants reported observing both technical and non-technical patient barriers. Clinicians reported concerns about the diagnostic safety of telephone (58.9%, 129/219) vs video (35.3%, 24/68). However, clinician comfort with telemedicine was high (89.3% (216/242) for telephone, 91.0% (61/67) for video), with many clinicians (220/239 or 92.1% telephone, 60/66 or 90.9% video) planning to continue telemedicine after COVID-19.

Conclusions:

Clinicians in a safety-net healthcare system report high comfort with and intention to continue telemedicine after the pandemic, despite patient challenges and safety concerns.

Introduction

The coronavirus Sars-Cov-2 (COVID-19) pandemic compelled health care networks across the United States to rapidly expand telemedicine services.^1,2 Telemedicine “seeks to improve a patient’s health by permitting two-way, real time interactive communication between the patient, and the physician or practitioner at the distant site.”³ Telemedicine provides means to conduct remote clinical visits, enabling decreased COVID-19 exposure risk. Remote approaches to clinical care are effective for the care of chronic medical and behavioral conditions ^4,5 and have high patient satisfaction;^6,7 video visits have equivalent perceived quality to in-person visits.⁸

Despite its favorable attributes, many payors have not previously reimbursed telemedicine services,⁹ a major implementation barrier for safety-net health networks caring for Medicaid- and Medicare-covered patients.^10–12 On March 27, 2020, the Coronavirus Aid, Relief, and Economic Security Act was signed into law, authorizing Federally Qualified Health Centers to furnish telehealth services to Medicare beneficiaries.¹³ Many state health agencies also authorized Medicaid payments for telemedicine services,¹⁴ including video and telephone visits. With these policy changes, institutions serving Medicare and Medicaid populations expanded their use of telemedicine.¹⁵ For example, New York City Health + Hospitals went from 500 billable “televisits” in February 2020 to nearly 83,000 in April 2020;¹⁶ Ohio State University Wexner Medical Center’s family medicine programs’ telemedicine volume grew from 4% to 92.5%.²

The urgency of the pandemic and shelter-in-place orders, along with shifts in reimbursement policy, propelled safety-net site clinicians to rapidly adapt to provide telemedicine care, often with little advance preparation or prior experience with this care modality. Clinicians are front-line stakeholders of telemedicine care and are key determinants of its implementation.¹⁷ As social distancing protocols and the need for telemedicine persists, clinician experience will be a key factor in ensuring overall care quality and outcomes.¹⁸ However, little research has queried clinician telemedicine experience in the safety-net.

Safety-net care settings serving low-income, ethnically diverse populations are unique. Patients from vulnerable populations can benefit from some of the advantages of telehealth, such as reduced transportation barriers and enhanced convenience of appointments so as not to miss work shifts.¹⁹ However, safety-net patients may have limited English proficiency, limited technology literacy, and barriers accessing smartphones and cellular or broadband service.^20,21,22 These barriers may make equitable telemedicine implementation particularly challenging in the safety-net.²³

We sought to understand clinician experience in the safety-net during implementation of telemedicine in the early phase of the COVID-19 pandemic. Highlighting frontline perspectives will be key for advancing telemedicine quality for safety-net populations, as well as long-term telemedicine sustainability.

Methods

Study setting

Our setting is a large urban safety-net hospital network, serving publicly-insured and low-income county residents. Our patient population is ethnically and linguistically diverse, with varying health literacy²⁴,²⁵ and digital literacy levels²⁶ (see Table 1 for detailed patient population demographics). Of county residents, 42.3% speak a language other than English at home and 19.1% have limited English proficiency.²⁷

Table 1.

Patient demographics of study setting^3,3

Hospital Network Demographics	% (N=107,434 patients)
Gender
Female	49%
Male	51%
Transgender	Not available
Age
<18	12%
18–24	8%
25–44	31%
45–64	32%
65+	17%
Race/Ethnicity
African American	15%
White	19%
Asian	21%
Hispanic/Latinx	38%
Payor Source
Medi-Cal (Medicaid)	55%
Medicare	23%
Private/Commercial	2%
Other	11%
Uninsured	10%

Our study population included clinicians (physicians, nurse midwives, nurse practitioners, physician assistants, behavioral health clinicians, and others) providing ambulatory care services in the healthcare network. We excluded residents and Accreditation Council for Graduate Medical Education fellows, as they rotate through different healthcare delivery systems and have a different telemedicine experience than staff and faculty clinicians.

The network officially began providing telemedicine care, primarily through telephone, on March 3, 2020, anticipating COVID-19 precautions. Prior to this date, telephone and video clinical visits were performed sporadically, depending on patient preference and clinician initiative. The official shelter-in-place mandate for the county began on March 17, 2020. An official electronic health record video encounter type and documentation workflow became available on June 15, 2020. The video visit software endorsed by the network was Zoom (San Jose, CA). Patients were required to download the Zoom mobile application on a mobile phone, tablet, or open a link on their computer device, and would receive a meeting ID from the clinical team to begin a video visit encounter. Zoom is usable by Windows, Apple, iOS or Android devices. Other clinicians informally used Doximity (San Francisco, CA) or other personal social media video platforms, although this was not sanctioned by the network. Each clinic received basic, introductory training materials on video visits using Zoom, and clinician and patient-facing websites to provide standardized guidance and to support onboarding. However, no system-wide infrastructure or device purchase occurred, and no system-wide standard work was implemented.

Outcomes and data collection

We conducted an online, anonymous quality improvement survey through Qualtrics software (Qualtrics, Provo, UT), provided via a secure university-based database (see Appendix for full survey). We compiled validated instrument questions adapted from the Consolidated Framework for implementation Research (CFIR),²⁸ pre-existing telemedicine assessment tools,^29–31 and front-line stakeholder recommendations. Study outcomes were to assess observed patient barriers (CFIR construct: outer setting, patient needs³²), satisfaction and comfort with telemedicine, ³³ and intention to use telemedicine in the future by choice.³⁰ We also developed a novel item to assess perceptions of telemedicine safety: “Compared with in-person visits, I’m concerned about the safety of telemedicine because of increased risk of missed or delayed diagnosis” (The complete survey is available as a supplemental file). Survey items were Likert-scale with optional free-text comments asking what had gone well with the telemedicine visits, what hadn’t gone well, and option to share additional perspectives and comments. As this survey was to inform quality improvement, the study was exempt from Institutional Review Board approval.³⁴ We distributed the survey via email, and department-specific study champions provided reminders and encouragements to participate. We collected responses from May 28 2020-July 14 2020.

Analysis

We dichotomized Likert-scale questions into binary variables. For example, for the item assessing clinician comfort providing telemedicine, we combined both “very comfortable” and “somewhat comfortable” into “comfortable”. To assess if outcomes differed by clinician specialty, we combined specialties into “primary/urgent care,” “medical specialty” and “surgical specialty.” We conducted cross-tabulations of key metrics and chi-squared analysis of dichotomized variables across clinician specialties using Stata 13.1 (College Station, TX). We reviewed free-text comments and provided exemplars based on a narrative review of free-text comments (led by author 1). Our study rationale was to provide a clear picture of the behaviors and perceptions of clinicians across specialty. Because it was cross-sectional, and there is little prior data on clinician telemedicine perceptions to guide a multivariate analysis, we elected to report key outcomes and specialty-based comparisons only. Because we were interested in clinicians’ direct experiences with these modalities rather than their beliefs or perceptions, we restricted analysis of telephone visits to those who reported on average at least 1 telephone visit per half-day clinic session in the prior month, and restricted analysis of video visits to providers who reported conducting at least 1 video visit on average per half-day session in the prior month.

Results

We had 311 final respondents, out of an eligible 643, for a response rate of 48.3%. Among participants, 37.0% (N=115) were medical specialists, 34.7% (N=108) were primary or urgent care clinicians, and 7.7% (N=24) were surgical specialists. Demographic characteristics of survey participants are listed in Table 2. The majority (57.2%, N=178) conducted 4 or more telephone visits per half-day session. Only 26.0% (N=81) conducted one or more video visits per half-day session. Below, we report summary statistics of key outcomes as well as exemplar quotes in italics from free-text responses that illustrate results. Additional free-text quotes are available in Table 3. There were no statistically significant differences in key outcome measures by clinician specialty.

Table 2.

Demographics of Clinician Survey respondents

Demographic	N (N=311)	%
Age
20–29	4	1.3
30–39	65	20.9
40–49	79	25.4
50–59	62	19.4
60–69	24	7.7
70+	4	1.3
Missing/not disclosed	73	23.5
Gender
Female	181	58.2
Male	51	16.4
Non-binary or non-conforming	3	1.0
Missing/not disclosed	76	24.4
Clinician type
Faculty/attending physician	144	46.3
Nurse practitioner/Physician Assistant	51	16.4
Licensed counselor/Social worker/marriage family therapist	9	2.9
Psychologist	9	2.9
Nurse midwife	7	2.3
Pharmacist	6	1.9
Non-ACGME Fellow	5	1.6
Occupational therapist/Speech language pathologist	3	1.0
Genetic counselor	2	0.6
Optometrist	2	0.6
Acupuncturist	1	0.3
Other	1	0.3
Missing/Not disclosed	71	22.8
Clinical Specialty
Primary and Urgent Care
Adult Urgent Care	1	0.3
Anticoagulation Clinic	3	1.0
Family Medicine	47	15.1
Internal Medicine Primary Care	32	10.3
Pediatrics	23	7.4
Pediatric Urgent Care	1	0.3
Geriatrics	1	0.3
Medical Specialty
Cardiology	6	1.9
Dermatology	3	1.0
Diabetes Clinic	4	1.3
Endocrinology	2	0.6
Gastroenterology	3	1.0
Hepatology	2	0.6
Infectious Diseases	6	1.9
Nephrology	3	1.0
Neurology	1	0.3
Ob/GYN/Midwifery	26	8.4
Pain Clinic	2	0.6
Palliative Care	5	1.6
Pediatric Asthma/Allergy	1	0.3
Oncology	9	2.9
Psychiatry	33	10.6
Pulmonology	4	1.3
Surgical Specialty
General Surgery & Trauma	5	1.6
Neurosurgery	2	0.6
Orthopedics	9	2.9
Ophthalmology	1	0.3
Optometry	2	0.6
Pediatric Urology	1	0.3
Podiatry	2	0.6
Urology	1	0.3
Vascular Surgery	1	0.3
Specialty not disclosed	64	20.6
Years since training
In training (non ACGME fellow)	7	2.3
1–5	47	15.1
6–10	42	13.5
11–15	50	16.1
16–20	43	13.8
21+	52	16.7
Average number of telemedicine visits per half-day clinic
Telephone
0	38	12.2
1–3	95	30.6
4–6	100	32.2
7–9	61	19.6
10+	17	5.5
Video
0	230	74.0
1–3	71	22.8
4–6+	10	3.2

Table 3.

Exemplar quotes from clinician telemedicine survey open-ended responses.

Survey domain	Example quotes
Observed patient barriers
Technical barriers	“cannot reach tons of patients that don’t have phones” “Losing the signal” “Poor patient cell phone service/difficulty communicating.” “Some low-income patients phone plans add cost when phone visits are used, cost that are not covered in the billing for care.” “Patients seem to miss appointments more often.”
Non-technical barriers	“…family being distracted.” “Lots of difficulty with interpreter logistics ESPECIALLY when working remotely.” “For low literacy patients explaining changes in medication doses without showing them which medicine and what the change is can be tricky/feel a little unsafe.”
Clinician safety concerns	“Audio quality can be poor. Facial cues and body language are missed. If caution is not used, diagnoses can be missed or delayed.” “Pt wanted a (diagnosis) of a lesion that she is having difficulty describing…and wants a diagnosis.” “Couldn’t do reflexes as part of neuro exam on a patient in whom it would have been helpful in terms of the differential diagnosis” “(Telemedicine) is not useful without known diagnosis or if date (physical exam) needs to be obtained.” “thought a patient was alone - didn’t know partner was there - and asked a question that shouldn’t have been asked in front of them”
Clinician comfort	“I actually felt like patients asked more questions and were in some ways more engaged in the visit. Fairly easy to incorporate telephone language interpreters into the call.” “I think it is a very convenient way to manage acute issues and very valuable for patients who have limited transportation or ability to take time off work to just get on the phone for follow up visits. “Can do more frequent, focused visits on topics that don’t require physical exam, especially on patients you already know well” “Linking in interested family members is easy.”
Clinician satisfaction	“I was pleasantly surprised and all we could accomplish with telemedicine. A great option for our patients!” “It is more efficient time wise and patients don’t wait as long. I don’t think there is anything better about a telephone visit but in some cases/limited cases it is as good.” “Patient appreciate being able to speak to the doctor while still being safe at home. This also is increasing our show-rates.” “I feel like for managing some chronic health conditions or doing social needs screening telemedicine is very useful and more convenient for families and I’m glad we can bill for it since we were doing it before anyway.” “For 80% of follow-ups where the neurological exam is established and not expected to change, the patient would benefit from a telemedicine visit as they are otherwise spending a lot of time, effort, and money coming into [hospital] from [the city].” “This significantly takes away from my satisfaction with providing patient care. I dread phone clinic days, sadly.”
Clinician likelihood to continue	“Though it has limitations, overall I think telemedicine is a great option, particularly for certain circumstances, such as discussions of lab results, and routine follow-ups with stable disease. I definitely would like to continue offering some telemedicine care long-term, and would appreciate more training opportunities on how to maximize the experience for the patients and myself.” “It will be vital for telemedicine visits to continue to provide improved access and efficiency for patients for the appropriate visits in a hybrid model with in person visits beyond the covid pandemic.” “I’m concerned that it won’t be financially feasible to do telephone visits once reimbursement for this stops, since I think video visits are not attractive to many low-income patients because of their resources and also their living situations.” “Would like to continue with telemedicine give it allows for more flexibility in scheduling and contacting patients around their appointments as well as the patients appreciate it more as well.”

We asked clinicians to select all applicable patient challenges that they had directly observed when conducting telemedicine visits over telephone and video (see Figure 1 for compilation of technical and non-technical barriers). The most common barriers for telephone visits included: speech, hearing or cognitive barriers (44.1%; example: “Patients that are hard of hearing are almost impossible to communicate with, very challenging to reach some patients by phone.”), communication quality (43.7%), and lack of having a phone (37.6%). The most common barriers for video visits included trouble using mobile applications (39.5%), lack of video (38.6%), lack of knowledge or skills to participate in the visit (37.9%: example, ““Difficulty setting up video access due to language and educational barriers.”), and lack of internet (35.0%).

Figure 1.

Clinician-observed patient barriers to telemedicine care in the safety-net (a) technical challenges and (b) non-technical challenges (N=311).

Note: Percentages are proportion of total survey participants (N=311) who indicated they had observed these challenges.

The majority of clinicians (129/219, 58.9%) conducting telephone visits agreed that they had concerns about the safety of telephone visits, with regards to a missed or delayed diagnosis. However, only 35.3% (24/68) of clinicians doing video visits had diagnostic safety concerns. Diagnostic safety concerns related to inability to gain objective vital signs or physical exam findings: “There are patients, particularly those with heart failure, who just physically need to be seen to evaluate them (volume status in particular). I’m really flying blind, and though I’m trying my best, there’s a good chance I’ll be wrong and cause harm.”

Despite patient challenges and safety concerns, clinician comfort with telemedicine services was high (Figure 3). The majority (89.3%, N=216/242) expressed being comfortable with telephone visits: “I think there is a lot we can do over the phone. It was a good way to show patients that we care about them and believe in the importance of sheltering in place. There is a lot that absolutely works over the phone.” For video visits, 91.0% (61/67) endorsed comfort conducting video visits. Along with comfort, satisfaction was also high; 72.3% (47/65) were satisfied with telephone visits and 89.2% (58/65) with video visits (Figure 4): “Providing information for results and following up on an in-person visit is satisfying to do over telephone.”

Figure 3.

Clinician comfort with providing telemedicine care by specialty.

Figure 4.

Clinician satisfaction with telemedicine care compared with in-person visits by specialty.

Clinicians stated high likelihood to continue telemedicine by choice once clinic operations had returned to normal; 92.1% (220/239) for telephone visits and 90.1% (60/66) for video visits: “I would love to be able to continue telemedicine but it depends on the department of public health’s decisions about our clinic. It saves patients and staff lot of time and hassle (commuting, childcare, parking). I think it’s great for visits that don’t need an exam.”

Discussion

To our knowledge, ours is the first assessment of the early experience implementing telemedicine across a multispecialty network of safety-net clinicians during Covid-19. Prior to COVID, assessments of clinician perceptions of telemedicine found high satisfaction and equivalent quality with in-person visits.⁸ Other surveys found clinician resistance to telemedicine implementation due to reimbursement, liability, and technical concerns.¹⁷ Since the pandemic, a survey of gastroenterologists found 88% rated video visits as better or as good as face-to-face, with only 41% rating telephone visits as better or as good as in-person visits.³⁵ Another post-COVID study found clinician satisfaction with telemedicine was correlated with stated likelihood to continue telemedicine; other predictors were perceived ease of conducting a telemedicine physical exam and a flexible personality style.³⁶ A study of primarily medical specialties found high likelihood to continue telemedicine but lower satisfaction with telephone visits than video.³⁷ These prior surveys have not highlighted the safety-net; nor safety-net specific patient barriers.

The telemedicine journey of rapid implementation, without dedicated resources or standard work, is common to many safety-net settings that disproportionately care for low-income populations.^1,27 Consistent with one prior study, we found that telemedicine implementation for a lower-resource patient population has been delivered primarily through telephone, although video visits are being piloted.¹⁶ This pattern is likely related to the higher complexity of technical infrastructure and clinic workflows required to support safety-net patients to successfully participate in video visits compared to phone visits. In the absence of team-based virtual workflows for video visits, particularly at lower-resourced healthcare sites, clinicians are required to instruct patients on how to install and use video software during their visits, cutting into the time to provide medical care. Best practices for transforming the health care systems, such as the use of standard work and teams, should be brought to bear on telemedicine practice. ²⁸

Both technical and non-technical challenges faced by patients in the safety-net may contribute to clinician concerns about reduced access for underserved patients. Clinicians raised equity-related concerns about patient speech, hearing or cognitive ability, access to a reliable phone number, video visit capacity at home which requires a smartphone and internet, reliable access to language interpreters, and patient education regarding expectations for a telemedicine visit. These barriers are closely linked to healthcare disparities based on disability, socioeconomic status, and educational attainment, and merit close attention for equitable implementation of telemedicine.^24,38–41

We can extrapolate strategies to overcome such barriers from research on improving use of online patient portals for vulnerable populations as well as telemedicine literature. For barriers accessing devices, programs to improve device access increase portal utilization⁴² and are recommended for telemedicine.⁴³ For barriers accessing data, low-cost and free high quality broadband access is a key component of telemedicine equity given neighborhood-based disparities.^21,44 For barriers in digital literacy, robust evidence exists for interventions to support individual-level patient preparation for portal access, which are likely to be of merit for telemedicine as well.^45–49 For limited English proficiency patients, dissemination of best practices for interpreter services must be standardized.²³ Finally, the design of telemedicine platforms and instructional tools must engage patients and stakeholders from vulnerable populations, utilizing user-centered design or other participatory methods.⁵⁰ Our group has created an online toolkit of examples and resources to support safety-net institutions seeking to overcome some of these barriers.⁵¹

Over half of respondents had concerns about the diagnostic safety of telemedicine care. Free-text comments shared the concerns of relying on a telephone visit when objective vital signs or exam findings are required, considering diagnoses of higher urgency such as acute abdominal pain, or privacy concerns limiting diagnostic information gathered. This can be compounded by language barriers or limited health literacy, particularly among patients in the safety-net. The current evidence base suggests that telemedicine video encounters have relatively equivalent diagnostic accuracy as in-person visits,^52,53 but it is unknown if this holds for telephone-based encounters or for safety-net populations. Additional resources to train providers on how to build diagnostic confidence via telemedicine are needed. Payor reimbursement for home self-monitoring devices, such as home blood pressure monitors, scales, and pulse oximeters for qualifying co-morbidities is another strategy to improve clinician confidence in safe, remote diagnosis and patient self-monitoring.⁵⁴

Despite safety concerns and challenges, there was very high interest in continuing to use telemedicine in the future by choice, after the COVID-19 pandemic has subsided. Our interpretation of the high likelihood to continue, despite safety concerns, is that the overall benefits of patient and staff convenience and satisfaction outweigh the potential risks. This is a promising finding, signifying that stakeholders are engaged in championing this service long-term. Since many state Medicaid programs model their Medicaid policy after Medicare, sustained Medicare telemedicine policy is key to shifting Medicaid nationwide. Moreover, as many safety-net patients do not have reliable video access, reimbursement for telephone-based visits will continue to be a priority to prevent widening the “digital divide”. Medicaid, Medicare and other safety-net insurers must establish reimbursement policies that facilitate long-term financial sustainability (or incentives) for provision of telemedicine.⁵⁵

Limitations

While our response rate was 48.3%, we had anticipated a 30–40% response rate for a non-compensated, voluntary survey conducted during a time of crisis; our response rate surpassed our expectations given the context at the time. Clinicians who chose to participate in the survey may differ from those who did not; for example, participants with more enthusiasm about telemedicine may have elected to participate in the survey. We lack detailed demographic data of all eligible clinicians in the network to compare responders to non-responders. We do not have data on whether patient characteristics, including language preference, differed by type of telemedicine visit or clinical specialty type. We do not know if factors such as amount of interpreter use or prior telemedicine experience may have affected some of the clinician ratings of their experience conducting telemedicine. Some respondents did not complete every question of this survey, and many did not disclose their specialty, leading to varying total denominators for each survey item and clinical specialty subgroup. Based on the survey structure, there may have been participants with less than 1 telemedicine visit per session on average which were categorized as zero; our results are therefore generalizable to clinicians who conduct at least one telemedicine encounter per half-day or more. Finally, as telemedicine was not widely implemented prior to COVID-19, we could not compare changes in utilization or perceptions pre- and post-pandemic. However, study strengths include a diverse range of clinician participants who practice in a safety-net setting, and our ability to capture front-line perspectives at a healthcare site relatively early (the first 6 months) of telemedicine implementation.

Conclusions

Our survey describes the realities of the rapidly changing ambulatory landscape during COVID-19. Safety-net clinicians are facing multiple patient barriers to engagement in telemedicine care during this first wave of the COVID-19 pandemic. However, they also appreciate the benefits for their patients and attest high interest in continuing telemedicine, despite concerns about safety and equitable access. Safety-net patients require access to devices, data, interpreter services, and technical support in order to participate equitably in telemedicine. Provider-level supports are needed to ensure sustainability, promote safety of care, and improve satisfaction to prevent long term burnout or adverse care outcomes. Best practices tailored for the safety-net will be key to building capacity if telemedicine is to remain for the long-term.

Figure 2.

Clinician safety concerns regarding telemedicine by specialty.

Figure 5.

Clinician reported likelihood to continue telemedicine after COVID-19 by specialty.