Telehealth Implementation Response to COVID-19 in the OneFlorida+ Clinical Research Network: Perspectives of Clinicians and Health Systems Leaders

Abstract

Introduction:

The COVID-19 pandemic forced health systems worldwide to make rapid adjustments to patient care. Nationwide stay-at-home mandates and public health concerns increased demand for telehealth to maintain patients' continuity of care. These circumstances permitted observation of telehealth implementation in real-world settings at a large scale. This study aimed to understand clinician and health system leader (HSL) experiences in expanding, implementing, and sustaining telehealth during COVID-19 in the OneFlorida+ clinical research network.

Methods:

We conducted semistructured videoconference interviews with 5 primary care providers, 7 specialist providers, and 12 HSLs across 7 OneFlorida+ health systems and settings. Interviews were audiorecorded, transcribed, and summarized using deductive team-based template coding. We then used matrix analysis to organize the qualitative data and identify inductive themes.

Results:

Rapid telehealth implementation occurred even among sites with low readiness, facilitated by responsive planning, shifts in resource allocation, and training. Common hurdles in routine telehealth use, including technical and reimbursement issues, were also barriers to telehealth implementation. Acceptability of telehealth was influenced by benefits such as the providers' ability to view a patient's home environment and the availability of tools to enhance patient education. Lower acceptability stemmed from the inability to conduct physical examinations during the shutdown.

Conclusions:

This study identified a broad range of barriers, facilitators, and strategies for implementing telehealth within large clinical research networks. The findings can contribute to optimizing the effectiveness of telehealth implementation in similar settings, and point toward promising directions for telehealth provider training to improve acceptability and promote sustainability.

Introduction

The COVID-19 pandemic presented challenges for health care delivery that demanded a rapid and decisive response from health care systems globally. To safely ensure continuity of care for patients, health care systems and clinics implemented or expanded digital strategies, such as videoconferencing platforms, which allow patients to remotely connect with their providers.

These tools are part of the broader approach of telehealth, defined by the World Health Organization as “delivery of health care services where patients and providers are separated by distance,” using “information and communication technologies for the exchange of information.”¹ While variations of telehealth services have existed since the 1960s, lack of treatment and vaccination options during the onset of COVID-19 necessitated the unprecedented expansion of these services.

Several studies have sought to describe and evaluate implementation of telehealth in response to COVID-19.^2–4 Facilitators to telehealth implementation included patient engagement, operational workflow, organizational readiness, and regulatory changes regarding reimbursement for telehealth appointments, which included the Coronavirus Aid, Relief, and Economic Security (CARES) Act of 2020 and a Centers for Medicare and Medicaid Services (CMS) 1135 Waiver.⁵ Barriers included patient limitations in access to and comfort with telehealth technologies, lack of clinical care telehealth guidelines, and training, technology, and financial constraints. Other literature has described the essential role of telehealth in maintaining access to and continuity of care during the pandemic.^2,6

Further research has shown that individual clinics within a single health system may struggle to deliver telehealth visits.⁷ Certain types of appointments, such as well-care visits, vaccinations, and screenings, can be challenging through telehealth because of the inability to conduct a physical exam. More broadly, United States (U.S.) providers specializing in endocrinology, gastroenterology, rheumatology, nephrology, cardiology, and psychiatry are the most likely to use telehealth, while specialists in optometry, physical therapy, and ophthalmology are the least likely, as these specialties rely more on physical exams and specialized equipment.⁸

However, there remain gaps in understanding the approaches that large health care systems have used to effectively implement telehealth during COVID-19, the impact of telehealth on health systems and clinics following implementation, and the sustainability of telehealth in these settings. Furthermore, perspectives of health system leaders (HSLs), such as chief operating officers (COOs) and chief medical officers (CMOs), are underrepresented in the literature on telehealth implementation during COVID-19. Two studies involving interviews with leadership and administrators in single health care settings did reveal system-level factors associated with telehealth implementation during the pandemic, including technology planning, leadership communication, goal setting, team collaboration, and sustained funding.^9,10

This study aimed to understand the circumstances influencing telehealth implementation across multiple health systems in the state of Florida, using qualitative interviews with HSLs, specialist clinicians, and primary care providers (PCPs) within or affiliated with the OneFlorida+ Clinical Research Consortium (OFCRC) network.¹¹ Specifically, we aimed to explore experiences and perspectives of HSLs and providers to characterize the readiness, implementation process, and sustainment of telehealth at OFCRC sites; identify the most salient factors that contributed to successful implementation; and explore the impact of implementation at the health system level after the lifting of stay-at-home restrictions.

Methods

This qualitative study was embedded in a mixed-methods study of telehealth implementation in OFCRC settings, with a focus on patients with diabetes and hypertension. We conducted semistructured, qualitative interviews with HSLs, PCPs, and specialist clinicians in five OFCRC health systems and two OFCRC-affiliated settings from April through October 2021 – ∼12 to 18 months after the beginning of the pandemic. This study was approved by the University of Florida Institutional Review Board.

STUDY SETTING AND POPULATION

The OFCRC is a partnership among 14 academic institutions and health systems across Florida, Georgia, and Alabama, formed to provide the clinical and data infrastructure to generate real-world evidence. This study included five OFCRC health care settings in Florida, including three academic health systems (University of Florida, University of Miami, and University of South Florida/Tampa General) and two private health systems (Tallahassee Memorial and the Villages Health), as well as two settings associated with Community Health IT (an OFCRC affiliate)––one private behavioral health system (Meridian Behavioral Health) and one private rural hospital (Doctors Memorial).

Principal investigators at each site were asked to provide names of HSLs and clinicians who had sufficient experience to offer insight on health system and clinical practices both before and during the pandemic. Potential respondents held roles of site principal investigator, executive (e.g., COO, CMO), PCP, or specialist clinician. A total of 37 individuals meeting these criteria were identified and contacted by e-mail to participate.

DATA COLLECTION

We developed separate interview guides for the HSL and clinician study groups following constructs in the Consolidated Framework for Implementation Research (CFIR).¹² The interview guides included questions about experiences with telehealth implementation and sustainment relevant to intervention characteristics (adaptability, complexity, relative advantage); inner setting (implementation climate, readiness); outer setting (patient needs and resources, external policies and incentives); individual characteristics (knowledge/beliefs, self-efficacy); and implementation process (planning, engaging opinion leaders, executing). Table 1 provides definitions of the CFIR constructs used in this study and example interview questions pertaining to each. The full interview guides are provided as a supplement.

Table 1.

Consolidated Framework for Implementation Research Constructs and Example Quotations

CFIR DOMAIN	CFIR CONSTRUCT	DEFINITION (TAILORED FOR STUDY)	EXAMPLE INTERVIEW QUESTION
Intervention characteristics	Adaptability	The degree to which telehealth can be adapted, tailored, refined, or reinvented to meet local needs	If you were already using telehealth before COVID-19, did you change the way you used different telehealth strategies?
	Complexity	Perceived difficulty of telehealth, reflected by scope, disruptiveness, intricacy, and number of steps required to implement	How easy is it to place orders when using telehealth?
	Relative advantage	Participants' perception of the advantage of implementing telehealth versus in-person care	Has patient adherence to recommendations for orders differed between those who use telehealth and those who come in person? How and why?
Inner setting	Implementation climate	Health system and clinic capacity for change, shared receptivity of participants to telehealth, and the extent to which use of telehealth will be rewarded, supported, and expected within health systems and clinics	How would you describe the support of your colleagues in your clinic or practice for telehealth?
	Readiness	Tangible and immediate indicators of health system and clinic commitment to the decision to implement telehealth	How would you describe the readiness in your clinic to implement or expand telehealth?
Outer setting	Patient needs and resources	The extent to which patient needs for telehealth, as well as barriers and facilitators to meet those needs, are accurately known and prioritized by health systems and clinics	Are there particular patient populations where you think telehealth is a challenge? Which ones? Why?
	External policies and incentives	External strategies to spread telehealth, including policies, regulations, and mandates; recommendations and guidelines; pay-for-performance initiatives; and public or benchmark reporting	Did you face any issues regarding cost and reimbursement for telehealth visits? What issues did you face?
Individual characteristics	Knowledge/beliefs	Participants' attitudes toward and value placed on telehealth, as well as familiarity with details related to telehealth	What do you see as the future of telehealth in your clinic, in your health system, and nationally?
	Self-efficacy	Participants' belief in their own capabilities to implement and use telehealth	How would you describe your ability to help patients meet their goals using telehealth compared with face-to-face?
Implementation process	Planning	The degree to which tasks for implementing telehealth were developed in advance, and the quality of those tasks	When you implemented or expanded telehealth in response to COVID-19, what was the planning process used?
	Engaging opinion leaders	Attracting and involving individuals in health systems and clinics who have influence on the attitudes and beliefs of their colleagues with respect to implementing telehealth	When you implemented or expanded telehealth in response to COVID-19, did you engage key health system or clinic leaders?
	Executing	Carrying or accomplishing implementation of telehealth according to plan	How successful was your clinic in implementing telehealth?

CFIR, consolidated framework for implementation research.

Six professional surveyors with the University of Florida Survey Research Center were trained in data collection by a qualitative methods expert (R.P.T.). Two additional trained interviewers collected data at the Villages Health. Interviews were conducted by means of distance communication (videoconference or telephone), lasting 26–99 min (mean: 48 min), and were audiorecorded and transcribed for analysis.

DATA ANALYSES

Data coding and analyses followed principles of team-based coding using template and framework approaches to data reduction, which are appropriate for the needs of implementation science research to rapidly synthesize findings for diverse stakeholders.^13–15 A team of coders (R.P.T., J.D., M.M., and R.G.) developed a deductive Excel template, which was used for transcript review and summary. Each transcript was summarized by a single coder, and the summary was then reviewed against the transcript by a second coder for agreement. Reviewed summaries were discussed in team meetings to resolve discrepancies.

The team compiled these reconciled summaries into a single respondent-by-domain matrix.¹⁶ Abstracted content underwent inductive coding in three broad categories: (1) specific approaches used by health systems for telehealth delivery, (2) barriers and facilitators to telehealth implementation and sustainment, and (3) impacts of telehealth implementation on health systems, clinics, providers, and patients. Inductive codes were compared across coders and compiled into code-by-property matrices, which allowed coders to assign groupings, CFIR constructs, and valence properties to each code, as well as maintain a coding audit trail. The team conducted thematic analysis of content in the coded matrices and identified meaningful groupings and differences by study site or group.

Data saturation, defined as the point at which no new codes emerged in successive interviews, was assessed in the coding audit separately for each of the three coding categories. For telehealth delivery approaches, saturation was reached after the 19th interview. Data saturation was not reached for determinants of telehealth implementation and sustainment, nor for impacts of telehealth implementation.

Results

Interviews were conducted with 24 participants: 5 PCPs, 8 specialist clinicians, and 11 HSLs (Table 2). Half of respondents were from academic health systems, and one-third were from private health systems. Three respondents were from a behavioral health system and one from a private hospital.

Table 2.

Participant Characteristics

PARTICIPANT CHARACTERISTIC	N	%
Study site
Academic health system	12	50%
Private health system	8	33%
Private BH system	3	13%
Private rural hospital	1	4%
Respondent group
PCP - Internal Medicine	4	17%
PCP - Family Medicine	1	4%
Specialist - Endocrinology	5	21%
Specialist - Neurology	1	4%
Specialist - OB/GYN	2	8%
Health system leader	11	46%
Years in practice/service
1–10 years	7	29%
11–20 years	6	25%
21+ years	11	46%

PCP, primary care provider.

Interviews revealed themes in three categories, highlighting that: (1) Implementation of telehealth during COVID-19 occurred rapidly; (2) Common barriers to effective telehealth use also impeded implementation; and (3) Unique features of telehealth influenced provider acceptability. Within each category, respondents described factors that were associated with success of implementation, as well as the impacts of implementation on practice, as summarized in Table 3.

Table 3.

Telehealth Implementation Themes and Quotations

CATEGORY	THEME	RESPONSE
Rapid implementation of telehealth during COVID-19	Fast implementation despite low readiness	“[The health center] started rolling it out pretty rapidly… and flipped the switch within a couple of weeks, I think, of when we had to shut it down… I think if you were to say the first day we had to go to telehealth, zero readiness. Nobody had ever done it before, so nobody knew how to do it. So, it was a concerted effort of trying to educate and reach out and create guidelines and stuff. But once we had a plan and had implemented it, I sat down with all of the providers individually, if needed, to make sure they were ready to do it.” [OB/GYN, academic health system, 1–10 years in practice]
	Responsive implementation plans	“So, we did not have anything prior to the lockdown. We had stuff developed on a priority basis, on an emergent basis. And there were already, I think, materials available from the local medical societies. And so, people could put together policies and procedures… They were willing to invest in equipment like cameras, and download the software, and enroll everybody in the software… This was all done pretty quickly within a few days.” [PCP, private health system, 11–20 years in practice]
	Rapid response to IT and EHR challenges	“[The university] just put in – they're called dot phrases – just a phrase that you could put in that just populates in the note you're writing. So, it says this was done via telehealth, and if you couldn't do it via telehealth, why that happened, and it was because of COVID-19. They're canned statements. So, you just put that at the end of your consult letter, it was done so fast. It was great. And I think probably maybe some other clinics were using it, and that's what made it happen so fast.” [OB/GYN, academic health system, 21+ years in practice]
	Changes and increases in resource allocation	“If I tried to do this before the pandemic, this would have been impossible. It's like suddenly what would have been impossible for me to do as a researcher, magically, every door opened, and all the resources were thrown at it. So, we were able to do it within less than a month, which would have taken me three years to do previously because of the amount of manpower needed.” [OB/GYN, academic health system, 1–10 years in practice]
	Provider and staff training	“In a matter of about four days, we onboarded all 39 practices and about 600 providers. Again, we already had a lot of the core process in place, it was just duplicating it big time. The learning curve was quick, it was messy. We had to engage ambassadors who could learn this quickly and then go to practices and teach people how to use it and how to teach the doctors how to use their cell phone, to connect with their patients… We kept coming up with tools and we created screenshots and videos of ‘this is how you do this’, and ‘this is how you do that’ and tried to make it very simple and create a resource library for everybody.” [HSL, private health system, 1–10 years in service]
Common barriers to effective telehealth use and implementation	Clinician challenges with technology	“For seasoned clinicians, I think it's a challenge to transition to an electronic world. It's difficult enough for certain generations to transition to an electronic medical record, but then when you add having to kind of navigate that telehealth portion… I think that that's challenging. I think that there's a level of frustration when the appointment does not go off well, when there's some kind of static or the call is dropped, or it's not working. They feel very guilty, that they're not able to engage with their patient because of some technology limitation.” [HSL, private health system, 21+ years in service]
	Patient challenges with technology	“The people who I was unable to provide what I would consider standard of care were the people without internet connection. They couldn't see their medication lists. They couldn't see test results. All these things that I could normally give them in their instructions, I would have to read the instructions as they would write them down on a piece of paper.” [HSL, academic health system, 20+ years in service]
	Lower reimbursement for telehealth	“I am very concerned that they will start soon to take away the reimbursements at an equal level for telehealth, which is going to be a detriment to patients, I feel, because it will limit what their providers want to offer. If I can get paid more to bring you in person and it takes you 3 h to get on the transportation to come here… I'm going to do it, because I have to keep my clinic and I need to have my operations satisfactory.” [HSL, academic health system, 20+ years in service]
	Inconsistent commercial insurance reimbursement	“There was some initial denials. So, we had double-duty on some of these to make sure we got reimbursed. In the state of Florida, [insurance company] tried to reduce the reimbursement to 75% and we weren't gonna sign the contract for that. And then I guess they got pushback from a lot of health systems, so they went back to 100%… So, we were able to keep it. I think it's just really having to watch it and track it by payor by payor to see… Initially… they all had different end dates. ‘We're going to stop reimbursing in 90 days, we're going to stop at 120 days,’ and then they push it out last minute… So, that moving target became very difficult when you have a whole panel of different payors and they all had different end dates and now they're still reimbursing. So it was a lot of work tracking.” [PCP, private health system, 11–20 years in practice]
Unique features of telehealth	Lack of in-person exams and procedures	“I think the new patient visit – [it] was probably more disappointing to have that converted to telehealth just because you're a new patient. You want to meet your physician in person, and they want to meet you in person, and we want to get an accurate set of vital signs, and weight, and just basically to get started with your care. So, not really a lot of pushback, but… just really that disappointment… Not being able to lay a stethoscope to the chest and be able to feel like you had thoroughly assessed the patient.” [HSL, private health system, 1–10 years in service]
	“Having eyes” in patients' homes	“The other bit is we can see their natural environment. That actually counts. If it's a chaotic environment that might change some of how a provider will guide that patient. That goes even beyond behavioral health. That can go for say an allergy clinic. If I'm your patient, and there's a dog running around behind me, you might be like, ‘…Did you ever think of maybe getting rid of the dog?’ I've had a couple of our nutritionists… when they're working with a patient on dietary habits, if the patient is really asking for help, they have been known to say, ‘Hey, are you comfortable if we take a tour of your refrigerator? Let's look at the foods you like and the foods you eat.’” [HSL, academic health system, 21+ years in service]
	Facilitating patient education	“I find the telemedicine is very helpful because they have the share screen so we can show them the results when I'm opening my EHR and they can see my screen and they understand things that I show them, videos like YouTube, how to use the medication correctly. So, I find that very helpful.” [PCP, academic health system, 21+ years in practice]

EHR, electronic health record; HSL, health system leader.

IMPLEMENTATION OF TELEHEALTH DURING COVID-19 OCCURRED RAPIDLY

The urgency of the pandemic led to the rapid implementation and expansion of telehealth, both to ensure the continued health and safety of patients and providers, and to maintain the viability of health systems and clinics. The vast majority of respondents (n = 22, 92%) reported that implementation occurred within a matter of weeks. Respondents cited several strategies that facilitated rapid implementation, including responsive planning, changes and increases in resource allocation, and provider and staff training.

Specialist clinicians in two academic health systems reported rapid implementation in their clinics even when readiness for implementation was low. While respondents at four health systems reported that no implementation plan had been in place before the pandemic, successful implementation under these circumstances was facilitated by rapid and responsive plans, and rapid response to information technology (IT) and electronic health record (EHR) challenges.

Respondents also described changes and increases in funding, staff, and equipment. At the health system level, rapid reallocation of resources was facilitated by support from leadership and availability of funding through grants and community initiatives. Consequently, reports of rapid and responsive implementation were more prevalent in academic health centers. As one PCP noted:

“We're lucky in the fact that we are a giant academic medical center and we have tons of people working on problems that can be scaled very quickly, that we were able to benefit from other groups that were fixing problems and get through things.”

Four clinics designated specific staff members to assist with scheduling and technical support. One clinic in an academic health system deployed a 3-person telehealth helpdesk for providers, patients, and support staff, which included an online collaboration platform to post updates, workflow changes, and documentation. In other clinics, these roles were taken on by staff less formally, such as having nurses, medical assistants, or front desk staff provide technical support for patients. Adaptability of staff was a key facilitator for effectively addressing these changes.

One-third of respondents (n = 8, 33%) reported needing additional telehealth equipment to support implementation. Clinicians turned to using personal computing devices to conduct telehealth in the initial stages of implementation. Some HSLs (n = 3, 13%) described other means of acquiring equipment at the health system level. One academic health system collected volunteer webcams from other parts of campus and purchased new webcams from a vendor. One private health system leveraged its contract with a wireless communications company to acquire equipment, and provided tablets to patients. Another private health system acquired equipment through foundation funding, a community drive for webcams, use of personal devices, and repurposing tablets that were previously used for wound care.

Nearly half of respondents (n = 11, 46%) reported that implementation was facilitated by telehealth training for providers and support staff. In some cases (n = 5, 21%), training efforts were immediate and rapid. At one academic health system, a team of 17 was deployed “to coordinate a mass deployment and training model” for providers:

“We were doubling capacity pretty much every week until we hit the peak of about 1800 visits a day. I think that shows a nimble response… We have over 1700 providers and practitioners set up to deliver telemedicine-based care today. That's going up from about, honestly, 40 before the pandemic…”

COMMON BARRIERS TO EFFECTIVE TELEHEALTH USE ALSO IMPEDED IMPLEMENTATION

Respondents cited several common barriers to telehealth use and implementation––in particular, difficulties with telehealth platforms and technology and with reimbursement for telehealth services.

Many respondents (n = 10, 42%) described technical challenges encountered by clinicians, including malfunctioning applications and equipment, connectivity issues, equipment that was inadequate for telehealth (e.g., small screens), poor integration with EHR, and difficulty getting electronic signatures. Two respondents noted that older clinicians, in particular, had difficulty adapting to telehealth platforms and technology. Most respondents (n = 19, 79%) also described challenges encountered by patients, including poor connectivity, particularly among rural patients, and limited access to technology for low-income patients. Respondents noted that elderly patients had difficulties understanding and interacting with telehealth technology. Patients were less likely to join or participate fully in telehealth visits because there was no privacy in their home or in public locations.

Implementation and sustainability of telehealth were strongly influenced by reimbursement. Many respondents (n = 10, 42%) stated that reimbursement was lower for telehealth than for in-person visits during the first year of the pandemic. Some respondents (n = 5, 21%) noted that telephone visits were not reimbursed or were reimbursed less than video visits. While expansion of reimbursement for Medicare beneficiaries (enabled by the CMS 1135 waiver) contributed significantly to the success of telehealth implementation and sustainment, reimbursement by private payors was inconsistent and subject to frequent changes. Two respondents from the same private health system noted that some payors began to fully reimburse for telehealth services only after receiving pressure from providers.

UNIQUE FEATURES OF TELEHEALTH INFLUENCED ACCEPTABILITY

Respondents remarked on features that are unique to telehealth, in comparison to in-person visits, such as the challenge of being unable to conduct physical exams and procedures, and the benefits of “having eyes” on patients' home environments and using tools to facilitate patient education. These features were described as impacting the delivery of effective care, and influenced acceptability of telehealth among HSLs and clinicians.

A common challenge was the inherent lack of opportunities to conduct physical exams during telehealth visits, which can lead to misdiagnosis. Furthermore, at the start of the pandemic, the inability to perform physical exams and procedures led to significant reductions in delivery of preventative services, which led to delays in diagnosis and care. As one PCP noted:

“We saw a big fall in incidence of preventive screening tests such as colonoscopies, mammograms. Most patients preferred to put that on hold. As such, we had a delay in potentially new diagnoses, serious diagnoses.”

Some respondents (n = 7, 29%) described the benefits of being able to see patients in their homes, which allowed them to better tailor patients' treatment plans by seeing their medications, the contents of their refrigerators, their physical and social environments, and health behaviors such as smoking.

Screen-sharing allowed providers to easily communicate results to patients, advise them on the proper administration of medications, give them step-by-step instructions, and share images and information from other sources. Two specialists related that for patients with diabetes, videoconference is a suitable mode for evaluation with dietitians, insulin pump specialists, and diabetes educators.

Discussion

This study revealed factors associated with the successful implementation of telehealth in large health systems in response to the COVID-19 pandemic, from the perspective of HSLs and clinicians. A common theme was the adaptability of systems, clinics, and providers to address the immediate need for telehealth. Rapid and responsive plans, even in clinics with low readiness, contributed to successful implementation by focusing on IT challenges, acquiring and reallocating resources, and training providers and staff. Lack of workflow integration and resources have been cited as barriers leading to physicians' reluctance to use telehealth during the pandemic,³ which in our study was found, in part, to be addressed through leadership support and mobilization of resources within health systems.

The study also found that known barriers to telehealth use also impeded implementation during the pandemic. Issues with telehealth technology generally, and among elderly, low-income, and rural patients in particular, are well established in the literature both before and during COVID-19.^3,17–20 Our study showed that technical issues encountered by clinicians were also a salient barrier, leading to an increased need for technical support. To accommodate the new platforms, technologies, and workflows, health systems and clinics instituted rapid and coordinated training efforts for clinicians and staff. Training programs improved system and clinic capacity for telehealth that facilitated sustainability, which in turn can carry benefits beyond the pandemic.

Issues related to reimbursement were common, despite the lifting of restrictions by public and private payors. Both temporary and permanent changes to CMS telehealth regulations after March 2020 removed several barriers to telehealth implementation and practice for providers treating Medicare patients, including restrictions on who can perform and receive telehealth, where telehealth can be done, the types of equipment and platforms that can be used, and telehealth reimbursement.¹⁷ While state and private payors were reported to have lifted the same restrictions, our study revealed that reimbursement practices by private payors were inconsistent and subject to frequent changes. Some private payors offered no or lower reimbursement for telephone-only encounters, which were important to maintain continuity of care for elderly, low-income, and rural patients.

Research has found that telephone-only encounters can overcome the barrier of low technological literacy among elderly patients.²¹ Unfortunately, with the end of the public health emergency, reimbursement will continue to be a challenge. CMS has proposed reductions in telehealth reimbursement for hospital-based clinics beginning in May 2023, and provider-based clinics will see 25–30% reimbursement reductions by January 2024.²² Furthermore, after December 2024 Medicare will only cover telehealth in rural areas and will no longer cover telephone-only encounters (except for behavioral health). Further growth and evolution of telehealth care delivery will require CMS and private insurers to enact consistent policies on coverage and reimbursement.

Acceptability of telehealth for providers was influenced by features that have implications for quality of care. Studies have raised concerns about eliminating face-to-face interactions during COVID-19,^3,4,19 which our study revealed could lead to errors and delays in diagnosis and care. Research also shows that patients and providers question the quality of communication and engagement with patients using a virtual platform.^3,17,23 While providers in our study expressed the same concerns, many also related that telehealth promoted continuity of care, maintained clinician/patient relationships, and helped patients meet disease management goals during the pandemic, which is consistent with other reports.⁶

Provider concerns about the relative effectiveness of telehealth may be addressed through training that emphasizes known evidence. A review of meta-analyses on telehealth before the pandemic found that telehealth is equally or more clinically effective than usual care across a range of disciplines, demonstrating positive outcomes for stroke management, blood pressure control, mental health support, pain management, blood glucose reduction, and diagnostic services in dermatology and ophthalmology.²⁴ A study of quality of care for patients with diabetes before and after March 2020 found that, compared with patients who used in-person visits, those using telehealth were more likely to maintain performance on indicators of diabetes control.²⁵ For chronic conditions that require patient engagement to effectively manage symptoms, enhancing patient education through videoconferencing tools may offset concerns about engaging patients remotely.

The advantage that telehealth brings to care by offering providers a “window” into patients' homes, and the influence this has on acceptability for telehealth, is consistent with other reports from practitioners during the pandemic.^18,19,26 In particular, the ability to ask for and view patients' medications during a telehealth visit, was found in our study and in other studies to enhance the management of chronic conditions.^18,19,26 Having the ability to observe a patient's physical and social home environment can introduce prompts that elicit more history than might be achieved in the clinic,²⁷ which in turn can help providers tailor treatment plans and deliver more person-centered care. Videoconferencing also allows providers to see and meet other family members of the patient, who would not otherwise join the patient for in-person visits, but who can offer valuable insight into the patient's health care needs.^18,19,26

This study adds to the existing literature on implementing organizational change during public health emergencies and crises,^28–30 including qualitative studies that address telehealth implementation and use during COVID-19 specifically.^{9,10,18,19,26,31} Including perspectives of both HSLs and clinicians allows for a more meaningful contribution to stakeholder-informed frameworks that address barriers and facilitators to telehealth implementation and guide adoption of telehealth in health systems.^32,33 Our team-based approach to coding and analysis also improved the dependability of findings.

The study also had several limitations. Underrepresentation of HSLs and clinicians practicing in rural parts of the state may have resulted in insufficient depth of findings regarding the influence of rurality on experiences with and acceptability of telehealth. A sample size of 24 respondents was sufficient to reach data saturation within the category of telehealth delivery approaches, but not for determinants of telehealth implementation and sustainment, nor for impacts of telehealth. It is possible that additional interviews may have revealed more information on these topics, as well as information pertaining to specific sites or provider types. Furthermore, while the study included perspectives of HSLs and clinicians in multiple health systems, these were limited to health systems in the state of Florida and findings may not be transferable to health systems in other states. It is important to note qualitative work is not designed to make inferences to the population, but rather to elicit the specific contexts in which experiences with health care unfold.

Conclusion

This study demonstrated that health systems and clinics within the OneFlorida+ Clinical Research Consortium encountered many established barriers to implementing telehealth during COVID-19. Strategies to facilitate implementation contributed to the sustainability of telehealth after stay-at-home restrictions were lifted. However, sustainability was adversely affected by concerns about quality of care and inconsistent reimbursement policies. Continued training of clinicians and staff in the use of telehealth tools, specifically focusing on the unique advantages of telehealth and the instances where the technology is best used, may help to ensure sustainability in years to come.

Authors' Contributions

R.T.: Conceptualization, Methodology, Writing—Original Draft, Review & Editing, Formal analysis, Resources, Data Curation, and Supervision. J.D.: Investigation, Writing—Original Draft, Review & Editing, Formal analysis, Resources, Data Curation, and Project Administration. M.M.: Investigation, Writing—Original Draft, Review & Editing, Formal analysis, and Data Curation. R.G.: Investigation, Writing—Original Draft, Review & Editing, Formal analysis, and Data Curation. S.W.: Review & Editing, and Formal analysis. T.D.: Conceptualization, Methodology, and Review & Editing. F.M.: Conceptualization, Methodology, Review & Editing, Supervision, and Funding Acquisition. O.C.: Conceptualization, Methodology, Review & Editing, Supervision, and Funding Acquisition. E.S.: Conceptualization, Methodology, Review & Editing, Supervision, and Funding Acquisition.

Disclosure Statement

All authors have reviewed the article and approve its submission, and none has potential conflicts of interest. The study was approved by the University of Florida Institutional Review Board (IRB-01).

Funding Information

Research reported in this publication was supported by the OneFlorida+ Clinical Data Network and funded by the Agency for Healthcare Research and Quality (No. R01HS028283). The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality, its Board of Governors or Methodology, the OneFlorida+ Clinical Research Consortium, or the University of Florida's Clinical and Translational Science Institute.

Supplementary Material

Supplementary Data