Community Health Centers Uptake of Telemedicine During the COVID-19 Pandemic: Trends, Barriers, and Successful Strategies

Abstract

Objective:

To describe telemedicine use patterns and understand clinic’s approaches to shifting care delivery during the COVID-19 pandemic.

Methods:

We used electronic health record data from 203 community health centers across 13 states between 01/01/2019 and 6/31/2021 to describe trends in telemedicine visit rates over time. Qualitative data were collected from 13 of those community health centers to understand factors influencing adoption and implementation of telemedicine.

Results:

Most clinics in our sample were in urban areas (n = 176) and served a majority of uninsured and publicly insured patients (12.8% and 44.4%, respectively) across racial and ethnic minority groups (16.6% Black and 29.3% Hispanic). During our analysis period there was a 791% increase in telemedicine visits from before the pandemic (.06% pre- vs 47.5% during). A latent class growth analysis was used to examine differences in patterns of adoption of telemedicine across the 203 CHCs. The model resulted in 6 clusters representing various levels of telemedicine adoption. A mixed methods approach streamlined these clusters into 4 final groups. Clinics that reported rapid adoption of telemedicine attributed this change to leadership prioritization of telemedicine, robust quality improvement processes (eg, using PDSA processes), and emphasis on training and technology support.

Conclusions:

In response to the COVID-19 pandemic, telemedicine adoption rates varied across clinics. Our study highlight that organizational factors contributed to the clinic’s ability to rapidly uptake and use telemedicine services throughout the pandemic. These approaches could inform future non-pandemic practice change and care delivery.

Introduction

The COVID-19 pandemic had a significant impact on the delivery of primary care practically overnight. By April 6, 2020, 42 States from the United States (US) had declared stay at home orders, ¹ and clinics ² that once delivered the majority of their care via in-person visits had to pivot to telemedicine-based visits, with varying degrees of success. Community Health Centers (CHCs) are of particular interest as they provide high quality health care to more than 30 million people, despite having limited resources and underfunding compared to larger health systems. ³

Prior to the COVID-19 pandemic, telemedicine use across the US remained low for both privately- and publicly-insured patients. ⁴ Poor reimbursement rates, lack of uniformity of policies, and lack of appropriate resources had stunted the growth of telemedicine across the US healthcare system at-large, especially in CHC settings.^{5 -7} However, in response to COVID-19 and the urgency to implement telemedicine services, federal and state agencies rapidly deployed policies to expand access to telemedicine services, waived cost-sharing for telemedicine visits paid by federal healthcare programs^8,9 and extended coverage to CHCs and Rural Health Centers (RHCs), ⁴ as well as dedicated $29 million per year over 4 years to invest in telemedicine infrastructure. ¹⁰ The results was a significant upward trend in telemedicine visit rates throughout the pandemic. ¹¹

Uptake of telemedicine was not uniform, it varied by clinic-level factors (eg, trainings, clinic staff and patient technological literacy, and provide perceptions) and community characteristics (eg, internet availability, median income, and race/ethnicity).^{2,12 -15} These studies^{4,12,13,16 -20} focus on describing overall adoption and barriers and facilitators of telemedicine. There is a paucity of research describing longitudinal patterns of telemedicine uptake and experiences associated with these patterns, particularly within a diverse group of primary care clinics serving socioeconomically disadvantaged and racially and ethnically diverse patients. Therefore, using mixed methods, this study examines patterns of telemedicine use over a 15-month period during the pandemic across a nationally representative sample of CHCs. Using latent class modeling and semi-structured interviews, we describe and explain the different approaches clinics took to shift care delivery to and sustain use of telemedicine. This approach can help provide an understanding of the unique strengths and challenges of CHCs to rapidly adopt to a new care delivery system, and the opportunities for learning on how to best support CHCs in rapidly adopting or adapting care delivery.

Methods

Study Setting and Sample

This study uses data collected for a larger project to evaluate the impact of the COVID-19 pandemic on the delivery of cancer screenings. ²¹ The larger project used a parallel mixed methods design and included CHC systems within the OCHIN practice-based research network. ²² OCHIN (not an acronym) is a nonprofit health information technology organization which provides a single, patient-linked instance of the Epic^® EHR. ²²

Given the parallel mixed-methods design of the larger project, ²¹ our study refers to 2 sample populations: (1) study sites of 203 CHCs (quantitative) and (2) qualitative sample sites of 13 CHCs (qualitative).

Quantitative study sites included in this study were CHCs listed as primary care and/or had records of preventive care visits, were live on the OCHIN EHR as of 1/1/2018, and had >10 telemedicine visits for each month of the post-study period. The sample included monthly clinic level data from 203 CHCs in 13 states (AK, CA, GA, IN, MA, MN, MT, NC, OH, OR, TX, WA, and WI).

Qualitative sample sites were purposively selected for the larger project to evaluate the impact of the COVID-19 pandemic on the delivery of cancer screenings. ²¹ As part of the larger project, CHCs health systems eligible for participation in the study had to vary by size (number of clinical sites in their health system), patient racial and ethnic diversity (ie, Black, Hispanic, and Asian), payer mix, including proportion of uninsured patients, and geographic location (eg, urban, suburban, and rural). These CHCs remained open during the pandemic and received information technology (IT) support from OCHIN.

Data Collection

Clinical level quantitative data were collected from January 2019 through March 2020 (pre-COVID) to assess pre-COVID, baseline data. These data continued to be collected for the study period, April 2020 through June 2021, a period of more than a year after the start of the pandemic. EHR data for study sites (n = 203) were extracted from the Accelerating Data Value Across a National Community Health Center Network (ADVANCE) Clinical Research Network (CRN) housed at OCHIN.

For qualitative sample sites, OCHIN recruited the health systems and worked with system leadership to identify1 to 2 clinics per health system to participate in semi-structured interviews. Recruited health systems ranged in size from 1 to 27 clinics. At the conclusion of recruitment, of the 203 CHCs in the sample, 13 agreed to participate. Because the recruitment of these clinics stemmed from a larger project ²¹ for which recruitment did not specifically target clinics based on the uptake of telemedicine, we did not monitor whether saturation was reached. Interviewees included clinic leaders and staff (eg, medical directors, clinicians, and quality directors) selected for their knowledge about operational changes made at the clinic during the pandemic. We conducted 1 to 4 interviews per clinic; this number varied based on clinic size and organizational structure. A total of 29 people were interviewed.

Interviews were conducted by 2 experienced qualitative researcher by phone between August 2020 and September 2021. Interviews followed a semi-structured guide developed inductively based on the original research question. The guide focused on operational and care delivery changes during the pandemic, including a question about telemedicine uptake and use. Interviews were approximately 60 min in length, were audio-recorded, professionally transcribed, and entered into Atlas.ti (Version 8.0, Atlas.ti Scientific Software Development GmbH, Berlin, Germany) for data management and analysis.

Measures

The primary quantitative outcome was telemedicine uptake and was defined as the percent of monthly telemedicine visits at the clinic level. Uptake was calculated by dividing the monthly number of telemedicine visits by the monthly number of total visits (in person or telemedicine) and multiplied by 100 to yield a rate of telemedicine visits per 100 total visits.

Other clinic-level covariates included patient panel characteristics (sex, age, race, ethnicity, Charlson Comorbidity Score Index, ²³ insurance status, and billable visits), number of medical doctors (MDs) and doctors of osteopathics (DOs) at the clinic and clinic geographic location (urban/rural).

Analysis

Quantitative Analysis

First, clinic level characteristics for the entire clinic sample were described. Then, to evaluate different trajectories in telemedicine uptake over the study period, a latent class growth analysis (LCGA) was performed to classify the clinics into the optimal number of clusters based on their distinct longitudinal trends of monthly telemedicine visits between April 2020 (start of pandemic restrictions) through June 2021. In brief, LCGA models have been increasingly utilized for their ability to identify homogeneous subpopulations within the larger heterogeneous population which can yield meaningful groups of clinics. ²⁴ The total number of unique mutually-exclusive clusters was selected by calculating the model fit indices, Bayesian Information Criterion (BIC) and the Integrated Completed Likelihood Criterion (ICL) for different number of clusters and selecting the groupings with the smallest values. ²⁵ For ease of interpretation and to further evaluate differences in telemedicine uptake trends, the clinic characteristics for each cluster were described. The quantitative team met to classify each of the clusters based on the distinct patterns of telemedicine uptake and trajectory of use based on the LCGA analysis. Analyses were conducted in RStudio 2023.06.0.

Qualitative Data Management and Analysis

The qualitative team followed the analytic process for health services research developed by Crabtree and Miller. ²⁶ As a team, qualitative analysts listened to and discussed interviews as a team. Interviews from the same clinic were reviewed as a set ²⁷ to understand each site’s setting, organizational structure, and process of telemedicine implementation. Through this process, the qualitative researchers developed a list of codes. All data related to telemedicine were tagged with a broad code, and additional codes were identified to capture emerging patterns. The qualitative team met weekly to analyze data, create, and refine codes based on team discussion and emergent findings, developed case summaries for each clinic, and identified factors at each site that appeared to influence the process of telemedicine uptake and use. Following within-case analysis we used the summaries to create a cross-case matrix display, ²⁸ allowing us to examine factors of interest across clinics (see Appendix Table A1 for the format, descriptive data have been removed). This cyclical and iterative immersion-crystallization ²⁹ process refined our understanding of how telemedicine uptake varied by site and over time, and helped to identify the leadership and organizational characteristics that influenced telemedicine uptake and use. To ensure analytic rigor, the qualitative team used a group analytic process to minimize researcher bias and debriefed the quantitative team about findings at least once to see if they had face validity. ³⁰ The qualitative team then systematically classified each clinic’s use of telemedicine based on these identified factors, independent of quantitative data (see Appendix Table A2).

Mixed Method Approach

To understand the factors that influenced the trajectories of telemedicine use, the quantitative and qualitative teams reviewed their independent classification together to identify overlap. Two of the clinics from the qualitative sample were excluded because they lacked the required threshold of 10 visits per month, which left 11 qualitative sites in the mixed methods analysis. Appendix Table A3 describes the distribution of study sample and qualitative sample sites before and after the mixed methods analysis (Appendix Table A3). It should be noted that the groups were quantitively determined and qualitatively harmonized, as such clinics within each group were quantitively aligned but could be qualitatively different.

Results

As seen in Table 1, most clinics in our sample were located in urban areas (n = 176) and served a majority of uninsured and publicly insured patients (12.8% and 44.4%, respectively) across racial and ethnic minority groups (16.6% Black, 29.3% Hispanic).

Table 1.

Characteristics of the 203 Study Clinics Over the Study Period (January 2019-June 2021).

Characteristics	Mean (IQR)
% Female patient	56.9 (53.5, 59.6)
% Age 30-49 years	31.0 (27.0, 36.6)
% Age 50-64 years	20.3 (15.7, 24.9)
% Age 65+ years	9.5 (4.5, 12.1)
% Black race	16.6 (1.1, 20.3)
% White race	64.5 (46.3, 85.7)
% Other race	3.0 (1.0, 4.4)
% Hispanic ethnicity (any race)	29.3 (7.2, 47.0)
% Patients with 5-6 Charlson Score	8.2 (5.0, 11.0)
% Patients with 7+ Charlson Score	10.0 (4.0, 14)
% Patient Uninsured	12.8 (4.0, 14.0)
% Patient with Medicaid	44.4 (34.3, 57.0)
% Patient mixed insurance	16.9 (11.9, 18.7)
Avg. # of MD/DO	13.4 (4.0, 14)
Avg. # of patients with billable visits	7669 (2918, 10 445)
# Rural clinics	27
# Urban clinics	176

The 203 clinics are located in AK, CA, GA, IN, MA, MN, MT, NC, OH, OR, TX, WA, and WI. The 13 recruited clinics were located in WI, MA, IN, CA, OH, OR, TX, and WA.

Prior to the COVID-19 pandemic, March 2019 to February 2020, 77 clinics out of the 203 clinics had a small number of telemedicine visits, which accounted for <1.0% of total visits in that timeframe. From April 2020 to June 2021, 200 out of the 203 clinics had any telemedicine visits, which accounted for 47.5% of all visits in that time period as illustrated in Figure 1.

Figure 1.

Percent of telemedicine visits by year (2019, 2020, and 2021).

The LCG clustering procedure resulted in 6 clusters represented in Figure 2. Cluster A clinics (n = 35) had no to low uptake in telemedicine. Cluster B (n = 34) had medium uptake and had a steady decline over the rest of the study period. Cluster C (n = 43) had low/medium uptake with steady decline over time, while Cluster D (n = 37) had medium/high uptake but with steady decline. Cluster E (n = 40) had high uptake of telemedicine with a steady decline. Cluster F (n = 14) had high initial and sustained uptake of telemedicine.

Figure 2.

Distribution of telemedicine adoption at the onset of the COVID-19 pandemic across study clinics by cluster (April 2020-June 2021).

The qualitative team identified 3 categories of initial telemedicine uptake: low, medium, and high (Appendix Table A2) based on variations of the leadership and organizational characteristics that influenced telemedicine uptake. This included the ways in which health system and clinic leadership were involved in telemedicine rollout, and details about telemedicine training. Three qualitative sites were identified as having low initial uptake; 6 qualitative sites were identified as having high uptake, and 4 were identified as having medium uptake.

The mixed methods analysis reviewed alignment between quantitative clusters and qualitative categories and identified 4 final Groups as seen in Figure 3. Group 1: Clusters A and B, including 4 qualitative sites; Group 2: Clusters C and E, including 5 qualitative sites; Group 3: cluster D including 2 qualitative sites; and Group 4: cluster F, which had no qualitative sites. [Figure 3]

Figure 3.

Final groups derived from the mixed methods “clustering of clusters” analysis.

Group 1: Low/Medium Uptake of Telemedicine With Steady Decline (A and B)

Overall, this group includes 69 clinics, 4 of them recruited for semi-structured interviews. Cluster A and B clinics are mostly urban, serve a similar proportion of Hispanic patients (~20%), and over 50% of patients were uninsured or had Medicaid coverage. These clusters differed in the proportion of Black patients served (23.6% vs 10.1%) and total number of visits (n = 228 456 vs 697 315; Table 2).

Table 2.

Characteristics of Study Clinics by Telemedicine Cluster Over the Study Period (April 2020-June 2021).

Mixed methods groups	Total	Group 1 low/medium uptake with steady decline		Group 2 medium/high uptake with steady decline	Group 3 high uptake with steady decline		Group 4 high uptake, remained high
Quantitative analysis clusters		(A)	(B)	(C)	(E)	(D)	(F)
N, Clinics	203	35	34	43	40	37	14
Rurality
Urban	176	26	27	43	35	31	14
Rural	27	9	7	0	5	6	0
% Female	56.9	55.1	61.7	55.8	57.1	55.9	55.2
% Hispanic	26.7	20.6	22.4	33.4	29.2	35.7	44.4
% ≥65 years	9.5	10.1	7.8	9.84	10.5	9.2	8.5
% Black	16.6	23.6	10.1	33.4	8.7	13.2	8.6
Charlson Score Index
5-6	8.1	8.4	7.8	7.3	8.9	8.2	8.1
7+	10.0	11.1	8.6	8.6	11.5	9.8	10.4
Insurance (04/20-06/21)
% Medicaid	44.5	34.4	45.9	43.7	49.2	44.8	52.1
% Uninsured	12.8	22.0	11	14.9	8.9	10.5	8.5
Number of health systems		22	21	19	24	27	9
Total # of visits 4/20-6/21	3 533 460	228 456	697 315	437 008	1 046 280	907 785	216 616
Avg % of telemedicine visits 3/19-3/20	0.6	0.2	1.5	1.0	0.5	0.3	0.0
Total # of telemedicine visits 3/19-3/20	15 351	837	2465	7706	2352	1987	2
Total # of patients with a visit in study period 4/20-6/21 (median (IQR))	5841 (2919, 10 437)	2697 (1157, 5688)	5846 (3547, 10 586)	5584 (2909, 8465)	7403 (5124, 11 538)	8665 (4065, 11 874)	5474 (3599 9873)

Prior to the pandemic lockdown, in February 2020, on average <1% of visits in this group were completed via telemedicine (Table 3). At the onset of the pandemic, this group averaged 37.7% of visits completed via telemedicine in April 2020 which declined to 10.9% by the end of our study period (June 2021).

Table 3.

Average Percent of Telemedicine Visits by Mixed Methods Grouping of Clusters.

Distinct time periods during the COVID-19 pandemic a	Group 1 (A and B)	Group 2 (C and E)	Group 3 (D)	Group 4 (F)
Average % telemedicine visits Feb 2020, mean (SD)	0.28 (1.3)	0.79 (1.8)	0.20 (0.7)	1.3 (2.3)
Average % telemedicine visits April 2020, mean (SD)	37.7 (29.9)	67.3 (26.7)	71.8 (13.4)	92.9 (3.4)
Average % telemedicine visits Nov 2020, mean (SD)	17.0 (14.5)	43.0 (22.4)	46.1 (8.1)	79.0 (6.3)
Average % telemedicine visits June 2021, mean (SD)	10.9 (11.0)	24.8 (17.9)	27.5 (10.8)	56.2 (14.2)
Average % telemedicine visits April 2020-June 2021, mean (SD)	19.0 (19.0)	43.2 (27.5)	48.9 (15.7)	78.0 (12.0)

^aTime periods were chosen based on last full month pre-COVID-19 restrictions (February 2020) and first full month under COVID-19 restrictions (April 2020). Intermedidate time periods were chosen based on equalivalent periods of months.

Semi-structured interviews with clinical staff highlighted that these sites had a more hands-off approach from clinic and health system leadership. This led to fragmented workflow and little to no telemedicine training or dedicated time for troubleshooting telemedicine-related challenges.

At this moment, we don’t have a telemed protocol for the organization. That’s one thing that we definitely need. . .Telemedicine protocol’s probably one area that is technically still deficient in terms of our team knowing what to do. They’ve been given the framework of it, so our team knows the different platforms that we use for audio and video. . . but we don’t have in the weeds [protocols]. – Chief Medical Officer, Site 4

I mean, in terms of any real formalized training, I would say no. I think that on most provider calls, definitely, challenges came up, and some protocols, potentially, on how to handle certain illnesses for example were discussed, but I don’t think that there was a real training on how to conduct telemedicine. –NP, Site 10

Further, clinicians in this group expressed resistance to telemedicine itself due to preferences or because they did not perceive that clinical visits via telemedicine offer sufficient patient information.

We also have some telemedicine yeah advice, but myself because I’m an old guy, I’d rather see people in-person than on video. – Medical Director, Site 8

It’s hard to do some of the OB visits, or even the GYN visits, online just because the exam is so important. It’s a very personal exam, too, so it’s not anything like people feel comfortable with over the telephone. – OBGYN, Site 3

There were some providers that were, especially for new patients, wanted to use non-telehealth, because it required an examination. – Medical Director, Site 3

Group 2: Medium/High Uptake of Telemedicine With Steady Decline (C and E)

Overall, this group included the largest number of study sites with 83 clinics, 5 of them represented in our qualitative sample. Both of these clusters are mostly urban, serve a similar proportion of Hispanic patients, patients ≥65 years, and most of their patients had Medicaid coverage or were uninsured. As seen in Table 2, Cluster C clinics served a larger proportion of Black patients (33.4%) than cluster E (8.7%), and had nearly 42% fewer visits during the study period than Cluster E.

In February 2020, prior to the pandemic, on average <1% of visits in this group were completed using telemedicine. At the onset of the pandemic, this rose to 67.3% (April 2020) and steadily declined to 43.0% in November of 2020 and to 24.8% at the end of the study (June 2021).

Qualitative data from semi-structured interviews with clinical staff highlight a strong prioritization from leadership on telemedicine-specific goals, such as the ratio of telemedicine versus in-person visits, and the ratio of telemedicine visits by phone versus video. Additionally, strong leadership drove much of the telemedicine implementation process, and influenced other key factors including formal training for all staff members, guided rollout of new workflows, convening a multidisciplinary team to discuss implementation and support from information technology (IT).

The executive team for [Site 1] starting in May made an organizational goal that 50 percent of our virtual care would be over video by October. We brought together an interdisciplinary group of folks across the organization. . .that full group met monthly and we developed several subcommittees that came out of that group of key priority areas around facilities and IT around workflow development. . . . We had a group focused on training. Staff training, from providers to front office, to medical assistants to make sure that everyone understood their part of the project or they’re part of onboarding a patient. . . so by October, we actually did meet our goal. – Chief Quality Officer, Site 1

Further, clinics that quickly adopted telemedicine in their practices exhibited processes found in Quality Improvement (QI) models for implementing a change in healthcare using rapid cycle improvement, similar to the Plan-Do-Study-Act (PDSA) cycle. These sites reported previous experience in implementing and applying QI tools and processes and directly applied this experience to their implementation of telemedicine services.

These are things that have just been tested out of the little PDSA cycles that had worked. We’ve also, for staffing issues with our medical assistants, if they’re able to help remotely, so right now while we’re having our meeting, my medical assistant is able to help other sites remotely. She doesn’t need to go in person, but she can call and help triage it and room the patient, patients at other sites. –Medical Director, Site 11

Another characteristic of clinics in this group was training clinic staff beyond clinicians, such as Medical Assistants (MA) and front desk staff, to provide technical support to patients, either during the virtual rooming or scheduling processes. Some clinics with large Hispanic populations were provided education about telemedicine in Spanish, and brought in Spanish-speaking interpreters to provide support during telemedicine visits.

Then one of the MAs, or sometimes we have a front desk dedicated to that when they were going through the schedule, to try to help all the patients have MyChart activation ready before the appointment. Then the providers don’t have to deal with that during that time. –Operations & Revenue Director, Site 5

Lastly, clinics in this cluster expressed a preference for keeping telemedicine in some capacity due to its accessibility for people with physical and mental health conditions or transportation barriers.

Again, telemedicine has been really important with my patient population and a lot of elderly folks, people dealing with dementia, people dealing with not being able to care for themselves as well at home. Telemedicine’s actually been wonderful. Twice now, at least, that out-of-town family member has been able to be part of the appointment on video. . .That’s really enhanced that patient’s care enormously. – Medical Director, Site 1

Group 3: High Uptake of Telemedicine With a Steady Decline (Cluster D)

This group included 37 study sites with 2 sites recruited for semi-structured interviews. Clinics in this group are mostly urban however, 20% of the clinics in this group were rural, the second most in our sample. One third of the patients identify as Hispanic (35.7%), nearly half of the patients have Medicaid insurance (44.8%), and saw an average of 8665 patients during the study period which accounted for 907 785 visits (Table 2).

February 2020, prior to the pandemic, on average <1% of visits were completed via telemedicine. At the onset of the pandemic, average telemedicine visits spiked to 71.8% of all visits (April 2020), slightly higher than the medium uptake seen in Group2. This steadily decreased to average 46.1% (November 2020) and 27.5% (June 2021) of all visits, respectively.

Qualitative data are limited in providing further insight into this group’s trajectory because of misalignment between the qualitative and quantitative data. The 2 qualitative sites included in this group were categorized as having either Low or Medium telemedicine adoption. These sites did not report on leadership prioritization or specific training for telemedicine, and instead preferred to remain open to in-person visits.

The low uptake qualitative sites is an employer-sponsored health system for agricultural workers. They already had telemedicine fully implemented prior to the start of the pandemic, but their priority was keeping the clinics open for the workers on-site.

We were never exclusively phone or virtual because people were always on site, and so my thought was if they’re on-site doing food stuff and they’re sick, I’d rather see ‘em in person, and pull them off the lines, and get them home than being like, “Oh, talk to us on the phone, even though you’re at work processing food.” We did have a in-person clinic throughout the entire pandemic. –Executive Medical Officer, Site 2

The medium uptake qualitative site reported a patient death from COVID-19 at the onset of the pandemic, which prompted them to quickly switch to telemedicine, despite limited guidance from leadership and no telemedicine training—although they did have access to IT support.

We weren’t really hit until end of March when my patient got sick and she died, and then another patient, also mine, came in coughing. We didn’t know what to do. Like, “Get her in a room. Isolate her. Do something.”. . .All of a sudden, it just arrived at [Clinic 6]. Also, at that same time, we started telemedicine, so—and honestly, once we started telemedicine, maybe the first week was a little slow, but once the patients got the hang of telemedicine, we were busy. – MD, Site 6

Group 4: High Uptake of Telemedicine and Remained High (F)

Group 4 included 14 clinics from the study sample, and none of our qualitative sites. All 14 sites were located in urban areas, served a large proportion of Hispanic patients (44.4%), and patients with Medicaid coverage (52.1%).

Prior to the pandemic, February 2020, clinics in this group averaged 1.3% of visits via telemedicine. These clinics switched almost entirely to telemedicine at the onset of the pandemic, with an average of 92.9% of all visits completed via telemedicine, which declined to 56.2% by the end of the study period (June 2021). We are unable to characterize the factors leading to this high uptake and sustained use of telemedicine, a limitation of our qualitative sampling design which was not based on initial quantitative groupings.

Discussion

This mixed methods study of primary care CHCs identified 4 distinct trajectories of telemedicine during the pandemic. To date, many previous studies have shown the uptake of telemedicine at the onset of the pandemic and the factors associated with use of telemedicine, but only use quantitative or qualitative work, or only focus on certain states or geographic regions.^{4,12,13,16 -20} This study adds to this body of evidence by examining sustained use, using qualitative semi-structured interviews to characterize the clinic experience associated with the patterns of adopting telemedicine, and using a national sample of clinics. The results show the critical importance of leadership support and experience with quality improvement to facilitate rapid uptake and support sustained use, and dedicated time for formal and informal trainings across all clinic staff members, rather than a specific subgroup (eg, clinician).

Experience with quality improvement was particularly indicative of a clinic’s familiarity with practice change management in our study. This primed clinics with the skills and potentially the resources to adapt their healthcare delivery rapidly (Group 2). These findings suggest the need for supporting increased capacity for QI in CHCs, provide trainings and internal or external support^{31 -34} to increase preparedness for rapid uptake of a new care delivery system, like telemedicine.

The importance of leadership support and training was evident especially among clinics who had lower uptake. Although the majority of our CHC’s rates of telemedicine increased dramatically at the onset of the pandemic, some clinics had lower rates of uptake (Group 1) and did not sustain use throughout the pandemic. Factors cited as limiting their uptake were the lack of formal training for clinical teams, clinician resistance to or preferences about adopting telemedicine, technology literacy, and a more hands-off approach from leadership which aligns with previously published literature.^{35 -37}

The results for sustainability highlight that clinics with supportive environments to adopt telemedicine and rapidly increase its use were more likely to continue to sustain use with some variations (Groups 2 and 4). These results cannot inform the specific circumstances that explain the variation in sustained use from 70% to 50% of the visits. It is likely that the reasons are multifactorial and based on patients and clinicians needs and preferences, ³⁵ payment reforms, and clinic resources. With the end of the Public Health Emergency (May 2023), Centers for Medicare and Medicaid and state programs are making changes to telehealth reimbursement.^38,39 The provisional nature of these policies could lead to changes in coverage for services or parity in payment, impacting the sustainability of telemedicine in CHCs. ³⁸ Specifically, for CHCs, changes to telehealth reimbursement mean fewer telehealth visits offered to their patients to ensure revenue. Many CHCs are in rural areas where lack of adequate broadband impedes the ability to conduct video visits. Reimbursement for telephone visits is more restricted in most states with some only reimbursing telephone visits occurring in underserved or rural areas and CMS not allowing reimbursement for Medicaid patients. Considering that telehealth significantly improved access to various services (eg, mental health care) to patients with access barriers, a reduction in the availability of telehealth in safety-net clinics is step backward to addressing health equity.

Strengths and Limitations

The strengths of this study include the nationally representative and highly varied sample of clinics in both the qualitative and quantitative samples. Additionally, many previously published studies about telemedicine use during the pandemic highlight only qualitative or quantitative findings,^{12,35,36,40 -43} while ours combines both methods.

However, this study has limitations. First, the qualitative data that we analyzed for this study was collected for a larger project about cancer prevention during the pandemic. While interviews contained useful data on telehealth implementation, the CHCs that participated in qualitative data collection were not selected specifically to inform the latent class analysis. As such, some of the clusters do not have sufficient clinics represented with interview data and we know little about the clinic’s telemedicine trajectory. Second, the qualitative data that we analyzed were collected at 1 time point, which limits our understanding of the factors that might explain the longitudinal trends we observed with regard to telehealth use in the quantitative data. Third, the interviews that we analyzed were not specifically focused on implementation and sustained use of telemedicine and the data we analyzed did not differentiate between phone and video visits, which is an important distinction with regard to reimbursement rates. Specifically, rural patients are more likely to rely on phone over video for telemedicine due to broadband issues.^44,45 Additionally, reimbursement for phone visits is a barrier for these clinics, especially as healthcare payors such as Medicare are changing their pandemic-era reimbursement, which is particularly challenging to CHCs in rural areas. ⁴⁶ To promote equity, policy makers need to identify reimbursement system to ensure that CHCs and their rural patients are not penalized for their geographic locations. Lastly, because the interview data were collected for another project, we could not monitor if saturation was reached, which may limit the transferability of our findings.

Conclusion

In this cohort of CHCs, experience with quality improvement, support from leadership, dedicated training, and IT assistance were found to be indicators of rapid uptake and sustained use of telemedicine during the COVID-19 pandemic. However, barriers to rapid uptake were also noted, including physician preferences and lack of support from leadership. In the event of a future need to rapidly adopt a new care delivery process, such as telemedicine, it would beseech health systems to consider these strengths and limitations to ensure the capacity of their clinics is adequate to make rapid changes to best support the communities they serve.

Appendix

Table A1.

Exploratory Qualitative Cross-Site Matrix.

Site ID	Staff outbreak of COVID-19	Leadership engagement in telemedicine uptake	Leadership support of staff	Previous QI experience	QI application during telemedicine uptake	Patient support through change	Telemedicine adoption	Care delivery adaptation
1	No	Engaged	Supportive	Experienced	Applied	Yes	High	High
2	[Unclear]	Unengaged	[Unclear]	Limited	Unapplied	No	Low	Low
3	Yes	Unengaged	Unsuportive	Limited	Unapplied	No	Medium	Low
4	Yes	Unengaged	Unsupportive	Limited	Unapplied	No	Medium	Medium
5	No	Engaged	Supportive	Experienced	Applied	Yes	High	High
6	Yes	Unengaged	Supportive	Limited	Unapplied	Yes	Medium	Low
7	[Unclear]	Engaged	Supportive	Experienced	Applied	Yes	High	Low
8	Yes	Unengaged	Unsupportive	Limited	Unapplied	No	Low	Low
9	No	Engaged	Supportive	Experienced	Applied	Yes	High	High
10	Yes	Engaged	Unsupportive	Limited	Unapplied	No	Low	Low
11	No	Engaged	Supportive	Experienced	Applied	Yes	Medium	Medium

Table A2.

Qualitative Categorization for Telemedicine Uptake.

Categorization	Features
High	Evidence of prioritization from leadership (eg, specific telemedicine goals) and all clinic staff trained formally
Medium	Limited formal training but evidence of some support (eg, regular discussion at clinician meetings, IT support available)
Low	Limited formal training and not all staff were trained and evidence of clinician resistance to telemedicine

Table A3.

Quantitative and Qualitative Harmonization for Mixed Method Clusters.

Qualitative site number a	Qualitative categorization	Quantitative cluster	Quantitative cluster name	Mixed method cluster
3	Medium	A	No or low Uptake	Group 1 Low/Medium uptake with steady decline
4	Medium	B	Medium uptake; steady decline
8	Low	A	No or low Uptake
10	Low	B	Medium uptake; steady decline
1	High	C	Low/Medium uptake; steady decline	Group 2 Medium/High uptake with steady decline
5	High	C	Low/Medium uptake; steady decline
7	High	C	Low/Medium uptake; steady decline
9	High	E	High uptake; steady decline
11	Medium	E	High uptake; steady decline
2	Low	D	Medium/High uptake; steady decline	Group 3 High uptake with steady decline
6	Medium	D	Medium/High uptake; steady decline

^aTwo clinics are excluded from quantitative analysis for not meeting the minimum required number of visits per month. There were no qualitative sites in Group 4, therefore it is excluded from this table.