“Beyond Just a Supplement”: Administrators’ Visions for the Future of Virtual Primary Care Services

Abstract

Purpose:

The COVID-19 pandemic resulted in unprecedented adoption and implementation of virtual primary care services, and little is known about whether and how virtual care services will be provided after the pandemic ends. We aim to identify how administrators at health care organizations perceive the future of virtual primary care services.

Methods:

In March-April of 2021, we conducted semistructured qualitative phone interviews with administrators at 17 health care organizations that ranged from multi-state nonfederal delivery systems to single-site primary care practices. Organizations differed in size, structure, ownership, and geography. We explore how health care administrators anticipate their organization will offer virtual primary care services after the COVID-19 pandemic subsides.

Results:

All interviewed administrators expected virtual primary care services to persist after the pandemic. We categorize expected impact of future virtual services as limited (n = 4); targeted to a narrow set of clinical encounters (n = 5); and a major shift in primary care delivery (n = 8). The underlying motivation expressed by administrators for providing virtual care services was to remain financially stable and competitive. This motivation can be seen in the 3 main goals described for their anticipated use of virtual services: (1) optimizing medical services; (2) enhancing the patient experience; and (3) increasing loyalty among patients.

Conclusions:

Health care organizations are considering how virtual primary care services can be used to improve patient outcomes, access to care, and convenience of care. To implement and sustain virtual primary care services, health care organizations will need long-term support from regulators and payers.

Introduction

The COVID-19 pandemic holds the potential to permanently reshape the delivery and configuration of health care in the US. In March of 2020, many health care organizations rapidly pivoted to providing virtual services, including telephone, video, and asynchronous care.¹ By the fall of 2020, nearly 2-thirds (up from 18% in prior years) of Medicare beneficiaries reported that their clinician offered virtual appointments and nearly half of those beneficiaries reported attending a virtual visit in the summer or fall of 2020.² In the spring of 2021, outpatient visits rebounded to volumes that exceeded levels before the pandemic and patients continued to engage in virtual care with the share visits which were virtual remaining elevated (9% in from March to August 2021).³

Before the COVID-19 pandemic, virtual services were often envisioned as ways to extend the reach of specialists and promote patient independence through remote, home-based monitoring, which largely excluded primary care.⁴ Previously, regulators and payers had strict requirements on when and to whom virtual services could be delivered.^4,5 For example, patients typically had to be within a health care facility to receive virtual services.⁴ As a result, virtual services were rare – only 0.3% of Medicare Part B enrollees had a telehealth service in 2016.⁴ Concerns about the safety of in-person services due to the COVID-19 pandemic forced payers to ease restrictions.^2,5–9 The combination of relaxed regulations and the hesitancy to offer in-person services spurred innovation in primary care.

Whereas the pandemic drove innovation in virtual service delivery out of necessity, it also provides an opportunity for sustained adoption of new virtual models. Informed by their experiences during the pandemic, which services will health care organizations choose to offer virtually? In this article, we use interviews with administrators from a diverse set of health care organizations to garner insights on their visions for how primary care service delivery might be transformed beyond the pandemic.

Methods

Data Collection

From March-April 2021 we conducted semistructured interviews with administrators at 17 health care organizations. The Institutional Review Board at Dartmouth College approved this study.

We identified organizations by selecting primary care practices and delivery systems that responded to the National Survey of Health Care Organizations and Systems (NSHOS).¹⁰ NSHOS is a suite of nationally representative surveys that were conducted in 2017 to 2018.^10–17 The NSHOS delivery system level survey (response rate = 57%) included systems with: (1) 1 or more hospitals and 1 or more physician practices; (2) no practices, but 2 or more hospitals; or (3) no hospitals, but 2 or more practices. NSHOS excluded systems owned by the federal government and systems focused on a single specialty (ie, cancer). The NSHOS practice level survey (response rate = 44%) surveyed primary care and multispecialty care physician practices that included 3 or more physicians. Practices were defined based on a single location. We used NSHOS to identify organizations for interviews because NSHOS includes a large, national sample of diverse health care organizations.

We emailed executives and asked them to connect us with the individual(s) at their organization who was best suited to speak about how their organization adapted care delivery during the COVID-19 pandemic. Individuals identified held a range of titles such as Chief Innovation Officer, Program Manager, and Director of Population Health (Online Appendix Table 1). Titles varied by organizational size and structure. Most of organizations (14 of 17) included an executive leader. We refer to interviewees as administrators because these individuals, regardless of title, were responsible for managing how their organization adapted care delivery during the COVID-19 pandemic (Online Appendix Tables 1 and 2). In most organizations (11 of 17), the interviewee was a physician leader. To ensure a diverse sample, we conducted outreach in waves, adjusting each wave as necessary to ensure diversity in size, urbanicity, and geography (Online Appendix Table 3). We continued data collection until we reached a point of saturation and no longer uncovered new themes in interviews.¹⁸

Interviews focused on how organizations adapted care delivery during the COVID-19 pandemic including (1) implementation of virtual services, (2) changes to in-person care, (3) expectations for virtual services postpandemic (Online Appendix Table 4). All interviews were conducted via telephone, lasted approximately 60minutes, and were reco-rded. Trained qualitative interviewers (T.F., a PhD level health services researcher with advanced training in qualitative research and L.B., an MPH-level researcher with expertise in health care delivery and qualitative methods) conducted the inter-views and analyses.

Data Analysis

We (T.F. and L.B.) first conducted iterative, unblinded double coding to ensure consistency between coders and to establish a deep understanding of the data.¹⁹ We used an established codebook that aligned with domains in the interview guide (Online Appendix Table 4). All coding was conducted using QSR NVivo.²⁰ Then we analyzed data initially coded as “expectations for use of virtual care services after the COVID-19 pandemic.” 1 team member (L.B.) conducted intermediate coding on all transcripts which the lead author (T.F.) reviewed. To understand organizational expectations for the future of virtual services, we applied an iterative memoing process using advanced coding and storylining to further develop themes.^21–23 We used a matrix coding approach to examine how each organization fits within each theme.²⁴ For each identified theme, we summarized how each organization did or did not support the given theme and documented supporting quotes. We met weekly to discuss coding and analysis. Online Appendix Figure 1 details our approach.

Results

We interviewed administrators at 17 health care organizations: 12 were health care delivery systems (5 of which included at least 1 federally qualified health center, (FQHC), or critical access hospital); 3 multi-practice physician organizations (2 of which included FQHCs); and 2 single site primary care practices (1 of which was a FQHC). Organizations were diverse in terms of geographic location, population density, and size. At 11 of the 17 organizations at least 1 of the interviewees was a physician.

All interviewed health care administrators believed that at least some virtual primary care services would continue beyond the pandemic. Administrators’ visions on the role of virtual primary care in the future were classified into 3 categories: (1) limited to encounters as necessary or requested (n = 4); (2) targeted to a narrow set of clinical encounters (n = 5); and (3) a major shift in primary care (n = 8) (Table 1).

Table 1.

Anticipated Future Use of Virtual Primary Care Services

Limited Use (n = 4)	Targeted Use (n = 5)	Major Shift in care delivery (n = 8)
Virtual primary care services were not expected to have a significant role in their organization.	Virtual primary care services were expected to continue in focused, defined areas.	Care delivery was expected to meaningfully change because of virtual care models.
Administrators noted that they would offer virtual services if a patient requested it, but it would not be their preferred modality for care delivery.	Examples included virtual urgent care, behavioral health services, and Annual Wellness visits.	Anticipated having a large share of all services into virtual modalities or developing robust, innovative virtual models to offer options to work in parallel to in-person care options.
“I would say it’s strongly preferred to have an in-person visits over telehealth. But it’s a nice tool to have it if you need.”	“Of course, we’re still doing some telehealth and we’re looking at trying to see how we might be able to provide an after-hours telehealth and may, maybe a little bit more so to either help supplement urgent care centers or emergency rooms.”	“I think with proper education, every single specialty has a portion of their work that is suited to telehealth.”

Administrators who anticipated the future of virtual primary care as limited described offering virtual services in specific circumstances or when requested by patients. These administrators did not believe virtual services would offer revolutionary changes within their organizations, nor more broadly within primary care. Others believed that virtual primary care would be integrated into primary care delivery for a targeted, narrow set of clinical encounters. These administrators typically set organization-wide goals for virtual services in the 10 to 15% range of patient interactions.

About half of the administrators envisioned that virtual services would meaningfully transform primary care delivery in their organization. One organization expected up to 70% of services to be offered virtually. Others viewed virtual primary care services as a way to significantly expand or optimize service delivery such as by offering virtual urgent care services or e-visits to provide patients with asynchronous care for lower severity visits. One administrator explained, “idea of that is offering a 24/7 care model eventually.”

Motivation for Virtual Services: Remain Financially Stable and Competitive

Administrators emphasized that providing virtual services was essential to their organization’s financial sustainability (Table 2). They believed virtual services were necessary to remain competitive not only with other health care delivery systems, but also with technology-based companies, retail clinics, and payers. They also viewed virtual services as an opportunity to generate revenue for activities that may not have been billable in the past, such as a follow-up call from a physician or a video visit before refilling a prescription. Organizations in value-based contracts noted that virtual services were likely to enable them to better manage costs because they could deliver care more efficiently and simultaneously meet quality metrics.

Table 2.

Motivations and Goals for Virtual Primary Care Services

Motivation		Goals
Financial sustainability Administrators believed virtual care services were necessary to ensure the organization remained competitive and financially viable. Virtual care service options offered by payers and technology-based companies motivated administrators to offer virtual care services within their organization.		1. Optimize care delivery Administrators were exploring which services may be best suited to virtual care. Behavioral health, Medicare Annual Wellness visits, and follow-up visits for some conditions were considered well suited for virtual care. 2. Enhance patient experiences Offering services that were convenient for patients and that increased access to care 24/7 was a goal for most virtual care programs (this includes options for asynchronous visits). 3. Build loyalty Administrators felt that younger, healthier patients may be inclined to seek out virtual care for their primary and urgent care needs, so they hoped to establish relationships with those patients.

Goals for Virtual Primary Care Services

The underlying financial motivation was seen across 3 goals that health care administrators described for their future use of virtual services: (1) optimizing medical care services; (2) enhancing the patient experience; and (3) increasing loyalty among patients (Table 2).

Goal 1: Optimize Medical Care Services

The core areas where administrators thought virtual services would succeed were: (1) treatment of minor acute illnesses; (2) behavioral health; (3) care coordination; and (4) care management; (5) follow-up visits; and (6) annual wellness visits (Table 3). Many services, such as behavioral health visits, were viewed as equally well (or better) suited for virtual versus in-person care were services that did not require physical exams. Administrators emphasized the importance of ensuring that the quality of patient care not be negatively impacted if the service were virtual. Table 3 provides rationale and examples for each area where virtual services were viewed as promising.

Table 3.

Goal for Future Virtual Services: Optimizing Primary Care

Rationale for virtual approach	Quote
Treatment of Minor Acute Illnesses • Allow faster access to primary care with the goals of preventing unnecessary emergency department visits or avoiding care outside of the health system (e.g., a visit to an independent urgent care). • Designated clinicians for after-hours reduces the need for all clinicians to be on call. • Asynchronous services (e-visits) use algorithm-derived questionnaires to assess patient concerns and can be converted to video visits, as needed.	“so for pink eye it’s a structured questionnaire that kind of you know describe your eye, describe the discharge, and it gives you options. You know any other symptoms and so you fill out this questionnaire, you can take a picture of your eyes and then send it […] then goes to the nurse practitioner who reviews it […] is able to then determine what the treatment would be so in the case of pink eye, if it’s clearly pink eye, then you know, being able to just provide that antibiotic prescription through e-prescribing and then close the loop with the patient to go pick up the medication at the pharmacy always kind of the purpose that that patient did not have to talk to anybody to get their care taken care of.”
Behavioral Health • Process mirrors in-person, just conducted via virtual modality which means there are no observed clinical downsides. • Does not require touching the patient. • Patient may feel more comfortable.	“Our [behavioral health] therapists are doing 100% virtual care, right now, still now and they report that it’s a very successful, you know, tool for them.”
Care Coordination • Increase interactions between primary care and specialist clinicians. • Facilitates data sharing across care settings. • Provides a financial incentive for collaboration. • Examples included: (1) e-consults (EHR-based tool) which may reduce need for specialist visits, (2) one clinician attended patients’ video visits with specialists.	“On a zoom call with the consultant, and the patient in the room and, you know, often the patient will go to the [specialist] visit and they’ll tell you something and then you know you might get a note and you might have questions, and you know being on the call and getting paid to be on the call for what you’re doing. You know, encourages you to do that stuff and you know you get much better patient care when you’re actually collaborating together rather than through letters or emails or things like that.”
Care Management • Provide the same care management services as previously, but via video. • Allows care management staff to visually assess patients’ homes.	“I also think that some of our support and ancillary services will use telehealth also as another way to connect with their patients are in care coordination”
Follow-up Visits • Alternating in-person and virtual follow up for patients with chronic illnesses can reduce travel. • Virtual visits may make the patient more likely to attend visits due to convenience.	“My plan in the future is to do, alternating telehealth and in person visits, so that that yeah and because of the distance to travel and things like that in a rural community patients really you know they like that not having to drive in and wait and make up ground and do this it’s a much too much shorter quick a visit for them.”
Annual Wellness Visits • Virtual Medicare annual wellness visits ensures the visit is focused on preventive care rather than diagnostic services (which can be addressed in a subsequent visit). • Allows clinicians to assess risks within the home.	“they [Medicare wellness visits] really lend themselves to telehealth because one they can be done with a nurse practitioner, and that allows the nurse practitioner […] because they can’t lay hands on the patient into because it’s really focused on preventive. And wellness questions to ensure that their visit does not convert to a diagnostic visit, which is so easy to do when you’re talking to patients with comorbidities.”

Abbreviations: EHR, Electronic health record.

Goal 2: Enhance the Patient Experience

Administrators emphasized that virtual services should give patients more choice, reduce unnecessary travel time, and allow patients to access care conveniently (Table 4).

Table 4.

Goal for Future Virtual Services: Enhance Patient Experiences

Increase Access to Care	Improve Convenience of Care
Administrators described efforts to improve patients’ access to care through: • Virtual urgent care services to allow patients increased access to care for acute needs. • Asynchronous e-visits where patients complete symptom-specific questionnaires and then receive a diagnosis and appropriate treatment. • Blending follow-up schedules (e.g., mix of in-person and virtual).	Administrators aimed to make care more convenient via: • Video appointments to reduce travel burden (which may be particularly useful in rural areas). • Expanded hours through virtual care (e.g., allowing patients to seek care on their schedule).
“We have an urgent care telehealth service. It dominated, 3:1 ratio, female to male. It is dominated by 20, 30, and 40-year-old women. That’s who is using it. It makes complete sense. The hours that they want it are completely different hours than what traditional services are. They want the service at 6:00 or 7:00 in the morning because they need to know first thing in the morning, not waiting until 9:00. It’s everything Starbucks has always known.”	“How do we provide that virtual urgent care in the most simplest fashion possible? How do we make it as convenient 24hours a day when that night shift worker gets off work, or when that day shift gets off work. We need to be able to provide convenient and accessible care and meet the patients where they are, which is on their mobile devices, which is on the go, which is synchronous as well as asynchronous.”
“Well, this patient is due for an A1C, but we haven’t actually seen them in two years. So let’s make sure that we outreach to them. And then that conversation’s a lot easier because we can say, ‘Hey, well, we do have telehealth available.’”	“However, we need to be thinking about it as a tool to truly be able to deliver on that 24/7 care so one of the things that we have been working with our teams for is to say your traditional clinic is open from 8 PM to four or 5 PM the majority of individuals are working at those same hours, so how do we meet consumer demand as more of a 24 seven approach, because you know when you are seeking care.”

Abbreviations: A1C test, also known as the hemoglobin A1C or HbA1c test is a simple blood test that measures your average blood sugar levels over the past 3 months.

First, administrators described enhancing the patient experience by reducing the number of in-person visits. Follow-up appointments, including postoperative and chronic conditions management, were often described as appropriate for virtual settings. Administrators thought this could be especially useful for patients in rural areas or patients:

In a rural community, patients really, you know, they like that not having to drive in and wait and make up ground and do this, it’s a much too much shorter quick a visit for them, so I think they enjoy that part and you know I think they really it’s worked out very well.

Second, administrators described how virtual platforms might increase patients’ access to services, especially outside of traditional business hours. For example, e-visits allowed patients to describe their symptoms and receive asynchronous care. In addition, virtual clinics could extend hours to make care more accessible.

In fact, it’s a matter of well how do we meet the patients where they are? The 20 something year old who’s really healthy. How do we provide that virtual urgent care in the most simplest fashion possible? How do we make it as convenient 24 hours a day when that night shift worker gets off work, or when that day shift gets off work. We need to be able to provide convenient and accessible care and meet the patients where they are, which is on their mobile devices, which is on the go, which is synchronous as well as asynchronous.

Goal 3: Increase Patient Loyalty

Administrators viewed virtual services as an opportunity to strengthen relationships with patients and ensure patients would choose their organization for future care (Table 5). First, administrators aimed to engage with younger patients who have infrequent health care interactions because these patients may be more likely to seek future care with their care delivery system if they had prior experience with virtual services. Second, they wanted to compete with external virtual only services by highlighting the value of having both virtual and in-person services available within the same organization. Administrators highlighted the limits (and potential added costs to patients) for first seeking care with virtual-only services and then needing an in-person visit.

Table 5.

Goal for Future Virtual Services: Build Loyalty Among Patients

Compete with external services	Appeal to younger patients
Health care organizations aimed to prove the value of seeking virtual care within an established care delivery system: • Having both physical locations as well as digital services allowed them to provide better, more comprehensive care. • There were concerns that patients may choose virtual care services from an alternative provider (such as CVS or a telemedicine only company).	Administrators used virtual care to engage with younger patients and foster life-course care. By encouraging the use of virtual care: • Administrators hoped to build and maintain primary care relationships with younger patients. • To engage younger patients in preventive care activities.
“the pledge we made is if you come to one of our virtual urgent cares, and we cannot resolve your visit digital, if you come the same calendar day to one of our physical locations, there’s no additional charge for that other one. [...] Now you’re starting to create a value proposition for people. Versus saying, “Well, I went to CVS and they weren’t able to resolve my problem. They gave me an antibiotic,” which maybe that’s what you wanted and that’s what you’re going to get. If we can make a deeper connection that we’re there to help you with other things that show up and take a Disney approach, sometimes Disney Plus is good enough. You can just watch Moana. But sometimes I need to physically experience Disney. I think that’s a recipe for us to potentially succeed.”	“One of the areas that that the team is focused on right now is how do we engage those commercial patients, so seniors engage with their care pretty steadily you know, for the most part 80/20 rule there but our commercial patients are younger. They are less likely to engage on a regular basis with their physician. I mean if you just if you feel healthy like what’s the point kind of thing, though, where we see video visits really helping with that is it a video visit connection with a commercial patient to their physician is low effort from a commercial patient perspective if they’re healthy but allows us to stay connected and ensure that they have that PCP relationship. In the event that something does happen.”
“So, as we think about that 24/7, it is creating access. That is always available for patients that can either resolve or then direct them to that next best level of care and not just direct them to it, but actually make that connection for them and guide them there.”	“On the patient side, I want it to provide options, so you don’t have to call your practice, you don’t have to wait for months, and you can receive care the way you want to receive it. So some of our patients really have strong relationships with their PCPs and they want to go in the practice. That’s great. I want them to have that option. But I also want the populations that tend to trend younger, that don’t really want to go to the practice, that don’t want to have to call the office, they want to get the answer, to have that option as well.”

Barriers

Although most administrators were optimistic about the future of virtual services, they also expressed 2 key concerns. First, nearly all administrators noted that continued payment parity between virtual and in-person visits was a key concern. Several administrators noted that if the current reimbursement rates were to decrease, they may not be able to provide virtual visits. As 1 administrator explained:

We need coverage and payment parity. If they cover it in the office, they should cover it at home. Whatever they pay for that service, they should pay the same at home. [. . .] The challenge is there’s a misconception that telehealth is cheaper for the health system to provide than in-person care, when it’s actually the opposite, and it will be for some years.

Administrators emphasized that offering virtual services often required upfront costs (purchasing new software and equipment, providing technical assistance, and training staff) and that the costs per visit often remained the same as most organizations were still using staff members, such as nurses and medical assistants, during the visit.

Second, administrators were concerned that the currently relaxed regulations around virtual services would be tightened. The perceived instability in requirements and the variability between payers made administrators nervous about investing in new and innovative programs. One administrator explained this concern when discussing a proposed policy change:

We have a payment parity bill that’s going [. . .] through our state senate. They just threw in an amendment, which I’m very much hoping that gets thrown out because it’s ludicrous, saying, “If you’re doing telehealth, you must offer the patient the option of. . . if they see a nurse practitioner, you must give them the option of seeing a physician.” I’m like, “Are you kidding me? I mean, we don’t even do that in person.”

Discussion

The health care industry has a long-held reputation as being hard to disrupt, slow to change, and difficult to transform.^25–27 Innovations that require adapting care delivery workflows within care teams are particularly challenging to implement.²⁸ But the COVID-19 pandemic illustrated that rapid health care transformation is possible. Although initial changes were implemented out of necessity, sustained adoption and further innovation within health care delivery will be a choice.^29,30 Our interviewees offer a glimpse into the roles administrators at health care organizations anticipate for virtual services. No administrators thought that health care delivery would completely return to prepandemic patterns. Roughly half of administrators predicted that virtual services will contribute to major shifts in the future of care delivery across settings.

So, what might the future hold? Administrators emphasized that unlike pre-COVID virtual services that were mostly tethered to specialists,³¹ such as tele-stroke services,³² some primary care visits could be especially well-suited for virtual formats. Virtual care services could mitigate well-known care delivery challenges such as labor shortages and uneven labor distribution across geographies,^33–35 coordinating care across care settings,^36–38 and expanding access to behavioral health.^33,39,40 For example, even before the pandemic, behavioral health was thought to be potentially well-suited for virtual formats.⁴¹ Virtual behavioral health services could reduce disparities in patient access to care due to the uneven distribution of clinicians.^33,42 As a second example, care coordination is considered a foundational pillar of high-quality primary care yet has been challenging to achieve.^43,44 When a primary care team member can virtually attend a specialist visit with their patient, this shared experience can ensure that everyone – primary care, specialists, and the patient – is included in comprehensive care planning.⁴⁵ Whereas primary care teams often communicated with patients via telephone before the pandemic (eg, providing test results), virtual care regulations could enable providers to bill for these interactions.⁴⁶

Inspired by patient feedback during the pandemic, administrators considered how to make primary care services more attractive to patients. This pushed health care to be more like other service industries, where patients have greater control over how they interact with and access care. A recent survey found that 97% of American adults own a cellphone and 85% own a smartphone,⁴⁷ suggesting that most patients could access virtual services, if that was their preference. Not only were administrators aligning service options with patients’ preferences, but they aimed to improve the experiences of care team members (eg, increasing patient access while reducing clinician on-call hours).Administrators wanted to build a health care delivery system that was all-inclusive and incentivized patients to receive all their care within that delivery system. The result then would be to improve the financial viability of the delivery system through increased market share, reduced patient churn, and enhanced competitiveness with nontraditional health care organizations. Further, administrators felt that this tightening of their connections with patients, across the lifespan, would improve outcomes.

As health care organizations explore the future of virtual care services, they need support from regulators and payers that incentivizes them to both implement and strengthen innovations. Regulatory barriers have been well-documented by others, including rules against practicing across state lines,^48,49 poor reimbursement for care coordination,^50,51 lack of payment parity for virtual services,⁵² and inconsistency across payers.^52,53 As states return to prepandemic licensing rules, for instance, some patients are driving across state lines for virtual visits.⁵⁴ A National Academies of Sci-ences, Engineering, and Medicine report recently advocated permanently adopting the changes made during the pandemic.⁵⁵ Until final regulatory decisions are made, health care delivery systems are in a holding pattern. Administrators are not confident enough about payment to make significant investments in virtual services, but they simultaneously worry they will lose the ability to compete in the growing virtual care market that is filled by nontraditional players, including organizations like Teladoc,⁵⁶ Amazon Care,⁵⁷ and others.⁵⁸ One risk is that the virtual models that are easiest to implement (eg, have the fewest regulatory barriers) will be sustained, even if those services are relatively low-value. This may exacerbate disparities in access to care^59–62 –for example, some patients may not be able to access all care options due to regulation (eg, requiring use of specific encrypted platforms rather than phones) and may need to access care via ways that are not aligned with their preferences and perhaps not as high-value. Whereas administrators emphasized that they would only offer virtual services when of comparable quality to in-person services, policy makers should facilitate monitoring outcomes by modality to ensure quality is maintained.

Our study has several key limitations. First, as a qualitative study, these findings are not meant to be generalized to all health care organizations. Rather, findings provide context to help guide payers, policy makers, and others as they consider the future of virtual care. Further, our qualitative sample is limited. For example, we excluded federally owned health care organizations (such as Veterans Affairs or Indian Health Services). Our sample did not include many independent practices. Although our study included several organizations with a safety net component (ie, FQHCs), we likely did not capture the specific issues faced by safety net organizations. Our study, with 17 participating organizations, was not designed to facilitate comparisons between different types of organizational characteristics. In addition, our interviews were conducted with health care administrators. Administrators have significant influence over the development and direction of organizational policies and were best positioned to discuss the organizational approach to virtual care as well as the business challenges. More practice-oriented stakeholders, such as physicians or other care team members, may have differing views on the future of virtual services. In addition all participating organizations had adopted some level of new virtual care services, though research shows that adoption of virtual services has not been universal.^1,63 These organizations may fundamentally differ from organizations that did not adopt virtual care services during the pandemic. Finally, interviews were conducted during the spring of 2021, when many individuals were getting vaccinated and before later waves of the COVID-19 pandemic, which may impact how administrators viewed the potential of virtual primary care services.

Virtual care services hold the potential for a win-win-win for patients, care teams, and delivery systems. Yet without regulatory and reimbursement decisions that advance virtual services, high quality, sustainable virtual service models may not come to fruition. Although rife with challenges, virtual care offers an opportunity for regulators, policy makers, and payers to invest in technological advancements that can strengthen primary care service delivery.

Appendix

Appendix Figure 1.

Overview of Analytical Approach.

Appendix Table 1.

Interviewee Types

Interviewee Categories	Description	Examples
Executive Leader*	Individuals primarily responsible for overseeing the operations of the entire organization	Chief Executive Officer, Chief Clinical Officer, Chief Innovation Officer
Program Management Staff	Individuals who oversee specific departments or services	Program Manager, Program Director
Practicing Clinician	Individuals whose primary role was the provision of medical care	Physician

^*Many executive leaders were also trained clinicians. If they had a clinical degree, but spoke primarily about the administrative role, we denoted this by adding their degree type after executive leader.

Appendix Table 2.

Number and Role of Interviewees Per Organization

Organization	Interviewee 1 Role(s)	Interviewee 2 Role(s)
1	Executive Leader and Practicing Physician	n/a
2	Executive Leader and Practicing Physician	n/a
3	Executive Physician Leader	Program Management Staff
4	Executive Physician Leader	Program Management Staff (PharmD)
5	Executive Leader	Practicing Physician
6	Executive Physician Leader	n/a
7	Practicing Physician	n/a
8	Program Management Staff	n/a
9	Executive Leader	n/a
10	Executive Leader	Program Management Staff (RN)
11	Executive Physician Leader	n/a
12	Program Management Staff	n/a
13	Executive Leader	n/a
14	Executive Leader	n/a
15	Executive Leader and Practicing Physician	n/a
16	Executive Physician Leader	n/a
17	Executive Physician Leader	n/a

Appendix Table 3.

Organizational Characteristics

Number	Region	Organization Type*	Composition	Ownership**	Rurality***
1	West	Practice (Federally qualified health center (FQHC))	1 practice	Independent	Rural
2	West	Multi-practice physician organization, including FQHCs	<10 practices	Independent	Rural
3	West	System, includes FQHCs	<10 practices	Not applicable	Urban
4	Northeast	System	10 to 50 practices	Not applicable	Urban, suburban
5	West	System	<10 hospitals, >100 practices	Not applicable	Mix
6	South	System	10 to 50 practices	Not applicable	Mix, largely rural
7	Northeast	Practice	1 practice	System	Rural
8	Northeast	System including critical access hospitals (CAHs)	>10 hospitals, >100 practices	Not applicable	Mix
9	South	System, includes CAHs and FQHCs	>10 hospitals, >100 practices	Not applicable	Mix
10	Northeast	Multi-practice physician organization, including FQHC	<10 practices	Independent	Rural
11	West	System	<10 hospitals, 10 to 50 practices	Not applicable	Mix
12	West	System	<10 hospitals, 10 to 50 practices	Not applicable	Urban
13	South	System	<10 hospitals, >10 practices	Not applicable	Mix
14	Midwest	Multi-practice physician organization	<10 practices	Independent	Suburban
15	Northeast	System, includes FQHC	<10 hospitals, 50 to 100 practices	Not applicable	Urban
16	Midwest	System, includes a CAH	<10 hospitals, 50 to 100 practices	Not applicable	Mix
17	Northeast	System	<10 hospitals, >100 practices	Not applicable	Urban, suburban

^*For practices and multi-site physician organizations, we noted if they were an FQHC. For systems, we noted if the system included FQHCs or CAHs, but this not a focus area of our interviews.

^**We only included ownership for practices. None of the included systems were federally owned.

^***Many systems covered large geographic areas and includes practices in a mix of rural, suburban, and urban areas.

Appendix Table 4.

Interview Guide Domains and Probes

Organizational Structure

Overview of organization (size, leadership)

Overview of patient population

Community approach to COVID-19 pandemic

Virtual Care Implementation

Prior programming?

Modality (phone, video)

Role of care teams

Patient Reaction and Engagement

Uptake

Patient challenges with use

Concerns from patients

Financial impacts

Reimbursement

Provider productivity

Questions for leaders

Strategy

Achieve and maintain clinician buy in

Changes to approach during pandemic

Questions for clinicians

Experience and views of virtual care

Relationships with patients

Patients without access

Internet connectivity

Hearing or vision impairment

Privacy

Patients with social needs

Seeing into homes

Care management

Clinic transformation

Lessons we can learn from virtual care

Other changes due to COVID?

What will stay/go?

Next steps?

Plans to keep any virtual care services?

Support needed from policymakers

Process of returning to in-person

Will virtual care be part of healthcare forever now?

What role will virtual care have in 2023? (post-pandemic)