Qualitative Assessment of Rapid System Transformation to Primary Care Video Visits at an Academic Medical Center

In response to the COVID-19 pandemic, primary care practices across the United States have transitioned from in-person visits to virtual visits. However, there is limited information regarding the facilitators and barriers to the implementation of such a transition. The objective of this study was to evaluate the short-term implications of rapid transition to video visits at Stanford Primary Care through qualitative interviews with key stakeholders.

Abstract

Background:

The coronavirus disease 2019 pandemic spurred health systems across the world to quickly shift from in-person visits to safer video visits.

Objective:

To seek stakeholder perspectives on video visits' acceptability and effect 3 weeks after near-total transition to video visits.

Design:

Semistructured qualitative interviews.

Setting:

6 Stanford general primary care and express care clinics at 6 northern California sites, with 81 providers, 123 staff, and 97 614 patient visits in 2019.

Participants:

Fifty-three program participants (overlapping roles as medical providers [n = 20], medical assistants [n = 16], nurses [n = 4], technologists [n = 4], and administrators [n = 13]) were interviewed about video visit transition and challenges.

Intervention:

In 3 weeks, express care and primary care video visits increased from less than 10% to greater than 80% and from less than 10% to greater than 75%, respectively. New video visit providers received video visit training and care quality feedback. New system workflows were created to accommodate the new visit method.

Measurements:

Nine faculty, trained in qualitative research methods, conducted 53 stakeholder interviews in 4 days using purposeful (administrators and technologists) and convenience (medical assistant, nurses, and providers) sampling. A rapid qualitative analytic approach for thematic analysis was used.

Results:

The analysis revealed 12 themes, including Pandemic as Catalyst; Joy in Medicine; Safety in Medicine; Slipping Through the Cracks; My Role, Redefined; and The New Normal. Themes were analyzed using the RE-AIM (reach, effectiveness, adoption, implementation, and maintenance) framework to identify critical issues for continued program utilization.

Limitations:

Evaluation was done immediately after deployment. Although viewpoints may have evolved later, immediate evaluation allowed for prompt program changes and identified broader issues to address for program sustainability.

Conclusion:

After pandemic-related systems transformation at Stanford, critical issues to sustain video visit long-term viability were identified. Specifically, technology ease of use must improve and support multiparty videoconferencing. Providers should be able to care for their patients, regardless of geography. Providers need decision-making support with virtual examination training and home-based patient diagnostics. Finally, ongoing video visit reimbursement should be commensurate with value to the patients' health and well-being.

Primary Funding Source:

Stanford Department of Medicine and Stanford Health Care.

As the 2020 severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic has expanded throughout the world, rapid adoption of virtual care has become a first line of defense (1, 2). Health systems have moved quickly to use video visits and other remote technologies to promote social distancing to protect patient and health care worker safety (3–6). Over 3 weeks, our academic medical center, like many others, converted almost all urgent and primary care patient visits to virtual visits and trained our front-line providers in a new series of best practices in clinical care.

The increase in video visits was only one of many adaptations to the pandemic at our center (7). In late February 2020, Stanford University and Stanford Health Care began implementing our pandemic plan (8). Researchers and health care system leaders collaborated with county health department, state, national, and international agencies to exchange epidemiologic information and clinical policies. The laboratory was among the first to receive U.S. Food and Drug Administration approval for a locally developed SARS-CoV-2 test (9). Local testing allowed Stanford to open the first drive-through coronavirus disease 2019 (COVID-19) testing operation in California (8). New clinical trials were launched for inpatient antiviral treatment and then planned for outpatient treatment.

Because video visits were a key part of our overall response, we sought to understand facilitators and barriers to program adoption in order to inform ongoing improvement (10–14). We also sought to understand the long- and short-term implications of this dramatic shift for ongoing medical care. To address these questions, beginning at the third week of video visit implementation, we conducted a primary care–wide evaluation of our video visit program through qualitative interviews with key stakeholders. We used the RE-AIM framework (reach, effectiveness, adoption, implementation, and maintenance) (15, 16) to identify critical issues that should be addressed for continued video visit program success.

Methods

Settings and Participants

Stanford's “general primary care” consists of 6 primary care and express care clinics at 6 sites in northern California, with 81 providers, 123 staff, and a total patient volume of 97 614 visits in 2019. Clinics transitioned to video visits within 3 weeks, starting on 28 February 2020 for express care and 3 March 2020 for general primary care. Four other Stanford specialty primary care groups (senior care, concierge care, coordinated care, and employer-based care) transitioned 1 week later and were not included in this program evaluation. From 23 March to 26 March 2020, qualitative interviews were done by 9 faculty assigned to interview medical assistants (MAs) and providers at each clinical site or to interview video visit program leads in technology and administration. The Stanford Institutional Review Board determined this evaluation to be nonresearch (protocol ID: 55692).

Program Description

Stanford Primary Care's rapid video visit program development and implementation related to COVID-19 screening have been described elsewhere (17). Briefly, during rollout, we rapidly created educational material, including patient care tools, virtual town halls, nursing and scheduling protocols, and mandatory video visit training (18). We obtained additional personnel, funding, and resources for rapid scaling, with Vidyo (Enghouse Systems) as our video visit vendor. Facilities operations set up new sites for drive-through testing. We created personal protective equipment training protocols to prevent workplace COVID-19 infections because initially a few staff and providers tested positive for COVID-19, and we changed communal workspaces to ensure social distancing (19). To ensure high-quality care, we created new provider training, quality assurance programs, and provider-focused daily COVID-19 updates. Medical assistants were recruited from ambulatory clinic float pools as patient COVID-19 test schedulers. We planned for “virtual rooming” in primary care, in which MAs would obtain home-based vital signs, help set patient agendas, update or preorder health maintenance and medications, and perform other previsit activities. Stanford's information technology department requisitioned and repurposed hundreds of computers to allow “sheltering-in-place” providers to work from home. Workflows were updated daily as new information about SARS-CoV-2 became available.

Over 2 weeks, Stanford Express Care converted more than 80% of its patient visits to video visits. The next week, the primary care clinics converted all possible visits—more than 75% of encounters—to video visits; 100% of providers had video visit training. In express care, new video visit providers had 5 charts reviewed to ensure care quality by 2 experienced clinicians. After their first video visit shift, they were given immediate feedback regarding several quality measures, including consent to participate, confirmation of geographic location, documented physical examination, and decision-making quality. At week 3, we began to evaluate early implementation efforts. We found that providers and staff would be less likely to complete surveys and more likely to share their program implementation viewpoints with site faculty through discussion.

Program Evaluation

Nine clinical faculty and administrators with research backgrounds were trained as qualitative interviewers through webinar sessions (20) using rapid qualitative analytic methods (21). Given the large number of site providers and staff, interviewers used a convenience sample of MAs, nurses, and medical providers (physicians, nurse practitioners, physician assistants) until thematic saturation was reached, typically 3 to 5 interactions per group. Interviewers did semistructured interviews using an interview guide with neutral opening questions and asked probing follow-up questions, inviting positive and critical comments (Supplement). Respondents were asked demographic questions and questions about comfort with new technology adoption (1 = not at all comfortable; 5 = extremely comfortable). With limited numbers of technologists and administrators involved in program rollout, interviewers purposefully sampled persons most likely to have programmatic insight, such as digital health administrative leads (architecting the program) or information technologists responsible for program implementation.

Interviews were done in person or by phone for 10 to 30 minutes using semistructured interviews focused on central guiding questions. At 2 sites, several MAs were interviewed together because of their time constraints. Interviewers took notes in the patient's own words whenever possible. Completed documents were uploaded to a Health Insurance Portability and Accountability Act (HIPAA)–compliant Stanford enterprise Box site. Five experienced qualitative researchers reviewed an initial subset of responses together to identify early themes using a grounded theory approach. Each researcher coded 10 to 20 interviews, working on a HIPAA-compliant collaborative file (Google Docs). Key quotes were identified, and each interview was read by at least 2 researchers. Final themes emerged after a consensus approach that included recoding notes, and disagreements were resolved via discussion or consensus. Thematic analysis was then critically reviewed and categorized according to the RE-AIM principles (15, 16). The team reviewed the resulting issues and posited next steps to address them.

Participant Demographic Characteristics

Fifty-three interviews were done across 6 sites with at least 5 participants represented in each site (range, 5 to 9 participants). Participants had (sometimes overlapping) roles as medical providers (n = 20), MAs (n = 16), nurses (n = 4), technologists (n = 4), and administrators (n = 13). Most respondents were women (79%). Most clinicians (providers and MAs) saw patients for greater than 50% of their time (average, 82% clinical full-time equivalents [range, 0.25 to 1.0 clinical full-time equivalent]). Twenty-six had full-time (n = 13) or part-time (n = 13 [range, 0.1 to 0.5 full-time equivalent]) technology or administrative roles. Respondents had completed college (n = 16); an MBA (n = 3); a non-MBA master's degree, including nurse practitioner or physician assistant (n = 15); or medical school (n = 15). Respondents were very comfortable (mean, 4.3 [range, 3 to 5]) adopting new technologies.

Role of the Funding Source

Stanford Health Care and Stanford University solely funded this video visit program implementation, and had no role in the design or conduct of the study.

Results

Twelve themes emerged from participant viewpoints of video visits, including both system-related (Table 1) and individual issues (Table 2). Themes are highlighted with quotes sharing relevant positive and critical viewpoints, with the quotation source in parenthesis (provider, MA, technologist, or administrator). Some comments were edited for grammar or ease of reading, but not content.

Table 1. Stanford's Video Visit Program Evaluation: Themes Regarding Systems*.

Table 2. Stanford's Video Visit Program Evaluation: Themes Regarding Individuals*.

Pandemic as Catalyst

“We needed a ‘Big Bang' approach when the pandemic happened” (technologist). The near total disruption of the pandemic created a desperate pressure to innovate, bringing together the university health care ecosystem to protect patient safety. “Barriers were wiped away during this pandemic…. [It] is exciting to see that we are finally able to implement projects that we have been planning” (administrator). Respondents commented that the sheer necessity for change forced leaders to move past their apprehension to trust in the system's ability to change as the situation evolved. It was a “very logical decision” (administrator). “[We] moved from a reactive approach to an enterprise approach” (technologist). In addition, technologists no longer needed to weigh competing priorities because leadership prioritized COVID-19–related work.

Collaborating Together With Meaning

“Academic medicine is not always known for being fast. This experience defied all that. We bonded, came together, and will be stronger on the other side” (administrator). Respondents shared that they felt a sense of teamwork, mutual respect, and meaning in the catalytic work of building a new care delivery system. “The important and amazing work helped to get [me] through it” (technologist). For many, participation in new system creation brought a sense of purpose and invigoration and helped them get through the long workdays. However, although teamwork seemed to deepen at the clinic and system level, one-to-one interactions, particularly interprofessional clinical interactions, were affected. For instance, “curbsiding” colleagues was not possible without seeing each other in a common workroom.

Safety in Medicine

“Everyone is scared to be coming to the doctor” (MA). “The patients' anxiety and fear was palpable” (provider). Respondents shared that they and their patients had not previously perceived that medical care involved deep personal hazard, including the risk for causing death. Without potentially sufficient personal protective equipment, respondents feared for their safety. Respondent safety concerns subsided with video visit program implementation, and they felt video visits were appreciated by patients. Under the hazardous circumstances that in-person visits represented, patients accepted video visits, reluctantly or willingly. “Patients are accepting and more accepting [of changes] than they would be due [to] the COVID scare” (provider).

Joy in Medicine

“I miss my contact with patients because I am used to greeting patients and learning more about their social life” (MA). Medical assistants' and providers' deep connection with their patients, often developed over years, provided enormous professional satisfaction. With video visits, MAs (including front desk staff) interfaced with patients in more administrative (not clinical) capacities. Some providers felt isolated in their video visit examination rooms without their usual collegial interactions and strong patient rapport facilitated by in-person visits. Others shared that they could spend longer with their patients and could connect quite well once they adapted. Some providers felt they had more schedule control without hectic running around and waiting for patient check-in and had a more peaceful pace, commenting that patients were also more peaceful and relaxed at their homes. Providers noted satisfaction in working from home, including flexibility with day care and scheduling meetings.

Growing Pains

“The work is different, and we are behind the curve in understanding the staff support that is needed for virtual visits” (administrator). Providers received training on how to use new technology, whereas staff training initially lagged behind. “[There] was not a lot of training on how staff should support the video visits at first” (administrator). Small technology issues, such as the inability to let patients check in early, as well as large issues, such as the video visit system crashing for a morning, were frustrating. As the system grew quickly, providers were also frustrated with having inappropriate patients scheduled for video visits versus in-person visits and wanted changes to the triaging and scheduling system. The rapid transition to video visits was facilitated by previous experience in some clinics. “The transition went smoothly because we were already using video visits; otherwise [I] could see the process being much more clunky” (provider). For clinics unfamiliar with video visits, this transition was facilitated in part by “templates, PowerPoint, [and] document handouts,” particularly “MD-generated content [that] was very helpful” (provider).

Slipping Through the Cracks

“[I] worry about missing some information when [patients] are just doing videos” (MA). Medical assistants and providers were concerned about critical clinical issues slipping through the cracks, including patient messages, laboratory results, and other staff messages with increasing patient volume. Providers expressed concerns about having patients scheduled incorrectly for video visits when they were actually quite ill and needed in-person visits. Some patients were reluctant to be seen over video. “My job has been to convince patients something to keep visits rather than cancelling the in-person to video [visit] conversion” (MA). For routine care in express care, patients without primary care providers were left on their own to follow up on scheduling of tests and procedures with radiology or other groups.

Doubt About My Video Examination and Decision Making

“Hopefully in a couple of months we won't have negative impacts: delayed diagnosis, guessed diagnosis” (provider). Providers were deeply concerned about their inability to do a physical examination, especially for patients presenting with more complicated chief symptoms, such as abdominal pain, where diagnosis is largely based on the examination. “[You] see what you see and that's it” (provider). Providers pointed out that some diagnoses rely exclusively on visualization (for example, seeing the tympanic membrane for inner ear pain). Other issues required critical physiologic information, such as oxygen saturation. Some providers and patients alike felt uncomfortable examining sensitive areas (for example, groin rash) through video. Providers were concerned that complicated patients would require “multiple visits” for issues usually resolved in 1 in-person visit. On the other hand, providers felt that uncomplicated patients could be managed well, including doing some examination maneuvers with the patient's help.

My Role, Redefined

“It is giving me fear that they don't need a lot of people in the clinics…[that] one day I might lose my job. But I try to make light of it and relax” (MA). In particular, MA responsibilities shifted. Medical assistants were initially pulled to schedule patients for video visits and COVID-19 testing. Without the need for patient check-in, rooming, and vital sign collection, MAs were fearful about their jobs, as they worried their services were less relevant now. Some providers were also dissatisfied: “All the onerous work is now on the provider. Staff used to check-in, room, vitals, med rec…and now we are tasked to perform every aspect of the visit.” In response, some clinics had MAs do “virtual rooming” previsits, similar to prior roles. Several administrators recognized the need to reexamine staffing models in the new environment.

Equitable Access for All

“Elderly patients have the most difficulty” (MA). Providers reported most video visits went well. However, they reported disparities in technology literacy and access for patients with cognitive, visual, or hearing difficulties or those who could not participate easily (like small children). Teenagers were initially unable to participate in video visits because they needed a health proxy. Initially, we could not have 3-party visits that included translators; providers used translators on their phone's speaker while conducting video visits by computer. Patients with poor or low internet bandwidth struggled with garbled audio, pixelated video, frequent dropped calls, and disruptive interruptions. Video visits excluded persons without means to pay for a smartphone or computer. However, access improved dramatically for others. Many vulnerable patients had transportation, childcare, or mobility issues, and video visits made obtaining health care easier. Providers reported increased satisfaction among these patients, including expressions of gratitude.

Is This Conversation Private?

“I had a patient whisper to me today, [who] had private things to share” (provider). Some patients were well resourced and positioned in comfortable, private spaces during the appointment, whereas others were in shared spaces out of necessity. In these cases, the private thoughts, stigmatizing conditions, and sensitive issues that were discussed could be overheard by the patient's family at home or by the video patient overhearing confidential conversations in clinic.

Is This Sustainable?

“I hope that we can keep this up” (provider). Previously, many insurers had not paid for video visits, and state licensing had limited the ability of providers to care for out-of-state patients. Stanford decided to offer video visits to all patients, regardless of their (or their insurance's) ability to pay, and to absorb the deficit. In addition, some Stanford patients were out of state when the pandemic occurred, and because of initial state-specific licensure requirements, providers could not give care to them or their families.

The New Normal and What Comes Next

“This has been transformational. We have been forced to imagine a new way of practicing medicine and we are realizing we should have been doing this all along” (administrator). “I would want my schedule 25% video visits after the pandemic is over” (provider). Respondents felt that video visits and telehealth should become a permanent part of clinical care after the pandemic. As the new normal developed, this would “also push us to ask, ‘What else can we do remotely for our patients?'” (administrator). Respondents shared their hopes for future digital health opportunities, including e-consults, new digital data sources (for example, photos), and remote monitoring, to provide convenient, high-quality care for persons everywhere. “[This should] open doors for other digital health technologies” (administrator).

Critical Lessons Learned: RE-AIM

Qualitative interview respondents strongly believed that video visits should be an ongoing part of medical practice after the pandemic, to improve health care access for patients who had video visit–appropriate health conditions. Using the RE-AIM framework of program evaluation, we examined respondent themes and concerns from an implementation perspective. Critical issues for virtual care program growth and sustainability over time were identified (22, 23) and are presented in the first column of Table 3, including issues related to provider or staff needs, health system evolution, and access and equity for patients (24).

Table 3. RE-AIM for Continued Video Visit Programmatic Success.

Discussion

Stanford's conversion to video visits to meet patient safety needs during the COVID-19 pandemic was remarkably rapid and pervasive. The primary program goal of ensuring patient safety through social distancing was achieved with greater than 80% of urgent care and 75% of primary care patients converted to safer video visits within 3 weeks. Our program participant evaluation was done throughout the third week of the video visit rollout, with analysis in the fourth week. Program participants were heartened by the intense collaboration to quickly promote social safety but were concerned that video visits would affect the quintuple aim, including their joy in practice and their role in medicine, and would exacerbate underlying patient inequities (24). Participants also raised issues related to video visit program sustainability should Medicare funding for emergency video visits be eliminated (3, 25–27).

Like other programs (13, 25), the Stanford video visit program rollout raised many critical issues for program participants. Video visits caused rapid changes in roles, workflow, patient interactions, and technology adoption and had unreimbursed costs (14, 25, 26). Overall, video visit adoption was facilitated by a greater sense of safety, joint purpose (ensuring public safety), and community (hard work leading to tangible change). However, the rapid program rollout also raised many critical issues, which must be addressed for sustainability (22). Many participants reported decreased role satisfaction, joy in medicine, and ability to make sound decisions and pointed out technology limitations. Other participants were concerned about program financial viability over time, given current temporary federal funding. Rapid qualitative evaluation was critical to make early program corrections to meet identified needs of program participants. For instance, in the weeks since evaluation, we have evolved our video visit program to meet some identified needs, such as creating continuing medical education videos on video visit physical examination, improving clinic workflow, developing best practices for telephone- and video-based physical examination and decision making, and working on video visit functionality.

Although the experience of other centers will be needed for robust recommendations, our findings suggest several next steps to ensure continued access to virtual care for patients. These are summarized in the second column of Table 3 and include the following:

1. Improve the ease of use of technology, including easier interfaces and support of multiparty interactions (12)

2. Clarify and construct new role responsibilities for all categories of providers, including MAs (28)

3. Build better video visit–specific quality assurance, including virtual physical examination training (29)

4. As video visits generate more ancillary communication demand, revise the triage system for incoming visits to ensure that providers are not overloaded

5. Revise insurer reimbursement policies to match the benefits of video visits (3, 22, 30)

6. Improve access to smartphones or internet for patients who are disenfranchised or otherwise impaired (31)

7. Develop permanent policies to allow clinical care across licensing jurisdictions (32, 33)

This study has many limitations. First, interviews were done immediately after implementation, so reactions include those of both new and experienced users of virtual care. Initial reactions may change over time. Second, given the press of COVID-19–related patient care, interviews were shorter than many qualitative interviews (<30 minutes). Although we feel we did reach thematic saturation, further reflection may have stimulated discussion of themes not considered initially. Third, interviewers were embedded faculty with preexisting relationships with the respondents and may have positively biased discussion. However, their deep collegial relationships may have also fostered honesty, and respondents raised numerous areas for improvement. Fourth, although providers at our northern California academic medical center were generally comfortable with adopting video visit technology, availability of the technology or comfort with adopting it may vary at other centers. Fifth, many video visits were converted to telephone encounters because of technical difficulties or technology literacy issues, necessitating different workflows and decision-making strategies. Finally, we interviewed our staff and providers as the lens to their patient's experiences and are planning patient interviews to understand their perspectives directly.

The Stanford video visit experience demonstrates that health systems can rapidly change to meet the needs of the public while maintaining standards of quality of care. If the barriers identified during our program evaluation can be overcome, video visits may become a viable and important part of our evolving health care system.