Evaluation of Patient Experience During Virtual and In-Person Urgent Care Visits: Time and Cost Analysis

Abstract

Urgent care (UC) is one of the fastest growing venues of health care delivery for nonemergent conditions. This study compared the patient experience during virtual and in-person urgent care visits. We conducted a cross-sectional study of patients with the same diagnosis during Virtual Urgent Care (VUC) and in-person UC visits with the same diagnosis with regard to time and cost over a period of one year. We recorded and analyzed 16 685 urgent care visits: In-person UC (n = 14 734), VUC (n = 1262). Significant differences were found in the average total time for a visit in an in-person UC (70.89 minutes), and VUC (9.38 minutes). The average total cost of VUC ($49) and in-person UC ($142.657) differed significantly. Significant difference was found between UC turnaround time and VC turnaround time (Dependent variable (DV): 53.77, P < .01). We found significant differences in cost and time between in favor of virtual visits. Our findings suggest additional policy reform to expand the use of virtual care among target populations to improve access, reduce costs, meet the needs of patients, and reduce emergency department visits.

Introduction

Urgent care (UC) is one of the fastest growing venues of health care delivery for nonemergent conditions (1). UC visits have shorter turnaround time, lower cost, higher satisfaction compared to emergency departments (ED) (2 –4). Nevertheless, the number of UC clinics is relatively limited with specific hours of operation during weekdays, and usually unavailable one weekend day, which presents geographical and temporal accessibility challenges (5). Long hours spent in travel and waiting for care reduce patients’ satisfaction with clinic visits (6 –8). On average, wait times in in-person urgent care vary from 22.5 to 55.27 minutes (9,10), and wait times in an ED vary from 2.5 to 17 hours (11,12). Because patients’ average travel time is 34 to 37 minutes, the combination of long travel times plus wait times can result in significant patient costs (13,14).

Virtual urgent care (VUC), a patient visit completed over the phone or online, can overcome current challenges within in-person urgent care clinics. In general, virtual visits can increase access in rural areas and improve health equity among vulnerable populations (15 –17). As an alternative to in-person visits, virtual visits can save rural patients an average of 142 minutes of travel time (18). Furthermore, patients avoid costs associated with travel and loss of work time, on average $32 in fuel costs, and $75 to $150 in family expenses (16,17). Importantly, patients are typically highly satisfied (94%-99%) with virtual visits (19 –21), specifically related to short wait times (7). Although it has been theorized that, in some cases, virtual urgent case visits may replace in-person visits, there is little evidence to inform this proposition.

To address this gap, we compared visits from a VUC center with in-person urgent clinics (UC). We compared 2 groups: (1) Virtual: patients who completed VUC visits, (2) In-person: patients who completed a face-to-face urgent care visit.

Objective

This study compared the patient experience (efficiency and cost) during virtual and in-person urgent care clinics by comparing patients with the same diagnosis over 1-year duration. We also measured patient satisfaction among virtual visits.

Methods

Settings and Participants

We conducted a cross-sectional study of patients’ experience during virtual and in-person UC visits regarding time, cost, and satisfaction. In-person urgent care visit data (demographics, diagnosis, and billing) were obtained from the Clinical Data Warehouse at a Southeastern Academic institution for the period from January 1, 2018, to December 31, 2018. The in-person data set represented 18 UC clinics associated with the same institution that provided VUC.

Virtual urgent care is a digital, on-demand telemedicine service built to connect patients having urgent care needs with physicians. Through the VUC web portal, patients can fill out their demographic information, choose from a list of board-certified physicians, schedule an e-visit immediately or for later, and make payments. At the end of the e-visit, the physician can prescribe medication, which the patient may get from their preferred pharmacy. Virtual urgent care went live in January 2018.

The satisfaction survey asked 3 questions regarding the overall experience, physician experience, and a possible alternative to VUC. A brief, voluntary satisfaction survey was provided to virtual patients after each visit. Survey response rate was 10%.

Outcomes

Primary outcomes were efficiency (turnaround time [wait time + service time], and total time [turnaround time + travel time]), cost (travel cost, visit cost, total cost). Secondary outcomes were patient satisfaction responses post-virtual care visit.

Data Sources and Cleaning

The original data sets included 2224 virtual visits and 91 766 in-person visits between January 2018 and December 2018. Each data set included patient’s age, gender, zip code, visit date and time, duration, wait time, diagnosis, and chief complaint. VUC data set included self-reported patient data on their preferred alternative care delivery option (in-person UC, emergency room [ER], PCP, Delay care).

We subset the data set according to the 10 most frequent diagnosis for VUC visits, that is, we only retained visits which are of one of these top 10 diagnosis codes (Appendix A). We used the top 10 diagnoses to ensure that we are comparing patients with the same diagnosis. Otherwise, if we included all diagnoses, our analysis would be biased by the type of diagnosis. For example, patient diagnosed with a headache has different needs from a patient diagnosed with upper respiratory infection. Therefore, we did not want to include the time and cost of encounter for patients with medical diagnoses not listed in the top 10 diagnoses.

The virtual and in-person data sets were subset by top10 diagnosis, with the resulting Virtual Top 10 (N = 1262), which was 56.75% (1262/2224) of the total VUC visits; and In-person Top 10 (N = 14 734) that made up 24.87% (14 734/59 250) of the total in-person data. All 5 data sets added up to 61 474 observations.

Virtual Urgent Care Materials and Measurements

Since VUC is available online, there were patients from outside North Carolina (NC). Based on zip codes, we deleted records of patients outside of NC from the data set. Of state encounters were excluded because all in-person encounters where for individuals in NC. Therefore, it would be inaccurate to compare a patient having an in-person encounter in NC to a patient doing a virtual visit from another state. The cost analysis from both patients would be very different.

For records with multiple diagnosis codes for the same patient, we kept the first one in the process of truncation as the primary diagnosis code. After truncation, the records were grouped on the basis of International Classification of Diseases diagnosis codes, recording the 10 most frequently occurring diagnosis codes, which account for 58.1% of the data points after deleting the records from outside NC.

Gas Cost Calculation

We calculated travel costs based on distance, deriving the monthly average price of gas from the US energy information administration (EIA) (22). In choosing a car, we determined the most popular personal car in NC in 2018, using ranking provided by a major insurance company (23). According to the Official US Government Source for Fuel Economy Information, the most popular car in NC in 2018 consumed approximately 0.03 gallons of gas per mile on city roads and highway (24). We then calculated the cost that VUC patients would have assumed if they had gone to the closest urgent care center in NC:

$$\text{Cost of gas }\left(\text{in dollars}\right)\text{ = distance traveled to the nearest UC}\times \text{gas price per mile}$$

In-Person UC Measurements and Materials

There were 91 766 visits to 18 in-person urgent clinics in 2018. We used the start time of the visit, the urgent center name where the visit took place, and the zip code of the patient’s residence. We eliminated visit entries that did not contain all the above information. Specifically, we first excluded entries with invalid zip codes or non-NC zip codes, and entries without recorded checkout times. Lastly, we extracted diagnosis code from the remaining UC visit entries. Like the diagnosis code record of VUC data set, there might be multiple diagnosis codes recorded for a single visit—we used the first one. The final UC data set included 59 250 records.

Turnaround Time Calculation

We measured the turnaround time, travel time, and distance, for every visit for the UC data set using zip code information. The turnaround time was calculated with R’s built-in function “difftime,” and the travel time and travel distance were found using R©.

Data Analysis

Welch t test is done between an in-person and virtual visits to analyze cost differences, in terms of time and money, VUC patients would have incurred if they had gone to the closest UC clinic in NC. This comparison is intended to show whether there is a statistically significant difference for VUC patients if they had chosen to go to in-person UC.

The huge difference in size between virtual and an in-person data set might suggest a difference in variance as well. This issue prevented us from using normal t test yet the Welch’s t test that we adopted does not require equal variance assumption (25).

Results

Of 93 990 total visits recorded, 16 685 visits were analyzed and compared: the Virtual UC (N = 1262), and In-person UC (N = 14 734), as shown in Table 1. Overall, most patients were females (76.6%), 35 to 64 years of age (51.4%), and insured (70.3%).

Table 1.

Summary of Patient Demographics for Virtual and In-Person Patients With the Same Diagnosis.

Patient Characteristics	Virtual, n (%)	In-person, n (%)	Total, N (%)
Gender
Female	1054 (83.52%)	10293 (69.86%)	11347 (76.69%)
Male	208 (16.49)	4441 (30.14%)	4649 (23.32%)
Age
0-18	88 (6.97%)	406 (2.76%)	494 (4.87%)
19-34	461 (36.53%)	4107 (27.87%)	4568 (32.20%)
35-64	699 (55.39%)	6986 (47.41%)	7685 (51.40%)
65+	14 (1.11%)	3235 (21.96%)	3249 (11.54%)
Insurance status
Insured	585 (46.35%)	13912 (94.42%)	14497 (70.39%)
Uninsured	677 (53.65%)	822 (5.58%)	1499 (29.62%)
Total	1262 (100%)	14734 (100%)	15996 (100%)

Female patients were more dominant in both virtual (83.5%) and in-person (69.8%). The proportion of male patient visits was double in in-person UC (30.14%) than in the virtual (16.4%) setting.

The age distribution shows that the highest number of patients were in the age 35 to 64 years of age. There were more visits from patients younger than 18 years in the virtual clinic compared to in-person; and there were more visits from patients older than 65 years at in-person clinics (21.9%) compared to virtual (1.11%). Overall, the virtual clinic served a greater percentage of uninsured patients (53.65%) compared to the in-person clinic (5.58%).

Time Analysis

Major differences were found in the average total time for in-person visits (70.89 minutes) and virtual visits (9.38 minutes), as shown in Table 2. The average turnaround time for an in-person visit was 55.31 minutes, whereas average turnaround time for virtual visit was 9.38 minutes, which was same as the average total time taken, since there was no travel time for VUC. The standard deviation for average total time for an in-person UC versus virtual visits varied significantly, 66.0 3 minutes for in-person and 4.46 minutes for virtual.

Table 2.

Descriptive Analysis of Time and Cost for VUC and In-Person UC.

Variable type	Modality	Mean (mins/$)	SD	Confidence level	Median	Range
Turnaround time
	Virtual	9.39	4.46	0.352	9.000	24.3
	In-person	55.31	27.66	0.459	49.133	139.72
Travel time
	Virtual	0	0	0	0	0
	In-person	13.79	12.98	0.21	13.42	56.07
Total time
	Virtual	9.39	4.46	0.35	9	24.3
	In-person	70.89	31.66	0.53	66.025	163.97
Service cost
	Virtual	$49	$0	0	$49	$49
	In-person	$141.96	$52.96	0.88	$154	$205
Travel (gas) cost
	Virtual	$0	$0	0	$0	$0
	In-person	$0.70	$0.68	0.01	$0.58	$4.99
Total cost
	Virtual	$49	$0	0	$49	$49
	In-person	$142.66	$53.02	0.878	$154.08	$209.98

Abbreviations: SD, standard deviation; VUC, Virtual Urgent Care.

The Turnaround Time distributions of visits in in-person and virtual visits were different in the shape, spread, and center. In-person visits had a range of 139.7 minutes, with a mode of 32 minutes in duration (Figure 1). Virtual visits had a much narrower range of 24.3 minutes, where visits between 4 and 10 minutes made up 68.33% of the total instances.

Figure 1.

Turnaround time distribution for in-person UC and Virtual Urgent Care (VUC).

The Total Time at the in-person and virtual visits was differently distributed, as shown in the density distribution histogram, Supplement A. Although the 2 figures were both unimodal and skewed to the left, the total time at the in-person visit was far more spread out (range = 163 minutes) as compared to the virtual visits (range = 52 minutes), suggesting greater variability in the former data set. In addition, the total time at the in-person visit was more uniformly distributed with the modes occurred multiple times over the 45 to 70 minutes interval. The Virtual visits Mode occurred at 8 to 10 minutes, which accounted for 19.13% of total visits.

T Test

Significant difference was found between in-person turnaround time and virtual visits turnaround time, derived from virtual visits turnaround time (Dependent variable (DV): 53.77, CI: 52.96-54.58, P < .01). Virtual and in-person visits also differed significantly in terms of travel time (DV: 3.68, CI: 2.92-4.45, P < .001), although the gap is not as big as it is between turnaround time. In light of these, it is not surprising that the total time spent on the 2 ways of seeking medical service, which is the sum of the turnaround time and travel time, is of prominent difference (DV: 57.46, CI: 56.34-58.58, P < .001). With respect to the travel cost, the almost zero P value indicates that the 2 modalities incur statistically divergent expenditures (DV: 0.16, CI: 0.12-0.21, P < .001).

Cost Analysis

As is demonstrated in Table 2, the average total cost of VUC ($49) and in-person UC ($142.657) differed vastly, the former being nearly one-third of the latter. In comparison, the average travel cost for an in-person visit was $0.69. Also, the range for travel cost for in-person visits was $4.99. The contrast between service costs of VUC ($49) and in-person UC ($141.959) was conspicuous, which influence the clear gap in the in total cost.

Virtual patient experience

Of all virtual visits, a total of 218 voluntary reviews were collected and analyzed. The overall satisfaction level with virtual visits was predominantly positive, 91.96% of respondents rated their experience as “Excellent,” “Very Good,” or “Good.” Among the rest, 5.12% of users referred to their experience as “Fair,” while only 3.72% rated their virtual care visit as “Poor.”

Among the respondents, 120 (55.3%) stated that they would have gone to in-person urgent care centers, 49 (22.58%) would have consulted primary care doctors, 19 (8.76%) would have sought care at ER, 9 (4.15%) considered commercial urgent care service, and 20 (9.22%) would have delayed care, Table 3.

Table 3.

Satisfaction Rating by Virtual Patients Destination.

Alternative care seeking choice	N	Q1: “How likely will you use VUC in the Future”	Q1 CI (95%)	Q2: “How likely will you recommend VUC to a peer or family member”	Q2: CI (95%)
Urgent care center	120	89.8%	85.7-93.8	89.5%	85.2-93.8
Primary care doctor	49	93.3%	88.7-97.8	93.5%	89.1-97.9
Emergency room	19	91.1%	78.8-103.3	90.5%	76.8-104.3
Convenient care/retail	9	84.4%	68.1-100.8	83.3%	66.1-100.5
Other	20	86%	76.2-95.8	85.5%	75.5-95.5
Total	218	90.1%		89.9%

Abbreviation: VUC, Virtual Urgent Care.

The responses were subsequently analyzed in subgroups, with respect to the patients’ alternative care seeking choices. For the Question 1, “On a scale of 1-10, how likely will you use VUC in the Future?,” the mean response was 9.01 (standard deviation [SD] = 2.09). For the Question “On a scale of 1-10, how likely will you recommend Virtual Care to a peer or family member?,” the mean rating was 8.99 (SD = 2.24).

Discussion

To our knowledge, this is the first study to investigate the differences in patient experience namely, time and cost, between virtual and in-person urgent care visits. More than 95% of patients visiting an in-person urgent care have insurance, where only 46% of virtual patients had insurance coverage. It is possible that the high cost of an in-person urgent care visit, which is significantly higher than the cost of a virtual visit, drove more uninsured patients without co-pays to use virtual care.

For the same diagnosis, we report significant differences in patient experience demonstrated by time and cost savings between virtual and in-person visits. The average total time for a visit at an in-person UC was 10 times more than the average total time taken by VC, showing there was a major difference in average time spent in-person UC versus a VC. The mean turnaround time of an in-person urgent care is 55 minutes versus 9.5 minutes in virtual, which could potentially relate to patients’ opportunity cost. Patients using VC saved an estimated mean travel time of 13 minutes per visit, which can reduce wait time in in-person urgent care clinics or ERs as well as minimize environmental pollution as a result of decreased car exhaust (26). We found that in-person UC and virtual visits differed significantly in terms of turnaround time, travel time, total time, and travel cost with the P values being approximately zero.

Difference in average service cost between in-person urgent care and virtual was $94, with the former being approximately 3 times bigger than the latter. Cost-effectiveness is a major factor in determining the preferred choice of the mode of visit. Travel cost of virtual visits was less that in-person visits. That difference indicated that in terms of travel expense, the opportunity cost of going to UC in person was not significant.

Patient satisfaction data show a strong correlation between likeliness of using virtual in future and recommending virtual to others. Of the survey respondents, 90% were satisfied with their virtual visit. Also, 40% of the patient’s preferred using an ER as a preferred alternative or to delay care if virtual was not available. Virtual visits can reduce ED visits, which can prevent overcrowding and increases cost. Also, delaying care for patients can have detrimental effect on mortality and ED visits (27). The low survey response rate may be due to the voluntary nature of the survey coupled with the clinic treating urgent care needs and hence, patients may not have the leisure to fill a post-visit satisfaction survey.

Although in-person urgent care clinics can absorb 30% to 50% of ED visits, virtual care can further improve health care quality by providing timely, equitable, and efficient care (16,28,29). Due to its novelty, we could not locate studies that investigated time and cost saving for VUC visits. Therefore, we present our findings in the light of similar studies in other medical specialties.

There are evidence showing that telehealth is effective to improving outcomes in some cases; and less strong or insufficient for others, which means there is a lack in gold standard on the effect of telehealth on clinical outcomes (30,31). Telehealth visits may be cheaper and more satisfying for patients but if the long-term costs are greater in terms of follow-up visits and negative health outcomes then the effectiveness of telehealth may be questionable. Therefore, comparing the impact of telehealth and in-person urgent care visits on clinical outcomes is yet to be investigated. This research explored the differences in time and cost; however, future work will utilize mixed methods to assess patient-reported outcomes through surveys and electronic health record (EHR) secondary data analysis. Our future work will analyze ED visits post telehealth and in-person visits. The hypothesis would be that ED visit rates will be similar between telehealth and in-person visits.

Report on Previous Findings

Virtual visits are preferred by patients over in-person visits because of low cost, decreased travel time, and high-quality care that resolve health issues (32,33). When compared with in-person visit, virtual visits were associated with lower costs (34 –37). Lower costs were partially associated to reduce travel burden by reducing travel miles, driving time, and travel reimbursement (35,38,39). Our study shows that the average virtual visits total cost was $160 lower than in-person visits; however, travel costs difference ($4.99) was not a substantial part of the cost reduction. This can be attributed to previous studies investigating rural regions coupled with the conservative travel cost calculations used in this study discussed in the limitations.

Virtual care visits were reported to reduce visit time and travel time. Studies showed reduction in visit time of 5 to 44 minutes when compared with in-person clinic visit (39,40). Rural patients reported that virtual care can reduce total time by 340 minutes when compared with in-person visit indicating time saving in travel and in clinic by 240 minutes and 65 minutes, respectively (41). Our study validates previous findings where VUC visits had a significant lower turnaround (115 minutes), travel (56 minutes), and total time (140 minutes) that in-person urgent care visits.

Patients’ experience with virtual visits was high than face-to-face visits (41,42). Virtual care patients report easier access to and better communication with the provider (43). Patient experiences with virtual care were generally in favor with a range of 47% to 95% satisfaction levels (41,44 –46). Reasons for high patient satisfaction with virtual visits included reduced wait times, availability of results, and impact on patient management (38). Our study validates previous findings that patient satisfaction with using virtual care was high (90%), plus 89% of patients indicated recommending virtual care to their family and friends.

Virtual Care Policy Reform

Virtual care programs are evolving faster than the evidence and policies supporting them, pressuring policy-makers to swiftly draft legislations. Findings from this study should encourage state policy-makers to consider the valuable potential virtual care has to offer to improve access at affordable costs. States play a critical role as regulators to establish commercial insurance laws, and as payers through Medicaid. There needs to be special focus on promoting virtual care in rural areas, among vulnerable populations, and within school-based health centers. Therefore, we believe that state laws that govern coverage and reimbursement of virtual care services will dictate the future of virtual care.

Study Strengths and Limitations

Virtual is an on-demand service that provides health access to all residents of NC. In-person UC data were obtained from 18 urgent clinics over a course of one year.

This study had several assumptions and limitations. Visit costs were based on the amount billed and do not reflect the co-pay or reimbursement rates; obtaining insurance policy information for each patient was out of the study scope. Travel calculations included one care model that is known for economical gas consumption, as well as a fixed gas price for the whole year. Travel calculations were based on zip code–level analysis which provides a conservative estimate of the distance traveled. The cost analysis included total price of visit and does not account for out-of-pocket payments in virtual and in-person visits. Although necessary for fair comparison, the study population was limited to the top 10 chief complaints only. The low satisfaction response rate is a weakness of the article, which may introduce selection bias where the extremely satisfied or unsatisfied patients will respond to the survey.

Another study limitation is the inability to compare face-to-face time with the physician in the in-person and virtual encounters, which was due to the absence of time spent with the physician in the in-person encounters. The data obtained from the Clinical Data Warehouse provided “Service time” for each in-person encounter. Service time was the time from when the patient is checked-in to the clinic until they checkout again, including wait time, time with the nurse, time with the physician, and checkout time. Therefore, it was not possible to compare face-to-face time with the physician in the in-person modality, which is why we used “turnaround time” to ensure a fair comparison in our analysis. Lastly, the in-person urgent care visits were from a single large health care system.

Conclusion

Our study provides the first evaluation, to our knowledge, of patient experiences between virtual and in-person urgent care visits. Among patients with the same diagnosis, we found significant differences in turnaround time, travel time, travel cost, and total cost in favor of virtual visits. We found high patient satisfaction regarding virtual use indicating preference over in-person visits. Our findings suggest that expanding the use of virtual care among target populations may save time, reduce costs, meet the needs of patients, and reduce ED visits.

Author Biographies

Saif Khairat, PhD, MPH is an assistant professor at the Carolina Health Informatics Program, University of North Carolina at Chapel Hill, NC, USA. As an informatics expert, Dr Khairat focuses on using novel mixed-methods and geospatial analysis to improve health access, outcomes, and satisfaction. He has authored more than 50 scientific articles in 30 different, peer-reviewed journals and conferences, and has served as an investigator for more than $6.5 million in research grants.

Xi Lin earned his MS from the Department of Statistics and Operations Research at University of North Carolina at Chapel Hill, NC, USA. He is currently a PhD student at the Paul Merage School of Business, University of California Irvine. He is interested in healthcare operations and management.

Songzi Liu earned her MSIS from the University of North Carolina at Chapel Hill, NC, USA. She is currently a PhD student in Biomedical Informatics at Northwestern University.

Zhaohui Man earned his MSIS from the School of Information and Library Science at the University of North Carolina at Chapel Hill, NC, USA. Mr Man is a research specialist at the Bioinformatics & Analytics Research Collaborative at the University of North Carolina at Chapel Hill.

Tanzila Zaman earned her PSM from the Carolina Health Informatics Program at the University of North Carolina at Chapel Hill, NC, USA.

Barbara Edson, RN, MBA is the executive director of UNC Health Care's Virtual Care Center at UNC Health Care, NC, USA.

Robert Gianforcaro, DO, FAAFP is the executive medical director for Population Health and Virtual Care at UNC Health, NC, USA.