Telemedicine vs In-Person Primary Care: Treatment and Follow-Up Visits

Abstract

Background:

Beyond initial COVID-19 pandemic emergency expansions of telemedicine use, it is unclear how well primary care telemedicine addresses patients’ needs.

Objective:

To compare treatment and follow-visits (office, emergency department, hospitalization) between primary care video or telephone telemedicine and in-person office visits.

Design:

Retrospective design based on administrated and electronic health record data.

Setting:

A large integrated delivery system with 1,300+ primary care providers between 4/2021–12/2021 (including the COVID-19 pandemic delta wave).

Patients:

1,589,014 adult patients, 26.5% were age 65+, 54.9% female, 22.2% Asian, 7.4% Black, 22.3% Hispanic, 46.5% White, 21.5% lived in lower socio-economic status neighborhoods, and 31.8% had a chronic condition.

Measurements:

Treatment outcomes included medication or antibiotic prescribing and lab or imaging ordering. Follow-up visits included in-person visits to the primary care office, emergency department, or hospitalization within 7 days. Outcomes were adjusted for socio-demographic and clinical characteristics and stratified by clinical area (abdominal pain, gastrointestinal, back pain, dermatology, musculoskeletal pain, routine, hypertension or diabetes, and mental health).

Results:

Of 2,357,598 primary care visits, 50.8% used telemedicine (19.5% video and 31.3% telephone). After adjustment, medications were prescribed in 46.8% of office visits, 38.4% of video visits and 34.6% of telephone visits. After the visit, 1.3% of in-person visits, 6.2% of video visits, and 7.6% of telephone visits had a 7-day return in-person primary care visit; 1.6% of in-person visits, 1.8% of video visits and 2.1% of telephone visits were followed by an emergency room visit. Differences in follow-up office visits were largest after index office vs. telephone visits for acute pain conditions, and smallest for mental health.

Limitations:

Setting telemedicine is fully integrated with ongoing EHR and clinicians, and study examines insured population and late COVID-19 pandemic period. Observational comparison lacks detailed severity or symptom measures, follow-up limited to 7 days. Clinical area categorization by diagnosis code rather than complaint.

Conclusions:

In-person return visits were somewhat higher after telemedicine compared with in-person primary care visits, but varied by specific clinical condition.

Introduction

While telemedicine use expanded greatly during the COVID-19 pandemic, driven primarily by social distancing efforts, longer-term use of telemedicine can continue to offer patients a convenient option and expand access to primary care.^1–3 Through video or telephone telemedicine, patients can access a clinician while avoiding barriers to access such as arranging for transportation, time-off from work, or clinic wait times. Telehealth delivery models that vary in level of integration with ongoing in-person primary care providers may influence their impact on health outcomes.⁴ U.S. long term telemedicine policy is still uncertain, and further evidence is needed to optimize telemedicine and outcomes.^5,6

Beyond initial pandemic-related telemedicine, there is limited evidence for whether longer term use of telemedicine visits in clinical practice adequately addresses patients’ needs, increases the likelihood of subsequent follow-up visits, or is more likely to be followed by a serious health event requiring an emergency room visit or hospital stay.^7,8 Prior studies have captured the clinical utility of telemedicine for a wide range of primary care concerns, with broad benefits for patient access to care and in maintaining continuity of care during the early pandemic period, and strong interest in longer term use after the initial pandemic emergency^9,10. Pre-pandemic, use of telemedicine in primary care was very limited, with early post-pandemic evidence showing follow up care and outcomes comparable to office visits.^11–13 Still, it is unclear whether telemedicine can address some clinical concerns better than others. Due to rapid recent practice changes, new evidence is needed to inform telemedicine policy in the context of much broader ongoing clinical use after the first year of the COVID-19 pandemic.

After the initial COVID pandemic emergency period, in a large integrated delivery system with mature audio-only (telephone) and video visit telemedicine offerings, we compared primary care telemedicine visits and in-person office visits on care processes and post-visit healthcare utilization, overall and by clinical concern areas.¹⁴ Compared with office visits, we hypothesized that some patients with telemedicine would require additional follow-up visits, that serious outcomes would be rare, and that outcomes may vary by clinical concern areas.

Methods

Study Setting

This study was conducted within Kaiser Permanente Northern California (KPNC), a large integrated health care delivery system with about 4.5 million members, 21 hospital medical centers, and over 9,000 physicians (1,300+ primary care providers), with patient membership representative of the insured Northern California region except at the lowest end of income.¹⁵ All primary care providers had access to use telemedicine for any patient since 2016. Starting in 2016, KPNC members initiating a visit with their own primary care provider could directly choose between an in-person, telephone, or video visit for any clinical condition (except for routine physical exams). In each visit type, the clinician had equivalent access to the patient’s full inpatient/outpatient EHR. Prior to the COVID-19 pandemic, in-person office visits were still the most common way for patients to visit primary care providers.¹⁶ After an initial COVID-19 pandemic emergency period during which telemedicine became the first line of contact between patients and primary care providers,¹⁷ in-person office visits returned to full availability for primary care appointments in April 2021, alongside continued availability of telephone and video visit appointments.

The Institutional Review Board of the Kaiser Foundation Research Institute approved the study protocol and materials and waived the requirement for written informed consent for participants. This cohort study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.

Study Population and Measures

Using the EHR, and other health system automated data sources, we identified study index visits among all primary care visits with primary care providers (PCP) by adult patients (18 years +) between April 1^st, 2021 (at return to full availability of in-person visits), and December 31^st, 2021, including office visits, video visits, and telephone visits. To focus the study population and visits on long-term general primary care practice with full access to in-person visits, we excluded visits with primary diagnosis of COVID or upper respiratory infection from the study sample. To define a relatively distinct care-seeking episode, we excluded visits from study if the patient had any visit within the previous 7 days.

Among all potential primary care visits, we defined subsets of visits with eight most common telemedicine clinical concern areas using ICD-10 code for primary diagnoses (categorized and validated by five physicians for a prior study¹⁸, with Kappa = 0.96, 95% CI 0.92–0.99, detailed list of definitions offered in Appendix Note 1): Dermatology, Musculoskeletal Pain, Back Pain, General Gastrointestinal (GI), Chronic Conditions (Hypertension (HTN) & Diabetes (DM)), Mental Health, Abdominal Pain and Routine Visits.

Our study primary outcomes were treatment during the primary index visit, and short-term follow-up health care utilization. For each study index visit, we identified any medication prescribing, antibiotic prescribing, lab orders and imaging orders during the visit as treatment outcome measures. To measure short-term follow-up health care utilization outcomes, we extracted all in-person office visits with primary care providers, or emergency department (ED) visits and hospitalizations that occurred within 7 days after each index visit.

Statistical Analysis

We used multivariable logistic regression to examine the association between index visit type (video, telephone vs. in-person office visit) outcomes, with a separate model for each treatment or utilization outcome. All models adjusted for patient characteristics including patient age, sex, race/ethnicity, lower neighborhood socioeconomic status (SES; at least 20% of households have household incomes below the federal poverty level or at least 25% of residents 25 years of age or older have less than a high school education in the census block group), preferred language for health care, lower neighborhood internet access (less than 80% of households have a residential fixed high-speed connection with at least 10 Mbps downstream and at least 1 Mbps upstream in the census tract based on FCC data), any mobile portal use in prior 365 days, any video visits in prior 365 days, whether the clinician was the patient’s own PCP and whether a family care partner had permissions to act for the adult patient to schedule an appointment (proxy), drive time from patient’s home to nearest facility, patient office visit cost-sharing, facility parking, appointment booking source, presence of each of four chronic conditions (using the health system’s clinical registries for asthma, diabetes, hypertension or heart failure), patient medical problem (ICD10 code grouping of primary diagnosis), medical center and calendar month. The health system clinical chronic condition registries are actively used in clinical care, designed to support national quality reporting, and have been extensively used in prior research. Standard errors were adjusted for repeated visits by the same patient by clustering observations by the patient with a robust variance estimator. To examine the association between visit type and outcomes for visits with similar diagnoses, we repeated all the analyses for each of the eight visit clinical concern areas i.e., abdominal pain, general GI, back pain, dermatology, musculoskeletal pain, routine, chronic conditions (hypertension [HTN]/ diabetes mellitus [DM]), and mental health.

For ease of interpretation, we calculated adjusted rates of each outcome by index visit type. We calculated this adjusted rate via marginal standardization by using Stata’s margins post-estimation command. All analyses were conducted using two-sided tests for significance, and p<0.05 as the threshold for significance, in Stata 17.0, StataCorp LLC, TX.

This study was funded by the Agency for Healthcare Research and Quality (AHRQ, R01HS25189). The funding source had no direct role in the design, conduct, and analysis of this paper, or in the decision to submit the manuscript for publication.

Results

Among all 2,357,598 primary care visits with over 1,300 PCPs by 1,589,014 adult patients during the study period, 49.2% were office visits, 19.5% video visits and 31.3% telephone visits. Overall, 28.4% were for patients 65 years and older; 56.1% were for female patients; 21.6% were Asian, 7.8% were Black, 23.0% were Hispanic, 46.2% were White race/ethnicity; and 21.8% lived in lower socio-economic status neighborhoods (Table 1). Nineteen percent of the visits were routine, 5.6% were for dermatology, 6.5% for chronic conditions (HTN/DM), 6.1% for musculoskeletal pain, 3.4% for general GI, 3.4% for back pain, 1.9% for abdominal pain, and 2.0% for mental health. The percent of visits conducted via telemedicine varied by clinical concern areas, and was highest in mental health (79.9%), followed by general GI (69.7%), back pain (58.1%), abdominal pain (57.2%), dermatology (55.2%), musculoskeletal pain (52.8%), chronic conditions (52.6%) and routine (15.2%) (Figure).

Table 1.

Study Primary care Index Visits by Visit Type: Patient Characteristics

		All	Office	Video	Telephone
		N=2,357,598	N=1,158,786	N=460,909	N=737,903
		Col %	Col %	Col %	Col %
Age	18–44	38.5	34.9	47.4	38.5
	45–64	33.1	32.9	31.4	34.5
	65–74	15.9	18.0	13.7	14.1
	75+	12.5	14.1	7.6	12.9
Female Sex		56.1	53.1	58.1	59.5
Race/ethnicity	Asian	21.6	22.4	24.6	18.4
	Black	7.8	7.0	7.4	9.2
	Hispanic	23.0	22.2	20.8	25.6
	White	46.2	47.0	45.7	45.2
Lower Neighborhood SES*		21.8	21.3	19.8	23.9
Lower Neighborhood internet**		30.0	29.8	26.8	32.1
Preferred language English		89.5	88.8	92.5	88.7
Visit with own PCP		70.4	77.2	65.1	63.2
Video visit in prior 365 days		45.3	39.8	61.0	44.3
Mobile portal access in prior 365 days		62.0	59.4	71.1	60.5
Having proxy for patient portal		2.6	2.7	2.8	2.3
Chronic conditions:	Diabetes	12.4	12.2	9.7	14.5
	Heart failure	1.5	1.5	1.0	1.8
	Hypertension	25.7	26.8	20.8	27.1
	Asthma	5.5	4.9	5.6	6.5

^*Lower neighborhood socio-economic status (SES): at least 20% of households have household incomes below the federal poverty level or at least 25% of residents 25 years of age or older have less than a high school education in the census block group

^**Lower neighborhood internet access: less than 80% of households have a residential fixed high-speed connection with at least 10 megabits per second (Mbps) downstream and at least 1 Mbps upstream in census tract-based Federal Communication Commission (FCC) data

PCP: primary care provider

Figure 1.

Percent of Index Visit Type by Clinical Concern Area

Percent shown in label on y axis is the percentage of all study visits in each diagnosis group.

HTN: Hypertension, DM: Diabetes, GI: Gastrointestinal

Treatment by Visit Type

Overall, and after adjustment for patient characteristics, medications were prescribed in 46.8% of office visits, 38.4% of video visits and 34.6% of telephone visits (Table 2, Appendix Figure 1, A). The adjusted rates of medication prescribing were higher in office visits visit than telemedicine visits for all visits and by all clinical concern areas. For example, patient visits for abdominal and musculoskeletal pain had the largest difference in medication prescribing rates after telemedicine compared with office visits (for general GI, −16.2%, 95% CI: −17.2% to −15.2% lower by video visit; for musculoskeletal pain: −19.8%, 95 %CI: −20.4% to −19.2% lower by telephone). Patients visits for dermatology and mental health had the smallest difference in medication prescribing rates after telemedicine compared with office visits (for dermatology −5.1%, 95 % CI: −5.8% to −4.5% lower by video; for mental health −9.5%, 95% CI: −10.6% to −8.3% lower by telephone visit).

Table 2.

Adjusted Rate of Medication and Antibiotic Prescribing by Index Visit Type and Difference between Telemedicine and Office visit

		Medication Prescribing		Antibiotic Prescribing
	N	Adj %	Adj Difference vs. Office (95% CI)	Adj %	Adj Difference vs. Office (95% CI)
Overall
Office	1,158,786	46.8%		6.6%
Video	460,909	38.4%	−8.3% (−8.5%, −8.2%)	6.6%	0.1% (0.0%, 0.1%)
Telephone	737,903	34.6%	−12.2% (−12.3%, −12.0%)	6.7%	0.2% (0.1%, 0.3%)
Routine
Office	387,550	37.6%		2.4%
Video	47,329	28.3%	−9.3% (−9.7%, −8.8%)	1.3%	−1.1% (−1.2%, −1.0%)
Telephone	22,014	20.1%	−17.4% (−18.0%, −16.9%)	1.0%	−1.4% (−1.6%, −1.3%)
HTN/DM
Office	72,812	59.6%		3.4%
Video	25,676	47.8%	−11.7% (−12.5%, −11.0%)	2.2%	−1.2% (−1.4%, −0.9%)
Telephone	55,172	45.9%	−13.6% (−14.2%, −13.0%)	0.9%	−2.5% (−2.7%, −2.3%)
Musculoskeletal Pain
Office	68,317	43.8%		1.4%
Video	31,387	28.3%	−15.5% (−16.1%, −14.8%)	0.8%	−0.6% (−0.7%, −0.5%)
Telephone	44,932	24.0%	−19.8% (−20.4%, −19.2%)	0.5%	−0.9% (−1.1%, −0.8%)
Back Pain
Office	33,607	62.3%		1.3%
Video	17,355	50.8%	−11.5% (−12.5%, −10.6%)	0.5%	−0.8% (−1.0%, −0.6%)
Telephone	29,178	47.1%	−15.2% (−16.0%, −14.4%)	0.3%	−1.0% (−1.1%, −0.8%)
Dermatology
Office	59,352	55.5%		16.0%
Video	42,574	50.3%	−5.1% (−5.8%, −4.5%)	16.6%	0.6% (0.1%, 1.1%)
Telephone	30,513	43.1%	−12.4% (−13.1%, −11.7%)	14.4%	−1.5% (−2.1%, −1.0%)
General GI
Office	24,430	61.3%		2.6%
Video	17,666	45.1%	−16.2% (−17.2%, −15.2%)	1.9%	−0.7% (−1.0%, −0.4%)
Telephone	38,424	42.4%	−18.9% (−19.7%, −18.1%)	1.8%	−0.8% (−1.0%, −0.5%)
Abdominal Pain
Office	19,151	43.8%		4.5%
Video	8,851	30.3%	−13.6% (−14.8%, −12.3%)	3.0%	−1.5% (−2.0%, −1.0%)
Telephone	17,082	26.5%	−17.4% (−18.4%, −16.4%)	3.3%	−1.2% (−1.6%, −0.8%)
Mental Health
Office	9,259	66.9%		1.9%
Video	11,228	60.6%	−6.3% (−7.6%, −4.9%)	0.7%	−1.2% (−1.5%, −0.8%)
Telephone	25,552	57.4%	−9.5% (−10.6%, −8.3%)	0.4%	−1.5% (−1.8%, −1.2%)

Adjusted rate for each visit type was calculated based on the coefficients from multivariate logistic regression via marginal standardization by using Stata’s margins post-estimation command.

Adj: Adjusted, HTN: Hypertension, DM: Diabetes, GI: Gastrointestinal

The adjusted rate of antibiotic prescribing was 6.6% in office visits, 6.6% in video visits and 6.7% in telephone visits overall (Table 2, Appendix Figure 1, B). The adjusted rates varied by clinical concern area (range 2.5% lower to 0.6% higher comparing telemedicine to office visits). The largest differences in antibiotic prescribing rates were found in visits with hypertension/diabetes or abdominal pain after telemedicine compared with office visits (abdominal pain: −1.5%, 95% CI: −2.0% to −1.0% lower by video visit; hypertension/diabetes: −2.5%, 95 %CI: −2.7% to −2.3% lower by telephone). Patients visits for musculoskeletal pain, dermatology, and general GI had the smallest difference in antibiotic prescribing rates after telemedicine compared with office visits (for musculoskeletal pain, −0.6%, 95 %CI: −0.7% to −0.5% lower for video; for dermatology 0.6%, 95 % CI: 0.1% to 1.1% higher by video; for general GI −0.8%, 95% CI: −1.0% to −0.5% lower by telephone visit).

The adjusted rate of lab test orders was 41.4% in office visits, 27.4% in video visits and 22.8% in telephone visits (Table 3, Appendix Figure 2, A). Differences varied by clinical concern area, with the greatest difference for abdominal pain −21.3%, 95% CI: −22.6% to −20.1% lower by video visit; −30.5%, 95% CI: −31.5% to −29.5% lower by telephone). The smallest difference was for musculoskeletal pain (−7.9%, 95% CI: −8.4% to −7.4% lower by video visit; −11.8%, 95% CI: −12.3% to −11.4% lower by telephone).

Table 3.

Adjusted Rate of Lab and Image Ordering by Index Visit type and Difference between Telemedicine and Office visit

			Lab Ordering		Image Ordering
		N	Adj %	Adj Difference vs. Office (95% CI)	Adj %	Adj Difference vs. Office (95% CI)
Overall
Office	1,158,786	1,158,786	41.4%		20.5%
Video	460,909	460,909	27.4%	−14.1% (−14.2%, −13.9%)	11.9%	−8.5% (−8.7%, −8.4%)
Telephone	737,903	737,903	22.8%	−18.6% (−18.7%, −18.5%)	8.7%	−11.7% (−11.8%, −11.6%)
Routine
Office	387,550	387,550	68.6%		15.0%
Video	47,329	47,329	57.8%	−10.8% (−11.2%, −10.3%)	8.6%	−6.4% (−6.6%, −6.1%)
Telephone	22,014	22,014	53.6%	−14.9% (−15.6%, −14.3%)	6.2%	−8.8% (−9.2%, −8.5%)
HTN/DM
Office	72,812	72,812	44.4%		16.2%
Video	25,676	25,676	32.8%	−11.6% (−12.3%, −10.9%)	7.1%	−9.1% (−9.5%, −8.7%)
Telephone	55,172	55,172	24.0%	−20.4% (−20.9%, −19.9%)	2.9%	−13.3% (−13.7%, −13.0%)
Musculoskeletal Pain
Office	68,317	68,317	21.4%		57.4%
Video	31,387	31,387	13.5%	−7.9% (−8.4%, −7.4%)	45.0%	−12.4% (−13.1%, −11.7%)
Telephone	44,932	44,932	9.5%	−11.8% (−12.3%, −11.4%)	38.9%	−18.5% (−19.1%, −17.9%)
Back Pain
Office	33,607	33,607	21.6%		41.9%
Video	17,355	17,355	10.8%	−10.8% (−11.5%, −10.1%)	25.6%	−16.3% (−17.1%, −15.4%)
Telephone	29,178	29,178	7.2%	−14.3% (−14.9%, −13.8%)	21.4%	−20.5% (−21.2%, −19.8%)
Dermatology
Office	59,352	59,352	22.2%		6.7%
Video	42,574	42,574	10.3%	−11.9% (−12.4%, −11.5%)	1.8%	−4.9% (−5.1%, −4.6%)
Telephone	30,513	30,513	9.5%	−12.7% (−13.2%, −12.2%)	1.2%	−5.5% (−5.7%, −5.2%)
General GI
Office	24,430	24,430	46.3%		11.0%
Video	17,666	17,666	32.7%	−13.7% (−14.7%, −12.7%)	4.7%	−6.3% (−6.8%, −5.8%)
Telephone	38,424	38,424	26.3%	−20.1% (−20.8%, −19.3%)	3.1%	−7.9% (−8.4%, −7.5%)
Abdominal Pain
Office	19,151	19,151	69.0%		39.8%
Video	8,851	8,851	47.7%	−21.3% (−22.6%, −20.1%)	22.9%	−16.9% (−18.1%, −15.8%)
Telephone	17,082	17,082	38.5%	−30.5% (−31.5%, −29.5%)	16.8%	−23.0% (−23.9%, −22.1%)
Mental Health
Office	9,259	9,259	31.7%		6.1%
Video	11,228	11,228	13.7%	−18.0% (−19.2%, −16.8%)	1.4%	−4.7% (−5.2%, −4.1%)
Telephone	25,552	25,552	7.7%	−24.1% (−25.1%, −23.0%)	1.0%	−5.1% (−5.6%, −4.6%)

Adjusted rate for each visit type was calculated based on the coefficients from multivariate logistic regression via marginal standardization by using Stata’s margins post-estimation command.

Adj: Adjusted, HTN: Hypertension, DM: Diabetes, GI: Gastrointestinal

The adjusted rate of imaging orders was 20.5% in office visits, 11.9% in video visits and 8.7% in telephone visits (Table 3, Appendix Figure 2, B). The adjusted rates varied by clinical concern area, with largest difference for abdominal pain (−16.9%, 95% CI: −18.1% to −15.8% lower by video visit; −23.0%, 95 %CI: −23.9% to −22.1% lower by telephone). The smallest difference was for mental health (−4.7%, 95% CI: −5.2% to −4.1% lower by video visit; −5.1%, 95%CI: −5.6% to −4.6% lower by telephone).

Follow-up care by Visit Type

After adjustment, 1.32% of in-person visits, 6.28% of video visits, and 7.6% of telephone visits had an in-person outpatient PCP visit in the following 7 days. The adjusted rates of follow-up PCP visits varied by clinical concern areas, ranging from 0.7% to 2.0% for office visits, 2.5% to 11.0% for video visits, and 2.1% to 14.9% for telephone visits (Table 4, Appendix Figure 3, A). For example, patient visits for abdominal and musculoskeletal concerns had the largest difference in follow up rates after telemedicine compared with office visits (abdominal pain: 9.1%, 95% CI: 8.4% to 9.8% higher by video visit; musculoskeletal: 13.2%, 95% CI: 12.8% to 13.5% higher by telephone). Patient visits for mental health had the smallest difference in follow-up rates after telemedicine compared with office visits (1.3%, 95% CI: 1.0% to 1.7% higher by video visit; 1.0%, 95% CI: −0.7% to 1.3% higher by telephone).

Table 4.

Adjusted Rate of 7-day Primary Care Office Visit, ED visit or Hospitalization by Index Visit Type and Difference between Telemedicine and Office Visit

		7-day PCP Office Visit		7-day ED Visit		7-day Hospitalization
	N	Adj %	Adj Difference vs. Office (95% CI)	Adj %	Adj Difference vs. Office (95% CI)	Adj %	Adj Difference vs. Office (95% CI)
Overall
Office	1,158,786	1.3%		1.6%		0.21%
Video	460,909	6.2%	4.9% (4.8%, 5.0%)	1.8%	0.2% (0.2%, 0.2%)	0.24%	0.03% (0.01%, 0.05%)
Telephone	737,903	7.6%	6.3% (6.2%, 6.4%)	2.1%	0.6% (0.5%, 0.6%)	0.25%	0.04% (0.03%, 0.06%)
Routine
Office	387,550	0.7%		0.5%		0.06%
Video	47,329	3.0%	2.3% (2.2%, 2.5%)	0.5%	0.0% (−0.1%, 0.0%)	0.07%	0.00% (−0.02%, 0.03%)
Telephone	22,014	5.9%	5.2% (4.9%, 5.6%)	0.8%	0.3% (0.2%, 0.4%)	0.13%	0.07% (0.02%, 0.12%)
HTN/DM
Office	72,812	1.2%		1.3%		0.19%
Video	25,676	4.3%	3.0% (2.7%, 3.3%)	1.5%	0.2% (0.0%, 0.3%)	0.23%	0.04% (−0.03%, 0.11%)
Telephone	55,172	4.4%	3.2% (3.0%, 3.4%)	1.3%	0.0% (−0.1%, 0.2%)	0.16%	−0.04% (−0.08%, 0.01%)
Musculoskeletal Pain
Office	68,317	1.7%		1.2%		0.11%
Video	31,387	9.9%	8.2% (7.8%, 8.6%)	1.4%	0.2% (0.0%, 0.4%)	0.10%	−0.01% (−0.06%, 0.04%)
Telephone	44,932	14.9%	13.2% (12.8%, 13.5%)	2.0%	0.8% (0.6%, 0.9%)	0.16%	0.05% (0.01%, 0.10%)
Back Pain
Office	33,607	1.4%		1.1%		0.10%
Video	17,355	5.9%	4.5% (4.1%, 4.9%)	1.4%	0.3% (0.1%, 0.5%)	0.16%	0.06% (−0.02%, 0.14%)
Telephone	29,178	8.1%	6.7% (6.3%, 7.0%)	2.0%	0.8% (0.6%, 1.0%)	0.10%	0.01% (−0.05%, 0.06%)
Dermatology
Office	59,352	1.7%		1.1%		0.13%
Video	42,574	8.6%	6.9% (6.6%, 7.2%)	1.3%	0.2% (0.0%, 0.3%)	0.14%	0.01% (−0.04%, 0.06%)
Telephone	30,513	12.4%	10.7% (10.4%, 11.1%)	1.7%	0.6% (0.5%, 0.8%)	0.11%	−0.01% (−0.06%, 0.03%)
General GI
Office	24,430	1.2%		2.1%		0.29%
Video	17,666	4.7%	3.4% (3.1%, 3.8%)	2.6%	0.4% (0.1%, 0.8%)	0.44%	0.15% (0.01%, 0.29%)
Telephone	38,424	5.5%	4.2% (4.0%, 4.5%)	2.9%	0.8% (0.6%, 1.1%)	0.40%	0.11% (0.02%, 0.21%)
Abdominal Pain
Office	19,151	2.0%		6.3%		0.80%
Video	8,851	11.0%	9.1% (8.4%, 9.8%)	6.1%	−0.2% (−0.8%, 0.4%)	0.96%	0.15% (−0.12%, 0.43%)
Telephone	17,082	14.5%	12.5% (12.0%, 13.1%)	7.0%	0.7% (0.2%, 1.3%)	0.91%	0.11% (−0.09%, 0.31%)
Mental Health
Office	9,259	1.1%		1.5%		0.21%
Video	11,228	2.5%	1.3% (1.0%, 1.7%)	1.4%	−0.1% (−0.5%, 0.2%)	0.06%	−0.15% (−0.27%, −0.04%)
Telephone	25,552	2.1%	1.0% (0.7%, 1.3%)	1.3%	−0.2% (−0.5%, 0.1%)	0.14%	−0.07% (−0.18%, 0.05%)

Adjusted rate for each visit type was calculated based on the coefficients from multivariate logistic regression via marginal standardization by using Stata’s margins post-estimation command.

Adj: Adjusted, HTN: Hypertension, DM: Diabetes, GI: Gastrointestinal

After adjustment, 1.6% of in-person visits, 1.8% of video visits and 2.1% of telephone visits were followed by an ED visit (Table 4, Appendix Figure 3, B). The adjusted rates of follow-up ED varied by clinical concern area, ranging from 0.5% −6.5% for office visits, 0.5% −6.1% for video visits, and 0.8% to 7.0% for telephone visits, with lowest among visits with routine and highest among those with abdominal pain for all type of visits. The differences in the follow-up ED visits were largest for general GI visits (0.4%, 95% CI 0.1% to 0.8% higher by video visit; 0.8% higher for musculoskeletal, back pain, and general by telephone). The differences in follow-up ED visits were smallest for routine and hypertension/diabetes (0.0%, 95% CI: −0.1%, to 0.0% difference by video visit; 0.0% 95% CI: −0.1% to 0.2% difference by telephone).

After adjustment, 0.21% of in-person visits, 0.24% of video visits and 0.25% of telephone visits were followed by a hospitalization (Table 4, Appendix Figure 3, C). The adjusted rates of follow-up hospitalization varied by clinical concern areas, ranging from 0.06% −0.8% for office visits, 0.06% −0.96% for video visits, and 0.10% to 0.91% for telephone visits. The differences in hospitalizations were largest for general GI, abdominal pain, and mental health (0.15% higher by video visit for general GI and abdominal pain; −0.15% lower by video visit for mental health; 0.11% higher by telephone for general GI and abdominal pain). The smallest difference in hospitalizations were for routine and dermatology visits (for routine visits 0.00%, 95% CI: −0.02%, to 0.03% difference by video; for back pain 0.01%, 95 %CI −0.05% to 0.06% higher by telephone; for dermatology −0.01%, 95 %CI: −0.06% to 0.03% lower by telephone).

Discussion

In a large integrated health care setting in 2021, beyond the initial pandemic emergency period, nearly half of primary care visits were still telemedicine visits. We found that medication prescriptions, and lab and imaging orders were highest among office visits and lowest among telephone PCP visits. Follow-up in-person primary care visits were lowest among index office visits and highest among telephone visits. The rates of ED visits and hospitalizations were low for all visit types, although similarly higher after telemedicine than office visits. For example, differences in telemedicine prescribing were largest among pain and GI conditions, and differences in follow-up visits were largest for telephone visits for abdominal and musculoskeletal concerns. While the specific rates of treatment and follow-up care utilization varied by clinical concern areas, patterns in the differences between office visits and telemedicine stayed mostly consistent.

This study offers an opportunity to examine a large volume of primary care telemedicine visits. Since primary care encompasses diverse clinical concerns, we examined patterns both overall and subsetted within eight common primary care clinical concern areas¹⁸. Telemedicine might be differentially useful depending on the specific primary care concern. Indeed, we saw frequent use of telemedicine in mental care and limited use of telemedicine for routine visits that might benefit from a general physical exam, suggesting that patients and clinicians are selecting visit types to fit the particular type of clinical concern. We found that treatment and imaging rates were higher in office visits than telemedicine, suggesting that despite early reports of potential excess clinical workup through telemedicine,⁴ this was not apparent in our study sample and integrated telemedicine care delivery setting.

Reasonably, telephone visits may not be able to address all types of patient issues as completely as office visits, resulting in a higher need for in-person follow-up care, particularly for musculoskeletal, abdominal pain, or skin conditions, where physical exam data are highly informative. Importantly, the differential office visit return rate after telephone visits was only modestly higher, suggesting that telephone or video telemedicine was still capable of addressing most patient clinical concern areas. Further, video visits can convey visual information and support more interaction between patients and providers, as compared with telephone visits. This may be contributing to the lower follow-up care utilization rates for video compared with telephone PCP visits. It is possible that patients and their clinicians may sometimes use telemedicine visits, especially telephone visits, as a first point of contact to determine if more care is needed, and the vast majority of telemedicine visits were not followed by in-person visits.¹⁹ While the specific rates of each treatment and follow-up outcome did vary, as we would expect, relative patterns between video, telephone, and office visits were remarkably consistent across clinical concern subgroups,⁷ and generally consistent with prior evidence finding generally comparable follow-up visit rates between telemedicine and office visits^7,12,13. In fact, the relative pattern of least follow-up return visits after office visits and higher return rates after telephone than video visits has been consistent in the pre-pandemic and early pandemic period as well.^16,17 Nonetheless, given prior evidence for differences in video visit use by patient sociodemographic characteristics, telephone may still represent a key low-tech telemedicine access tool.^17,20,21

We found relatively low rates of follow-up ED visits or hospitalizations generally, but slightly higher after telemedicine than in-person primary care office visits. The relative differences in follow-up ED visits varied by clinical concern and were higher for visits for musculoskeletal issues, general GI problems, back pain, or skin conditions. While not directly captured in our study data, it is possible that emergency room follow-up visits could result if patients who use telemedicine also face nuanced barriers to office visit care-seeking (such as clinic hours), or if patients using telemedicine were clinically recommended to visit the emergency room. Further exploration of clinical situations and patient circumstances is needed to potentially inform clinicians or operational groups further about to what extent these events might have been avoided, or the potential cost-impacts of visit outcomes.

Limitations

This study was conducted in a large integrated health care setting where video and telephone telemedicine were already widely available long before the pandemic, thus the findings may not directly generalize to other stand-alone less-integrated telemedicine delivery settings or to patients without access to insurance or primary care. Also, since the setting is an integrated setting, we assume that nearly all of a patient’s health care is captured within in-system administrative data, but it is possible that some out-of-system care-seeking was not captured in study data. Since this is an observational study, we cannot rule out unmeasured confounding, and results cannot be interpreted as causal. Nonetheless, we collected a wide range of covariates including patient, technology and clinical characteristics previously found to be associated with telemedicine use and outcomes,⁸ and accounted for them in the analyses. Still, unobserved factors, such as differences in symptoms, severity, comorbidity, clinical condition content, pandemic-related patient care-seeking preferences that are not captured within the structured EHR-based measures studied may persist between visit types and may explain some of the differences in outcomes identified in our results.

Also, our analyses did not account for differences in individual physician practice patterns or examine geographic regions outside of the study setting. And while we identified orders during the index visit and health care utilization within a 7-day follow-up window, we did not account for follow-up beyond 7 days, during which further care may still occur, including ED visits or hospitalizations. Likewise, we categorized visits by diagnosis codes and not the patient’s initial complaint, thus follow-up outcomes may differ by chief complaint. Also, while we defined visits within a primary clinical concern area, patients may well have multiple clinical concern areas and severity of complaints can differ within the same clinical concern area by visit type. Further, although our study period is lagged after the onset of the COVID-19 pandemic by one year or more, it was conducted during the delta variant surge which may have affected care-seeking and care delivery during our study – further research is needed to continue to examine telemedicine use and outcomes in the longer term.

Conclusions

Patients and physicians will continue to use telemedicine visits to some degree, for convenient primary care access after the pandemic emergency ends,^3,9,10 especially for patients with barriers to in-person visits such as taking time-off from work, transportation, and travel distance. In this observational cohort study during the second year of the COVID-19 pandemic, we found primary care telemedicine visits resulted in lower treatment rates and higher rates of follow-up healthcare utilization, as compared with in-person office visits. For example, differences in follow-up office visits were largest after index office vs. telephone visits for acute pain conditions, and smallest after office vs. telephone visits for mental health. While event rates varied by clinical area, telemedicine patterns were generally consistent across type of patient clinical concern. Overall, the rates of follow-up ED visit and hospitalizations were low, and differences between in-person visits and telemedicine visits were small. Telemedicine care can offer a convenient option to address patient primary care needs without raising substantial safety concerns.

Reproducible Research Statement:

Protocol:

Interested readers may contact the corresponding author for more information.

Statistical Code:

Interested readers may contact the corresponding author for more information.

Data:

Individual-level data may not be made publicly available due to IRB and privacy concerns.