Antibiotic Prescribing During Pediatric Direct-to-Consumer Telemedicine Visits

Using claims from a large national commercial health plan, we examine antibiotic prescribing for acute respiratory illnesses during pediatric DTC telemedicine visits.

Abstract

Download video file ^{(20.7MB, mp4)}

DOI: 10.1542/5995525523001

Video Abstract

BACKGROUND AND OBJECTIVES:

Use of commercial direct-to-consumer (DTC) telemedicine outside of the pediatric medical home is increasing among children, and acute respiratory infections (ARIs) are the most commonly diagnosed condition at DTC telemedicine visits. Our objective was to compare the quality of antibiotic prescribing for ARIs among children across 3 settings: DTC telemedicine, urgent care, and the primary care provider (PCP) office.

METHODS:

In a retrospective cohort study using 2015–2016 claims data from a large national commercial health plan, we identified ARI visits by children (0–17 years old), excluding visits with comorbidities that could affect antibiotic decisions. Visits were matched on age, sex, chronic medical complexity, state, rurality, health plan type, and ARI diagnosis category. Within the matched sample, we compared the percentage of ARI visits with any antibiotic prescribing and the percentage of ARI visits with guideline-concordant antibiotic management.

RESULTS:

There were 4604 DTC telemedicine, 38 408 urgent care, and 485 201 PCP visits for ARIs in the matched sample. Antibiotic prescribing was higher for DTC telemedicine visits than for other settings (52% of DTC telemedicine visits versus 42% urgent care and 31% PCP visits; P < .001 for both comparisons). Guideline-concordant antibiotic management was lower at DTC telemedicine visits than at other settings (59% of DTC telemedicine visits versus 67% urgent care and 78% PCP visits; P < .001 for both comparisons).

CONCLUSIONS:

At DTC telemedicine visits, children with ARIs were more likely to receive antibiotics and less likely to receive guideline-concordant antibiotic management compared to children at PCP visits and urgent care visits.

What’s Known on This Subject:

Authors of previous studies identified concerns about antibiotic prescribing for adults using direct-to-consumer (DTC) telemedicine. No study has examined antibiotic prescribing during pediatric DTC telemedicine visits.

What This Study Adds:

Compared with urgent care and primary care visits, children receiving care for acute respiratory infections during DTC telemedicine visits are more likely to receive antibiotics and less likely to receive guideline-concordant antibiotic management.

Many commercial companies offer direct-to-consumer (DTC) telemedicine visits in which patients are connected to physicians outside of the medical home through audio-only or audio-video conferencing on their personal devices.¹ DTC telemedicine companies highlight the convenience of this care and report high patient satisfaction.^2–4 Access to telemedicine has expanded, with 96% of large business insurance plans now offering coverage for employees and their families.⁵ As a result, millions of children now have access to DTC telemedicine, and use is rapidly increasing.⁶

The American Academy of Pediatrics discourages the use of DTC telemedicine outside of the medical home for acute pediatric care, citing concerns such as limited physical examination capabilities, lack of ongoing patient-provider relationships, and lack of access to patient records, all of which may lead to lower-quality care.⁷ The American Academy of Pediatrics further notes that these issues may be particularly important with children who may have difficulty verbalizing their symptoms.⁷ The American Telemedicine Association echoed many of these concerns,⁸ adding that the use of personal devices may also impact technical quality and privacy of care. The American Telemedicine Association further suggests that DTC telemedicine outside of the medical home should not be used for children <2 years old.⁸ Both organizations emphasize the need for telemedicine visits to adhere to the same standards expected of in-person care, including adherence to professional guidelines for appropriate examination, diagnostic testing, and treatment.^7,8

To date, there is limited evidence regarding the quality of care during pediatric DTC telemedicine visits. Early studies of adult patients using DTC telemedicine compared with physician office visits have identified more inappropriate antibiotic prescriptions for bronchitis⁹ and more broad-spectrum antibiotic use.¹⁰ The quality of DTC telemedicine care for children may differ for several reasons. First, there is heightened reliance on the physical examination among children who cannot fully articulate symptoms.⁷ Second, care requires knowledge of pediatric-specific guidelines.^11–13 Third, rates of guideline-concordant antibiotic management at physician visits is higher for children than for adults¹⁴; therefore, the relative difference in quality of care at DTC telemedicine visits may be different.

To fill this gap in knowledge about the quality of commercial DTC telemedicine care for children, we used data from a large national health plan to compare the quality of antibiotic prescribing for acute respiratory infections (ARIs) at pediatric DTC telemedicine, urgent care, and primary care provider (PCP) visits. We focus on ARIs because authors of previous studies identified concerns about antibiotic prescribing during pediatric ARI visits in other settings,^14–19 and because ARIs account for the majority of DTC telemedicine visits.⁶

Methods

Study Population

We performed a retrospective cohort study using 2015–2016 claims data from a large national commercial health plan, which provides coverage for roughly 4 million children annually from all 9 census divisions and contracts with a single national DTC telemedicine vendor. Children age 0 to 17 years were included, with no minimum enrollment duration requirement. We excluded children who lacked pharmaceutical coverage during the month of their visit to ensure record of pharmacy claims.

Identifying ARI Visits

ARI visits were identified by using previously published methods on the basis of International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes.^15,20,21 Consistent with previous work,^14,15,20,21 ARI diagnoses included diagnoses for which antibiotics may be warranted (sinusitis [461.x], pneumonia [481.x–483.x, 485.x–486.x], streptococcal pharyngitis [034.x], acute otitis media [382.x]) and diagnoses for which antibiotics are not warranted (eg, viral upper respiratory infection [URI], bronchiolitis, viral pharyngitis, serous otitis media [381.x, 460.x, 462–466.x, 480.x, 490.x]). We excluded bacterial pneumonia visits because of the small number of DTC telemedicine visits identified (n = 11). We converted International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM) codes to ICD-9-CM codes using a modified crosswalk created by the Centers for Medicare and Medicaid Services (see Supplemental Table 5).²²

To examine the first ARI visit within an episode of illness, we identified “index” ARI visits with no ARI visits at any location in the previous 21 days, which is consistent with previous studies.^10,23 Subsequent ARI visits within 21 days of an index visit were considered follow-up care. We excluded episodes as detailed in Supplemental Fig 2. Using a previously published methodology, we excluded episodes for which the index visit included a diagnosis indicating chronic conditions (eg, history of transplant) or additional codiagnoses (eg, urinary tract infection) that could impact antibiotic prescribing decisions (n = 60 947; 4.7%) (Supplemental Table 6).^15,21 These exclusion criteria were originally developed for adult populations. Although not all of these exclusion criteria are reasons to deviate from pediatric ARI guidelines, we applied them to study a set of visits consistent with other studies.

Identifying Setting of Care

ARI episodes were categorized as DTC telemedicine, urgent care, or PCP on the basis of the index visit setting. We identified setting using place of service codes, Current Procedural Terminology codes, insurer provider specialty codes, and organizational national provider identifiers and tax identification numbers for urgent care and DTC telemedicine companies through previously described methods (additional details in Supplemental Information).²⁴ ARI episodes that began with index visits to settings other than DTC telemedicine, urgent care, or PCP offices (eg, emergency department, retail-based clinics) were not included.

Outcomes

Our 2 primary outcomes, compared across settings, were (1) the proportion of visits that resulted in an antibiotic prescription and (2) the proportion of visits that resulted in antibiotic management that was concordant with guidelines for the visit diagnosis.

We identified all filled prescriptions on the day of the index visit and subsequent 2 days and assigned these to the index visit, consistent with previous methods.^1,10,25 We included all systemic antibiotics (ie, oral, intramuscular) and excluded topical, ophthalmic, or otic antibiotics. We determined guideline concordance of antibiotics with visit diagnosis, assigning each visit to 1 of 5 categories (Supplemental Table 7).^21,26 “Antibiotic prescribed” visits included visits categorized as guideline-concordant antibiotic use, guideline–non-concordant antibiotic use, and unnecessary antibiotic use. “Guideline-concordant antibiotic management” included visits categorized as guideline-concordant antibiotic use for sinusitis, otitis media, or streptococcal pharyngitis (ie, amoxicillin or amoxicillin-clavulanic acid for sinusitis,¹² amoxicillin or amoxicillin-clavulanic acid for otitis media,¹¹ and penicillin or amoxicillin for streptococcal pharyngitis²⁷), no antibiotic use for sinusitis or otitis media (also guideline-concordant management for these diagnoses^11,12), or guideline-concordant nonuse of antibiotics for diagnoses that do not warrant antibiotics (eg, bronchiolitis¹³).

As secondary outcomes, we compared the occurrence of streptococcal testing within 1 day for children diagnosed with streptococcal pharyngitis²⁷ as well as return visits within 2 and 21 days of an index visit to any setting (eg, PCP, DTC telemedicine, urgent care, retail clinic, emergency department). We did not have streptococcal testing results.

Patient and Visit Characteristics

Patient age, sex, and high-deductible health plan (HDHP) status were determined from insurance enrollment files. Using the patients’ zip code, we determined census region and rural-urban commuting area codes.²⁸ We determined patient chronic medical complexity using the Pediatric Medical Complexity Algorithm Version 2 applied to 2014–2016 inpatient and outpatient claims.²⁹ This algorithm is used to identify children with 3 levels of chronic medical complexity: complex chronic disease, noncomplex chronic disease, and no chronic disease.²⁹

Matching

To address differences in patients and conditions managed in the 3 settings, we used coarsened-exact matching,³⁰ which prunes observations to achieve covariate balance between groups. Instead of matching to a fixed number of visits, coarsened-exact matching matches DTC telemedicine visits to many matched urgent care visits and PCP visits and then weights each stratum or matched set.³⁰ Specifically, we matched DTC telemedicine visits to urgent care visits and separately to PCP visits on the basis of age (in years), sex, chronic medical complexity, state, rural or urban area, HDHP status, and diagnosis category (eg, otitis media, viral URI).

Statistical Analysis

Across ARI visits for the 3 settings of care, we described patient characteristics and diagnoses. In the unmatched sample and the matched weighted sample, we examined antibiotic prescribing and guideline-concordant antibiotic management for DTC telemedicine visits compared with urgent care and compared with PCP offices using χ² tests. We include both unmatched and matched results because of potential subjectivity in diagnoses, which may then drive variability in antibiotic prescribing.¹⁶ In the matched weighted sample, we also compared streptococcal testing and return visits by setting using χ² tests. We adjusted for multiple comparisons in our main results using the Benjamini-Hochberg false discovery rate,³¹ yielding a P value of .0342.

To examine whether differences in antibiotic prescribing varied by patient characteristics, we performed further analysis that was focused on 2 variables that may impact guideline adherence: child age and chronic medical conditions. Within each category of child age and medical complexity, we examined antibiotic testing and guideline concordance for DTC telemedicine compared with urgent care and PCP offices using χ² tests and then tested the significance of an interaction term between age or medical complexity and visit setting using logistic regression.

We performed 2 sensitivity analyses to examine the robustness of our findings to specific assumptions. First, we conducted a sensitivity analysis in which we redefined guideline-concordant antibiotic management so that it no longer included “no antibiotic use” for sinusitis and otitis. Because guidelines allow for observation without antibiotic use in specific circumstances for sinusitis and otitis media.^11,12 we considered no antibiotic use to be a guideline-concordant management option for these diagnoses in our main analysis but not in this sensitivity analysis. Second, we examined antibiotic prescribing and guideline-concordant management after excluding ARI visits with a follow-up visit within 2 days because visits with such early follow-up might result in miscategorization of the site of antibiotic prescribing.

Analyses were conducted in SAS version 9.4 (SAS Institute, Inc, Cary, NC). Deidentified data were used in this analysis, which was determined to be exempt by Harvard Medical School’s Institutional Review Board.

Results

In the unmatched sample, there were 5431 index ARI visits to DTC telemedicine, 87 555 to urgent care, and 1 132 116 to PCPs in 2015–2016 that met inclusion criteria. Compared with urgent care and PCP visits, DTC telemedicine visits were more likely to involve children who were older, enrolled in HDHPs, and in the South (Table 1). Among DTC telemedicine visits, 5% were for children <2 years old. At DTC telemedicine visits, children were more likely to receive a diagnosis of viral ARI (68% vs 54% urgent care; 59% PCP visits) or sinusitis (18% vs 8% urgent care; 10% PCP visits) and less likely to receive a diagnosis of otitis media (8% vs 26% urgent care; 23% PCP).

TABLE 1.

Characteristics of ARI Visits Before and After Matching, 2015–2016

	Pediatric Visits
	Before Matching					After Matchinga
	DTC Telemedicine, No. (%)	Urgent Care, No. (%)	Standardized Difference,b %	PCP, No. (%)	Standardized Difference,b %	DTC Telemedicine, No. (%)	Urgent Care, No. (%)	Standardized Difference,b %	PCP, No. (%)	Standardized Difference,b %
Visits, N	5431	87 555	—	1 132 116	—	4604	38 408	—	485 201	—
Child age, y
0–1	245 (5)	8234 (9)	59.0	195 692 (17)	89.6	224 (5)	1956 (5)	4.7	24 192 (5)	2.5
2–5	1305 (24)	24 845 (28)	21.5	390 699 (35)	46.4	1132 (25)	9528 (25)	1.2	121 932 (25)	2.9
6–11	1930 (36)	30 018 (34)	5.6	331 716 (29)	30.1	1614 (35)	13 392 (35)	0.8	168 036 (35)	1.9
12–17	1951 (36)	24 458 (28)	39.3	214 009 (19)	110.7	1634 (35)	13 532 (35)	1.1	171 041 (35)	1.0
Sex
Female	2755 (51)	44 181 (50)	1.1	555 277 (49)	6.7	2339 (51)	19 513 (51)	0	246 500 (51)	0
Male	2676 (49)	43 374 (50)	—	576 839 (51)	—	2265 (49)	18 895 (49)	—	238 701 (49)	—
Chronic medical complexityc
No chronic conditions	3993 (74)	65 931 (75)	9.5	835 897 (74)	1.6	3542 (77)	29 548 (77)	0	373 280 (77)	0
Noncomplex chronic condition	1062 (20)	14 823 (17)	18.5	197 934 (17)	14.3	803 (17)	6699 (17)	0	84 626 (17)	0
Complex chronic condition	376 (7)	6801 (8)	11.9	98 285 (9)	22.2	259 (6)	2161 (6)	0	27 295 (6)	0
Rural or urband
Metropolitan	4967 (91)	81 028 (93)	15.7	1 052 233 (93)	22.7	4415 (96)	36 839 (96)	0.5	465 266 (96)	0.1
Micropolitan	284 (5)	4421 (5)	3.7	48 849 (4)	22.1	140 (3)	1161 (3)	0.6	14 771 (3)	0.1
Small town	109 (2)	1449 (2)	21.4	20 540 (2)	10.8	34 (1)	284 (1)	0	3583 (1)	0
Rural	71 (1)	657 (1)	71.7	10 494 (1)	41.3	15 (0)	125 (0)	0	1581 (0)	0
Region
Northeast	627 (12)	5691 (7)	79.7	346 792 (31)	90.0	490 (11)	4088 (11)	0	51 640 (11)	0
Midwest	929 (17)	16 393 (19)	10.7	161 878 (14)	22.9	701 (15)	5833 (15)	0.3	73 876 (15)	0
South	2943 (54)	42 860 (49)	21.0	445 160 (39)	62.3	2587 (56)	21 757 (57)	1.9	275 657 (57)	2.5
West	932 (17)	22 611 (26)	45.8	178 286 (16)	10.6	826 (18)	6730 (18)	2.9	84 029 (17)	4.3
HDHP
No	4575 (84)	81 720 (93)	136.2	1 062 214 (94)	164.3	4119 (89)	34 362 (89)	0	434 088 (89)	0
Yes	856 (16)	5835 (7)	—	69 902 (6)	—	485 (11)	4046 (11)	—	51 113 (11)	—
Diagnosis categorye
Sinusitis	966 (18)	7101 (8)	122.1	113 392 (10)	85.9	646 (14)	5389 (14)	0	68 080 (14)	0
Streptococcal pharyngitis	326 (6)	11 745 (13)	65.6	103 066 (9)	37.5	250 (5)	2086 (5)	0	26 347 (5)	0
Otitis media	429 (8)	22 838 (26)	96.9	263 743 (23)	86.4	369 (8)	3078 (8)	0	38 888 (8)	0
Viral ARIf	3710 (68)	46 979 (54)	59.1	664 912 (59)	39.5	3339 (73)	27 855 (73)	0	351 887 (73)	0

—, not applicable.

^aSample characteristics before and after matching. In comparison of matched samples, each stratum of matched sets is weighted to account for differences in the number of matched urgent care and PCP visits. Counts within the matched categories may exceed column total because of rounding of weighted frequencies. Visits were matched by age (in years) and state, but results are reported in the table by larger categories of age group and census region.

^bStandardized differences comparing the characteristics of urgent care or PCP visits (in the column to the left) to DTC telemedicine, before and after matching.

^cPatient chronic medical complexity was determined by using the Pediatric Medical Complexity Algorithm applied to all 2014–2016 inpatient and outpatient claims for each patient.²⁹ This algorithm is used to identify children with 3 levels of chronic medical complexity: complex chronic disease, (ie, significant multisystem chronic disease, progressive conditions with decreased life expectancy, technology dependence, or malignancy), noncomplex chronic disease (ie, chronic conditions such as asthma that do not meet the above criteria), and no chronic disease.

^dRural or urban status was determined by using the US Department of Agriculture rural-urban commuting area codes.

^eBecause a small percentage of visits have >1 of these diagnoses (1.1%), percentages for diagnosis category may sum to >100%.

^fViral ARI category consists of diagnoses for which antibiotics are not warranted (eg, viral URI, bronchiolitis, viral pharyngitis, and serous otitis media) in the absence of another diagnosis.

In the unmatched cohort (Table 2), DTC telemedicine visits were associated with substantially increased antibiotic prescribing (52% vs 46% urgent care; 33% PCP; P < .001 for both) and decreased guideline-concordant antibiotic management compared with other settings (60% vs 71% urgent care; 80% PCP; P < .001 for both).

TABLE 2.

Antibiotic Prescribing and Guideline-Concordant Antibiotic Management in Matched Sample, 2015–2016

	Unmatched Visits					Matched Visits
	DTC Telemedicine	Urgent Care	Pa	PCP	Pb	DTC Telemedicine	Urgent Care	Pa	PCP	Pb
Total ARI visits, N	5431	87 555	—	1 132 116	—	4604	38 408	—	485 201	—
Any antibiotics prescribed,c n (%)d	2846 (52)	40 594 (46)	<.001	376 638 (33)	<.001	2381 (52)	16 198 (42)	<.001	152 687 (31)	<.001
Guideline-concordant management,e n (%)d	3246 (60)	61 842 (71)	<.001	907 450 (80)	<.001	2712 (59)	25 788 (67)	<.001	376 259 (78)	<.001
Specifics category of guideline concordance and nonconcordance among ARI visits, n (%)d
Guideline-concordant use	761 (14)	18 613 (21)	<.001	189 970 (17)	<.001	561 (12)	4260 (11)	.026	52 776 (11)	.005
Guideline nonconcordant use	369 (7)	7514 (9)	<.001	83 297 (7)	.11	271 (6)	2507 (7)	.095	31 138 (6)	.14
Unnecessary antibiotic use	1716 (32)	14 467 (17)	<.001	103 371 (9)	<.001	1549 (34)	9431 (25)	<.001	68 773 (14)	<.001
No antibiotic use	591 (11)	14 449 (17)	<.001	193 937 (17)	<.001	433 (9)	3787 (10)	.33	49 401 (10)	.083
Guideline-concordant nonuse	1994 (37)	32 512 (37)	.54	561 541 (50)	<.001	1790 (39)	18 424 (48)	<.001	283 114 (58)	<.001

—, not applicable.

^aP value determined from χ² test for DTC telemedicine versus urgent care visits.

^bP value determined from χ² test for DTC telemedicine versus PCP visits.

^cVisits with antibiotic prescribing include those categorized as guideline-concordant use, guideline nonconcordant use, and unnecessary antibiotic use.

^dPercentages reported are the proportion of visits in each setting meeting the specified criteria divided by the total number of ARI visits at that setting (indicated by the top row).

^eVisits with guideline-concordant antibiotic management include those categorized as guideline-concordant antibiotic use, no antibiotic use (for acute otitis media and sinusitis), and guideline-concordant nonuse of antibiotics.

Antibiotic Prescribing in Matched Visits

After matching, the weighted sample included 4604 DTC telemedicine, 38 408 urgent care, and 485 201 PCP visits with improved balance on observed characteristics, including child age, geographic region, and visit diagnosis (Table 1, Supplemental Table 8).

At DTC telemedicine visits, children were more likely to receive an antibiotic compared with matched visits at other settings (52% vs 42% urgent care; 31% PCP; P < .001 for both comparisons; Table 2). At DTC telemedicine visits, children were less likely to receive guideline-concordant antibiotic management (59% vs 67% urgent care; 78% PCP; P < .001 for both comparisons). The difference in guideline-concordant antibiotic management rates was primarily driven by antibiotic prescribing for visits with viral ARI diagnoses that do not warrant antibiotics. For such visits, antibiotics were appropriately not prescribed in only 54% of DTC telemedicine visits compared with 66% of urgent care visits (P < .001) and 80% of PCP visits (P < .001; Table 3).

TABLE 3.

Antibiotic Prescribing Outcomes by Diagnosis

	Pediatric Visits
	DTC Telemedicine	Urgent Care	Pa	PCP	Pb
Total ARI Visits, N	4604	38 408	—	485 201	—
Summative categories
Visits for which antibiotics may be indicated, n	1265	10 553	—	133 314	—
Any antibiotic use, n (%)	832 (66)	6766 (64)	.25	83 914 (63)	.038
Guideline-concordant antibiotic management,c n (%)	922 (73)	7364 (70)	.023	93 145 (70)	.020
Sinusitis visits, n	646	5389	—	68 080	—
Any antibiotic use, n (%)	410 (63)	3415 (63)	.96	43 219 (63)	.99
Guideline-concordant antibiotic management, n (%)	471 (73)	3837 (71)	.36	49 611 (73)	.98
Streptococcal pharyngitis visits, n	250	2086	—	26 347	—
Any antibiotic use, n (%)	178 (71)	1403 (67)	.21	17 315 (66)	.069
Guideline-concordant antibiotic management, n (%)	136 (54)	974 (47)	.021	11 424 (43)	<.001
Otitis media visits, n	369	3078	—	38 888	—
Any antibiotic use, n (%)	244 (66)	1948 (63)	.28	23 380 (60)	.019
Guideline-concordant antibiotic management, n (%)	315 (85)	2553 (83)	.24	32 109 (83)	.16
Visits where antibiotics not indicated, n	3339	27 855	—	351 887	—
Guideline-concordant antibiotic management, n (%)	1790 (54)	18 424 (66)	<.001	283 114 (80)	<.001
Bronchiolitis visits, n	51	263	—	4037	—
Guideline-concordant antibiotic management, n (%)	31 (61)	182 (69)	.25	3197 (79)	.001
Bronchitis visits, n	363	2686	—	27 308	—
Guideline-concordant antibiotic management, n (%)	160 (44)	1114 (41)	.344	11 270 (41)	.28
Visits for other viral URIs, n	2925	24 906	—	320 542	—
Guideline-concordant antibiotic management, n (%)	1599 (55)	17 128 (69)	<.001	268 647 (84)	<.001
Visits where antibiotics prescribed, n	2381	16 198	—	152 687	—
Guideline-concordant antibiotic management, n (%)	561 (24)	4260 (26)	.005	52 776 (35)	<.001

—, not applicable.

^aP value determined from χ² test for DTC telemedicine versus urgent care visits.

^bP value determined from χ² test for DTC telemedicine versus PCP visits.

^cVisits with guideline-concordant antibiotic management include those categorized as guideline-concordant antibiotic use, no antibiotic use (for acute otitis media and sinusitis visits), and guideline-concordant nonuse of antibiotics.

Streptococcal Testing and Return Visits

In the matched sample, among visits with a diagnosis of streptococcal pharyngitis, streptococcal testing was performed for 4% of DTC telemedicine visits compared with 75% of urgent care and 68% of PCP visits (P < .001 for both; Table 4). Subsequent visits within 2 days occurred for a slightly higher percentage of DTC telemedicine visits (5% vs 2% urgent care; 1% PCP; P < .001 for both).

TABLE 4.

Streptococcal Testing and Return Visits in Matched Sample, 2015–2016

	Pediatric Visits
	DTC Telemedicine	Urgent Care	Pa	PCP	Pb
Visits with diagnosis of streptococcal pharyngitis, N	250	2086	—	26 347	—
Streptococcal testing performed, n (%)	9 (4)	1557 (75)	<.001	17 818 (68)	<.001
Total ARI visits, N	4604	38 408	—	485 201	—
Return ARI visit within 2 d, n (%)	226 (5)	869 (2)	<.001	5875 (1)	<.001
Return ARI visit within 21 d, n (%)	525 (11)	3719 (10)	<.001	45 629 (9)	<.001

—, not applicable.

^aP value determined from χ² test for DTC telemedicine versus urgent care visits.

^bP value determined from χ² test for DTC telemedicine versus PCP visits.

Stratified Analysis by Patient Characteristics

Compared with urgent care and PCP visits, DTC telemedicine visits had significantly increased antibiotic prescribing and decreased guideline-concordant antibiotic management within each category of age and chronic medical complexity (Fig 1). The 1 exception was that DTC telemedicine and urgent care results were not statistically different for the youngest children (0–1 year old), for which the sample size was small. Interaction terms indicated that the degree of difference between DTC telemedicine and PCP visits varied significantly with age for antibiotics prescribed (interaction term P < .001) and guideline concordance (interaction term P = .004); other interaction terms were not significant.

FIGURE 1.

Antibiotic prescribing and guideline-concordant antibiotic management for DTC telemedicine compared with urgent care (UC) and PCP visits, stratified by child age and chronic medical complexity. A, Antibiotic prescribing associated with DTC telemedicine visits compared with UC and PCP visits, stratified by 4 levels of child age (left) and 3 levels of chronic medical complexity (right). B, Guideline-concordant antibiotic management associated with DTC telemedicine visits compared with UC and PCP visits, stratified by 4 levels of child age (left) and 3 levels of chronic medical complexity (right). Within each age or medical complexity category, the values for UC and PCP visits differed significantly from DTC telemedicine with 1 exception: DTC telemedicine and UC results were not statistically different for the youngest children (0–1 year old). Interaction terms indicated that the degree of difference between DTC telemedicine and PCP visits varied significantly with age for antibiotics prescribed (interaction term P < .001) and guideline concordance (interaction term P = .004); other interaction terms were not significant.

Sensitivity Analyses

When no longer classifying “no antibiotic use” as guideline-concordant for otitis media and sinusitis, rates of guideline-concordant antibiotic management remained lower overall at DTC telemedicine (51% vs 59% urgent care; 69% PCP; P < .001 for both; Supplemental Table 9). Analysis excluding index visits with follow-up visits within 2 days did not differ meaningfully from main results.

Discussion

Children treated for ARIs at commercial DTC telemedicine visits were substantially more likely to receive an antibiotic and less likely to receive guideline-concordant antibiotic management than children presenting to urgent care or primary care. These quality differences were observed in metrics agnostic to visit diagnosis (antibiotic prescribing rate) and conditional on diagnosis (guideline-concordant antibiotic management). These differences were primarily driven by increased use of antibiotics for visits receiving viral diagnoses.

These differences in antibiotic prescribing for children contrast with previous studies of DTC telemedicine quality among adult patients in which quality differences have been smaller or nonexistent.^9,10 For example, using data from the same health plan, we observed similar antibiotic management for adults seeking care for ARIs in these 3 settings.³² Professional groups such as the American Academy of Pediatrics and the American Telemedicine Association have previously raised concerns about DTC telemedicine care outside of the medical home for pediatric patients.^7,8 Our results support these concerns and underscore the importance of pediatric-specific evaluation and guidelines. Targeted improvement initiatives have the potential to substantially improve the quality of care delivered via DTC telemedicine.³³ Pediatric-specific guidelines and metrics should be incorporated into such efforts.

Several mechanisms could contribute to increased antibiotic prescribing and decreased guideline concordance in pediatric DTC telemedicine visits. First, the potentially rich information that telemedicine can provide in other models of telemedicine, such as when the patient is at a facility designed for telemedicine consultation, is relatively limited in DTC telemedicine. Specifically, the information transmitted in DTC telemedicine is limited in that personal devices used by parents for DTC telemedicine visits rarely incorporate peripheral attachments designed to enhance remote visits (eg, tele-otoscopy) and also do not incorporate trained telepresenters (ie, nurses or medical assistants with the patient). The limited availability of otoscopy and streptococcal testing in DTC telemedicine may contribute to the different diagnosis mix at DTC telemedicine (increased viral ARIs and sinusitis and decreased otitis media and streptococcal pharyngitis) compared with in-person settings. Additionally, personal devices may vary in the quality of microphones, cameras, and Wi-Fi, and some DTC telemedicine visits are completed by using only telephone. These issues reduce the data available to the treating DTC telemedicine provider, which may be of greater concern in pediatric care because of the more limited ability of children to communicate symptoms. Second, there may be differential expectations for antibiotics among children and parents who use DTC telemedicine versus in-person care. Although we matched on a number of observed variables, parental expectations may still differ across settings in ways that we were unable to address. Third, DTC telemedicine visits outside of the medical home lack 3 types of continuity: informational (lack of medical records), relationship (lack of ongoing relationship between the provider and patient), and clinical management (lack of opportunity to manage the patient’s care over time),³⁴ which has the potential to impact quality of care. However, this is unlikely to be the only explanation given that authors of previous evaluations of retail-based clinics, which are also a model of in-person care with limited continuity, have found comparable quality to physician office visits.^15,19 Fourth, the majority of commercial DTC telemedicine visits are for adults, such that DTC telemedicine providers may have variable pediatric training or experience, which could impact providers’ knowledge of and adherence to guideline-based pediatric care.³⁵ Prior studies indicate that only 9.6% of urgent care centers employ any pediatricians,³⁶ and pediatricians may also be underrepresented in DTC telemedicine. Together, these issues may increase clinical uncertainty during pediatric DTC telemedicine visits, prompting physicians to prescribe antibiotics “just to be safe.”^37,38

We note additional quality concerns raised by our analysis beyond antibiotic use. First, the American Telemedicine Association has recommended that commercial DTC telemedicine should not be used for children <2 years old.⁸ We note that 5% of studied DTC telemedicine visits for ARIs were within this age group but that gaps in quality existed for children of all ages. Second, accurate diagnosis of otitis media in children requires visualization of the tympanic membranes,¹¹ and accurate diagnosis of streptococcal pharyngitis requires streptococcal testing.²⁷ Making these diagnoses via telemedicine in the absence of reliable visualization and testing could in itself be considered a quality concern. Third, increased rates of follow-up visits after DTC telemedicine raise questions about the effectiveness and system-level efficiency of these visits.

Quality of care for ARIs may be better in other models of acute telemedicine care. For example, acute telemedicine visits in which peripheral devices such as tele-stethoscopes or tele-otoscopes (such as offered at telemedicine kiosks³⁹) are integrated may allow for enhanced examination compared with DTC telemedicine. Using telemedicine technology to connect patients to their PCP’s office would allow telemedicine use within the context of continuity of information, relationships, and care management.⁷ School-based or day care–based models of acute telemedicine can be used to integrate peripherals and telepresenters.^40,41 Finally, pediatric-specific improvement initiatives may also result in improved quality.³³ Examining the variation in quality across different models of acute telemedicine care may inform optimal use of technology for acute pediatric concerns.

Although our focus was DTC telemedicine, the quality of antibiotic prescribing was also lower at urgent care visits compared with PCP visits (59% vs 67% guideline concordant). Urgent care clinics have grown rapidly in the United States,⁴² and urgent care and DTC telemedicine share some characteristics that might impede quality, such as limited continuity. Across all 3 settings, our findings emphasize the need for improvement in guideline-concordant antibiotic use and ongoing antibiotic stewardship efforts in outpatient settings.^43–48

Our analysis had several limitations. As an analysis of insurance claims, we did not have additional sociodemographic or clinical data and thus could not account for variables such as patient race, severity of illness, time constraints, or family expectations, which might drive differences in prescribing.^38,49 Given the data available, we could not independently confirm the diagnosis made. The differences in diagnosis mix in the unmatched data could be due to true differences in case mix (perhaps due to self-selection to specific settings on the part of parents) or biases in diagnosis (perhaps due to differences in information available to clinicians or differences in desire to select a diagnosis that justifies prescription of antibiotics). Insurance claims also do not have information on patient allergies. Although we would not expect substantial differences in rates of allergies across settings, we may be underestimating overall guideline concordance in the absence of allergy data. Also, because some children were enrolled for brief periods, we may not be capturing all children with higher medical complexity. However, we confirmed that identified rates of chronic complex disease in the full population of pediatric health plan beneficiaries (6.4%) were relatively similar to rates identified in other population studies (6% in previous claims analysis²⁹). We used claims data rather than electronic health record or provider-reported data. Although this means we examined antibiotics filled rather than prescribed, in-person visit antibiotic use appeared consistent with previous studies of antibiotic use. Specifically, our rates of antibiotic use among unmatched PCP visits were comparable to previous studies for diagnoses for which antibiotics may be indicated (59% vs 50%–80% in previous literature) and diagnoses for which antibiotics are not indicated (16% vs 20%–21% in previous literature).^14,20,50 We did not include retail-based clinics in our analyses. Authors of other studies have found that among adults, retail-based clinics have higher rates of guideline-concordant care relative to PCPs.^15,19 Physician specialty was not available for DTC telemedicine visits, so we could not examine whether pediatric-specific physician training explained any quality differences. We note also that our analysis is limited to a specific health plan and visits by its members to their contracted DTC telemedicine vendor, and our analysis also did not include Medicaid beneficiaries. Antibiotic prescribing among other DTC telemedicine companies, models, and populations may differ. Understanding the impact of physician training and telemedicine models on quality of care during DTC telemedicine should be a priority in future work.

Conclusions

Children who receive care via DTC telemedicine visits were significantly more likely to receive antibiotics and less likely to receive guideline-concordant antibiotic management than children who visit either urgent care or PCPs.

Glossary

ARI

acute respiratory infection

DTC

direct-to-consumer

HDHP

high-deductible health plan

ICD-9-CM

International Classification of Diseases, Ninth Revision, Clinical Modification

ICD-10-CM

International Classification of Diseases, 10th Revision, Clinical Modification

PCP

primary care provider

URI

upper respiratory infection