Clinical Effectiveness of Telemedicine-Based Pediatric Genetics Care

Abstract

Background and Objectives

Telemedicine is one proposed solution to increase access to medical genetics care. However, in the pediatric setting, it is unknown how telemedicine may impact the diagnostic rate due to the importance of the dysmorphology physical exam. Therefore, we studied the clinical effectiveness of telemedicine for patients with suspected or confirmed genetic conditions.

Methods

We conducted a retrospective cohort study of outpatient encounters before and after the widespread implementation of telemedicine (n=5854 total). Visit types, diagnoses, patient demographics and laboratory data were acquired from the electronic health record. Patient satisfaction was assessed using survey responses. New molecular diagnosis was the primary endpoint.

Results

Patients seen by telemedicine were more likely to report non-Hispanic white ancestry, prefer to speak English, live in zip codes with higher median incomes, and have commercial insurance (each p<0.01). Genetic testing was recommended for more patients evaluated by telemedicine than in-person (79.5% vs 70.9%, p<0.001). However, patients seen in person were more likely to have a sample collected, resulting in similar test completion rates (51.2% for telemedicine vs. 55.1% for in-person, p=0.09). Ultimately, there was no significant difference in molecular diagnosis rate between visit modalities (13.8% for telemedicine vs 12.4% for in-person, p=0.40).

Conclusions

Telemedicine and traditional in-person evaluation resulted in similar molecular diagnosis rates. However, improved methodologies for remote sample collection may be required. This study demonstrates the feasibility of telemedicine in a large academic medical genetics practice and is applicable to other pediatric specialties with perceived importance of physical examination.

Table of Contents Summary

Following the implementation of telemedicine-based genetics care at an academic center, we analyze the diagnostic timeline and molecular diagnosis rate for in-person versus virtual evaluations.

Introduction

A major issue in medical genetics is access to care due to a shortage of providers and frequent affiliation with large academic medical centers located in urban areas. Even prior to the COVID-19 pandemic, the field of genetics recognized the potential value of implementing telemedicine, a care model which has been termed “telegenetics.”^1–3 Telemedicine can broadly be defined as the use of “information and communication technologies for the exchange of valid information for diagnosis, treatment and prevention of disease.”⁴ Genetic care can be provided remotely through multiple platforms, but live synchronous videoconferencing has become the most commonly used.⁵ Clinical genetics involves physical examination and precise anthropometric measurement, as well as the provision of diagnostic testing, patient counseling, and medical management of rare inherited diseases. All of this is orchestrated by highly specialized genetic counselors, advanced practice providers, and physicians. A telegenetics-based care model could alleviate geographic constraints, thereby increasing patient access.^6–9

Previous studies have evaluated multiple factors surrounding telemedicine. Depending on the clinical setting, use of telemedicine seems to vary among different racial and socioeconomic groups in potentially disparate ways.^10–12 Compared to in-person visits, telegenetics care has been shown to have similar outcomes as measured by patient satisfaction, genetic knowledge, and psychosocial outcomes.^13–16 In general, these studies have largely been performed in adult practices and lack assessment of clinical diagnostic efficacy^17,18,5,9,19

Thus, we evaluated the effect of telemedicine on medical genetic care in a pediatric setting. Here, we describe our findings, which have numerous implications for the implementation of telegenetics and telemedicine more broadly.

Methods

Human Subjects Research

The Institutional Review Board at Children’s Hospital of Philadelphia (CHOP) determined that this study met exemption criteria per 45 CFR 46.104(d) 4(iii). A waiver of HIPAA authorization per 45 CFR 164.512(i)(2)(ii) was granted for accessing identifiable information from the medical records.

Setting

This study was performed by the Division of Human Genetics at CHOP, which is comprised of Clinical Genetics (including the Individualized Medical Genetics Center and the 22q and You Center) and Metabolism (also known as Biochemical Genetics and including the Mitochondrial Medicine program). The division sees approximately 5500 outpatient encounters annually. The time periods analyzed for this study were April 1, 2019, through October 1, 2019, and April 1, 2020, through October 1, 2020. In 2019, the division included 21 attending physicians, 10 fellows, 19 genetic counselors and 3 advanced practice providers. In 2020, the division included 20 attending physicians, 12 fellows, 24 genetic counselors and 5 advanced practice providers.

Data Collection and Analysis

Data extracted from the electronic health record (EHR, Epic Systems) included visit type, patient demographics (age, sex, race and ethnicity, primary language, ZIP code, payor), ICD-10 diagnosis codes, and amounts billed and reimbursed for each encounter. Median income by ZIP code was determined from United States 2019 census data. Distance to CHOP was calculated “as-the-crow-flies” from the GPS coordinates of the patient’s home address to the GPS coordinates of the main hospital. A random amount of deviation between −0.05º and + 0.05º was added to figures to protect privacy. Press Ganey scores and free text comments were compiled to assess patient satisfaction.

To assess the diagnostic process in Clinical Genetics, we manually reviewed 2240 new patient encounters during our study periods. For each new patient encounter, we recorded the date and type of test (single gene, SNP microarray, gene panel and exome) recommended, the date the sample was collected, the date the test resulted, and the date results were disclosed to the patient. A test was “recommended” if a clinician from the division documented their intention to perform it. The date a sample was collected or a test resulted was determined from the EHR or the test report form if resulting from an external reference lab. The disclosure date was obtained from clinical notes. Finally, we recorded whether the recommended test(s) resulted in a new molecular diagnosis for the patient.

We calculated a metric of test breadth recommended at each initial evaluation as a proxy for clinician’s confidence in their diagnostic assessment. We asked the clinically trained coauthors to assign each class of test an integer value from most targeted (one) and to most broad (five). We determined the mean value for each test and rounded to the nearest integer. Based on this, Fragile X and single gene testing were assigned one, panels were assigned three, microarray was assigned four, and exome was assigned five. The breadth of the recommended test(s) for a given patient was equal to the integer value of most broad test recommended.

Statistics

The R statistical language and software environment was used to visualize and analyze data. Sentiment analysis of free text comments was performed using the “tm” and “syuzhet” packages. Equivalence and non-inferiority testing were performed using the “TOSTER” package. Equivalence was claimed if both bounds of the 95% confidence interval of the proportion of patients receiving a molecular diagnosis was within a predetermined margin of equivalence (−2% to +2%). Non-inferiority, which we refer to as “comparable” throughout the manuscript, was claimed if the lower bound of the 95% confidence interval was within the margin but the upper bound exceeded it.

Results

Patient volume with transition to telegenetics-based care

In April 2020, we rapidly transitioned to telemedicine-based provision of care, with 99% of encounters that month conducted virtually (Figure 1A, n = 430 virtual encounters of 433 total). Overall, 78% of visits in the 2020 study period were conducted virtually, compared to 2% in 2019. We found that genetic counselor-only visits in our 2020 study period (n = 206) were nearly quadruple that of the same period in 2019 (n = 53), demonstrating the important role of genetic counselors in staffing telegenetics appointments.

Figure 1. Distribution of in-person vs. video encounters in 2019 and 2020.

(A) Distribution of in-person versus video visits for new and follow-up appointments across the Division of Human Genetics. (B) Distribution of in-person versus video visits for each section within the Division of Human Genetics.

As 2020 progressed, there was variable return to in-person appointments across the division. While Clinical Genetics increased in-person encounters to 53% of overall encounters by September 2020, Metabolism appointments remained largely virtual. Of note, more than 95% of all visits performed exclusively by a genetic counselor remained virtual, regardless of section.

Demographics of patient populations by encounter methodology

We compared demographic data for patients who were seen virtually, in person or both during our study periods (Table 1). We found significant differences in patient age, race/ethnicity, preferred language, median income by ZIP code, and payor type based on encounter type. Interestingly, we did not find a significant difference in distance from the patient’s home to the hospital based on encounter type, although those evaluated only in person lived an average of 15 kilometers closer to the hospital (Supplemental Figure 1). There was no significant difference in patient sex.

Table 1.

Patient Demographic Data

	In-person (n = 2642)	Telehealth (n = 1685)	Both (n = 556)	Total (n = 4883)	p value
Age (years)					< 0.001
Mean (SD)	8.307 (9.280)	8.795 (11.370)	11.067 (12.743)	8.789 (10.505)
Sex					0.973
Female	1216 (46.0%)	782 (46.4%)	260 (46.8%)	2258 (46.2%)
Male	1425 (53.9%)	902 (53.5%)	296 (53.2%)	2623 (53.7%)
Race / Ethnicity					< 0.001
Hispanic or Latino	281 (10.6%)	200 (12.0%)	47 (8.5%)	528 (10.9%)
Non-Hispanic Black	299 (11.3%)	146 (8.8%)	34 (6.1%)	479 (9.8%)
Non-Hispanic White	1574 (59.6%)	1034 (62.0%)	386 (69.4%)	2994 (61.6%)
Other	485 (18.4%)	288 (17.3%)	89 (16.0%)	862 (17.7%)
Preferred Language					< 0.001
Arabic	25 (1.0%)	2 (0.1%)	2 (0.4%)	29 (0.6%)
English	2429 (92.5%)	1573 (95.0%)	527 (95.1%)	4529 (93.7%)
Spanish	105 (4.0%)	60 (3.6%)	18 (3.2%)	183 (3.8%)
Other	66 (2.5%)	21 (1.3%)	7 (1.3%)	94 (1.9%)
Median Home ZIP Code Income (USD)					0.007
Mean (SD)	83336 (38957)	87160 (40546)	84287 (36688)	84769 (39297)
Distance (km)					0.188
Mean (SD)	118 (312)	133 (351)	139 (281)	126 (323)
Payor Type					< 0.001
Commercial	1628 (61.6%)	1095 (65.0%)	362 (65.1%)	3085 (63.2%)
Medical Assistance	642 (24.3%)	468 (27.8%)	95 (17.1%)	1205 (24.7%)
Other	372 (14.1%)	122 (7.2%)	99 (17.8%)	593 (12.1%)

Patient age, sex, and self-reported race and ethnicity and language were abstracted from Epic. Income was approximated using the median income by ZIP code from 2019 census data. Distance was measured “as-the-crow” flies from the patient’s home address to the CHOP main hospital building. Patients were grouped based on the encounter types they had during the study periods. P values were generated by ANOVA or chi-squared tests where appropriate. Note that these numbers do not equal the total encounters because some patients were seen multiple times.

We also explored the distribution of the most common ICD-10 diagnosis categories seen during our study periods and found no evidence for a significant relationship between evaluation method and 10 most common ICD-10 diagnosis categories (Supplemental Figure 2).

Patient satisfaction with virtual vs. in-person care

To assess for differences in patient satisfaction levels, we analyzed survey responses for encounters within our study periods (Supplemental Figure 3). We found similar rates of overall satisfaction, with 89.2% of 2019 respondents and 88.4% of 2020 respondents selecting the highest possible score (p = 0.76, Wilcoxon test). Importantly, 2020 respondents expressed substantially increased satisfaction with access, with 74.3% selecting the top score, compared to 60.8% in 2019 (p < 0.001, Wilcoxon test). Interestingly, we found that patient satisfaction with their care provider was decreased in 2020, with 87.7% rating the top score, compared to 91.8% in 2019 (p = 0.01, Wilcoxon test). We also analyzed the sentiment (negative, neutral, or positive) of free-text comments made by respondents and found no significant difference between the time periods (p = 0.62, chi-squared test).

Diagnostic timeline and diagnostic efficacy

Anecdotally, providers felt that videoconferencing can introduce challenges to the dysmorphology physical examination and anthropometric measurement. We wondered how diagnostic uncertainty caused by these limitations might affect recommendations for genetic testing and if there was a difference in the proportion of new patient evaluations resulting in a molecular diagnosis. Metabolism encounters were excluded to minimize influence of newborn screening results and pre-evaluation biochemical testing.

We found that providers who evaluated patients initially by video recommended at least one genetic test for 8.6% more patients (79.5% for virtual vs. 70.9% for in-person evaluation, p < 0.001, Fisher’s exact test, Figure 2A). We found that clinicians completing evaluation in person were significantly more likely to recommend only the most targeted (single gene) tests as the initial step of genetic diagnosis (p < 0.01 Fisher’s exact test with Bonferroni correction, Supplemental Figure 4). However, averaged over all recommended genetic tests, there was no significant difference in the breadth of testing for patients seen in person compared to those seen virtually (Wilcoxon test, Supplemental Figure 4).

Figure 2. Test recommendation and ultimate completion rates by initial visit method.

(A) Percentage of patients recommended to undergo a given diagnostic test and percentage completed. Error bars indicate the 95% confidence interval of the proportion. (B) Time required between test recommendation and return of results to the patient. (C) Analysis of the steps in diagnostic testing, including time between recommendation and sample receipt by the laboratory, time between sample receipt and test report, and time between test report and documentation of disclosure. Note that the sample collection time may also include time required for insurance authorization or benefits investigation.

Interestingly, while more total tests were recommended for patients evaluated by video, we found no significant difference in ultimate genetic test completion rate between the two evaluation methodologies (51.2% for virtual vs. 55.1% for in-person evaluation, p = 0.09, Fisher’s exact test, Figure 2A). Similarly, we found no difference in breadth of completed testing (Wilcoxon test, Supplemental Figure 4). However, patients evaluated in person completed testing in a shorter amount of time (p < 0.001, Kolmogorov-Smirnov test, Figure 2B). Analysis of the steps leading to result disclosure revealed sample collection as the bottleneck, with a DNA sample drawn a median of 40 days sooner for patients seen in person (Figure 2C). For a considerable proportion of these patients, a DNA sample was collected on the day of their visit, whereas genomic studies for telegenetics patients required distribution and return of a saliva collection kit or subsequent presentation to a laboratory.

Given similar test utilization regardless of encounter method, we wanted to understand potential effects of telehealth on ultimate molecular diagnosis rate. Strikingly, we found that our overall molecular diagnosis rate for patients seen virtually was comparable to that for patients seen in person (13.8% vs 12.4% respectively, p = 0.40, Fisher’s exact test, Supplemental Figure 5).

Given the delay in sample collection for patients undergoing telemedicine evaluations within Clinical Genetics, we wondered if similar delays affected care for established patients with inborn errors of metabolism. We evaluated time to sample collection in patients with maple syrup urine disease (MSUD) and phenylketonuria (PKU), as amino acid levels are used to guide management decisions in both conditions. In person evaluation permits same-day sample collection, and next-day results are available via our in-house metabolic laboratory. When patients with MSUD were evaluated virtually, plasma samples were collected later and results became available a median 13.5 days later (p < 0.001, Wilcoxon test, Figure 3). In contrast, monitoring for patients with PKU can be achieved through dried blood spots collected on filter paper and mailed to the laboratory. We also identified a significant but smaller delay in monitoring laboratory results in this patient population (median 6.5 days, p < 0.001, Wilcoxon test, Figure 3).

Figure 3. Metabolic monitoring lab result timeline by follow-up visit method.

Left: Time between follow-up visit and next monitoring amino acid result stratified by visit method for patients with MSUD. Right: Same analysis for patients with PKU. In contrast to MSUD, monitoring for patients with PKU can be performed by state newborn screening laboratories by mail. Circles indicate plasma monitoring performed by our hospital metabolic laboratory and triangles indicate those performed by the state newborn screening facility.

Reimbursement by encounter methodology

Finally, we asked whether charges and reimbursement amounts were different for in-person versus virtual care encounters (Figure 4). For new patient visits, the median amount charged was 203 USD higher for in-person encounters, but the median amount reimbursed was only 53 USD higher. Across the 1235 new video evaluations, this represented 65455 USD in potentially lost reimbursement. For follow-up appointments, the amounts charged and reimbursed were similar between appointment modalities.

Figure 4. Amount charged and reimbursed by visit method.

Amount charged and reimbursed for new and follow-up appointments for in-person versus video encounters. The dashed line indicates the median amount for each visit type and methodology. The percentage reimbursed for in-person and video visits were similar (37% versus 41%, respectively).

Discussion

Here, we analyze one academic medical center’s experience in the provision of in-person and virtual clinical care before and after the onset of the COVID-19 pandemic. Telegenetics has long been considered an attractive option to increase patient access to subspecialty care but remained relatively underexplored in pediatric genetics prior to 2020.

We found significant differences in patient race/ethnicity, preferred language, median income by ZIP code, and payor type based on encounter type. Patients evaluated by telehealth were more likely to report non-Hispanic White background, report English language preference, live in areas with high median income, and have commercial insurance. These findings are consistent with some previous studies demonstrating disparities in telemedicine use, particularly during the COVID-19 pandemic^11,20–23. In historic clinical contexts, telemedicine has improved access to rural communities and those with lower annual household incomes^7,12,24–26. Therefore, our findings should be taken in the context of one academic medical center in a global pandemic.

In 2020, use of telemedicine allowed us to maintain a consistent patient volume despite the limitations imposed by the beginning of the COVID-19 pandemic. Analysis of the most common ICD-10 diagnosis categories showed no evidence that patients with a particular diagnosis category were being systematically triaged to in-person or telemedicine evaluation. Interestingly, Clinical Genetics had increasingly more in-person encounters, representing at least 50% of appointments by October, whereas Metabolism continued to provide care mostly via telemedicine. Most in-person Clinical Genetics encounters were for new patient appointments; clinicians reported feeling that in-person evaluations generally provided a better opportunity for phenotyping than could be achieved virtually. In contrast, most video appointments for patients with metabolic and mitochondrial disorders were follow-up encounters. Telemedicine was an attractive option for families, as it eliminated challenges of coming to the hospital for care and potential infectious disease exposure that could lead to metabolic decompensation.

While telemedicine will continue to play an essential role in patient care, it is important to consider the impact of virtual appointments on the acquisition of monitoring labs. We found that that the time between evaluation and amino acid lab monitoring for MSUD and PKU patients was significantly longer for patients seen virtually compared to in-person evaluation. This may represent a trend among monitoring labs for many metabolic conditions. The lag for PKU patients was shorter than for MSUD. An attractive hypothesis is that the PKU families’ longstanding use of mail-based monitoring through state newborn screening labs may have primed them for continued remote monitoring.

Because video evaluation does not allow for a comprehensive dysmorphology physical examination and anthropometric assessment, previous opinion pieces express the community’s hesitation with this medium⁵. Indeed, the results of modern genetic testing, such as those from exome sequencing, appear to be influenced by the amount and quality of phenotype information submitted with testing requisitions.²⁷ Surprisingly, we found that the molecular diagnosis rate for patients seen virtually was comparable to that achieved for patients seen in person.

Our analysis did not address all potential impediments to sample collection, including steps in prior authorization for genetic testing, notification of testing authorization, out-of-pocket cost for genetic testing, access to diagnostic labs for blood draws, or staffing difficulties affecting this process. However, patients had the opportunity to submit saliva samples by mail from the earliest days of the pandemic. Additional studies are needed to better understand impediments to sample collection for diagnostic evaluation imposed by the pandemic.

There are other potential limitations of our diagnostic efficacy analysis. There was slight variation in clinicians providing care and our clinical efficacy results only consider diagnoses with a Mendelian genetic etiology and exclude other diagnoses within our purview, including teratogenicity and malformation associations such as vertebral defects, anal atresia, cardiac defects, tracheo-esophageal fistula, renal anomalies, and limb abnormalities (VACTERL) or omphalocele, extrophy, imperforate anus and spinal defects (OEIS).

Historically, one source of hesitation in implementing telemedicine has been limited reimbursement of telehealth services.^28,29 We found only minor differences in reimbursements for in-person and virtual encounters. This likely reflects an increase in insurance coverage of telemedicine during the COVID-19 pandemic. Additionally, we found that providers charged a similar amount or less for video encounters than for in-person care, while the diagnostic rate was similar. Together, these data suggest that telemedicine is a clinically and cost-effective mode of care and lends support to continued insurance coverage of telegenetics beyond the current global health crisis.

Conclusions

Unexpectedly, considering the presumed importance of the dysmorphology physical exam, we find the clinical efficacy of pediatric telegenetics evaluation to be comparable to that of in-person evaluation; however, delays in sample collection may affect timely diagnosis and management of existing conditions. Additionally, we find high levels of patient satisfaction with telehealth and similar level of reimbursement. Overall, telemedicine appears to be an appropriate care delivery platform for genetics. Our findings may be applicable to other pediatric subspecialties in which physical examination is presumed to be highly important, but diagnostic testing can be broad and accurate, such as Endocrinology or Rheumatology.

What’s Known on This Subject

Previous studies have identified high levels of patient satisfaction with telemedicine but disparities in its use. Telemedicine-mediated provision of pediatric genetic care has massively expanded in response to COVID-19 but the diagnostic efficacy of virtual evaluation remains unknown.

What This Study Adds

This study identified that the molecular diagnostic rate achieved via telemedicine evaluation is comparable to that of in-person evaluation in pediatric clinical genetics, however a potential bottleneck in evaluation is sample collection.

Abbreviations:

COVID-19

coronavirus disease 2019

EHR

electronic health record

SD

standard deviation

USD

United States dollar