Skip to main content
PLOS ONE logoLink to PLOS ONE
. 2022 Oct 27;17(10):e0276461. doi: 10.1371/journal.pone.0276461

Healthcare utilization in children across the care continuum during the COVID-19 pandemic

Alan R Schroeder 1,*, Alex Dahlen 2, Natasha Purington 2, Francisco Alvarez 1, Rona Brooks 1, Lauren Destino 1, Gayatri Madduri 1, Marie Wang 1, Eric R Coon 3
Editor: Tai-Heng Chen4
PMCID: PMC9612476  PMID: 36301947

Abstract

Objectives

Healthcare utilization decreased during the COVID-19 pandemic, likely due to reduced transmission of infections and healthcare avoidance. Though various investigations have described these changing patterns in children, most have analyzed specific care settings. We compared healthcare utilization, prescriptions, and diagnosis patterns in children across the care continuum during the first year of the pandemic with preceding years.

Study design

Using national claims data, we compared enrollees under 18 years during the pre-pandemic (January 2016 –mid-March 2020) and pandemic (mid-March 2020 through March 2021) periods. The pandemic was further divided into early (mid-March through mid-June 2020) and middle (mid-June 2020 through March 2021) periods. Utilization was compared using interrupted time series.

Results

The mean number of pediatric enrollees/month was 2,519,755 in the pre-pandemic and 2,428,912 in the pandemic period. Utilization decreased across all settings in the early pandemic, with the greatest decrease (76.9%, 95% confidence interval [CI] 72.6–80.5%) seen for urgent care visits. Only well visits returned to pre-pandemic rates during the mid-pandemic. Hospitalizations decreased by 43% (95% CI 37.4–48.1) during the early pandemic and were still 26.6% (17.7–34.6) lower mid-pandemic. However, hospitalizations in non-psychiatric facilities for various mental health disorders increased substantially mid-pandemic.

Conclusion

Healthcare utilization in children dropped substantially during the first year of the pandemic, with a shift away from infectious diseases and a spike in mental health hospitalizations. These findings are important to characterize as we monitor the health of children, can be used to inform healthcare strategies during subsequent COVID-19 surges and/or future pandemics, and may help identify training gaps for pediatric trainees. Subsequent investigations should examine how changes in healthcare utilization impacted the incidence and outcomes of specific diseases.

Background

Early trends since the start of the COVID-19 pandemic suggested large-scale decreases in non-COVID-19 related healthcare utilization. These changes have been largely attributable to a decrease in transmission of common infections [18] and/or fear of COVID-19 exposure in the healthcare setting [913]. Though various investigations have described these changing patterns of utilization in children, most have analyzed specific conditions or settings of care (e.g. hospitalizations or prescription medication use), and few have assessed how patterns align across different types of utilization. Furthermore, the investigations into hospitalization trends have focused only on US children’s hospitals, where fewer than half of all pediatric hospitalizations occur [14]. Additionally, the majority of these published studies address the time period immediately following the pandemic onset without evaluating ongoing effects as the pandemic evolved.

Insurance claims databases offer the opportunity to examine all types of utilization across the continuum of care for large populations of patients throughout the United States (US) [1519]. Major shifts in health care use and diagnosis patterns as we have observed are important to quantify as we monitor child health following the pandemic. Additionally, quantification of these shifts help to identify potential gaps in clinical exposure for students, residents, and fellows who underwent training during the pandemic. Furthermore, characterization of utilization changes during this pandemic can inform healthcare strategies during future pandemics. Finally, trends in diagnoses across various utilization settings can identify the extent to which conditions are sensitive to changes brought about by the pandemic. Such information provides a “natural experiment” to investigate how viral transmission and healthcare utilization impact the incidence and outcomes of specific diseases, and quantifying any changes in incidence is a necessary first step. We aimed to use claims data to compare the rate of primary care, urgent care, and emergency department (ED) visits, hospitalizations, and outpatient prescription medications during the first year of the pandemic as compared to prior years. Additionally, we analyzed the pandemic’s impact on common diagnoses and prescription medication use.

Methods

Design

Using a cross-sectional approach, we used Optum’s De-identified Clinformatics Data Mart database to measure changes in utilization during vs before the pandemic. The Optum database contains administrative health claims for large commercial and Medicare Advantage health plans covering approximately 15–18 million lives annually, but does not include patients with Medicaid. The database includes patient demographics, pharmaceutical claims, inpatient and outpatient diagnosis codes, and procedure claims from enrollees in 50 states, with more than 2 million children enrolled each month [20]. The age and sex distribution of the plan members is similar to that reported by the US Census Bureau [21]. Although deidentification of patient information makes formal validation of claims data challenging, some investigations have demonstrated encouraging reliability, though specificity of diagnostic codes is likely higher than sensitivity [2224]. The Stanford University institutional review board approved the use of this deidentified database in this study under Stanford’s Center for Population Health Sciences umbrella protocol. Given the deidentified nature of the dataset, informed consent was not required for this study.

Time periods

We compared two time periods. The COVID-19 pandemic period covered mid-March, 2020 through March 31, 2021 while the pre-pandemic period included January 1, 2016 through mid-March, 2020 [25]. We elected to use 2016 as a starting point to allow sufficient time to account for secular trends in utilization that may have begun prior to the pandemic. For de-identification purposes, Optum only provides the month of service, not the specific date. Because most lockdowns began around mid-March 2020, we labeled March as being 50% in the pandemic period and 50% in the pre-pandemic period, and the pandemic period as starting in “mid-March”.

We began the pre-pandemic period with the year 2016 because the International Classification of Diagnosis (ICD)-9 to ICD-10 conversion occurred in 2015, and because we surmised that 4 years is a sufficient time period for comparison. As utilization changes were dynamic during the pandemic, we further divided this period into two segments: “early pandemic” covered the first 3 months (mid-March 2020 –mid-June 2020) and “middle pandemic” covered mid-June 2020 through March 2021 (the most recently available data at the time of this investigation).

Subjects

We included all subjects under 18 years of age who were enrolled in Optum during the study periods. Events were only included when occurring prior to the subject’s 18th birthday. Census region and income quartile were determined by linking the patient’s zip code to 2019 five-year American Community Survey US Census data.

Utilization types

We compared primary care, urgent care, and ED visits, hospitalizations, and prescription medication claims. Our strategy for distinguishing visit types built upon prior investigations [17, 19, 26] and direct communication with the vendor. A combination of Current Procedural Terminology (CPT) and additional codes were used to distinguish utilization types as described in S2 File. Hospitalizations were characterized as psychiatric and non-psychiatric based on the designated provider category.

Classifying diseases and medications

In order to categorize diagnoses, we grouped ICD-10 codes using the Pediatric Clinical Classification System (PECCS) classification system [27], and list the top 30 diagnoses by visit type in both the pre-pandemic and the pandemic time periods. We used the American Hospital Formulary Service (AHFS) classification system to categorize prescription medications and additionally categorize prescriptions as new or refill based on that designation in Optum. The top 30 prescription categories are assessed in each time period.

Statistical analysis

For each visit and prescription type we calculated a rate by dividing the total number of visits or prescriptions in a month by the total number of patients (<18 years of age) who were covered by Optum during that month. We used an interrupted time series (ITS) approach to evaluate changes in healthcare utilization. Percent changes are measured with respect to a counterfactual where the pandemic had not occurred and the pre-pandemic seasonal fluctuations and linear trends had continued. Full details of the model specification are provided in (S2 File).

Finally, we also consider visit types associated with specific diseases using this same model. For ED and primary care provider (PCP) sick visits, and for hospitalizations, we report the changes in the number of visits associated with the most common PECCS disease classifications. For medication, we report the changes in the number of prescriptions written for the most common AHFS categories. For tables depicting the leading disease and prescription categories, we include any disease/medication that was in the top 30 during either the pre-pandemic period or the pandemic period. All analyses were conducted in Python, version 3.8.5. Regression parameters were estimated using the statsmodels package, version 0.12.0.

Results

The mean number of child enrollees per month was 2,519,755 in the pre-pandemic period and 2,428,912 in the pandemic period. Demographic characteristics of the pediatric Optum cohort are provided in Table 1, which depicts comparisons of the pre-pandemic and pandemic periods. Overall the demographics varied minimally, with standardized mean differences less than 0.2. S1 Table in S1 File details results of an ITS analysis examining shifts in the demographic breakdown throughout the study period. Of note, the proportion of missing data on race in Optum has increased over recent years; however, the representation by category remained fairly constant for patients without missing data (S1 Fig in S1 File).

Table 1. Demographic characteristics of the study population.

Pre-Pandemic* Pandemic* Standardized Mean Difference**
n 7,048,075 3,387,469
Female 48.9% 48.9% 0.00
Age
    < 1 2.2% 2.1% -0.04
    1–5 25.9% 25.6% -0.01
    6–12 40.6% 40.4% -0.01
    13–17 31.3% 31.9% 0.02
Race/Ethnicity (excluding unknown)
    non-Hispanic White 69.7% 70.5% 0.04
    non-Hispanic Black 8.1% 7.9% -0.04
    Hispanic 15% 14.4% -0.05
    Asian 7.2% 7.2% 0.01
Unknown Race/Ethnicity 16.5% 42.3% -
Census Region
    New England 2.5% 2.8% 0.09
    Middle Atlantic 6.4% 6.4% 0.00
    East North Central 15.2% 15.6% 0.03
    West North Central 11.4% 13% 0.15
    South Atlantic 20% 20.4% 0.03
    East South Central 3.7% 3.8% 0.03
    West South Central 17.5% 15.7% -0.13
    Mountain 10.9% 10.8% -0.01
    Pacific 12.4% 11.6% -0.07
Income quartile (of patient zip code)
    Lowest (< $55,019) 21.8% 21% -0.04
    Second ($55,020 - $72,759) 25.1% 25.1% 0.00
    Third ($72,759 - $96,969) 26.7% 27% 0.02
    Highest (> $96,969) 26.5% 26.8% 0.02

*Pre-pandemic period corresponds to January 2016 until mid-March 2020. Pandemic period corresponds to mid-March 2020 through March 2021.

**Standardized mean differences can be interpreted using Cohen’s guideline: d = 0.2 corresponds to a small difference; d = 0.5, to a medium difference; and d = 0.8 to a large difference.

Early pandemic utilization

During the early pandemic period, utilization decreased across all visit types and for hospitalizations and prescription fills (Figs 1 and 2). The largest drop occurred for urgent care visits (76.9% decrease, 95% confidence interval [CI] 72.6–80.5%), PCP sick visits (61.6% decrease, 95% CI 59.4% - 63.6%) and ED visits (59.6% decrease, 95% CI 54.8–63.9%) (Fig 2).

Fig 1. Time series depicting trends in utilization and prescriptions per 100 child-years over the study period.

Fig 1

Dotted vertical lines represent the starts of the early pandemic (mid-March) and mid-pandemic (mid-June).

Fig 2. Pandemic changes in healthcare utilization with heat map.

Fig 2

There were 288.9 prescription fills per 100 child years during the pre-pandemic period (with 54% of patients receiving at least one new or refill prescription in 2019). While prescription refills were unchanged during the early pandemic period, new prescriptions decreased by 47.1% (95% CI 43.7–50.3%).

Hospitalizations in non-psychiatric hospitals decreased by 43% (95% CI 37.4–48.1%), with similar decreases for hospitalizations that did and did not require an ICU stay. Hospitalizations in psychiatric facilities decreased by 36.6% (95% CI 31.9–40.9%).

Middle pandemic utilization

During the mid-pandemic period, utilization patterns increased from the early pandemic period (Fig 2). However, with the exception of PCP well-visits, utilization during the middle pandemic period was still below the pre-pandemic rates, with the greatest sustained decreases seen in ED visits (38.4% decrease from pre-pandemic period, 95% CI 30.9–45.2%) and PCP sick visits (39% decrease, 95% CI 32.3–45.1%). After the initial decrease at the pandemic onset, new medication fills (Fig 1) recovered over the next 6 months, decreased again from December 2020 –Feb 2021, and then started to increase again in March 2021. Over the entire middle pandemic period, new prescriptions were still 34.7% (95% CI 25.7–42.6%) lower than the pre-pandemic period.

Hospitalizations in non-psychiatric facilities returned closer to pre-pandemic rates during the middle-pandemic period but were still 26.6% (95% CI 17.7–34.6%) below baseline. Non-ICU hospitalizations recovered slightly more than hospitalizations with ICU stays, and hospitalizations in psychiatric facilities went nearly back to baseline (Fig 2).

In our sensitivity analysis, adjusting for the demographics portrayed in Table 1 (age, sex, race, census region, and income quartile) did not have a meaningful impact on the changes in utilization in the early or mid-pandemic periods (S6 Fig in S1 File).

Changes in common diagnoses

Common infectious diagnoses dropped substantially across utilization types during the early pandemic period and only increased modestly as the pandemic evolved. Fig 3 displays heat maps representing the changes in the most common diagnoses associated with hospitalizations in non-psychiatric facilities. Notably, hospitalizations in non-psychiatric facilities for major depressive disorder (29.5% increase, 95% CI 16.6–43.8%), bipolar disorder (118.2% increase, 95% CI 40.5–239%), substance-related disorders (83.3% increase, 95% CI 6–217%) and suicide and intentional self-inflicted injury (82.1% increase, 95% CI 51.2–119.3%) were substantially increased from pre-pandemic levels during the middle pandemic period. Hospitalizations for some conditions that should not have been impacted by the reduction in transmissible viruses occurred less frequently during the pandemic. For example, neonatal hyperbilirubinemia hospitalizations were 25.2% (95% CI 16.1–33.4%) lower during the early pandemic period and 31.8% (95% CI 23.1–39.5%) lower during the middle pandemic period.

Fig 3. Pandemic changes in leading causes of hospitalization with heat map.

Fig 3

S2 Fig in S1 File displays trends in common causes of hospitalizations in non-psychiatric facilities, PCP sick visits, ED visits, and prescriptions. S3 Fig in S1 File (PCP sick visits) and S4 Fig in S1 File (ED visits) display heat maps representing proportional changes in diagnoses during the early and middle pandemic periods as compared to the pre-pandemic periods. Large decreases in infectious diagnoses were observed in both settings. However, in contrast to hospitalizations for mental health conditions, mental health diagnoses were generally not increased above pre-pandemic norms for ED or sick visits

Changes in common new prescriptions

During the pre-pandemic period, antibiotics were the most commonly prescribed new medication, constituting 5 of the top 10 AHFS codes (Fig 4). During the pandemic, antibiotic prescriptions decreased substantially, and respiratory and central nervous system stimulants, amphetamines, selective serotonin reuptake inhibitors, and anxiolytics comprised 4 of the top 5 new prescription class codes. However, the vast majority of prescription medications were filled at rates below pre-pandemic baselines. Rates of selective-serotonin reuptake inhibitors (anti-depressants) were unchanged. The only medication that increased significantly was opiate agonists, which were 24.7% higher (95% CI 6.3–46.2%) during the middle-pandemic period.

Fig 4. Pandemic changes in leading prescriptions with heat map.

Fig 4

Telehealth

Prior to the pandemic, billed telehealth visits were rare (S5 Fig in S1 File). During the early pandemic, telehealth visits increased considerably, especially for PCP sick visits, where nearly half of these visits were conducted via telehealth. Telehealth visits decreased fairly quickly as the pandemic evolved, though remained more frequent than the pre-pandemic period.

Discussion

In this analysis of over 2 million children enrolled per year in a US insurance claims database, we report striking changes in healthcare utilization by children during the COVID-19 pandemic. Utilization decreased across all sites of care but was more pronounced for sick visits (at PCPs, urgent cares, and EDs) than for well-child visits. Decreases persisted throughout the first 12 months of the pandemic, with only well-child visits returning to their pre-pandemic rates. Common diagnoses associated with these healthcare encounters shifted away from infectious conditions, with lesser decreases or even increases (hospitalizations in non-psychiatric facilities) in mental health conditions. Similarly, prescription fills decreased with especially large drops in antibiotics.

Drivers of the observed patterns are likely multifactorial. Physical distancing and other non-pharmacologic interventions (NPIs) to prevent the spread of SARS-CoV-2 led to a decrease in circulating infectious organisms that are common causes of sick visits, hospitalizations, and antibiotic use in children [18]. Additionally, families may have been more reluctant to pursue healthcare for fear of exposure to SARS-CoV-2. Lastly, the stresses imposed by the pandemic and measures to mitigate its severity likely triggered the observed increases in mental health hospitalizations in non-psychiatric facilities. However, hospitalizations in psychiatric hospitals did not increase over the first year, perhaps because psychiatric hospitals often operate at or near maximal occupancy [28].

There have been multiple related investigations describing pandemic associated patterns of utilization. However, the majority focus on unique utilization types and have shorter pandemic analysis periods. Using claims data, Schweiberger et al. [19] demonstrated similar initial drops in utilization, with a more pronounced decrease in infectious-related visits. Similar decreases have been demonstrated consistently in studies of ER visits [2932].

Multiple investigations have demonstrated a reduction in hospitalizations [2, 3, 3336], but most are limited to children’s hospitals despite the fact that the majority of pediatric hospitalizations in the US occur outside of children’s hospitals [14]. Our alternative approach may explain why we found different trends for certain conditions. For example, we found that urinary tract infection (UTI) hospitalizations remained fairly stable, whereas Gill et al. [3] demonstrated a reduction of UTI hospitalizations in children’s hospitals. Whether this discrepancy might be explained by inclusion of non-children’s hospitals in our investigation, inclusion of public insurance in their investigation, or other factors warrants further investigation.

For prescription fills, Chua et al. [37] analyzed a national prescription audit of US pharmacies and demonstrated a similar pattern of a dramatic drop in prescriptions early on in the pandemic, a gradual rise over the next several months, but then a drop again in December. We demonstrate that this drop (which may have been attributable to heightened NPI efforts during that December surge) continued through January and February of 2021 but then increased again. Similar to that investigation, we observed large decreases in antibiotics and no significant change in anti-depressant use (though mental health medications in general comprised a higher proportion of the most common prescriptions). The fact that the rise in hospitalizations for mental health disorders was not accompanied by an increase (in absolute terms) in prescriptions for depression and other mental health disorders is noteworthy and warrants further investigation. We did observe a significant increase in opiate prescriptions during the mid-pandemic period, along with a significant increase in hospitalizations for substance-related disorders, concerning findings that warrant further investigation and continued monitoring.

This comprehensive assessment of healthcare utilization in children during the pandemic can inform the literature in several ways. First, in the event of future lockdowns for this or other pandemics, our data can be used to assist with predictions surrounding pediatric healthcare needs. Given the persistence of SARS-CoV-2, the emergence of new variants, and the possibility of future pandemics caused by other organisms, our findings may be useful to anticipate where and how scarce resources are most needed. For example, early appropriation of resources towards mental health services may help to mitigate this important morbidity. Additionally, the near-disappearance of circulating viruses and gradual return may inform immunization strategies (e.g. for influenza or respiratory syncytial virus immunoprophylaxis).

Second, this work provides the foundation to further explore the necessity of care for various conditions [38]. For diseases that may have been impacted by a decreased pursuit of healthcare, changes in diagnostic patterns may provide further insight into the natural history of untreated disease and the extent to which the timing of diagnosis impacts outcomes. For example, delays in the pursuit of healthcare may have contributed to the increase in hospitalizations for diabetic ketoacidosis (Fig 3), which highlights the need for increased vigilance around this condition. However, for other conditions, the decreased pursuit of healthcare may not have had adverse consequences. Notably, hyperbilirubinemia hospitalizations decreased considerably during the pandemic (Fig 3). Because lockdowns and NPIs (and the resulting decrease in transmission of circulating viruses) should have had minimal if any impact on the true prevalence of neonatal hyperbilirubinemia, this reduction in hospitalizations may be explained by heightened efforts to keep young infants out of the hospital. Whether this reduction was associated with an increase in severe outcomes warrants further evaluation that may ultimately inform strategies for treatment of jaundice. Similarly, we noted that UTI diagnoses decreased substantially in the ED setting (S4 Fig in S1 File), though hospitalization rates remained similar. Because UTIs are not thought to be transmissible, these changes are also worth exploring to better understand the epidemiology of UTI–a condition which may self-resolve without treatment [39] and where uncertainty over the gold standard for diagnosis may contribute to high rates of false positives [40].

Third, our data can be used to inform our understanding of certain diseases in children where the etiology remains uncertain and where viruses may play a role. Examples of such conditions include Type I diabetes [41], Henoch-Schoenlein Purpura [42], appendicitis [43], and certain malignancies [44]. Because the overall incidence of viral infections decreased so dramatically during the pandemic, especially early on, conditions that are acutely or sub-acutely triggered by viruses should have decreased as well. We noted, for example, that appendicitis (with or without peritonitis) hospitalizations decreased (Fig 3). Possible explanations for this finding include a potential role of viruses in triggering appendicitis or the possibility that milder cases of appendicitis may have gone undetected and self-resolved. Although the current investigation will not provide definitive answers to these questions, we hope that our data will be used to stimulate hypotheses and further investigations.

Fourth, we provide general descriptive information on overall healthcare use in children before and during the pandemic. This information can be used to guide and target quality improvement and de implementation efforts. We found that privately insured US children, on average, have 3 prescriptions filled per year, and that just over half of children have at least one prescription filled per year is quite striking. In comparison, only ~ 23% of Canadian children had at least one prescription fill per year between 2012–2017 [45]. Whether this contrast reflects differences in health, health care practices, or other factors warrants further investigation.

This investigation has several limitations. Most importantly, the claims database only includes children with commercial insurance and utilization patterns may differ for publicly insured or uninsured children. Although there are active efforts to procure Medicaid claims database at Stanford, the investigators did not have access to public insurance databases at the time of this investigation. Understanding how these utilization patterns differed across demographics is an important next step. Similarly, investigations comparing shifts in utilization patterns between different countries and in the context of varying states of transmission of COVID-19 (and related restrictions) may also help to inform policy.

We relied on ICD-10 codes for diagnoses. Although investigations into the validity of diagnostic codes in claims data have mostly been promising [2224], there is still potential for miscoding. Additionally, as our intent for this investigation was to provide a broad overview of utilization and diagnostic shifts, we only included the first diagnosis in each claim. Inclusion of all diagnoses should be considered to comprehensively investigate changes for specific diseases. Furthermore, we did not attempt to discriminate between new versus recurrent problems. Such discrimination is important to better understand some of the changes we observed in conditions such as diabetic ketoacidosis. Lastly, our division of the first year of the pandemic into two periods was based on our initial visualization of the data. Separation of the pandemic into different periods may have yielded different point estimates for utilization changes and may have impacted the list of top diagnoses. For utilization changes, the monthly rates depicted in Fig 1 and S2 Fig in S1 File provide the most detailed and accurate picture of utilization trends.

Conclusion

The COVID-19 pandemic has been associated with substantial decreases changes in healthcare utilization in children. These changes have important implications for healthcare policy and disease epidemiology.

Supporting information

S1 File. Additional supporting information is provided in the supplement, which includes S1 Table and S1-S6 Figs.

(PDF)

S2 File. Supplemental methods PUPPI resubmission clean.

(DOCX)

Abbreviations

US

United States

ED

Emergency Department

ICD

International Classification of Diagnosis

CPT

Current Procedural Terminology

PECCS

Pediatric Clinical Classification System

AHFS

American Hospital Formulary Service

ITS

Interrupted Time Series; POS: Place of Service

PCP

Primary Care Provider

ICU: CI

Confidence Interval; Intensive Care Unit

NPI

Non-pharmacologic Interventions

UTI

Urinary Tract Infection

Data Availability

Database is proprietary (owned by Optum) and only available for purchase through the company. The Population Health Sciences Department of the Stanford University School of Medicine obtained this data for use by Stanford investigators. Because of this contractual relationship, Stanford investigators were allowed direct access to data. Investigators outside of Stanford interested in replicating this study should contact Optum directly for access. The Optum claims database website is https://www.optum.com/business/life-sciences/real-world-data/claims-data.html. Contact information can be found on the product sheet: https://cdn-aem.optum.com/content/dam/optum4/resources/pdf/clinformatics-data-mart.pdf.

Funding Statement

ARS received funding for biostatistical support through a private fund (Michael Enright Fund). The donors of this fund played no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

  • 1.Ae R, Shibata Y, Kosami K, Nakamura Y, Hamada H. Kawasaki Disease and Pediatric Infectious Diseases During the Coronavirus Disease 2019 Pandemic. J Pediatr. 2021. Epub 20210726. doi: 10.1016/j.jpeds.2021.07.053 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Antoon JW, Williams DJ, Thurm C, Bendel-Stenzel M, Spaulding AB, Teufel RJ 2nd, et al. The COVID-19 Pandemic and Changes in Healthcare Utilization for Pediatric Respiratory and Nonrespiratory Illnesses in the United States. J Hosp Med. 2021;16(5):294–7. doi: 10.12788/jhm.3608 ; PubMed Central PMCID: PMC8086992. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Gill PJ, Mahant S, Hall M, Berry JG. Reasons for Admissions to US Children’s Hospitals During the COVID-19 Pandemic. JAMA. 2021;325(16):1676–9. doi: 10.1001/jama.2021.4382 ; PubMed Central PMCID: PMC8080216. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Haapanen M, Renko M, Artama M, Kuitunen I. Systemic antibiotics and asthma medicines dispensed to 0–12 year olds significantly decreased during the COVID-19 pandemic in 2020. Acta Paediatr. 2021. Epub 20211008. doi: 10.1111/apa.16144 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Haddadin Z, Schuster JE, Spieker AJ, Rahman H, Blozinski A, Stewart L, et al. Acute Respiratory Illnesses in Children in the SARS-CoV-2 Pandemic: Prospective Multicenter Study. Pediatrics. 2021;148(2). Epub 20210513. doi: 10.1542/peds.2021-051462 ; PubMed Central PMCID: PMC8338906. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Katz SE, Spencer H, Zhang M, Banerjee R. Impact of the COVID-19 Pandemic on Infectious Diagnoses and Antibiotic Use in Pediatric Ambulatory Practices. J Pediatric Infect Dis Soc. 2021;10(1):62–4. doi: 10.1093/jpids/piaa124 ; PubMed Central PMCID: PMC7797736. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Luo Z, Li S, Li N, Li Y, Zhang Y, Cao Z, et al. Assessment of Pediatric Outpatient Visits for Notifiable Infectious Diseases in a University Hospital in Beijing During COVID-19. JAMA Netw Open. 2020;3(8):e2019224. Epub 20200803. doi: 10.1001/jamanetworkopen.2020.19224 ; PubMed Central PMCID: PMC7445592. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Kadambari S, Goldacre R, Morris E, Goldacre MJ, Pollard AJ. Indirect effects of the covid-19 pandemic on childhood infection in England: population based observational study. BMJ. 2022;376:e067519. Epub 20220112. doi: 10.1136/bmj-2021-067519 ; PubMed Central PMCID: PMC8753487. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Eswaran V, Wang RC, Vashi AA, Kanzaria HK, Fahimi J, Raven MC. Patient reported delays in obtaining emergency care during COVID19. Am J Emerg Med. 2021. Epub 20210808. doi: 10.1016/j.ajem.2021.08.007 ; PubMed Central PMCID: PMC8349312. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Sun C, Dyer S, Salvia J, Segal L, Levi R. Worse Cardiac Arrest Outcomes During The COVID-19 Pandemic In Boston Can Be Attributed To Patient Reluctance To Seek Care. Health Aff (Millwood). 2021;40(6):886–95. Epub 20210526. doi: 10.1377/hlthaff.2021.00250 . [DOI] [PubMed] [Google Scholar]
  • 11.Steele M, Howells L, Santer M, Sivyer K, Lawton S, Roberts A, et al. How has the COVID-19 pandemic affected eczema self-management and help seeking? A qualitative interview study with young people and parents/carers of children with eczema. Skin Health Dis. 2021:e59. Epub 20210623. doi: 10.1002/ski2.59 ; PubMed Central PMCID: PMC8420339. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Shaw KG, Salton RL, Carry P, Hadley-Miller N, Georgopoulos G. Multi-day delay to care identified in pediatric trauma cases during COVID-19. J Pediatr Orthop B. 2021. Epub 20210817. doi: 10.1097/BPB.0000000000000910 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Lynn RM, Avis JL, Lenton S, Amin-Chowdhury Z, Ladhani SN. Delayed access to care and late presentations in children during the COVID-19 pandemic: a snapshot survey of 4075 paediatricians in the UK and Ireland. Arch Dis Child. 2021;106(2):e8. Epub 20200625. doi: 10.1136/archdischild-2020-319848 ; PubMed Central PMCID: PMC7371571. [DOI] [PubMed] [Google Scholar]
  • 14.Leyenaar JK, Ralston SL, Shieh MS, Pekow PS, Mangione-Smith R, Lindenauer PK. Epidemiology of pediatric hospitalizations at general hospitals and freestanding children’s hospitals in the United States. J Hosp Med. 2016;11(11):743–9. Epub 20160704. doi: 10.1002/jhm.2624 ; PubMed Central PMCID: PMC5467435. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Caswell-Jin JL, Shafaee MN, Xiao L, Liu M, John EM, Bondy ML, et al. Breast cancer diagnosis and treatment during the COVID-19 pandemic in a nationwide, insured population. Breast Cancer Res Treat. 2022;194(2):475–82. Epub 20220527. doi: 10.1007/s10549-022-06634-z ; PubMed Central PMCID: PMC9140322. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Engelbrecht K, Roy S, Capkun G, Kahler K, Olson M. Impact of the COVID-19 pandemic on healthcare resource utilization across selected disease areas in the USA. J Comp Eff Res. 2022;11(11):815–28. Epub 20220614. doi: 10.2217/cer-2022-0059 ; PubMed Central PMCID: PMC9251631. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Ray KN, Shi Z, Ganguli I, Rao A, Orav EJ, Mehrotra A. Trends in Pediatric Primary Care Visits Among Commercially Insured US Children, 2008–2016. JAMA Pediatr. 2020;174(4):350–7. doi: 10.1001/jamapediatrics.2019.5509 ; PubMed Central PMCID: PMC6990970. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Rodrigues A, Jin MC, Pendharkar A. Epidemiological and Treatment Trends for Acute Ischemic Stroke Preceding and during the COVID-19 Pandemic. Cerebrovasc Dis. 2022;51(2):165–8. Epub 20210922. doi: 10.1159/000518935 ; PubMed Central PMCID: PMC8678258. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Schweiberger K, Patel SY, Mehrotra A, Ray KN. Trends in Pediatric Primary Care Visits During the Coronavirus Disease of 2019 Pandemic. Acad Pediatr. 2021;21(8):1426–33. Epub 20210511. doi: 10.1016/j.acap.2021.04.031 ; PubMed Central PMCID: PMC8561008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Datamart OC. [cited 2021 12/15/2021]. Available from: https://www.optum.com/business/solutions/life-sciences/explore-data/advanced-analytics/claims-data.html.
  • 21.Jain A, Marshall J, Buikema A, Bancroft T, Kelly JP, Newschaffer CJ. Autism occurrence by MMR vaccine status among US children with older siblings with and without autism. JAMA. 2015;313(15):1534–40. doi: 10.1001/jama.2015.3077 . [DOI] [PubMed] [Google Scholar]
  • 22.Cullen MR, Vegso S, Cantley L, Galusha D, Rabinowitz P, Taiwo O, et al. Use of medical insurance claims data for occupational health research. J Occup Environ Med. 2006;48(10):1054–61. doi: 10.1097/01.jom.0000241049.23093.a4 . [DOI] [PubMed] [Google Scholar]
  • 23.Tessier-Sherman B, Galusha D, Taiwo OA, Cantley L, Slade MD, Kirsche SR, et al. Further validation that claims data are a useful tool for epidemiologic research on hypertension. BMC Public Health. 2013;13:51. Epub 20130118. doi: 10.1186/1471-2458-13-51 ; PubMed Central PMCID: PMC3565904. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Thyagarajan V, Su S, Gee J, Duffy J, McCarthy NL, Chan KA, et al. Identification of seizures among adults and children following influenza vaccination using health insurance claims data. Vaccine. 2013;31(50):5997–6002. Epub 20131019. doi: 10.1016/j.vaccine.2013.10.026 ; PubMed Central PMCID: PMC6636628. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Moreland A, Herlihy C, Tynan MA, Sunshine G, McCord RF, Hilton C, et al. Timing of State and Territorial COVID-19 Stay-at-Home Orders and Changes in Population Movement—United States, March 1-May 31, 2020. MMWR Morb Mortal Wkly Rep. 2020;69(35):1198–203. Epub 20200904. doi: 10.15585/mmwr.mm6935a2 ; PubMed Central PMCID: PMC7470456. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Poon SJ, Schuur JD, Mehrotra A. Trends in Visits to Acute Care Venues for Treatment of Low-Acuity Conditions in the United States From 2008 to 2015. JAMA Intern Med. 2018;178(10):1342–9. doi: 10.1001/jamainternmed.2018.3205 ; PubMed Central PMCID: PMC6233753. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Gill PJ, Anwar MR, Thavam T, Hall M, Rodean J, Mahant S. Pediatric Clinical Classification System for Use in Inpatient Settings. JAMA Pediatr. 2021;175(5):525–7. doi: 10.1001/jamapediatrics.2020.6007 ; PubMed Central PMCID: PMC7922237. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Bebinger M. Shortage of Mental Health Beds Keeps Kids Trapped Inside ERs. In: WebMD, editor. 2021. [Google Scholar]
  • 29.Finkelstein Y, Maguire B, Zemek R, Osmanlliu E, Kam AJ, Dixon A, et al. Effect of the COVID-19 Pandemic on Patient Volumes, Acuity, and Outcomes in Pediatric Emergency Departments: A Nationwide Study. Pediatr Emerg Care. 2021;37(8):427–34. doi: 10.1097/PEC.0000000000002484 ; PubMed Central PMCID: PMC8327936. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Haddadin Z, Blozinski A, Fernandez K, Vittetoe K, Greeno AL, Halasa NB, et al. Changes in Pediatric Emergency Department Visits During the COVID-19 Pandemic. Hosp Pediatr. 2021;11(4):e57–e60. Epub 20210112. doi: 10.1542/hpeds.2020-005074 . [DOI] [PubMed] [Google Scholar]
  • 31.Pines JM, Zocchi MS, Black BS, Carlson JN, Celedon P, Moghtaderi A, et al. Characterizing pediatric emergency department visits during the COVID-19 pandemic. Am J Emerg Med. 2021;41:201–4. Epub 20201123. doi: 10.1016/j.ajem.2020.11.037 ; PubMed Central PMCID: PMC7682424. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Roland D, Harwood R, Bishop N, Hargreaves D, Patel S, Sinha I. Children’s emergency presentations during the COVID-19 pandemic. Lancet Child Adolesc Health. 2020;4(8):e32–e3. Epub 20200627. doi: 10.1016/S2352-4642(20)30206-6 ; PubMed Central PMCID: PMC7319915. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Clark NA, Rodean J, Mestre M, Rangarajan HG, Samuels-Kalow M, Satwani P, et al. Pandemic-Related Shifts in New Patients Admitted to Children’s Hospitals. Hosp Pediatr. 2021;11(8):e142–e51. Epub 20210601. doi: 10.1542/hpeds.2021-005876 . [DOI] [PubMed] [Google Scholar]
  • 34.Manzoni P, Militello MA, Fiorica L, Cappiello AR, Manzionna M. Impact of COVID-19 epidemics in paediatric morbidity and utilisation of Hospital Paediatric Services in Italy. Acta Paediatr. 2021;110(4):1369–70. Epub 20200724. doi: 10.1111/apa.15435 ; PubMed Central PMCID: PMC7361715. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Williams TC, MacRae C, Swann OV, Haseeb H, Cunningham S, Davies P, et al. Indirect effects of the COVID-19 pandemic on paediatric healthcare use and severe disease: a retrospective national cohort study. Arch Dis Child. 2021;106(9):911–7. Epub 20210115. doi: 10.1136/archdischild-2020-321008 ; PubMed Central PMCID: PMC8380881. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Markham JL, Richardson T, DePorre A, Teufel RJ 2nd, Hersh AL, Fleegler EW, et al. Inpatient Use and Outcomes at Children’s Hospitals During the Early COVID-19 Pandemic. Pediatrics. 2021;147(6). Epub 20210323. doi: 10.1542/peds.2020-044735 . [DOI] [PubMed] [Google Scholar]
  • 37.Chua KP, Volerman A, Conti RM. Prescription Drug Dispensing to US Children During the COVID-19 Pandemic. Pediatrics. 2021;148(2). Epub 20210720. doi: 10.1542/peds.2021-049972 ; PubMed Central PMCID: PMC8344340. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Moynihan R, Johansson M, Maybee A, Lang E, Legare F. Covid-19: an opportunity to reduce unnecessary healthcare. BMJ. 2020;370:m2752. Epub 20200714. doi: 10.1136/bmj.m2752 . [DOI] [PubMed] [Google Scholar]
  • 39.Newman TB, Bernzweig JA, Takayama JI, Finch SA, Wasserman RC, Pantell RH. Urine testing and urinary tract infections in febrile infants seen in office settings: the Pediatric Research in Office Settings’ Febrile Infant Study. Arch Pediatr Adolesc Med. 2002;156(1):44–54. doi: 10.1001/archpedi.156.1.44 . [DOI] [PubMed] [Google Scholar]
  • 40.Schroeder AR. UTI and Faulty Gold Standards. Pediatrics. 2017;139(3). doi: 10.1542/peds.2016-3814A . [DOI] [PubMed] [Google Scholar]
  • 41.Filippi CM, von Herrath MG. Viral trigger for type 1 diabetes: pros and cons. Diabetes. 2008;57(11):2863–71. doi: 10.2337/db07-1023 ; PubMed Central PMCID: PMC2570378. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Trnka P. Henoch-Schonlein purpura in children. J Paediatr Child Health. 2013;49(12):995–1003. Epub 20131018. doi: 10.1111/jpc.12403 . [DOI] [PubMed] [Google Scholar]
  • 43.Alder AC, Fomby TB, Woodward WA, Haley RW, Sarosi G, Livingston EH. Association of viral infection and appendicitis. Arch Surg. 2010;145(1):63–71. doi: 10.1001/archsurg.2009.250 . [DOI] [PubMed] [Google Scholar]
  • 44.Alibek K, Mussabekova A, Kakpenova A, Duisembekova A, Baiken Y, Aituov B, et al. Childhood cancers: what is a possible role of infectious agents? Infect Agent Cancer. 2013;8(1):48. Epub 20131210. doi: 10.1186/1750-9378-8-48 ; PubMed Central PMCID: PMC4029297. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Servais J, Ramage-Morin PL, Gal J, Hales CM. Prescription medication use among Canadian children and youth, 2012 to 2017. Health Rep. 2021;32(3):3–16. doi: 10.25318/82-003-x202100300001-eng ; PubMed Central PMCID: PMC8647816. [DOI] [PMC free article] [PubMed] [Google Scholar]

Decision Letter 0

Tai-Heng Chen

8 Jul 2022

PONE-D-22-14626Healthcare Utilization in Children Across the Care Continuum During the COVID-19 PandemicPLOS ONE

Dear Dr. Schroeder,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Aug 22 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Tai-Heng Chen, M.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified (1) whether consent was informed and (2) what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

If you are reporting a retrospective study of medical records or archived samples, please ensure that you have discussed whether all data were fully anonymized before you accessed them and/or whether the IRB or ethics committee waived the requirement for informed consent. If patients provided informed written consent to have data from their medical records used in research, please include this information.

3. Thank you for stating the following in your Competing Interests section:  

"NO authors have competing interests"

Please complete your Competing Interests on the online submission form to state any Competing Interests. If you have no competing interests, please state ""The authors have declared that no competing interests exist."", as detailed online in our guide for authors at http://journals.plos.org/plosone/s/submit-now 

 This information should be included in your cover letter; we will change the online submission form on your behalf.

4. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability.

Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized.

Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access.

We will update your Data Availability statement to reflect the information you provide in your cover letter.

5. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

6. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Healthcare Utilization in Children Across the Care Continuum During the COVID-19 Pandemic

Overall:

The paper focused on the essential topic of “ utilization of pediatric services during Covid”. It is week written.

Introduction:

The information included in the second paragraph does not have any reference. In the second paragraph, “Major shifts in health care use and diagnosis patterns are important to describe as we monitor child health following the pandemic” needs to be rephrased. What shift the authors are referring to is unclear. How the training can help the shift has also not been clarified. The statement “how viral transmission and healthcare utilization impact the incidence and outcomes of specific diseases” is beyond the scope of this study as the source of data is insurance claim administrative data.

Method

The major concern is why include four years of data for comparison rather than taking previous year's data (March 2019-Feb 2020) for analysis? There was no justification given. The authors focused on primary care, urgent care, and ED visits, hospitalizations, and prescription medication claim but did not provide the operational definition of those terms.

Results

It would have been useful for the policymakers to show the utilization rate change according to the income quintile, race, and gender.

Discussion

The utilization rate over four years period might create a bias toward utilization rate. The conclusion about future policy or strategy the differentiation if any, across gender, race, and income quintile would have been helpful.

Reviewer #2: This is a very well-written manuscript describing changes in healthcare utilization related to the pandemic across different sectors in a population-based sample of US children. It adds to the literature by broadening the US data to include outpatient care and prescriptions and by including emergency department and admissions to community hospitals. Given the broad nature of the outcomes, the analyses raise more questions that are relevant both for future pandemics but also to potentially extraneous patterns of health care use at baseline. The authors did an excellent job in the discussion of highlighting which findings require more investigation.

I have only minor comments to potentially strengthen the manuscript.

Methods:

• As per RECORD guidelines, a comment on whether the dataset has ever been validated would be helpful (even if it hasn’t been – there was only a comment in the limitations in general related to ICD-10 codes).

Analysis:

• it would be important for the authors to justify why the models weren’t age/sex adjusted – the fact that age and sex distribution were similar amongst enrollees across the time periods doesn’t mean that there would not have potentially been effects of either across the outcomes.

Discussion:

• The authors cite studies from other jurisdictions (UK, Canada etc) and it would be helpful to comment that the consistencies of many findings, despite different 1) health systems, 2) levels of COVID transmission at various time points and 3) restrictions (esp school closures) is also worthy of more comparative investigation.

• The inability to distinguish new vs recurrent problems (because of no ability to link across episodes) is a limitation that should be included. This is esp. true for the findings around DKA as the implications of higher rates of DKA in incident vs prevalent cases are different. For the former this is related to delays in care seeking or issues in misdiagnoses in primary care whereas for children with known diabetes, this would be related to problems with specialized care.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Astrid Guttmann

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Attachment

Submitted filename: Comments PON_ June 2022.docx

PLoS One. 2022 Oct 27;17(10):e0276461. doi: 10.1371/journal.pone.0276461.r002

Author response to Decision Letter 0


23 Aug 2022

Dear Editors, PLoS One:

We appreciate the favorable reviews for our manuscript and thank the reviewers for their thoughtful comments and suggestions, which have strengthened the manuscript. Below is a point-by-point response to each suggestion.

Reviewer #1:

Comment 1: The information included in the second paragraph does not have any reference.

Response: Multiple additional references were added following the first sentence.

Comment 2: In the second paragraph, “Major shifts in health care use and diagnosis patterns are important to describe as we monitor child health following the pandemic” needs to be rephrased. What shift the authors are referring to is unclear.

Response: The sentence has been modified and now reads as follows: “Major shifts in health care use and diagnosis patterns as we have observed are important to describe quantify as we monitor child health following the pandemic.”

Comment 3: How the training can help the shift has also not been clarified.

Response: The sentence has been modified and now reads: “Additionally, quantification of these shifts help to identify potential gaps in clinical exposure for students, residents, and fellows who underwent training during the pandemic.”

Comment 4: The statement “how viral transmission and healthcare utilization impact the incidence and outcomes of specific diseases” is beyond the scope of this study as the source of data is insurance claim administrative data.

Response: We agree that we are not analyzing outcomes in this paper. However, this paper will serve as a guide for our group (and hopefully other investigators) to know which conditions to target for these types of investigations. We have modified the sentence as follows: “Such information provides a “natural experiment” to investigate how viral transmission and healthcare utilization impact the incidence and outcomes of specific diseases, and quantifying any changes in incidence is a necessary first step.”

Comment 5: The major concern is why include four years of data for comparison rather than taking previous year's data (March 2019-Feb 2020) for analysis? There was no justification given.

Response: Thank you for this comment – we agree that clarification would help. We felt that we needed at least a few years of pre-pandemic data to account for trends that may have predated the pandemic (for example, SSRI prescriptions were rising steadily from 2016-2019) and to mitigate the impact that an unusual occurrence (e.g. a bad influenza year) would have if only one year were used. We did not go back further than 2016 given the ICD9 to ICD10 transition in late 2015. We have added the following sentence: “We elected to use 2016 as a starting point to allow sufficient time to account for secular trends in utilization that may have begun prior to the pandemic.”

Comment 6: The authors focused on primary care, urgent care, and ED visits, hospitalizations, and prescription medication claim but did not provide the operational definition of those terms.

Response: The first page of Supplementary Information describes how all of these types of utilization were defined.

Comment 7: It would have been useful for the policymakers to show the utilization rate change according to the income quintile, race, and gender.

Response: We agree with the reviewer that our analysis leaves open the possibility that different demographic groups were affected differently during the pandemic—heterogeneity of treatment effect. While a full analysis of this heterogeneity would be really interesting, we decided that it was outside the scope of our analysis, and would require multiple additional tables and figures, as well as potentially lengthy explanations for any notable findings. We are in the process of procuring a Medicaid database and do intend to compare and contrast utilization patterns for that.

Nonetheless, given the related comment from Reviewer 2 below, we did do an additional analysis adjusting for these covariates. See comments below.

We have added a sentence to limitations section after mentioning the private insurance only limitation: “Understanding how these utilization patterns differed across demographics is an important next step.”

Comment 8: The utilization rate over four years period might create a bias toward utilization rate.

Response: The utilization rate described is a count per 100 child years, and should not be biased by the number of years prior to the pandemic. As stated above, it is important to account for trends that predated the pandemic, and the ITS model is used precisely for that purpose.

Comment 9: The conclusion about future policy or strategy the differentiation if any, across gender, race, and income quintile would have been helpful.

Response: thank you for this insightful comment. Please see above responses and response to reviewer comment 2.

Reviewer #2:

We thank Dr. Guttman for her very kind comments and suggestions for improvement.

Comment 1: As per RECORD guidelines, a comment on whether the dataset has ever been validated would be helpful (even if it hasn’t been – there was only a comment in the limitations in general related to ICD-10 codes).

Response: We have added a sentence to Methods, with 3 references investigating this question: “Although deidentification of patient information makes formal validation of claims data challenging, some investigations have demonstrated encouraging reliability, though specificity of diagnostic codes is likely higher than sensitivity.22-24”

The section in Limitations reads: “Although investigations into the validity of diagnostic codes in claims data have mostly been promising,22-24 there is still potential for miscoding.”

Comment 2: it would be important for the authors to justify why the models weren’t age/sex adjusted – the fact that age and sex distribution were similar amongst enrollees across the time periods doesn’t mean that there would not have potentially been effects of either across the outcomes.

Response: A similar comment was made by Reviewer 1, see our related response above. We focused on unadjusted differences in part because presentation in this fashion can be easier to interpret. Additionally, we wanted our data to depict actual “real world” changes. When we performed the full interrupted time series analysis for trends in demographic makeup cohort (results in the supplement), we found that no demographic variable made a step change of >1%. Nonetheless, in response to this suggestion, we’ve also added a sensitivity analysis to the Supplement where we adjust for demographic shifts, and indeed the results are pretty much the same.

The Supplementary Methods now ends with: “Lastly, as a sensitivity analysis, we included demographic covariates (age, sex, and race, census region, and income quartile) in the ITS model to examine the extent to which adjustment for these variables led affected the unadjusted rates.”

The Results section now has a new paragraph reading: “In our sensitivity analysis, adjusting for the demographics portrayed in Table 1 (age, sex, race, census region, and income quartile) did not have a meaningful impact on the changes in utilization in the early or mid-pandemic periods (Supplementary Figure 6).”

The supplementary tables and figures now has an additional Supplementary Figure 6 portraying the results of this sensitivity analysis.

Comment 3: The authors cite studies from other jurisdictions (UK, Canada etc) and it would be helpful to comment that the consistencies of many findings, despite different 1) health systems, 2) levels of COVID transmission at various time points and 3) restrictions (esp school closures) is also worthy of more comparative investigation.

Response: We added the following sentence to Limitations: “Similarly, investigations comparing shifts in utilization patterns between different countries and in the context of varying states of transmission of COVID-19 (and related restrictions) may also help to inform policy.”

Comment 4: The inability to distinguish new vs recurrent problems (because of no ability to link across episodes) is a limitation that should be included. This is esp. true for the findings around DKA as the implications of higher rates of DKA in incident vs prevalent cases are different. For the former this is related to delays in care seeking or issues in misdiagnoses in primary care whereas for children with known diabetes, this would be related to problems with specialized care.

Response: We added the following two sentences to Limitations: “Furthermore, we did not attempt to discriminate between new versus recurrent problems. Such discrimination is important to better understand some of the changes we observed in conditions such as diabetic ketoacidosis.”

Attachment

Submitted filename: PLoS_Response_PUPPI_7-21-22.docx

Decision Letter 1

Tai-Heng Chen

7 Oct 2022

Healthcare Utilization in Children Across the Care Continuum During the COVID-19 Pandemic

PONE-D-22-14626R1

Dear Dr. Schroeder,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Tai-Heng Chen, M.D.

Academic Editor

PLOS ONE

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: All comments are addressed. There are several limitations of this study which have been acknowledged by the authors!!

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Malabika Sarker

**********

Acceptance letter

Tai-Heng Chen

20 Oct 2022

PONE-D-22-14626R1

Healthcare Utilization in Children Across the Care Continuum During the COVID-19 Pandemic

Dear Dr. Schroeder:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Tai-Heng Chen

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 File. Additional supporting information is provided in the supplement, which includes S1 Table and S1-S6 Figs.

    (PDF)

    S2 File. Supplemental methods PUPPI resubmission clean.

    (DOCX)

    Attachment

    Submitted filename: Comments PON_ June 2022.docx

    Attachment

    Submitted filename: PLoS_Response_PUPPI_7-21-22.docx

    Data Availability Statement

    Database is proprietary (owned by Optum) and only available for purchase through the company. The Population Health Sciences Department of the Stanford University School of Medicine obtained this data for use by Stanford investigators. Because of this contractual relationship, Stanford investigators were allowed direct access to data. Investigators outside of Stanford interested in replicating this study should contact Optum directly for access. The Optum claims database website is https://www.optum.com/business/life-sciences/real-world-data/claims-data.html. Contact information can be found on the product sheet: https://cdn-aem.optum.com/content/dam/optum4/resources/pdf/clinformatics-data-mart.pdf.


    Articles from PLOS ONE are provided here courtesy of PLOS

    RESOURCES