A database of pediatric drug effects to evaluate ontogenic mechanisms from child growth and development

Abstract

Background:

Adverse drugs effects (ADEs) in children are common and may result in disability and death, necessitating post-marketing monitoring of their use. Evaluating drug safety is especially challenging in children due to the processes of growth and maturation which can alter how children respond to treatment. Current drug safety signal detection methods do not account for these dynamics.

Methods:

We recently developed a method called disproportionality generalized additive models (dGAMs) to better identify safety signals for drugs across child development stages.

Findings:

We used dGAMs on a database of 264,453 pediatric adverse event reports and found 19,438 ADEs signals associated with development and validated these signals against a small reference set of pediatric ADEs. Using our approach, we can hypothesize on the ontogenic dynamics of ADE signals, such as that montelukast-induced psychiatric disorders appears most significant in the second year of life. Additionally, we integrated pediatric enzyme expression data and found that pharmacogenes with dynamic childhood expression, like CYP2C18 and CYP27B1, are associated with pediatric ADEs.

Conclusion:

We curated KidSIDES, a database of pediatric drug safety signals, for the research community and developed the Pediatric Drug Safety portal (PDSportal) to facilitate evaluation of drug safety signals across childhood growth and development.

Funding:

This study was supported by grants from the National Institutes of Health (NIH).

eTOC

Enzyme activity and hormone levels are examples of dynamic processes during child growth and development that can significantly affect drug safety. However, pediatric drug safety signal algorithms currently ignore these dynamics. Giangreco et al. developed a data-driven approach to capture dynamic safety signals and show their database provides robust drug safety signals across childhood.

Introduction

Adverse drug events (ADEs) are responsible for up to 10% of pediatric hospitalizations¹ with up to 45% as life threatening². Knowledge of side effects in children is limited, fragmented, and often anecdotal. Randomized controlled trials (RCTs) are the gold standard for establishing the safety and efficacy of treatment. However, pediatric RCTs are challenging due to low recruitment rates, lack of established endpoints, inflated placebo effects, and ethical concerns^3–6. As a result, drugs are often used off-label – for an unapproved indication - without strong evidence and with a significant burden of adverse drug events⁷.

The processes of growth and maturation during childhood complicates the prediction, detection, and evaluation of pediatric adverse drug effects. Dynamics in activity and expression of pharmacogenes during childhood may alter pharmacokinetics and pharmacodynamics of administered drugs⁸. For example, cytochrome P450 enzymes, which metabolize 70–80% of drugs, are known to vary up to 100-fold within the first weeks of life⁹. Activating metabolizers such as monooxygenases, aldehyde dehydrogenases, and amidases exhibit dynamic changes during infancy and early childhood¹⁰. During puberty, hormonal dynamics can drastically reduce or accelerate the bioavailability of drugs¹¹ and regulate receptor availability and downstream signaling^12,13. Despite strong evidence for molecular dynamics across child development and notable drug safety examples, such as doxorubicin-induced cardiotoxicity¹⁴ and methylphenidate-induced mental disorders¹⁵, its role in pediatric drug effects remains largely a mystery. Preclinical toxicity studies do not sufficiently identify ontogenic drug safety risks nor do they significantly influence the design of trials involving children¹⁶. When pediatric trials are conducted (recently incentivized or required by regulation¹⁷), the studies often include few patients in specific age groups to sufficiently evaluate drug effects¹⁸. Real-world observational data, such as electronic health records (EHR), administrative claims data (ACD), and spontaneous reporting systems (SRS), can capture a wide range and large amount of drug event outcomes that result from clinical practice and represent an opportunity to further elucidate potential ontogenic-related drug safety risks⁸. Current approaches leveraging these resources, however, treat child development periods as independent or model all children under 18 as one homologous group, both of which limit their power and usefulness¹⁹.

The choice of which real-world data resource to use to study pediatric drug safety is driven by the goal of the research. Electronic health records and administrative claims data capture a more complete set of patient exposures and therefore would be more appropriate for risk estimation (comparing the rate of an adverse event for patients exposed to a study drug compared to the rate of the unexposed). However, these systems are not designed to accurately or systematically capture adverse drug event outcomes. For example, very few adverse drug events are coded by clinical billing terminologies, like ICD10, and the terminology used for describing adverse events, MedDRA, is not used in these systems. Further, the process of identifying good proxies for adverse events in EHRs and ACD is an open and unsolved research question. Ultimately, each adverse event outcome requires manual revaluation of patient records to establish and validate proper endpoints. As a result, drug risk estimation using these systems are limited in their focus to one or a few adverse drug events. SRS databases, like the FDA’s Adverse Event Reporting System (FAERS), and the World Health Organization’s Vigibase, on the other hand, are designed specifically to capture potential adverse events and utilize the MedDRA terminology to encode these outcomes. This makes systematic evaluation of thousands of adverse events possible. The limitation in SRS is that they only capture data from patients who experienced adverse events and therefore have no denominator upon which to estimate risk²⁰. Analysis of these systems is limited, therefore, to signal detection²¹. Drug safety signals are identified by comparing the rates of reporting across drugs. If the ADE is found to be “disproportionately” reported with the study drug, the ADE is flagged as a potential safety concern. This analysis is sensitive to unique biases of SRS databases including sampling variance, under-reporting biases, recency biases, and notoriety biases (see Table S1 for definitions) – all of which can also be time varying. Pharmacovigilance data mining methods, like the Empirical Bayes Geometric Mean (EBGM)²² and the Statistical Correction of Uncharacterized Bias (SCRUB)²³, address some of these challenges and can improve the false discovery rate of drug safety signal detection. Ultimately, since estimates of risk cannot be directly obtained from these data, follow up clinical analysis on a case-by-case basis is required to identify which signals may be causal^24–26. Recently, such analysis has led to the discovery of pediatric drug effects such as severe anxiety disorders by selective serotonin reuptake inhibitors²⁷ and life-threatening arrhythmias by short-acting beta-2 agonists²⁸. There has not yet been a systematic evaluation of potential adverse drug events associated with development stages.

We invented a new pharmacovigilance signal detection algorithm using generalized additive models (GAMs) that can identify potential safety concerns at each stage of childhood development without sacrificing much statistical power²⁹. GAMs are common in evaluating environmental and ecological characteristics over space and time, such as black smoke particulate exposures across the UK over four decades³⁰ and artificial light density on bird stopover in different habitats during autumn migration³¹. We showed GAMs allow for sharing information between development stages that reveal drug effect dynamics even when there may be scant evidence at a particular stage²⁹. We found that GAMs produced more stable safety signals and are more precise in low data situations than traditional drug safety methods²⁹. Moreover, GAMs are computationally efficient enough to apply in a high throughput manner to identify pediatric drug safety signals and evaluate the biology that may be associated.

To systematically study pediatric drug safety, we mined a quarter of a million pediatric adverse event reports from the Food and Drug Administration’s Adverse Event Reporting System (FAERS) for pediatric drug safety signals (Figure 1). We applied disproportionality GAMs (dGAMs) to all the observed pediatric ADEs and generated a database of drug safety signals across the stages of child development. Our dGAMs mitigated reporting bias through covariate adjustment and increased the detection of rare adverse events by more than two-fold. We identified 19,438 significant pediatric drug safety signals, including known pediatric-specific adverse drug events and ADE profiles (time-series of signals across development stages) that are consistent with physiological growth and development. We investigate the relationship between development stage and known pediatric drug effects, such as montelukast-induced psychiatric disorders where we found significant signal (odds ratio 8.77 [2.51, 46.94]) within the second year of life. To incorporate a potential molecular explanation to our safety signals, we evaluated and found evidence that signals were associated with differences in cytochrome P450 enzymes expression. Finally, we built a database and web application for the pediatric drug safety community to further evaluate clinical and molecular hypotheses for drug safety signals in the context of child development.

Figure 1: Study overview.

Adverse drug events (ADEs) occur throughout childhood. The dynamic biological processes of growth and maturation, however, make them difficult to detect. We mined the Food and Drug Administration Adverse Event Reporting System (FAERS) database to identify pediatric adverse drug events across child development stages. Our analysis is based on 460,837 adverse event reports for pediatric patients (up to 21 years of age) submitted by physicians, consumers, and other healthcare workers between 1994 and 2019. We present a new disproportionality analysis method that can identify dynamics of adverse drug events that depend on age (or in this case, development stage) using generalized additive models (GAMs). These disproportionality GAMs, or dGAMs, estimate the log odds for an adverse event given a pediatric development stage and can account for comedications, or other confounders, as covariates in the model. We identified 19,438 putative pediatric adverse drug events. We evaluated signal significance using 90% confidence intervals and through permutation analyses. We compared the pediatric adverse events identified by our dGAM against a reference standard of pediatric ADEs and found a 2-fold higher detection rate by sharing drug event information across child development stages. We discovered dynamic reporting patterns of ADEs using unsupervised time-series based clustering of dGAM signal estimates across development stages. We evaluated available expression data across development stages for cytochrome P450s and discovered correlations between enzyme expression and adverse drug event signal. We use these correlations to form putative hypotheses regarding potential adverse event mechanisms. We developed the Pediatric Drug Safety portal (PDSportal) as a public resource of pediatric ADE signal estimates across development stages.

Results

Pediatric FAERS adverse drug event reporting

There were 264,453 pediatric reports ranging from term neonates through late adolescents in the Pediatric FAERS dataset (Table 1). There were 460,837 unique drug-event (ADE) pairs reported over three decades. Majority of reports listed Female sex (52.9%). The most frequently reported drugs were from nervous system (35.3%), antineoplastic (26.8%), and alimentary tract and metabolic (13.5%) pharmacological classes. Each report listed 2.28 drugs, on average, and 95% listed 8 or fewer drugs. We observed 94% of ADEs had 10 or fewer reports (Figure S1A) and reporting factors such as stage, sex, reporting date, type of reporter, and class of drugs varied across childhood (Figure S1B–F).

Table 1: Adverse drug events across child development stages in Pediatric FAERS.

The number and proportion of reports with the subject or drug characteristics. The drug class of the reported drug is in descending order.

	Pediatric FAERS
Number of drug-events	460,837
Number of reports	264,438
Sex = Male (%)	124,578 (47.1)
NICHD child development stage (%)
Term Neonatal	6,185 (2.3)
Infancy	13,689 (5.2)
Toddler	10,432 (3.9)
Early childhood	22,063 (8.3)
Middle childhood	53,046 (20.1)
Early adolescence	104,747 (39.6)
Late adolescence	54,291 (20.5)
Reporter qualification (%)
Consumer or non-health professional	106,346 (40.2)
Lawyer	1,449 (0.5)
Other health professional	64,815 (24.5)
Pharmacist	14,189 (5.4)
Physician	77,639 (29.4)
Polypharmacy (mean (SD))	2.28 (2.24)
Anatomical/Pharmacological drug class
NERVOUS SYSTEM	93,407 (35.3)
ANTINEOPLASTIC AND IMMUNOMODULATING AGENTS	70,904 (26.8)
ALIMENTARY TRACT AND METABOLISM	35,698 (13.5)
DERMATOLOGICALS	33,011 (12.5)
SYSTEMIC HORMONAL PREPARATIONS, EXCL. SEX HORMONES AND INSULINS	31,148 (11.8)
RESPIRATORY SYSTEM	29,976 (11.3)
ANTIINFECTIVES FOR SYSTEMIC USE	29,791 (11.3)
GENITO URINARY SYSTEM AND SEX HORMONES	21,985 (8.3)
CARDIOVASCULAR SYSTEM	17,981 (6.8)
MUSCULO-SKELETAL SYSTEM	16,184 (6.1)
BLOOD AND BLOOD FORMING ORGANS	11,203 (4.2)
ANTIPARASITIC PRODUCTS, INSECTICIDES AND REPELLENTS	3,074 (1.2)
SENSORY ORGANS	2,538 (1.0)
VARIOUS	1,889 (0.7)

dGAMs mitigate confounding and reporting bias

Confounding and reporting biases are a known issue in observational analyses. To evaluate our models with respect to confounding, we compared the number of spurious associations (see Methods) between the base model (i.e. no covariate adjustment) and a model that adjusted for reported sex, type of reporter, date of reporting, and the number of co-medications by pharmacological drug class (Figure S2). We found that adjusting for confounding factors resulted in a 2-fold decrease in spurious drug safety signals (linear regression intercept, unadjusted dGAM: 3.1 [3.0, 3.2]; adjusted dGAM: 1.17 [1.1, 1.24]) overall and corresponding with stage-specific drug usage (linear regression slope, unadjusted dGAM: 2.36 [1.81, 2.9]; adjusted dGAM: 0.72 [0.33, 1.11]) (Figure S3).

dGAMs identify drug safety signals across child development

We evaluated 460,837 drug safety signals across childhood for 10,770 and 1,088 unique adverse events and drug exposures, respectively. In comparison to the commonly used proportional reporting ratio (PRR), our dGAMs identified signals by sharing information across all stages (Figure 2A–B). We then summarized the dGAM safety signal distribution across child development stages (Figure 2C). The model identified 152,919 or 33.2% drug-events with at least one nominally significant signal (GAM 90% lower bound beta coefficient>0) across child development stages (Figure 2D). To reduce false positives, we further defined ADE significance by comparing signals to a null signal from random drug and event associations (see Methods). This narrowed our findings to 19,438 or 4.2% of drug-events passing the 99^th percentile threshold at each stage. We defined signal significance as having at least one signal passed nominal and null model significance thresholds (Figure 2E). The significant signals were more often identified during earlier child development stages (Figure 2F).

Figure 2: dGAMs estimate dynamic drug safety signals that share information across stages to identify pediatric adverse drug events.

A) The spearman correlation between drug-event signals at child development stages from a random set of 2,000 drug-events, compared between our dGAM and the popular proportional reporting ratio. B) The normalized scores across child development stages (normalized between [0,1] for each drug-event) from a random set of 2,000 drug-events, compared between our dGAM and the popular proportional reporting ratio. C) The percent of nominally significant drug-event signals (90% lower bound above 0 or the null association) across child development stages for the dGAMs. Error bars represent the 95% confidence interval for percentages calculated across 100 bootstraps of drug-events from Pediatric FAERS. The dashed redline indicates the null association between an adverse event and drug exposure across child development stages. D) The percent of nominally significant drug-event signals (90% lower bound above 0 or the null association) across child development stages. E) The percent of significant signals by the null model, out of all nominally significant signals, across child development stages. F) The percent of significant signals by the null model out of all drug-events in Pediatric FAERS.

dGAMs identifies signals for known ADEs in a pediatric reference standard

We evaluated dGAM safety signals of known drug-events and compared them to unassociated drug-events as negative controls. We examined nominal signals across childhood for 187 drug-events (known N=75, negative controls N=112), which were specific to the pediatric population, curated by the Global Research in Pediatrics (GRiP) consortium. Our dGAM achieved a precision of 0.56 and a recall of 0.31 corresponding to a 1.9-fold higher rate (odds ratio 90% CI [1.08, 3.38]) for detecting known ADEs at child development stages. In comparison, the PRR achieved a precision of 0.39 and a recall of 0.95 corresponding to a 0.96-fold rate (odds ratio 90% CI [0.68, 1.37]) to detect known ADEs. The known drug-events in our reference did not contain information on which development stage is affected. Using our safety signals, we assigned a development stage to each of the known ADEs and found that 22 (29.3%) drug-events had significant differences across childhood development stages (Figure 3).

Figure 3: Known pediatric drug effects show dynamic signal across child development stages.

Drug-event signal across child development stages for drug-events in the GRiP drug-event reference set with either epidemiological or mechanistic evidence in children.

Pediatric drug safety signal profiles vary by drug effect class

We found differences in the safety signals generated between childhood development stages when ADEs were grouped by drug class and adverse event type (Figure 4A). We found that 9 of 14 high-level pharmacological drug classes (ATC 1^st classes, Figure S4) had higher than expected reporting (odds ratio>1 and adjusted p-value<0.05) at least one development stage. Similarly, we found 20 of 27 adverse effect system organ classes (MedDRA SOCs, Figure S5) were significantly associated with at least one stage.

Figure 4: Drug effect class associations and signal profiles of 32 developmentally-sensitive medications.

A) The number of significant associations of MedDRA adverse event classes conditioned on ATC drug class exposures in child development stages. B) The 95% lower-bounded odds for enriched adverse event classes (HLT, HLGT, and SOC) by medications (ATC 5^th class) with at least one significant enrichment across child development stages. The presence of the line indicates putative ADEs were observed at that stage, and line absence indicates non were observed. The red dashed line indicates the null enrichment threshold. Abbreviations: ATC: Anatomical Therapeutic Class; MedDRA: Medical Dictionary of Regulatory Activities; ATC1-5: ATC 1^st – 5^th level; SOC: System Organ Class, HLGT: Higher-Level Group Term, HLT: Higher-Level Term.

In total, 393 out of the 1,517 drug classes were significantly associated with at least one stage and majority (N=169) were associated with both early and late adolescent stages, such as blood glucose lowering drugs excluding insulins (Table S2).

We identified 302 out of 8,674 adverse event classes associated with at least one stage. Adverse event classes were twice as likely (109 or 36%) to be reported within the first 28 days in neonates than at any other stage of development (Table S3).

We identified 306 out of 212,917 significant associations of adverse effect classes conditioned on drug class exposure across child development stages. We discovered 32 “development-sensitive” medications, such as montelukast signal for psychiatric disorders, exhibiting dynamic in safety signals across childhood (Figure 4B and Table S4). Moreover, as detailed in the following section the 32 drug safety signals had varying patterns (increase: 31.4%, plateau: 31.0%, decrease: 36.5%, inverse plateau: 0.01%) from one to multiple dynamics dependent on effect type (Figure S6).

Dynamics of pediatric safety signals cluster into a small number of patterns

We clustered dGAM estimates to evaluate safety signals for drug-events in Pediatric FAERS (Figure 5). After tuning hyperparameters to maximize, sensitivity, precision, and parsimony (Figure 5A–C and see Methods), we found that our pediatric drug safety signals clustered into four groups: increasing (N=300,697; signal increases with age, highest in young adults), decreasing (N=137,008; signal decreases with age, highest in neonates), plateau (N=20,127; signal peaks at a particular stage of development), and inverse plateau (N=3,005; signal troughs at a particular stage of development) (Figure 5D). While majority (65%) of drug-events were assigned to the increase dynamic cluster, only 5,397 or 2% were significant by the null model. In contrast, 5,885 or 29% of putative ADEs were assigned to the plateau dynamic cluster (Figure 5E). Furthermore, we found plateau-assigned ADE signal profiles resembled a normal distribution peaking during early childhood with fewer signal at younger and older stages (Figure 5F). For example, we discovered montelukast signal profiles predominantly followed the plateau dynamic for psychiatric adverse effects across child development stages (Figure 5G).

Figure 5: Clustering of ADE profiles (time-series of signals) categorizes drug signals into dynamic patterns across child development.

A) Clustering metrics dynamics localization versus cluster purity scores across clustering models for (Number of clusters, centroid, distance) triplet hyperparameter sets. B) Cluster metric scores and their 95% confidence interval versus the number of clusters to fit in the clustering model. C) Cluster metric scores and their 95% CI versus the drug-event distance used to fit in the clustering model. See Methods for details on the clustering strategy and metrics. D) Cluster assignments assigned to putative or significant ADE signal dynamics categories after fitting top cluster model with all drug-events in pediatric FAERS. The dGAM coefficients were normalized between [0,1] producing scores across child development stages for each drug-event. E) Percent of drug-events assigned from the top cluster model that were significant by the null model. F) The number of putative ADEs assigned the plateau dynamic within each child development stage. G) Montelukast-psychiatric disorder drug-events assigned signal dynamics clusters.

Cytochrome P450 expression dynamics were predictive of drug safety signals across child development stages

We hypothesized the expression of CYP enzymes across child development stages was associated with the observed dynamics of drug safety signals (Figure 6A). Out of 23 cytochrome P450 enzymes, we observed over-representation (FDR<0.05 and odds ratio>1) in the pediatric safety signals for drugs metabolized by CYP2A6 (Fisher exact test odds 1.30 [1.11, 1.53]; p-value=1.19E-03) and CYP2D6 (odds 1.15 [1.04, 1.26]; p-value=4.08E-03) during the neonatal stage, CYP26A1 (odds 6.22 [2.36, 19.16]; p-value=2.51E-05) during the infancy stage, and CYP3A43 (odds 2.98 [1.52, 6.0]; p-value=6.55E-04) during early childhood (Figure 6B). We then curated a dataset of gene expression across childhood (Table S5), previously identified for age-associated genes by Stevens et al. (Figure S7 and see Methods). We identified 22 CYP genes comprising 50 probes, with concordant probe expression within genes, that showed dynamics across child development stages.

Figure 6: Drug signals from real-world observations associate with gene expression dynamics across childhood.

A) Putative metabolic models of development-dependent signal. Adverse drug reactions (ADRs) can be influenced by dynamic metabolic processes during growth and development. In Model A, the safety-concern of an ADR stems from increased concentration of the drug. The decreased metabolism, such as by the enzyme in green, increases the bioavailability of the drug resulting in the observed ADR. In this case, drug signals are inversely proportional to expression dynamics of this enzyme across child development stages. In Model B, the safety-concern of an ADR stems from the aberrant modification of the drug such as by the enzyme in green. The concentration of the metabolite results in an observed ADR. Drug signals are directly proportional to expression dynamics of this enzyme across child development stages. B) Enrichment of significant drug-events at child development stages out of all drug-events with cytochrome P450-metabolized drugs. The red dashed line indicates the null enrichment threshold, and the error bars are the 95% confidence intervals of the odds ratio. Shown are CYP enzymes where at least one stage was significantly enriched for CYP-metabolized drug signals across child development stages. C) Volcano plots of the average mutual information (MI) for drug systemic disorders by CYP substrates versus the −log10 t-test false discovery rate (FDR) between substrate and non-substrate mutual information. A dashed red line for FDR=0.05 is shown. D) The average drug substrate signal (dGAM score) for systemic disorders for each CYP enzyme sharing significant information (t-test FDR<0.01) within systemic disorder categories. E) The residual probe expression across child development stages for the CYP enzymes in D).

We evaluated the correlation between drug safety signals and CYP enzyme expression across child development stages. Specifically, we compared the mutual information between CYP gene expression and safety signals for CYP substrates compared to the signals for non-substrates across child development stages. We restricted this analysis to signals for drug side effects that are known and already present on drug product labels. Out of 780 significant drug-events present on drug product labels, there were 429 drug-events where the drugs were substrates of the 22 CYP enzymes. Overall, we found enzyme expression was significantly associated with (t-test FDR<0.05) ADE signals across multiple system organ classes (Figure 6C). Specifically, we found dynamic expression of 16 CYP enzymes with 196 systemic drug safety signals across child development stages (Figure 6D,E). For example, the dynamic expression of CYP2C18 (t-test FDR<5.57E-23) associated with 21 drug signals, such cyclophosphamide and infertility (Figure S8). Moreover, dynamic expression of CYP27B1 (FDR<1.69E-31) associated with signals by ergocalciferol, including Hypervitaminosis D, Hypercalciuria, and Polyuria adverse events across child development stages.

Discussion

Study overview

We systematically evaluated 460,837 pediatric drug safety signals across child development stages. Unlike signal detection measures that use stratification to quantify signal within age groups, our approach using generalized additive models (GAMs) shares information across age groups to mitigate power loss. We assessed the impact of different factor combinations and constructed dGAMs that produced robust safety signals. For the first time, we have systematically identified pediatric drug safety signal dynamics within and across all stages of child development.

dGAMs reveal temporal dynamics of known pediatric drug effects

We produced safety signals for known pediatric drug effects for those events curated by the Global Research in Pediatrics (GRiP) consortium^32,33. Notably, these ADEs are not annotated with stage information but a simple binary (yes/no) indicator for children overall. Using dGAMs we can provide a developmental context to the safety concerns for these pediatric drug-events that was unspecified previously. For known culprits such as montelukast, our approach identified significant signals during mid-childhood (Figure 3) which is consistent with studies from the Swedish ADR database^34,35 and the World Health Organization’s Vigibase³⁶. In addition, our unsupervised clustering approach further evaluates the dynamics of these and thousands of drug safety signals that were previously unknown. We found cluster categories contained ADEs with a visually distinct signal profile and corresponded to clinical and developmental trends across development stages. For example, endocrine disorders became significant drug effect concerns (Fisher exact test odds ratio range 1.36 – 1.58) during the stages of early childhood through early adolescence which parallels the biological processes of puberty (Figure S3). Moreover, we found safety signals for antineoplastic agents in each development stage after the first month of life (odds ratio range 1.12 – 1.23) (Figure S5). Importantly, our systematic approach generates signals for classes of medications resulting in seemingly distinct but temporally-related side effects.

dGAMs facilitate evaluating ontogenic-mediated pediatric drug effects

Our database enables for the first time the ability to investigate pediatric, ontogenic biology from observational data. For example, the cytochrome P450 enzymes, which metabolize about 70–80% of marketed drugs, exhibit dynamic changes in activity across child development that result in altered drug actions and effects^37,38. Notably, these enzymes show characteristic activity patterns where it is generally thought CYP enzymes surge in activity during the first few years of life and then gradually decline to mature levels³⁹. From our approach, we showed drug safety signals within stages throughout childhood strongly associated with their CYP metabolizing enzymes. We curated a gene expression dataset to evaluate a biological underpinning or shared information between our safety signals and the expression of drug enzymes across child development stages. We found evidence for ergocalciferol, also known as vitamin D2, resulting in signals for Polyuria, Hypercalciuria, and Hypervitaminosis D from CYP27B1 metabolism, where ergocalciferol treatment was previously shown to result in abnormally high calcium levels in younger pediatric patients⁴⁰. We also identified evidence of drug signals associated with CYP2C18 expression, where the ontogeny of this gene isoform is not known but drug binding was found to be different compared to other CYP2C gene family isoforms⁴¹. There were four drug-event substrates different from other CYP2C family substrates, comprising the three drugs cyclophosphamide, ifosfamide, and omeprazole. The alkylating agents ifosfamide and its parent compound cyclophosphamide are known to share a toxicity profile of myelosuppression and urotoxicity⁴², but our data suggest and corroborate the long term effect of severe cellular damage and possible infertility in pediatric cancer-survivors⁴³. Moreover, our observational analysis corroborates omeprazole safety concern for the benign adverse effect of enlarged breast tissue in neonates that stems from increased oestrogen production⁴⁴ (Figure S8).

dGAMs enable data-driven control of confounding effects

An advantage of dGAMs includes the ability to include confounders within the signal generation. As the dGAM is a composition of regression models, confounders can be included as covariates. Our results suggest that modeling the covariates this way mitigates the confounders’ effects (Figure S3). In contrast, the PRR and other disproportionality techniques cannot natively control for confounding but require a preliminary step such as propensity score matching.

Pediatric drug effect database

We created KidSIDES a first-of-its-kind database of half a million pediatric drug safety signals across growth and development stages. We have shown how a massive data mining effort captures known pediatric drug effects and reproduces system-wide development patterns. We included standardized drug and adverse event vocabulary information as well as derived clustering and stage enrichment data to further investigate both clinical and molecular pediatric ADE hypotheses. Importantly, we include report-level data and covariates used in all dGAMs to provide the data-context of our signal generation. To increase accessibility of our resource, we developed the Pediatric Drug Safety portal (PDSportal) for the research and drug safety community to view population-level evidence of medications associated with safety signals throughout childhood. Our web application allows for downloading our entire database and to generate targeted mechanistic hypotheses such as metabolic associations on observed drug signal dynamics as shown here for substrates of CYP enzymes. We include additional data for evaluating shared information between a drug’s safety signals and their targets, carriers, transporters, and enzymes.

Limitations of Study

The observations of drug reporting over time in FAERS contains biases and confounding factors that may impact our signal detection and, due to the lack of systemic reference standard for pediatric effects, it is challenging to estimate the false discovery rate of our approach accurately. While we mitigate potential biases by including confounding factors and show dGAMs reduce spurious associations, we acknowledge that our results may still be subject to these biases and many false signals will be generated. For example, we accounted for drug administration route but were unable to account for drug dosage and frequency, which are known factors in drug toxicity, due to lack of available data. We did show, however, reduced bias and increased signal when accounting for known adverse drug event safety factors.

In addition, our method relies on age ranges defined by NICHD to standardize consistent age groups for randomized clinical trials. Other stage definitions or modeling strategies could reveal different ADE hypotheses.

Our approach generated nonzero signals at stages for drug-events with no reporting. This is both an advantage of our method – that we can share information from adjacent stages to boost power when primary evidence is lacking – and a weakness in that we can produce estimates when they are physiologically unlikely. For example, our method estimates a psychological safety signal for neonatal hallucinations which would be difficult or nonsensical to validate. We acknowledge the signals that we identify require further experiments – either review of the original clinical trials, gathering of more observational data, or prospective laboratory experiments – to establish a potential causal link.

Conclusion

Children may be prescribed medications at any point during childhood, and we provide the first resource to identify and evaluate drug safety signals across child development stages. We generated signals for half a million adverse drug events by sharing information across child development stages. The generated signals differentiated known pediatric drug effects from a set of unknown drug and event associations. These signals spanned across child development stages to allow for examining differential effects across childhood, such as by systemic disorders and drug classes. Moreover, our real-world evidence was shown to contain dynamic information on putative biological mechanisms that supports evaluating adverse drug effect hypotheses. We provide the PDSportal as an accessible web application as well as a bulk download of our KidSIDES database for the community to explore from identifying safety endpoints in clinical trials to evaluating known and novel developmental pharmacology.

STAR Methods

RESOURCE AVAILABILITY

Lead Contact

Further information and requests for resources such as data or code should be directed to and will be fulfilled by the lead contact, Nicholas Tatonetti (npt2105@cumc.columbia.edu).

Materials and availability

This study did not generate new unique reagents.

Data and Code Availability

• This paper analyzes existing, publicly available data downloaded from the openFDA platform. Links are listed in the key resources table. The stable web link generated for the web application PDSportal is listed in the key resources table.

• All original code has been deposited on Github and at Zenodo and is publically available as of the date of publication. URLs and DOIs are listed in the key resources table.

• Any additional information to reanalyze the data reported in this paper is available from the lead contact upon request.

EXPERIMENTAL MODEL AND SUBJECT DETAILS

We created “Pediatric FAERS” as a subset of the existing Food and Drug Administration’s Adverse Event Reporting System (FAERS) database limited to those patients aged 21 or younger. We retrieved drug event reports from the Food and Drug Administration’s openFDA⁴⁵ download page, utilizing an API key with extended permissions, containing the FAERS data. The data is comprised of safety reports listing at least one drug and at least one adverse event. Using custom python notebooks and scripts available in the ‘openFDA_drug_event-parsing’ github repository (DOI: 10.5281/zenodo.4464544), we extracted and formatted all drug event reports prior to the third quarter of 2019. Data fields included the safety report identifier, age value, age code e.g. year, adverse event the Medical Dictionary of Regulatory Activities concept code (preferred terms), and drug RxNorm code (various) used in our analyses. Race, ethnicity, as well as other socioeconomic factors were not available for this analysis. The age value was standardized to year units for categorizing reports into the 7 child development stages according to the Eunice Kennedy Shriver National Institute of Child and Human Development⁴⁶. Adverse drug event MedDRA codes were mapped to standard concept identifiers using concept tables⁴⁷ from the OMOP common data model. The drug RxNorm code was similarly translated to the standard RxNorm concept identifier (ingredient level) in OMOP and was further mapped to the equivalent Anatomical Therapeutic Chemical (ATC) Classification concept identifier (ATC 5^th level) using the concept relationship table. The occurrence of an adverse drug event is defined as any safety report where both the adverse event and drug concepts are reported together. The pediatric report space for any adverse drug event is all reports which have age above zero and less than or equal to 21 years old which is the upper bound for the late adolescence child development stage. All pediatric reports reported either Female or Male sex, contained the type of reporter (Physician, Consumer, lawyer, or other health professional), and the date of the report. Additionally, we joined the higher-level ATC class for drugs from the drugbank database⁴⁸, with code to generate the database on GitHub (DOI: 10.5281/zenodo.4464604).

METHOD DETAILS

ADE detection models

We compared two different models for detecting adverse drug events from spontaneous reports. First, we applied the logistic generalized additive model⁴⁹ (GAM) to all unique drug-event pairs in Pediatric FAERS. Each drug-event GAM was used to quantify adverse event signal due to drug exposure versus no exposure across child development stages. We refer to the ‘base’ GAM formula as:

$$g(E(\mathit{\text{Event}}))=s(\mathit{\text{nichd}})*\mathit{\text{Drug}}$$

where g is a logit link function, E(Event) is the expected value of event reporting, s is a spline function with a penalized cubic basis, nicℎd is the child development stage of the report’s subject, and Drug is an indicator i.e. 0 or 1 of drug reporting. Details for GAMs can be found at references^50,51 and we specified the model using the mgcv package in R.

Briefly, the GAM is a flexible statistical model that captures nonlinear effects of covariates onto a response. In this paper, we model the effect by the child development stage interacting with drug reporting on the reporting of an event where the event is the reporting of the MedDRA preferred term and the drug is the reporting of the ATC 5^th level drug concept. The s() function is a spline function where the interaction of the child development stage (main effect) and the drug (interaction using the ‘by’ variable) is modeled according to a set of basis functions. Each development stage defines the knot (7 in total) in which the expectation of event reporting is quantified. In the spline function, a penalized cubic spline basis (bs=‘cs’) is used for fitting the basis functions where the first and second derivative of the event expectation is zero at each knot, resulting in a smooth event expectation across stages. To mitigate overfitting or ‘wiggliness’, we used a penalized iterative restricted likelihood approach, called ‘fREML’, with a wiggliness penalty in the objective function. Fitting the GAM model (using the ‘bam’ function and discrete=T) produces coefficient terms, similar to beta coefficients in logistic regression, for each child development stage for the association of the adverse event being reported in interaction with reporting the drug. We generated GAM scores for each child development stage resulting in 7 scores for each drug-event pair. It is important to note that all GAM scores produced were finite, nonzero values.

In addition, we made a comparison to the Proportional Reporting Ratio (PRR):

$$\frac{\frac{a}{a+b}}{\frac{b}{b+d}}$$

where ‘a’ is the number of reports with the drug and event, ‘b’ is the number of reports without the drug and with the event, ‘c’ is the number of reports with the drug and without the event, and ‘d’ is the number of reports without the drug or event of interest. The resulting score is the event reporting prevalence with the drug compared to without the drug. We generated PRR scores for each child development stage resulting in 7 scores for each drug-event pair.

We determined the lower confidence bound in which the population-based score would be greater than 90% of score replicates. A signal for a drug-event at a stage was nominally or statistically significant if the score had a 90% lower bound above the null association (null association: GAM==0, PRR==1). The drug-events from Pediatric FAERS were nominally significant if at least one signal, the coefficient’s 90% lower bound, was above the null association. The GAM coefficients and PRR scores were normalized between [0,1] producing scores across childhood for each drug-event to generate normalized scores.

We derived a null GAM to evaluate signal significance for drug-events compared to random drug and event reporting. We randomized drugs and events for reports, maintaining the report characteristics, and then recalculated the drug-event GAMs for 10,000 randomly selected drug and event pairs. The null GAM coefficients for each stage resulted in a null distribution of signals for randomly-associated drugs and events. A signal was significant by the null model if the score had a 90% lower bound above the 99% percentile of the null GAM coefficient distribution at a stage. The drug-events from Pediatric FAERS were significant drug safety signals by the null model if at least one dGAM score, the coefficient’s 90% lower bound, was above the 99% of the null distribution for that stage. This ensured that at least one dGAM score at a child development stage was nominally significant as well.

Unless otherwise specified, all statistics in brackets are the lower and upper 95% confidence intervals. Throughout the manuscript, we refer to this disproportionality GAM implementation as the dGAM.

dGAM development and evaluation

We evaluated different model types or dGAMs including different combinations of covariates: the smooth interaction effect between the report’s subject being in a child development stage and reporting female or male sex (‘Sex’), the date of first reporting the drug-event (‘Date’), the type of reporter for the drug-event (‘Reporter’), the number of drugs in ATC level 1 pharmacological drug classes for the report’s subject (‘ATC’), the number of drugs in ATC level 2 therapeutic drug classes for the report’s subject (‘ATC3’), the exposure of a drug within ATC level 1 pharmacological drug classes for the report’s subject (‘ATCbin’), the exposure of a drug within ATC level 2 therapeutic drug classes for the report’s subject (‘ATC3bin’), and a smooth effect for the number of drugs taken by the report’s subject (‘NdrugsS’).

We evaluated the dGAM generalizability and likelihood i.e. fit of a drug-event GAM, including each model type, on a random sample of 2,000 drug-events from Pediatric FAERS. We quantified model statistics using a proportion of the 2,000 drug-events that were reported 50 or more times and an additional, complementary set of drug-events with less than 50 reports. We quantified the fit of the dGAMs using the Akaike’s Information Criterion (AIC), which is a measure of the tradeoff between model likelihood and complexity. We quantified the generalization of the dGAMs by fitting each model on 80% of the dataset, termed the training set, and quantifying the area under the receiver operating characteristic (AUROC) curve on both the training set and the unseen testing set. The training and testing sets were balanced in having the same proportion of reports with (20%) and without (80%) the adverse event. Each model type was fit on the same training set.

We selected the model, which accounted for sex, report date, reporter, and the number of drugs within pharmacological classes, that produced the most improved model likelihood, generalization performance, and drug-event probability in a more modest time frame (Figure S2). We assessed the relationship of log odds scores and reporting of drug-events, only at the stage of highest reporting, between the base model and after covariate adjustment.

Drug-event reference sets

GRiP pediatric reference set

We extracted the drug-event pairs observed in Pediatric FAERS listed within the pediatric drug-event reference standard from the Global Research in Pediatrics consortium³². A machine-readable dataset can be found at the ‘GRiP_pediatric_ADE-reference_set’ github repository (DOI: 10.5281/zenodo.4453379). We assigned drug-event pairs with epidemiological or mechanistic evidence in children (Control==‘C’ and Control==‘B’) as the positive class (N=179 and 75 in Pediatric FAERS), and the cross-product of all drugs and events that were complementary to drug-event pairs in the reference set as the negative class (N=397 and 112 in Pediatric FAERS). In total, we evaluated 187 positive and negative drug-events observed in Pediatric FAERS. To generate predicted scores for evaluating detection performance, we averaged dGAM scores across child development stages for each drug-event that had at least one nominally significant score.

Pediatric adverse events

We downloaded and joined standard concepts to the pediatric adverse event term list from the MedDRA website (https://www.meddra.org/paediatric-and-gender-adverse-event-term-lists). This term list is no longer supported but we provide a machine-readable version in our resource.

ADE clustering

We identified clusters of signal patterns across development stages for all drug-events in Pediatric FAERS. We considered ADE signal patterns as time series, representing temporal drug-event signal across child development stages. The temporal signals were normalized dGAM scores between 0 and 1. We used the R package dtwclust⁵² to compare different distance and centroid methods due to ease of implementation and optimization of computationally expensive methods. We performed an iterative procedure to evaluate the clustering by different combinations of distance and centroid methods.

We used a partitional clustering strategy, which minimizes the intra-cluster distance while maximizing the inter-cluster distance by iterative greedy descent to converge to a local optima⁵³. The distance methods evaluated were dynamic time warping (DTW), which is a fast implementation to find the optimum warping path between two drug-events (‘dtw_basic’); the shape-based distance (‘sbd’), which is a shift and scale-invariant comparison of time series based on the k-Shape algorithm⁵⁴; and the triangular global alignment kernel (‘gak’) which is a kernel method unlike DTW that has been shown faster and more efficient in classification tasks⁵⁵. The centroid methods that evaluated cluster assignment for drug-events were the average signals between drug-events across childhood (‘mean’); the partition-around-medoids (‘pam’), which utilizes one of the series as the cluster centroid; DTW barycenter averaging (‘dba’), which finds the optimum average drug-event between drug-event series in DTW space; and shape averaging (‘shape’) based on the k-Shape algorithm, which extracts the most representative drug-event dynamic to utilize as the centroid⁵⁴.

We fit the clustering model, with hyperparameter sets that included a distance metric, centroid method, and number of clusters K. We randomly sampled with replacement the 10,000 drug-events and applied the clustering algorithm with each unique triplet set of hyperparameters (N=216). Additionally, we spiked-in ‘canonical’ dynamics patterns at each bootstrap, asserting that reporting dynamics during childhood reflect ontogenic profiles observed on molecular, functional, and structural levels^10,56,57. The canonical drug-events previously studied, including filtering for the ranked pattern of interest, were categorized as ‘increase’ (N=237), ‘decrease’ (N=224), or ‘plateau’ dynamics (N=106)²⁹. We quantified clustering performance for each hyperparameter set using the drug-events in each canonical dynamics’ category and their cluster assignment (see Figure S7 for details and illustrations of the strategy). Overall, we developed two custom metrics: 1) Cluster purity, which is the clustering precision or the score for drug-events from a canonical dynamics category assigned to a cluster, and 2) Dynamics localization, which is the clustering sensitivity or the score for drug-events within a cluster from a particular canonical dynamics’ category. This ensured the clustering performance, both the cluster purity and dynamics localization, for each hyperparameter set scores the homogeniety in both the assignment of dynamics and type of dynamics within clusters. We only considered the predominant or most frequent cluster of each of the three dynamics to compute the above metrics, allowing for comparing performance for K>3 (see Figure S9 for details and illustrations of the strategy). We computed the cluster purity and dynamics localization score for each bootstrap and averaged the scores per metric.

ADE stage enrichment

We evaluated the enrichment of (also referred to as association) putative ADEs within child development stages and categories of drug-events. We calculated the fisher’s exact test to evaluate enrichment of drug-events within a specific category to also have a significant signal, by the null model, at a specific stage.

Pediatric gene expression dataset

Dataset processing

We extracted expression data from GEO and EBI microarray datasets utilized by Stevens et al.¹² to derive gene expression across child development stages (Table S5). We compiled datasets’ raw image (CEL) files (affymetrix images only) and integrated annotation to the microarray probe sets. We used the Bioconductor R package affy to load the microarray data. Samples were preprocessed together per the same assay (hgu133a, hgu133b, and hgu133plus2) closely aligning to the procedure in Stevens et al. Specifically, we used Robust Microchip Average (RMA) background correction (‘rma’), quantile normalization (‘quantile’), perfect match correction algorithm (‘pmonly’), and mean probe set summarization (‘avgdiff’). We used the R package ROMOPOmics (DOI: 10.5281/zenodo.4463257) to extract phenotypic data from each GEO dataset (the ‘TABM666’ dataset was downloaded from the EBI website) and convert the age of each sample from the datasets to year units. We defined the NICHD child development stages using the age of the samples within the stage age boundaries⁴⁶. We mapped probe set IDs to uniprot IDs and to gene symbols using the libraries of each assay’s annotation R package within Bioconductor. We evaluated the average difference in log2 probe expression values, using 10,000 samples with replacement, by Students t-test between each adjacent child development stage.

Dataset validation

We performed a validation analysis to evaluate the processed gene expression data to reproduce the significant age-associated findings published by Stevens et al¹². Specifically, there were 690 genes comprising 927 probes with expression that was significantly associated to age. We then performed an association analysis to evaluate significant stage-associated genes from our gene expression dataset (we only utilized the datasets from their main analysis dataset: GSE11504, GSE9006, and TABM666). We quantified stage-association of each probe’s expression using a GLM where the probe value was the dependent variable and the NICHD stage, where each category was an integer, and the first six principal components and GEO series indicator minus the intercept term were predictors or covariates. We computed the odds ratio using the hypergeometric test to compare overlap of age-associated and stage-associated genes, compared to those that were not, at different alpha significance thresholds (we required at least one probe to be significantly stage-associated per gene). We found that the enrichment of genes in our data to be robust at varying significance levels (Figure S6C). The robust enrichment provided evidence that our gene expression dataset was capturing accurate dynamic expression patterns across childhood.

Pediatric cytochrome P450 gene expression dynamics evaluation

We evaluated whether drug-event signal dynamics were dependent on expression dynamics of the drug’s substrate using our pediatric gene expression dataset. Again, we only utilized the datasets from their main analysis dataset: GSE11504, GSE9006, and TABM666.

First, to account for observed batch effects (Figure S6B), we performed a regression for every probe to mitigate bias of gene expression dynamics across child development stages and all datasets. We used the residual probe levels from a GLM where the dependent variable were the observed values, the observations were each sample, and the covariates were the first six principal components and the GSE series indicator. Importantly, the GLM did not include association to development stages so that the residuals capture the difference between observed probe values and the batch-predicted probe values (again, we only utilized the datasets from their main analysis dataset: GSE11504, GSE9006, and TABM666). We used these probe residual values in our downstream analysis.

We identified cytochrome P450 gene products using a regex expression ‘^CYP’ on gene symbols to extract probe-level expression data. We performed a correlation analysis between pairs of probes within non-random CYP gene products and removed probes that showed a negative correlation (Pearson r<0) in at least one pairwise comparison.

We manually scraped drugbank webpages to determine the mapping between drug enzymes and uniprot IDs. We then filtered for drugs that were annotated as substrates for CYP gene products, again using the gene symbol pattern matching to the regex expression ‘^CYP’. We only considered drug signals where side effects were listed on the drug’s label according to SIDER 4.0⁵⁸ (N=780).

We hypothesized expression of CYP enzymes across child development stages influence the adverse event signals of drugs they metabolize versus do not metabolize. In other words, drugs that were substrates for CYP enzymes generated drug-event signals with more shared information with expression dynamics than drugs that were not substrates. We generated the two distributions by computing the mutual information (MI), using the maigesPack R package in Bioconductor, between the CYP probes’ residual expression, averaged across samples, and drug-event’s scores for each child development stage, if both expression and signal were present. We generated a distribution of mutual information from (drug-event, CYP gene probe, z score) triplets. The drug-event score at each stage varied according to a randomly selected z-score from a standard normal distribution. We used the mean (mu) and the standard error (SE) of the GAM estimates to generate a new score at each child development stage:

$$\mathit{\text{risk}}\mathit{\text{score}}=z*SE+mu$$

These permutations generated substrate MIs and non-substrate MIs for each CYP enzyme. The two MI distributions were compared using Student’s t-test to evaluate a greater difference in average mutual information for substrate compared to non-substrate drug-event signals for their substrate’s gene expression. Also, we computed a Mann Whitney one-sided test to evaluate whether the substrate scores comparisons were greater in rank than non-substrate scores, on average across possible score variations for a drug-event and CYP probe residual expression. We derived an AUROC statistic by normalizing the Mann Whitney U statistic for the number of comparisons between substrate (n₀) and non-substrate (n₁) doublets:

$$\mathit{\text{AUROC}}=\frac{U}{n_0*n_1}$$

We made these comparisons across all event disorders, detailed above, as well systemic events of a system organ class by the same procedure.

QUANTIFICATION AND STATISTICAL METHODS

Briefly, we constructed and fit generalized additive models to detect drug safety signals at child development stages for each drug and adverse event (ADE) occurrence, including covariates from Table 1, within Pediatric FAERS. We employed unsupervised time series clustering to derive temporal signal relationships between ADEs (7 signals for each stage per ADE). We compared ADE temporal signal to averaged gene expression of enzymes across child development stages using mutual information. Analyses were written in the R programming language. Statistical descriptions can be found in figure legends and results. Further details can be found in the method details section of the STAR Methods. The complete code base and packages used can be found in the KRT table. We listed the URL and DOI here as well for convenience: https://github.com/ngiangre/pediatric_ade_database_study. (DOI: 10.5281/zenodo.5719309).

Supplementary Material

3

Table S1: Top drug classes sensitive at developmental stages. The top results (highest odds ratio) for putative ADEs grouped by each ATC level drug class enriched at child development stages. The columns are ordered by child development stage. Abbreviations: ATC1-5: ATC 1^st – 5^th level. Related to Figure 4.

4

Table S2: Top adverse event classes frequently reported at developmental stages. The top results (highest odds ratio) for putative ADEs grouped by each MedDRA adverse event class enriched at child development stages. The columns are ordered by child development stage. Abbreviations: PT: preferred term, HLT: higher level term, HLGT: higher level greater term, SOC: system organ class. Related to Figure 4.

5

Table S3: Medications pose safety signals for systemic disorders at child development stages. Enrichment results for the ATC 5^th level drugs enriched for MedDRA SOC-level events at a child development stage. The columns are ordered by child development stage then FDR. Related to Figure 4. Abbreviations: ATC5: ATC 5^th level; SOC: System Organ Class, HLGT: Higher-Level Group Term. Related to Figure 4.

6

Table S4: The dataset of gene expression across childhood is made up of various datasets across development stages and tissues. Related to Figure 6.

Context and Significance.

Side effects are significant safety concerns in pediatric drug treatment but are rarely captured during clinical trials, and severely underreported post-market. Moreover, variations in metabolism and physiology as children grow and develop complicate detection of drug safety signals across child development. Researchers at Columbia University addressed this problem by inventing a novel algorithm for generating drug safety signals associated with and across child development stages. They showed their methodology reduced artificial drug and adverse event relationships, increased discovery of both known and rare side effects, and found evidence of different drug safety signals through development. They made their database of drug safety signals, called KidSIDES, freely available and browsable by the PDSportal web application.

Highlights.

• Adverse drug events in children are common and can have lasting adverse effects

• Our method identified drug safety signals across all child development stages

• These signals were robust to noise and identified known pediatric adverse reactions

• We developed KidSIDES and the PDSportal to access and view our pediatric safety signals