Data mining differential clinical outcomes associated with drug regimens using adverse event reporting data

To the Editor:

The US Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS) contains millions of clinical records that detail drug exposure(s), disease indications and clinical outcomes. However, a variety of challenges limit the ability of these data to be exploited for the extraction of clinically meaningful information beyond single-agent safety signal associations. Here, we report an open access web data mining tool, AERSMine, that enables data exploration and hypothesis generation among alternatively medicated cohorts to discover patterns of differential outcomes. Cohort groups can be formed as a function of demographics, underlying disorders, drug classes and other user-driven constraints across the entire range of human diseases and approved drugs.

Understanding short-term and long-term clinical outcomes associated with drug therapies is challenging. Virtually all drugs can cause unwanted side effects, therapeutic efficacy can vary widely between individuals, and long-term outcomes of chronic regimens are highly confounded by indication-associated risks. In some cases, severe drug-associated adverse effects have become apparent only after the onset of treatment, and have led to withdrawal or restriction of drugs by the FDA. For example, troglitazone was withdrawn by the FDA in 2000 owing to increased risk of hepatotoxicity, whereas cerivastatin was withdrawn in 2001 owing to increased risk of rhabdomyolysis¹. Furthermore, our ability to identify differential patterns of drug responses and clinical outcomes among population subgroups has been limited.

To address these challenges, the FDA and the World Health Organization (WHO; Geneva) conduct pharmacovigilance and monitor safety standards of approved drugs on the market. The FDA maintains the FAERS (http://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Surveillance/AdverseDrugEffects/default.htm), which stores manually reviewed adverse event (AE) reports received by the FDA from healthcare professionals, drug manufacturers and consumers from around the world. Although the FAERS provides patient demographic details such as age, gender, clinical indications, drugs, AEs and outcomes that can be used to identify latent risks of approved therapeutics and their combinations^2–6, accurate mining of this information remains difficult.

AERSMine (https://research.cchmc.org/aers) is a tool that effectively mines the FAERS data through systematic normalization, unification and ontological aggregation of the drugs, clinical indications and AEs (Online Methods, Supplementary Tables 1 and 2 and Supplementary Figs. 1–5). This allows analysis of large clinical cohorts and comparison of differential long-term outcomes between treatment regimens. AERSMine facilitates aggregation, subcategorization and simultaneous comparison of multiple patient groups, based on explicit combinations of demographics, clinical indication(s), and exposure to different drug(s) or drug classes. For instance, AERSMine allows users to create mutually exclusive treatment cohorts (e.g., drugs a AND b, OR c, NOT x, y, z) with normalized row values of clinical events (AEs) and generate data matrices for pattern-based analyses of differential AE risks and incidence rates (Fig. 1). These high-resolution analyses can enable the detection of differential effects between varied drug classes and within specific, defined patient subgroups that would be otherwise confounded by mixing population subgroups that differ in their relative risks of specific clinical events or outcomes.

Figure 1.

Schematic overview of AERSMine, a multi-cohort data mining platform to analyze millions of reports from the FAERS and identify subgroup-specific differential responses to therapeutics. (a) AERSMine allows user-driven selection and partition across any of the FAERS data elements (i.e., drugs, clinical indications, demographics and outcomes). (b) AERSMine also facilitates ontology-based multiple cohort selection and comparative analyses of differential effects. (c) Multiple cohorts can be created, saved and simultaneously compared for cohort-specific signatures of differential effects.(d) Output data matrices can be pruned and analyzed for quantitative safety signals, relative risks and known drug-associated adverse effects. (e) The analyzable matrices can be displayed as absolute counts, normalized reports per 1,000 patients, relative risks, drug–AE and drug–drug–AE signals, and they can be exported in tab-separated files or readily visualized within AERSMine.

AERSMine analyses can be displayed and/or downloaded as matrices that show the absolute counts, normalized reports per 1,000 patients, relative risks, drug AEs and drug–drug–AE safety signals^7–9, and they can also be visualized in a variety of formats (e.g., heat map and tag cloud). Analyses can be saved (as ‘.aers’ text files) and loaded to permit reproducibility (especially when sharing analyses), continuity and portability across multiple workstations. Additionally, AERSMine uses the Fisher’s exact test for significance (to generate the uncorrected P value) and multiple modes o false-discovery rate (FDR) correction (e.g., Benjamini–Hochberg and Bonferroni) as multiple hypothesis testing correction to limit the number of false positives.

Currently available FAERS-mining resources, such as DrugCite (http://www.drugcite.com), AdverseEvents (http://adverseevents.com/index.php), FDAble (http://www.fdable.com), OpenVigil¹⁰ and AERS Spider¹¹, do not provide the capability of ontological aggregations or high-dimensional cohort-based analyses to identify patterns of differential risks associated with alternative drug classes in population subgroups (Supplementary Table 3). This unique ability to both create multiple cohorts across any dimension (drugs, clinical indications and AEs) and study their differential profiles allows AERSMine to not only generate testable hypotheses that address differential outcomes among relatively similar patients following alternative regimens (e.g., comparing rheumatoid arthritis patients on methotrexate versus anti-tumor necrosis factor (TNF) biologics), but also identify potential new drug uses or combinations that may reduce risks of developing end-stage disease outcomes.

We briefly illustrate the use of AERSMine for hypothesis generation using three examples: first, detecting lithium-associated AEs and potential drug candidates that could minimize lithium toxicities; second, investigating the role of TNF drugs and glucocorticoids in the treatment of TNF-elevated disorders, their interactions and exacerbation of pulmonary complications; and third, stratifying risks of non-steroidal anti-inflammatory drugs (NSAIDs) in hypertension, arthritis and chronic pain management. A more step-wise tutorial for executing these queries is available in the AERSMine help section (https://research.cchmc.org/aers/help).

In the first example, searching for ‘lithium’ on AERSMine shows that 22,575 patients reported lithium use and a total of 4,180 AEs, of which 327 AEs were significantly correlated with lithium use, and 57 of the 327 AEs are known, on-label, lithium-related complications, whereas the others may represent potentially new lithium-related AEs. To better approximate an accurate lithium toxicity profile, AERSMine allows selective exclusion of known interacting drugs (Supplementary Table 4) and comorbidities (e.g., diabetes^12–14). Using this putative set of lithium-correlated neurological AEs, we identified inversely correlated drugs that included angiotensin receptor blockers and antithrombotics, which may represent reduction in the risk of lithium AEs. A comparative analysis of the differential risks across these drugs showed significant reduced risks of anger (RR = 0.501) and aggression (RR = 0.331), self-injurious behavior (RR = 0.14) and suicidal ideation (RR = 0.428) in patients on angiotensin receptor blockers (Fig. 2a and Supplementary Figs. 6 and 7). Patients on a combination of lithium and an angiotensin receptor blocker show reduced risks and favorable safety signals for lithiumspecific AEs, suggesting potential benefit of combinatorial therapy (Supplementary Fig. 8, Supplementary Note 1 and Supplementary Table 4).

Figure 2.

Illustration of AERSMine in hypothesis generation. (a) Candidate lithium toxicity protective agents generated by clustering differential AE risks of lithium toxicities as a function of drug class exposures and identifying inversely correlated drugs. Results indicate that angiotensin receptor blockers (ARBs) are good candidates for combination with lithium, presenting a reduced risk (RR ≤ 1, P < 0.001, safety signal < 0) of various AEs, including anger, aggression, self-injurious behavior and suicidal ideation; ARBs also matched the safety profile of other cardiovascular system drugs, including antithrombotic agents and beta blockers. (b) Differential AE risks among population subgroups as a function of indication, demographic and the use of anti-TNF agents, glucocorticoids (GCS), methotrexate (MTX) and combinations, suggesting significant elevated risk for some groups compared with baseline of patients on anti-TNF therapy (safety signal > 0, red). The heat map shows that concomitant use of glucocorticoids with anti-TNF therapy significantly exacerbates the risk of groups of AEs (P ≤ 0.05, two-tailed Mann-Whitney-Wilcoxon test) compared with baseline anti-TNF monotherapy. The risks of interstitial lung disease, pulmonary edema and fibrosis are elevated in patients (particularly elderly) on a combination of anti-TNF agents and glucocorticoids (P ≤ 0.05, two-tailed Mann-Whitney-Wilcoxon test). (c) Pattern analysis of AEs associated with different classes of NSAIDs among different indication subgroups. Differential AE risks in management of pain, arthritis and hypertension suggests higher risks (more than twofold) for cardiovascular events associated with COX-2 inhibitor (coxib) use. Some of the AEs that showed differential risk patterns across clinical indications and demographic subgroups are highlighted AEs (Supplementary Table 6).

In the second example, we use AERSMine to stratify patient subgroups based on anti-TNF-associated AEs and identify drug combinations that exhibit specifically correlated complications¹⁵. We defined four key treatment groups for indications of TNF-responsive inflammatory and autoimmune disorders (rheumatoid arthritis, psoriasis, psoriatic arthropathy and ankylosing spondylitis), excluding any patients with malignancies: anti-TNF agents (aTNFs only; n = 267,158), glucocorticoids only (gcs; n = 11,521), methotrexate only (n = 9,313) and anti-TNF agents plus glucocorticosteroids (aTNFs + gcs; n = 26,346). Analyzing adult and elderly subgroups for differential therapy-associated effects resulted in a list of 118 AEs, including pulmonary edema, fibrosis, interstitial lung disease, pleural effusion and infections. The relative risks of developing these AEs appear to be significantly higher (P < 0.05, two-tailed Mann-Whitney-Wilcoxon test) in patients on corticosteroids than the baseline of patients receiving anti-TNF agents (Fig. 2b, Supplementary Fig. 9 and Supplementary Table 5). Additionally, patients on a combination of anti-TNF agents and glucocorticoids are at significantly increased risk (at least 2.5-fold, P < 0.05, two-tailed Mann-Whitney-Wilcoxon test) of these pulmonary AEs, which is an important observation because this combination therapy is encouraged in treatment of autoimmune disorders¹⁶. The combination-therapy-associated risks of pulmonary complications are further exacerbated in the elderly, who are at most risk of life-threatening AEs¹⁵ (Supplementary Note 2).

In the third example, we demonstrate the utility of AERSMine in detecting differential drug effects through clinical-indication-based construction and grouping of distinct patient subsets. Analyzing differential rates of AEs in patients with arthritis, hypertension and pain management issues and reporting use of propionic acid derivatives (e.g., ibuprofen), salicylic acid derivatives (e.g., acetylsalicylic acid) or cyclooxygenase (COX)-2 inhibitors (e.g., celecoxib), we noted that patients (both adults and elderly) on COX-2 inhibitors are at increased risk¹⁷ (over twofold) for cardiovascular system–related AEs compared with those on either propionic acid or salicylic acid derivatives (Fig. 2c, Supplementary Fig. 10 and Supplementary Table 6). On the other hand, the risks of interstitial lung disease¹⁸ and abnormal hepatic function¹⁹ are higher (RR > 2, safety signals > 0) with propionic acid derivatives. Additionally, for subgroups of all indications, we observed an increased risk of COX-2 inhibitor-correlated AEs (of which 106 are known COX-2 inhibitor-related AEs, Supplementary Table 6), whereas the risk patterns for similar indications across other non-steroidal anti-inflammatory drugs (NSAIDs) varied. Such clinical-indication-based constructio and grouping of distinct patient subsets not only improve our ability to detect differential drug effects but also deepen ou understanding of drug-related differential long-term therapeutic outcomes.

AERSMine has certain limitations due to the nature of the FAERS data, the numerous variables, potential reporting biases and confounding^20–23, which may affect understanding true correlations and present a major challenge for drug–AE hypothesis generation. AERSMine currently relies on a priori clinical knowledge for exclusion of known confounders; however, automatic confounder control methods²⁴ can be potentially applied. Second, underlying indications, including comorbidities, have the potential to drive differential reporting of AEs, leading to indication-specific signatures and differential reporting rates. Third, reporting peaks for a drug may vary after release, and the trend may have an effect on pharmacovigilance activities^6,25,26; however, FAERS does not appear to be affected by reporting biases, such as the ‘Weber effect’ and stimulated reporting following issuance of FDA alerts^25,26. Also, FAERS lacks the denominators (the total number of patients using the drug globally who did not experience an event) to estimate the true attributable risks. Lastly, determining confidence in the relative risk scores requires sufficient cohort sizes and drug–AE reports. Because of dynamic querying resulting in variable cohorts sizes, comparison of RR signals becomes context specific to the events, therapeutic classes or clinical indications. AERSMine uses other quantitative measures, which may mitigate some of the RR-related limitations²⁷.

AERSMine continues to develop as a platform and will be updated periodically with FAERS data releases, newly approved or trial drugs and combinations, and drug label–AE metadata. AERSMine-facilitated hypotheses and safety signals can be further reviewed by consortia, such as SONAR⁶, and enable additional AE-specific investigations (Supplementary Note 3). Rich data segmentation and pattern analyses enabled by the tool allow comparative studies across indication groups to identify differential signatures that have the potential to improve our understanding of AE patterns across patient genotype–phenotype–demographic subgroups and generate testable hypotheses^15,28. The ultimate goals of these analyses are to protect patients by improving therapeutic selections and monitoring strategies, and conserve valuable therapeutics by minimizing harmful interaction choices. This open aperture approach—based on AERSMine’s ontological aggregation of individual cases—enables the creation of focused/virtual cohorts whose comparisons allow differential treatment effects (either better or worse) to be identified essentially as emergent properties of the database.

METHODS

Methods and any associated references available in the online version of the paper.

ONLINE METHODS

Drug name normalization and concept aggregation.

Because FAERS allows drug entries as free text, a lack of strict regulation allows spurious and redundant data to be inserted into patient drug reports. FAERS drugs were unified to their known generic name concepts using a semi-supervised iterative approach, and further categorized into therapeutic classes using the WHO Anatomical Therapeutic Chemical (ATC) Classification System nomenclature. Normalizing the drug entries to generic names is a multi-step process that involves primarily consolidating free-text drug entries with the KEGG-enriched INN-USAN ATC generics, and subsequently using RxNorm (https://www.nlm.nih.gov/research/umls/rxnorm/index.html), Drugbank²⁹, PharmGKB³⁰ and other publicly available drug label information to further enrich the mapping. After unification of each FAERS drug entry to its corresponding generic name, a unique concept identifier (CUI) was obtained through the Unified Medical Language System (UMLS) (https://www.nlm.nih.gov/research/umls/). These CUIs were aggregated into therapeutic classes using the ATC classification. A separate AERS-curated ontology was created for drug entries that could not be mapped to ATC; these include homeopathic and herbal medications, non-ATC unmapped combinatorials, drugs awaiting ATC classification, trials (Supplementary Figs. 1 and 2). In the next ATC revision, these new drugs will be allocated ATC codes and their ontological aggregations will be reflected within AERSMine.

The FDA codes the clinical indications and adverse events using the Medical Dictionary for Regulatory Activities Terminology (MedDRA) Preferred Terms or in some older reports to the Lower Level Terms. Both Preferred Terms and Lower Level Terms are lower-level granular concepts. To allow uniform data mining, FAERS data released since 2004 were unified to MedDRA version 16.1. Unifying the FAERS data to a single version of MedDRA allows uniformity and aggregation across multiple levels of the ontology, which is not possible using the raw FAERS data. AERSMine thus facilitates analyses across any of the MedDRA classification levels, namely PT (Preferred Terms), HLT (High Level Terms), HLGT (High Level Global Terms) and SOC (System Organ Class).

AERSMine utilizes the power of aggregation and provides PTs, in addition to higher-level MedDRA concepts including HLTs, HLGTs and SOCs, and thus offers an ontology-based hierarchical grouping of indications and AEs that allow focusing on a group of disorders, indications or reactions, such as pulmonary vascular disorders, red blood cell disorders (Supplementary Fig. 3). The PTs are further consolidated with drug labels³¹ to create a ‘Known Drug Reactions’ (KDR) repository, which is extensively used to indicate label-AEs and non-label AEs for each analysis. This integration of KDR further allows AERSMine to systematically partition analyses to focus on both known and previously unknown associations.

Quantitative safety signals, pruning matrices and ranking.

The quantitative safety signals are detected by measuring the disproportionality between the observed and the expected reporting frequency of a drug–AE pair (IC)^8,9 or drug–drug–AE triplet (Ω)⁷, with a positive score indicating a potential safety concern requiring further review. The higher the score, the more the combination stands out from the background and represents a suspected ‘outlier’ association within the FAERS data. These quantitative metrics are exploratory analysis tools for recognizing potential pharmacological interactions and the significant associations (outliers) highlighted by IC and Ω scores require further validation and review to establish a causal relationship. To identify significant associations within a large result data set, Analysis Filters allow the analysis matrix to be pruned using a combination of absolute counts, normalized counts per 1,000 patients, relative risks, relative risk P values (based on Chi-squared test), and quantitative safety signal metrics (IC or Ω). Ranking of the result matrices is done using a combination of risk and incidence frequencies, high incidence/high risk, low incidence/high risk or high incidence/low risk, and allows capturing and representation of differential effects across multiple cohorts.

AERSMine database structure, design and query framework.

AERSMine has three primary data sources (FDA’s AERS data, ATC and MedDRA) that are used by the querying engine for facilitating large-scale studies. The ATC and MedDRA data sources are used to annotate the FAERS drugs, indications and adverse events with concept IDs that are used for AERSMine’s ontological aggregation. The KEGG-ATC ontology provides International Nonproprietary Name (INN), United States Adopted Names (USAN) and salt names for the drugs, which are used as alternative names that facilitate searching across multiple variants of a drug, for example, paracetamol (INN) and acetaminophen (USAN). Additionally, the alternative names table is used to store drug name variants including spelling variants and foreign names as available in the original FAERS data (Supplementary Fig. 4). The node counts are precomputed for optimal performance and are refreshed as new data are added, upon release by the FDA, for example, losartan 47,653, and stored in ontology tables. Enriching the raw FAERS data with ATC/MedDRA facilitates the construction of ontology trees (as displayed on search by clicking Show Ontology), unifies drug variants to generic concepts and also provides the data sources for dynamic auto complete tools for ontology search (Supplementary Fig. 5).

AERSMine is a hybrid system that is built with a Java core designed to run within a Java Servlet Container and provide interactive user sessions based upon HTML5. This hybrid architecture enhances parallel processing of multiple sets, allowing data high-dimensional analysis resulting from numerous internal queries. The query-data model leverages the construction of multi-dimensional ontologies that provide multi-linked nodular indexing for reducing the complexity and provides highly optimal query runtimes that efficiently facilitate complex multi-layered analyses.

AERSMine is primarily programmed in Java with Servlet technologies for the backend whereas the front end uses HTML5, CSS and JavaScript in combination with various JavaScript tools such as JQuery and AngularJS to provide an improved user experience. On the server, the ontologies that map the core data are kept memory resident, with persisted states within a MySQL database server and a SOLR server. The memory-resident ontologies can be dumped to the local hard drive and reloaded directly to bypass the dynamic reconstruction of the ontologies and their multi-dimensional inheritances.

Supplementary Material

Note: Any Supplementary Information and Source files are available in the online version of the paper (doi:10.1038/nbt.3623).