Adverse drug reactions – examples of detection of rare events using databases

Abstract

It is recognised that randomised controlled trials are not feasible for capturing rare adverse events. There is an increasing trend towards observational research methodologies using large population-based health databases. These databases offer more scope for adequate sample sizes, allowing for comprehensive patient characterisation and assessment of the associated factors. While direct causality cannot be established and confounders cannot be ignored, databases present an opportunity to explore and quantify rare events. The use of databases for the detection of rare adverse events in the following conditions, sudden death associated with attention deficit hyperactivity disorder (ADHD) treatment, retinal detachment associated with the use of fluoroquinolones and toxic epidermal necrolysis associated with drug exposure, are discussed as examples. In general, rare adverse events tend to have immediate and important clinical implications and may be life-threatening. An understanding of the causative factors is therefore important, in addition to the research methodologies and database platforms that enable the undertaking of the research.

Introduction

While randomised controlled trials (RCTs) are considered the gold standard in clinical research, they are conducted in controlled settings and are less informative for the assessment of rare events or long term effects. Observational studies are a useful complement. In comparison with RCTs, observational studies explore rare events among much larger numbers of patients in the real-life setting with longer follow-up time.

Large databases provide an important platform for the undertaking of observational studies to generate clinical data on the effectiveness and safety of drugs. Useful pharmacoepidemiological data, including drug usage trends, incidence and prevalence of rare events, are commonly generated from large population databases, electronic clinical and administrative-claimed databases.

One particular advantage of observational studies is the ability to explore adverse drug events for both new and old drugs. Importantly, they facilitate the assessment of rare (between ≥1/10 000 and <1/1000) to very rare (<1/10 000) events 1.

This review discusses database use in the study of rare adverse events, highlighting examples of previous pharmacovigilance studies. We discuss the features of the study methodologies and the merits and challenges of the databases used. Table 1 summarises the databases used in three examples below for detecting rare ADRs.

Table 1.

Examples of databases for ADR studies

Database [Reference]	Type	Population, Country	Current population size
General Practice Research Database (GPRD) [5]	Non-administrative medical records	Primary care attenders, UK	Over 9 million patients with over 44 million years of follow-up
Medicaid [6], [7]	Administrative health insurance database	Social welfare recipients, US	∼50 million since 1980s
HealthCore [6]	Administrative health insurance database	Insurance plans enrollees, US	43 million since late 1990s
Kaiser Permanente California [7]	Health Maintenance Organization (HMO)	Healthcare services payers, US	2.8 million since 1994
MarketScan Research Database [8]	Administrative health insurance database	Social welfare recipients, US	150 million since late 1980s
British Columbia Linked Health Database [17]	Non-administrative research database	Residents, British Columbia, Canada	4.5 million since 1985
Clinical Data Analysis and Reporting System (CDARS) database [19]	Clinical databases	Residents, Hong Kong	7 million since 1993
National Health Insurance Research Database [19], [25]	Administrative claimed database/insurance	Residents, Taiwan	23 million (99%) since 1995
The Health Improvement Network [20]	Non-administrative medical records	Primary care attenders, UK	<11 million since 2003
MarketScan Commercial Claims and Encounters Database [21]	Administrative claimed database/insurance	Insurance plans enrollees, US	138 million since 1995
Optum ClinFormatics [21]	Administrative claimed database/insurance	Insurance plans enrollees, US	34 million
Rochester Epidemiology Project [22]	Non-administrative research database	Residents, Olmsted County, Minnesota, US	<490 000 since 1966
Central Person Register/Civil Registration System [24]	Administrative registry linked with medical records	Residents, Denmark and Greenland	8 million since 1968
Dokumentationszentrum schwerer Hautreaktionen (dZh) [36], [38]	Registry database	Residents, Germany	>900
Health Insurance Review and Assessment Service (HIRA) database [37]	Administrative database of national medical insurance	Residents, Korea	>40 million
RegiSCAR [39]	Registry database	Patient with severe cutaneous adverse reactions, Europe	>500

Example 1 Serious and rare adverse events in children: sudden death associated with attention deficit hyperactivity disorder (ADHD) treatment

Sudden death related with ADHD medications is rare in children. The risk of sudden death among children and adolescents, estimated at 0.6–6 in 100 000 per year 2, is much lower than that for adults, whose risk sits at 1 in 1000 per year 3. For the detection of such rare adverse events, standard RCTs are unfeasible. Thus, observational studies with large population databases would be more useful and applicable in this context.

Methodology of the included studies

Observational study designs, such as case-control and cohort studies, are among the most relevant choices for investigating rare events under these circumstances where incidence of outcome is so rare that the rate of events cannot be estimated within a short study period. This example summarises the studies that used different databases for detecting sudden death in children. Gould et al. 4 conducted a case-control study on 564 cases of sudden death in children and young people aged 7 to 19 years old vs. matched controls of fatalities in motor vehicle accidents. ADHD medications used in this study, including amphetamine, dextroamphetamine, methamphetamine and methylphenidate, were associated with an increased risk of sudden death (odds ratio [OR] = 7.4, 95% confidence interval [CI] 1.3, 9.4). McCarthy et al. 5, Schelleman et al. 6, Cooper et al. 7 and Olfson et al. 8 carried out retrospective cohort studies among different age groups under 25 years old. Neither detected an increased risk for sudden death nor any significant differences between exposure and non-exposure to ADHD medications. However, as acknowledged by the authors, some important limitations, such as unestablished direct causal relationships and inadequate sample size regarding observational study designs cannot be ignored.

Types of databases

Administrative health databases are used frequently by US researchers. For instance, Schelleman et al. 6 and Cooper et al. 7 identified eligible subjects using Medicaid, a health insurance database for social welfare recipients. Other examples include HealthCore 6 and MarketScan Research Databases 8, while some researchers incorporated databases from American Health Maintenance Organizations (HMO), such as the Kaiser Permanente California 7. These administrative health databases were initially developed to reimburse healthcare providers in nationally founded healthcare systems or managed care organisations 9, and were not intended for research purposes. The information recorded was a by-product of financial transactions that included the type of insurance coverage, dates of medical service and associated diagnoses, tests performed and prescriptions dispensed by community pharmacies 10,11.

Physician-driven medical records are another type of database used more often in Europe. McCarthy et al. 5 used the UK General Practice Research Database (GPRD), which is different from those in North America, and is non-administrative in nature. The GPRD is one of the largest medical record databases in routine use for medication investigation and contains the data of over 9 million patients and 44 million years of follow up time. Information is entered by general practitioners and it is widely used for research due to the comprehensive data covering demographics, medical diagnoses, prescriptions, referrals to hospitals, smoking status, height, weight, immunisations and laboratory results 12. With these strengths, McCarthy et al. were able to identify children and young people aged 2 to 21 years from 1993 to 2006 who were prescribed ADHD medications resulting in a cohort of 18 637 person years 5. Once again, the study drew the conclusion that sudden death was a rare event in children taking stimulant medication for ADHD where the best case scenario gave an incident rate ratio of 0 (95% CI 0, 5.35), excluding one unconfirmed death. If this case was included as sudden death, the worst case scenario gave an incident rate ratio of 1.63 (95% CI 0.004, 9.71) 5.

Merits and challenges

Population-based databases enable the investigation of rare events like sudden death in real-life conditions due to their high quality longitudinal data on a large population. They carry more accurate information than that which is self-reported, thus avoiding recall bias. Simultaneously, selection bias can be reduced without mandatory patient informed consent. Nevertheless, several factors need to be considered when applying these databases to different study designs and objectives. First, the quality of data needs further validation. Second-hand data requires researchers to screen, retrieve and confirm all the information necessary for their particular purposes, especially diagnostic information which needs timely updates 12. However, there are difficulties due to privacy and legal issues 13. Second, confounding variables are limited. Most population-based databases, especially administrative health databases, lack personalised information related to the diseases of interest. For instance, disease severity, drug regimes, patients’ medication taking behaviours and lifestyles are frequently unknown. In addition, data on medications prescribed elsewhere or without prescriptions were unavailable across databases 12. In particular, populations covered under some health insurance plans are unstable due to high patient turnover, for example, the American HMO 14. Therefore, the data may not be representative of the study population. As seen in the studies above, although population-based databases contain large cohorts ranging from 18 637 to 2 579 104 person-years 5,7,8, these studies still failed to achieve sufficient statistical power to detect rare adverse events such as sudden death in children.

Example 2 Serious adverse events after short term treatment: retinal detachment (RD) associated with the use of fluoroquinolones

In contrast to example 1 where long term use of ADHD medications is required, the detection of RD following short term treatment with oral fluoroquinolones (FQ) is examined. RD is not a rare event but a medical emergency for which re-attachment is required to prevent permanent vision loss. The incidence of RD varies among ethnicities 15,16. The duration of a course of antibiotics is very short (approximately 1 week). Therefore, the probability of a RD event during the FQ prescription period was very low, making this a rare adverse event.

Methodology of the included studies

Etminan et al. published a nested case-control study in 2012 reporting a significant association between current oral FQ use and the development of RD in Canadian subjects. The mechanism behind the development of RD is unknown 17. An adjusted rate ratio of 4.5 (95% CI 3.56, 5.70) was reported among current users of FQs, suggesting that the development of RD was an acute consequence of FQ use. Kuo et al. published a cohort study afterwards but the findings did not concur 18. They reported a positive association (adjusted hazard ratio 2.07 [95% CI 1.45, 2.96]) compared with amoxicillin users but the duration between the use of oral FQ and the occurrence of RD was 35.5 days, which contradicted the findings of an acute association reported by Etminan et al. However, the follow-up period in the study by Kuo et al. 18 was 90 days which differs from the definition of current users (i.e. event occurring within prescription period) in the Etminan et al. study. This discrepancy may account for the differences in the findings. Chui et al. 19, Eftekhari et al. 20, Fife et al. 21, Kapoor et al. 22 and Pasternak et al. 23 conducted studies on the same association using different study designs such as cohort, case-control or self-controlled case series. They shared a similar definition of current FQ use (within prescription period or within 10 days after the beginning of prescription period) with Etminan et al. but all reported an insignificant association between FQ and RD.

Types of databases

Several databases from different countries were used in the above-mentioned studies, which, similar to those in example 1, consisted mainly of administrative and non-administrative databases. Pasternak et al. 23 used an administrative registry database, the Central Person Register/Civil Registration System, which includes demographic data and the vital status (death and birth records) of residents in Denmark and Greenland. The registry can be linked to other medical record databases such as The National Prescription Registry and The Danish National Patient Register 24. Kuo et al. 18 and Chui et al. 19 used the National Health Insurance Research Database 25, also an administrative-claimed database maintained by the National Health Research Institutes in Taiwan. This contains data from the National Health Insurance programme, and is a mandatory enrolment and single payment programme covering over 99% of Taiwan’s population. Fife et al. 21 used administrative claims databases, the MarketScan Commercial Claims and Encounter database and Optum ClinFormatics database whose purpose and use are similar to the health insurance databases mentioned in example 1.

In North America, non-administrative databases such as the British Columbia Linked Health Database was used by Etminan et al. 17 and data from the Rochester Epidemiology Project were used by Kapoor et al. 22. These are non-administrative databases and are designed to capture longitudinal data 26,27. Chui et al. 19 used the Clinical Data Analysis and Reporting System, another clinical database that is managed by the Hong Kong Hospital Authority, which provides primary, secondary and tertiary healthcare services to all residents. The Health Improvement Network 28 in the UK is a non-administrative electronic medical record database used by Eftekhari et al. 20, and is similar to the GPRD mentioned in example 1 5. At least 4000 RD cases or 90 000 FQ prescriptions were identified from these databases which enabled the investigation of the association between oral FQ and the subsequent occurrence of RD.

Merits and challenges

Based on the current literature, the development of RD whilst exposed to oral FQ is a rare event. Similar to the limitations in example 1 regarding study design, which failed to control fully residual confounders, factors such as the degree of myopia and other genetic factors may increase the risk of RD. However, the self-controlled case series study design used by Chui et al. 19 and Fife et al. 21, which depends on within-person comparison, controlled such confounding. To interpret the association between RD and oral FQ, a long follow-up period of the cohort is required to capture sufficient FQ exposed periods to make a comparison. Such factors are particularly important when the exposure time of antibiotics is very short, as demonstrated in this example. Large population databases contain enormous quantities of longitudinal healthcare information, thus providing sufficient short term prescriptions for the interpretation of such events. Although most of the studies included in this example failed to find a significant result, this is probably due to the weak association of the event itself, rather than insufficient sample size of the databases.

Example 3 Serious rare adverse events that are known to be drug induced: toxic epidermal necrolysis (TEN)

Compared with example 1 and 2, TEN is an even rarer and more serious adverse event. The incidence of TEN is only 0.4 to 1.9 per million people worldwide 29–31 with a mortality rate of 25–40% 32,33. Due to the extreme rareness, there is little information on the epidemiology of TEN. Although it is widely known that most TEN events are drug induced, data on the estimated risk of various medications are limited.

Methodology of the included studies

Descriptive and case-control studies are two common approaches for investigating TEN using databases. Case-control study is preferred as little is known about the associated risk factors. Moreau et al. 34, Li et al. 35 and Choon et al. 36 conducted descriptive analyses to determine the incidence, clinical patterns and associated medications for various cutaneous adverse reactions, including TEN. Moreau et al. 34 found an association between ophthalmologic disease and severity of cutaneous adverse reactions in US children. Li et al. 35 estimated the incidence of TEN in the Haidian district of Beijing as no less than 0.05 per million-person-year. Li et al. 35 and Choon et al. 36 identified the most common drugs associated with serious cutaneous adverse reactions (SCARs) to be anticonvulsants, allopurinol, and antibiotics in the Chinese and Malaysian population, respectively, with the results being comparable with other studies.

Lin et al. 33 and Kim et al. 37 conducted case-control studies to estimate the risk of TEN induced by different medications. Lin et al. 33 identified 35 TEN/SJS cases and 105 controls from the database of a university teaching hospital in Taiwan. Carbamazepine, phenytoin and allopurinol were the drugs frequently associated with TEN/SJS with a crude relative risk of 33.0, 9.6 and 18.0, respectively. Another case-control study by Kim et al. also utilised databases to evaluate the risk of SCARs induced by multi-antiepileptic (AED) drugs in Korean elderly patients 37. Kim et al. showed that among the AED drugs, only carbamazepine was significantly associated with SCARs in elderly patients aged ≥65 years with a 10-fold increase of risk (adjusted OR = 10.39, 95% CI 2.64, 40.86).

Types of databases

Administrative databases, such as in hospitals, are commonly used in the above-mentioned studies. The database used by Moreau et al. contained the 2005 discharge information from hospitals in 11 US states, comprising 37.6% of the 2005 US paediatric population 34. Li et al. 35 used the database from Peking University Third Hospital, the only hospital in the Haidian district of Beijing with a dermatology ward. Lin et al. 33 also obtained data from the admission database of a university teaching hospital in Taiwan.

Kim et al. conducted their study with the Health Insurance Review and Assessment Service (HIRA), another administrative claims database, which contains information from Korea’s National Health Insurance (NHI) programme. The insurance is mandatory and therefore covers almost all Korean residents 37.

Choon et al. used a research driven registry database for cutaneous adverse drug events. The registry was developed in 2001 at the Department of Dermatology, Hospital Sultanah Aminah, a tertiary referral centre for the Johor Bahru district, Malaysia. All cases were validated and information was obtained from patients’ clinical records and case reporting forms, which were completed by dermatologists 36.

Merits and challenges

TEN is a severe condition requiring hospitalisation. Therefore, hospital databases have the advantage of capturing most of the TEN cases in the region. Similar to the UK GPRD in example 1 5, the information contained in hospital databases is ideal for the investigation of epidemiological patterns in different patient groups, as in Moreau et al.’s study 34. They are also valuable for the identification of causative drugs and their subsequent risk analysis, enabling researchers to obtain detailed medication records critical to the investigation of a case. In addition, hospital databases have a large pool of patients where researchers can easily search for matched tolerant controls for the medication risk estimate.

However, a major challenge of using hospital databases for epidemiological study is the generalisability of results. Hospital databases record patients in a designated region only and are usually unable to access information from other hospitals. As a result, analysis of the pattern and distribution of the disease, such as incidence, is limited to a certain geographic region. Li et al. only presented the incidence of TEN in the Haidian district of Beijing and this incidence was relatively low compared with other studies. The researchers explained that not all patients would attend the hospital being investigated and thus, the incidence was underestimated 35. For epidemiological studies, population based databases, such as the administrative claims database used by Kim et al. 37, are preferred.

Although databases can contribute to the identification of rare ADRs with extremely low incidences, it remains a challenge to obtain adequate sample sizes for TEN studies. It is even more challenging if the cases are further divided into various drug-induced cases for medication risk analysis. Kim et al. used a database with over 4 million people during the study period but found only 23 TEN cases 37. This number is clearly insufficient for the case-control analysis of each specific drug-induced TEN. Therefore, it is usual to combine TEN with other cutaneous adverse reactions for analysis.

To obtain sufficient cases, some studies have established a registry database for serious rare events, as in Choon et al. 36. Registry databases collect cases across regions or even worldwide and, therefore, have no geographic limitation. ‘Dokumentationszentrum schwerer Hautreaktionen’ (dZh) is an early SCARs registry established in Germany since 1990 38 and it has collected information on hospitalised SCARs across the nation up to the year 2000 with more than 950 reported cases. The International Registry of Severe Cutaneous Adverse Reactions (RegiSCAR) is the largest registry of SCARs 39. It contains cases across Europe and the network is being extended to Asia. It is expected that, through multinational networks, there will be sufficient TEN cases to conduct risk analysis on various drugs, especially those new on the market. However, registry databases are unable to estimate the incidence of TEN.

Conclusion

Databases are an important tool used in the detection and assessment of unconfirmed adverse drug reactions, including rare or very rare events, enhancing our understanding of new and old drugs to safeguard public health. Large databases will continue to provide an important platform for the detection of rare events. An understanding of the strengths and limitations of the available population-based databases are important in the design and undertaking of observational studies.

Competing Interests

All authors have completed the Unified Competing Interest form at http://www.icmje.org/coi_disclosure.pdf and declare no support from any organisation for the submitted work, no financial relationships with any organisations that might have an interest in the submitted work in the previous 3 years and no other relationships or activities that could appear to have influenced the submitted work.