Research Techniques Made Simple: Pharmacoepidemiology Research Methods in Dermatology

Abstract

Clinical trials have several important limitations for evaluating the safety of new medications, leading to many adverse events not being identified until the postmarketing period. Descriptive studies, including case reports, case series, cross-sectional, and ecologic studies, help identify potential safety signals and generate hypotheses. Further research using analytic study methods, including case-control studies and cohort studies, are necessary to determine if an association truly exists and to better understand the potential for causation. Pharmacoepidemiology research examines the use and effects of drugs when used in large populations of patients, using a variety of study designs and biostatistical techniques to reduce the confounding and systematic error associated with observational research. Understanding the strengths and limitations of pharmacoepidemiology research techniques is necessary to interpret the validity of drug safety studies, guiding both individual patient decisions and broader public health decisions.

INTRODUCTION

Pharmacoepidemiology applies the basic science of clinical epidemiology to study the use and effects of drugs in large populations (Strom, 2012). According to data from the National Health and Nutrition Examination Survey, in 2012, 59% of all adults reported the use of at least one prescription drug in the previous 30 days, and 39% of adults over 65 years of age reported the use of five or more prescription medications (Kantor et al., 2015). Pharmacoepidemiology research, using a variety of study designs and biostatistical techniques to reduce the confounding and systematic error (i.e., bias) associated with observational research, is necessary to understand the effects of medications in large, heterogeneous populations over time and guides both individual decisions for patients and broader public health decisions.

CLINICAL TRIALS HAVE SIGNIFICANT LIMITATIONS FOR DETECTING DRUG SAFETY

Drugs are approved by the US Food and Drug Administration (FDA) based on data from preclinical animal studies and safety and efficacy studies in humans, typically from randomized controlled trials (RCTs). Although RCTs are the criterion standard for showing the efficacy of a drug, they have limitations with regard to fully understanding the safety of a medication. These limitations represent a tradeoff between the need to bring new, effective medications to market and the duration of exposure and sample size required to detect rare adverse effects that are important to patients and society. As a result, many potentially serious adverse effects of medications are detected in the postmarketing phase. For example, among prescription drugs approved between 2000 and 2009, 26.7% received a black box warning after approval (Frank et al., 2014).

There are several reasons why preapproval clinical trials do not definitively address safety issues. First, clinical trials are performed in relatively healthy subjects with minimal comorbidities and frequently do not include an ethnically diverse patient population. Therefore, the generalizability of safety data from RCTs is often uncertain in a diverse patient population and in combination with other medications or comorbidities. Also, RCTs typically monitor exposure to the medication over a period of only weeks to months, providing minimal information on the safety of long-term exposure. Finally, RCTs are designed to detect relatively common adverse effects. When a drug is approved by the FDA, typically only several thousand patients have been treated with the drug for a relatively short time period. The “rule of three” states that if an event was not observed in a clinical trial with N participants, it can be concluded with 95% confidence that fewer than 3/N people will experience the event (Strom, 2012). As a result, trials can usually accurately describe only adverse event rates that occur in about 1 in 100 patients and often cannot detect rare adverse events that occur in fewer than 1 in 1,000 people.

CLASSIFICATION OF ADVERSE REACTIONS TO MEDICATIONS

Adverse reactions to medications are divided into three types (Strom, 2012). Type A reactions are pharmacological effects of the drug and are generally well described in RCTs by the time a drug is approved for marketing. They are usually common and dose related, and they can be mitigated by using doses that are appropriate for an individual patient. An example is isotretinoin-related cheilitis. Cheilitis is common and expected based on the pharmacology of isotretinoin and typically improves with a decreased dose. Type B reactions are idiosyncratic or allergic, occur in close proximity to drug initiation, and are rare (<1 in 1,000). Type B effects are usually discovered through descriptive studies (spontaneous reports) after approval, given their rarity. Agranulocytosis from diaminodiphenyl sulfone (dapsone) is an example of a Type B reaction because agranulocytosis is a rare, non-predictable reaction not associated with the predicted mechanism of action of the drug. Type C reactions introduce new morbidities by altering the risk of diseases that occur over time and can often have substantial impacts on public health. Because they are statistically rare and often delayed, they are typically not detected before drug approval. Type C adverse events typically require analytic studies, including cohort or case-control studies, to investigate the association of the drug with the effect in question. An example is squamous cell carcinoma induced by psoralen plus UVA that was identified in a cohort study of over 1,300 patients who were followed for more than 5 years (Stern et al., 1984).

THE DETECTION OF ADVERSE SAFETY EVENTS

Given the limitations of clinical trials for evaluating drug safety, many adverse events are identified in the postmarketing period, beginning with spontaneous reports. Prescribers, patients, pharmacists, and drug manufacturers can all file MedWatch reports that are collected in the FDA Adverse Event Reporting System. Spontaneous reporting programs also have important limitations. Studies have shown severe underreporting of adverse events, with only about 1% of adverse effects reported (Khong and Singer, 2002). Additionally, the number of people exposed to a medication in a population captured by a spontaneous reporting system is not well defined. As a result, the incidence of a potential adverse effect of a medication cannot be reliably determined. There also exists substantial bias in the reporting of adverse events. Adverse event reporting is more likely to occur within the first 2 years of drug approval or if there is media attention related to a specific adverse event (Tsintis and La Mache, 2004). Finally, it is often difficult to determine true causation from an individual case report. Therefore, spontaneous reports should be considered as hypothesis generating and require confirmation through further studies. To address the limitations of the FDA Adverse Event Reporting System, in 2008 the FDA launched the Sentinel Initiative to improve the ability of the FDA to test safety signals identified through spontaneous reporting, using de-identified electronic health care data from multiple sources. A safety signal is defined as information on a possible causal relationship between an adverse event and a drug, the relationship being unknown or incompletely documented previously. The full Sentinel System is now operational and allows the FDA to rapidly access information from more than 193 million patients in the United States and efficiently perform further studies when safety signals are identified (Psaty and Breckenridge, 2014).

PHARMACOEPIDEMIOLOGY STUDY DESIGNS

Descriptive studies, including case reports, case series, cross-sectional and ecologic studies, are best used to identify potential signals and generate hypotheses. Further research using analytic study methods, including case-control studies, cohort studies, and clinical trials, are necessary to determine if an association truly exists to confirm or refute a safety signal identified through descriptive studies. Although RCTs are the criterion standard for causality, case-control and cohort studies are often more appropriate for addressing the hypotheses generated by case reports. Meta-analyses combine the data from multiple studies and are considered the highest level of evidence; however, it is important to understand that the data are only as accurate as the individual studies that have been included, and therefore the individual studies must be scrutinized to understand the accuracy of a meta-analysis. An overview of pharmacoepidemiology study designs is presented in Table 1.

Table 1.

Overview of pharmacoepidemiology study designs

Study Design	Description	Strengths	Limitations
Case report/case series	A description of a single patient or a series of patients	Efficient source for hypothesis generation	Cannot rule out chance/bias Unable to determine incidence Observation may not be generalizable to other patients
Cross-sectional study	The presence or absence of both exposure and disease are assessed at a single point in time	Establish prevalence Hypothesis generation	Cannot establish temporal relationship
Ecological or secular trend study	A study comparing geographic and/or time trends of illness versus trends in risk factors	Rapid and easy support for or against a hypothesis	Associations made at the aggregate population level may not apply to individuals
Case crossover study	A study comparing the pattern of exposure between an event time and a control time with each patient serving as his/her own control	Minimizes confounding by indication	Exposure must be transient Outcome must be an acute event that increases sharply and then subsides Recall bias
Case-control study	A study that selects patients with the disease of interest (cases) and individuals without the disease of interest (controls). The case and control participants are evaluated for differences in prior exposure to various risk factors, yielding odds ratios as a measure of association.	Can study multiple risk factors for a single disease, especially useful for rare diseases Time efficient	Bias in measurement of exposure Confounding by indication
Cohort study	A study that selects subjects on the basis of the presence (exposed population) or absence (control population) of exposure to a factor of interest. Researchers then follow subjects over time, looking for differences in a variety of outcomes, yielding relative risks as a measure of association.	Can study multiple outcomes from an exposure Can measure incidence (risk) of outcome	Selection bias Confounding by indication Prolonged duration Costly
Clinical trial	The investigator determines which patients receive an exposure and then follows the patients for the outcome.	Randomization controls for confounding, selection bias, and confounding by indication. Blinding controls for information bias Criterion standard to establish causality	Generalizability Ethical issues Statistical power Costly Prolonged duration

Adapted from Gelfand and Langan (2013).

Both descriptive and analytical studies have important limitations that must be considered when interpreting study results. Table 2 summarizes potential sources of error and methodological issues that must be considered when interpreting studies looking at adverse drug events. Bias is any systematic error in the design, conduct, or analysis of a study that results in an incorrect estimate of the exposure’s effect on the outcome. A special type of bias that affects pharmacoepidemiology studies is confounding by indication (channeling bias, protopathic bias), which occurs when the disease or symptoms of the disease being treated are also independent risk factors for the outcome being studied. An example of confounding by indication encountered in dermatology research is the increased risk of lymphoma, particularly cutaneous T-cell lymphoma, in patients with more severe psoriasis. A hypothetical study comparing the rate of cutaneous T-cell lymphoma in patients with more severe psoriasis receiving an investigational medicine to the rate of cutaneous T-cell lymphoma in the general population (SEER, for example) might erroneously conclude that the drug increases the risk of lymphoma, when the association is due to the underlying treatment indication (more severe psoriasis). Finally, after understanding any potential methodological issues, when determining the causal nature of an association, one needs to consider time sequence, biologic plausibility, dose-response, strength of study design, strength of association, and consistency with previous research (Table 3).

Table 2.

Sources of error in pharmacoepidemiology studies

Source of Error	Definition	Question To Be Answered
Confounding	An observed association, or lack of association, that is due to a mixing of effects between the exposure, the outcome, and a third confounding variable	Is there a third factor associated with both the treatment (exposure) and the outcome?
Confounding by indication	Systematic error that occurs when the disease itself, or symptoms of the disease, are risk factors for the outcome being studied.	Is the underlying disease being treated in the study a risk factor for the outcome?
Selection bias	Systematic error that arises from methods to select participants for a study that is related to the probability of developing the outcome of interest.	Were the two study groups selected into the study similar, with the exception of the exposure of interest?
Information bias	Systematic error that is associated with the measurement of the exposure or outcome.	Were data on the exposure and outcome measured/collected the same way in both groups?
Generalizability (external validity)	The applicability of the results to other populations	Do the results apply to the general population? Your patient population?
Type I (alpha error)	The probability of finding a significant association when the association is actually due to chance	Were the observed results due to chance alone?
Type II (beta error)	The probability of concluding that there is no difference when a real difference exists	What magnitude of effect was the study powered to detect?
Confidence interval	The range within which the true magnitude of the effect exists	Does the confidence interval include/exclude the relative risk that is important to detect?
Precision	The accuracy of the measured results, including the width of the 95% confidence interval	What was the range of results statistically consistent with the observed finding?

Adapted from Gelfand and Langan (2013).

Table 3.

Factors to consider when understanding causal associations

Time sequence	Does the time sequence between the exposure and the outcome make sense?
Biological plausibility	Is the relationship between the exposure and the outcome biologically plausible?
Dose-response	Is there a dose-response relationship?
Strength of study design	Clinical trials provide more strength for a causal association than observational studies (case-control or cohort studies), which in turn provide more strength for a causal association than descriptive studies.
Strength of association	How high is the point estimate? How wide is the confidence interval?
Consistency with previous research	Are there other studies with strong study designs showing an association?

Adapted from Gelfand and Langan (2013).

ADVANCED BIOSTATICS METHODS APPLIED TO PHARMACOEPIDEMIOLOGY

In clinical trials patients are randomized to a treatment group and there are minimal systematic differences in observed or unobserved covariates between treated and untreated patients. In observational studies, patients in the treated group may differ from those who are untreated in ways that affect the likelihood that the outcome under study will occur. Traditional multivariable regression can be used to adjust for measured covariates, but multivariable regression can be problematic when the outcome is rare. Propensity score methods improve statistical efficiency by creating a single covariate that estimates the probability of receiving a specific treatment. Propensity scores create a balance of baseline clinical characteristics, allowing for direct comparison of similar individuals, but cannot adjust for unmeasured confounders (Strom, 2012). Instrumental variables (IVs) are secondary analysis techniques that can be used to address the effects of unmeasured confounding. An IV tries to mimic randomization, using a variable associated with variations in treatment but not the outcome. Use of an IV assumes that the IV is predictive of the treatment (exposure), is independent of the outcome, and is not associated with measured or unmeasured confounders. Not all studies have an appropriate IV, but common IVs include calendar time, provider treatment preference, geographic distance to a hospital, and insurance plan. An IV analysis should be used with caution because these assumptions are often difficult to fulfill (Strom, 2012). Finally, external adjustment methods can be used to determine the likelihood that unknown or unmeasured confounding may explain an association observed between an exposure and an outcome (Schneeweiss, 2006). If external data sources contain information about a relationship between potential unmeasured confounders and the outcome of interest, the numeric measurement of this relationship can be used to estimate the amount of unmeasured confounding necessary to meaningfully alter the conclusions.

AN EXAMPLE OF PHARMACOEPIDEMIOLOGY IN DERMATOLOGY: THE ASSOCIATION BETWEEN ISOTRETINOIN AND INFLAMMATORY BOWEL DISEASE (IBD)

Isotretinoin was approved by the FDA for severe, recalcitrant nodulocystic acne in 1982, based on studies of the drug in fewer than 100 people. Two years after approval, a warning was added to the package insert about a possible association between IBD and isotretinoin, based on spontaneous reports to the FDA, but the safety signal was not investigated further. Two decades later, lawsuits started to emerge related to a possible association between isotretinoin and IBD. This spurred a large increase in the number of cases being reported. An analysis of cases reported to the FDA between 2003 and 2011 concluded that attorneys reported 87.8% of cases, physicians reported 6.0%, and consumers reported 5.1%. In the entire FDA Adverse Event Reporting System during that same period, only 3.6% of reports were made by attorneys (Stobaugh et al., 2013).

Decades after the initial safety signal was identified by case reports, analytical studies emerged. Initial observational studies offered conflicting results on the relationship between isotretinoin and IBD. These studies were limited by a small number of cases and lack of adjustment for concurrent medications associated with the development of IBD, mainly tetracycline antibiotics that are also used to treat moderate to severe acne. Alhusayen et al. (2013) performed a retrospective cohort study using population-based electronic health data from British Columbia. The study included information from over 4.5 million people and found no association between IBD and the use of isotretinoin (relative risk = 1.14; 95% confidence interval [CI] = 0.92–1.41). In secondary analyses, there was a weak but significant association between isotretinoin and IBD in people aged 12 through 19 years (relative risk = 1.39; 95% CI = 1.03–1.87). There was also a weak but significant association in people who used topical acne medications only and the development of ulcerative colitis (relative risk = 1.19; 95% CI = 1.00–1.42). Taken together, these associations suggest that IBD may be associated with acne itself, not isotretinoin (i.e., confounding by indication). Strengths of this study over previous research include a large, population-based design; adjustment for oral tetracycline antibiotics; and the use of a control group of patients using topical acne medications to address an association between acne itself and IBD. The study suggests that it is unlikely that isotretinoin causes IBD, but the 95% CI indicates that a potentially clinically significant increased risk cannot be ruled out statistically. A meta-analysis of six observation studies confirmed no increased risk of IBD in patients exposed to isotretinoin (OR = 1.08, 95% CI = 0.82–1.42) (Lee et al., 2016).

CONCLUSIONS

New medications are being developed at an increasingly rapid rate in current clinical practice. Therefore, pharmacoepidemiology research is increasingly important to provide a fuller understanding of drug safety in the postapproval setting. Understanding pharmacoepidemiology study design, validity, and the complexity of causal associations is crucial to guide physician decisions for the individual patient and public health and public policy decisions.

CME Activity Dates: January 19, 2018

Expiration Date: January 18, 2019

Estimated Time to Complete: 1 hour

Planning Committee/Speaker Disclosure: Joel Gelfand, MD receives research support from AbbVie, Jansen, Novartis AG, Celgene, Pfizer, Inc., Sanofi-Aventis, and Regeneron and is a consultant/advisor for Bristol-Myers Squibb Sanofi Pharmaceuticals Partnership, Coherus, Jansen, Novartis AG, Dermira, Menlo, Pfizer, Inc., Sanofi-Aventis, Dr. Reddy’s Labs, and Regeneron. All other authors, planning committee members, CME committee members and staff involved with this activity as content validation reviewers have no financial relationships with commercial interests to disclose relative to the content of this CME activity.

Commercial Support Acknowledgment: This CME activity is supported by an educational grant from Lilly USA, LLC.

Description: This article, designed for dermatologists, residents, fellows, and related healthcare providers, seeks to reduce the growing divide between dermatology clinical practice and the basic science/current research methodologies on which many diagnostic and therapeutic advances are built.

Objectives: At the conclusion of this activity, learners should be better able to:

• Recognize the newest techniques in biomedical research.

• Describe how these techniques can be utilized and their limitations.

• Describe the potential impact of these techniques.

CME Accreditation and Credit Designation: This activity has been planned and implemented in accordance with the accreditation requirements and policies of the Accreditation Council for Continuing Medical Education through the joint providership of Beaumont Health and the Society for Investigative Dermatology. Beaumont Health is accredited by the ACCME to provide continuing medical education for physicians. Beaumont Health designates this enduring material for a maximum of 1.0 AMA PRA Category 1 Credit(s)™. Physicians should claim only the credit commensurate with the extent of their participation in the activity.

Method of Physician Participation in Learning Process: The content can be read from the Journal of Investigative Dermatology website: http://www.jidonline.org/current. Tests for CME credits may only be submitted online at https://beaumont.cloud-cme.com/RTMS-Feb18 – click ‘CME on Demand’ and locate the article to complete the test. Fax or other copies will not be accepted. To receive credits, learners must review the CME accreditation information; view the entire article, complete the post-test with a minimum performance level of 60%; and complete the online evaluation form in order to claim CME credit. The CME credit code for this activity is: 21310. For questions about CME credit email cme@beaumont.edu.

SUMMARY POINTS.

• Pharmacoepidemiology research uses a variety of study designs and biostatistical techniques, including propensity scores, instrumental variables, and external adjustment, to reduce the confounding and systematic error associated with observational research.

• Descriptive studies, including case reports, case series, cross-sectional, and ecologic studies are best used to identify potential safety signals and generate hypotheses.

• Analytic study methods, including case-control studies, cohort studies, and clinical trials, are necessary to determine if causation can be inferred from an association and to confirm or refute a safety signal identified through descriptive studies.

MULTIPLE CHOICE QUESTIONS.

1. A new drug has been studied in 3,000 patients before approval. The upper limit for the detection of rare adverse reactions in this safety database would be

   1. 1 in 100.
   2. 1 in 1,000.
   3. 1 in 10,000.
   4. 1 in 1,000,000.
2. Who can report a potential adverse drug reaction to the FDA?

   1. Patients
   2. Physicians
   3. Drug manufacturers
   4. All of the above
3. Which of the following is an example of a type A adverse reaction?

   1. Agranulocytosis after starting diaminodiphenyl sulfone (dapsone)
   2. Cheilitis associated with isotretinoin
   3. Squamous cell carcinoma after psoralen plus UVA treatment
   4. Progressive multifocal leukoencephalopathy after efalizumab
4. Which of the following is true about spontaneous reporting of adverse drug events?

   1. Most adverse drug events that occur are reported to the FDA.
   2. Spontaneous reports can be used to calculate the incidence of an adverse event.
   3. Information generated from spontaneous reports should be subjected to further studies.
   4. Events are reported more commonly for older drugs.
5. Which of the following is an advantage of using propensity scores over traditional regression analysis?

   1. Propensity scores improve the efficiency of the analysis.
   2. Propensity scores can adjust for unmeasured confounding.
   3. Propensity scores randomize patients to a treatment arm.
   4. Propensity scores adjust for confounding by indication.

Abbreviations

CI

confidence interval

FDA

US Food and Drug Administration

IV

instrumental variable

RCT

randomized controlled trial