Use of the self-controlled case series method in drug safety assessment

Introduction

The randomized double-blind controlled clinical trial (RCT) is the undisputed gold-standard for assessing drug efficacy. However the RCT is usually insufficiently powered, or too brief, to detect rare but serious adverse effects or modest but important increases in the risk of common disease outcomes (such as coronary heart disease events) that can have a major population impact in absolute terms. Pooling individual trials via meta-analyses sometimes helps but the reporting of information on adverse effects in clinical trials is often incomplete or poorly quantified, particularly when compared to the efficacy endpoints. Clinically significant and unexpected abnormal laboratory values may not be detected as not all are routinely included in trial protocols. Most RCTs also tend to exclude the elderly, patients with co-morbidity or pregnancy, and this reduces the generalisability of these data. Therefore, at the time of product launch, there are often limited safety data of any new drug, in both the short- and longer-term which is directly applicable to that of the target population. Drugs in use therefore need to remain under constant surveillance (pharmacovigilance) and study by observation (pharmacoepidemiology) Pharmacovigilance systems identify safety signals (signal detection) and thus serve to generate hypothesis. Pharmacoepidemiology tests such hypothesis (signal validation) and quantifies the risk. Both pharmacovigilance and pharmacoepidemiology have limitations, are complementary and only partially overlap.

What is the problem?

Current pharmacovigilance systems rely heavily on spontaneous case reports. Although there have been some notable successes, this system has deficiencies. First, the onus on reporting is placed on patients (in some countries only such as the US and UK) and on already busy health professionals so under-reporting is likely. Several studies have shown that some adverse effects are more likely to be under-reported for a variety of other reasons.¹ Second, the system is better placed to detect rare and unusual adverse effects (such as tendon rupture from ciprofloxacin), than for the detection of modest increases in the risk of more common clinical outcomes already prevalent in the patient groups being treated (such as myocardial infarction in patients with diabetes and obesity receiving new therapies for these conditions). Third, such case reports lack appropriate denominator information and therefore cannot be effectively used to estimate the likely incidence. Fourth, analysis of spontaneous reporting may be influenced by external factors such as media interest or safety alerts (notoriety bias). Fifth, there is also the requirement for the reporter to make a [possible] link between the drug and an event

Concerns about possible adverse effects often prompt cohort studies where the risk of the suspected adverse effect is compared among those exposed and unexposed to the drug in question. These can be effective, but this design is limited by the non-randomized exposure to the drug so that exposed and unexposed groups can differ systematically in the risk of the adverse effect in question. These systematic differences can confound interpretation of the causal relationship between [drug] exposure and [adverse] outcome which can be both difficult to measure and/or control for. One potentially important source of confounding relates to the decision to prescribe a new drug or not (confounding by indication). If, for example, the adverse effect is already suspected, doctors may be less likely to prescribe the drug in question to those at higher risk and a cohort study may thus bias the estimate of the risk of the adverse effect towards the null. Alternatively, a drug could be [incorrectly] considered to be safer and clinicians may be more likely to select higher risk patients for treatment which would thus inflate the overall risk estimate. Case-control studies are another alternative where the exposure status is compared in patients with the outcome of interest versus patients without. The case-control approach is more effective at identifying relatively newly suspected adverse effects when the adverse effect being studied is otherwise rare in the group being treated (such as phocomelia from thalidomide exposure). Case-control studies are relatively rapid and inexpensive to conduct as compared to cohort studies, however, also suffer from recall bias and face difficulties in selection of appropriate controls.

Can some of these limitations be overcome?

Is there an alternative way of reliably evaluating modest increases in the risk of common adverse effects using routinely collected clinical data? The self-controlled case series method developed by Farrington² and described by Whitaker³ was designed to assess the association between a time-varying exposure and an outcome event and was originally applied to adverse effects of vaccines. More recently, we have applied this method to the investigation of drug safety using information on drug prescription and clinical diagnoses from the UK General Practice Research Database (GPRD);^4-6 although other sources capturing clinical and exposure data can be used (e.g. The Health Improvement Network [THIN] or health insurance databases). Figure 1 provides a pictorial representation of the self-controlled case series safety analysis of drugs. Briefly, the method involves comparison of events occurring in pre-defined intervals during exposure relative to all other observed time [baseline period] for a sampled individual.

Figure 1.

Pictorial representation of a self-controlled case series method where age-adjusted incidence rate ratios of the outcome of interest are derived using Conditional Poisson Regression by comparing defined intervals during the exposure period relative to all other observed time [baseline period] for each person.

This method is advantageous as it only requires assessment of cases and it eliminates fixed confounders as the comparisons are intra-person. This technique will thus remove variation between individuals in risk factors for the disease outcome of interest. Hence, fixed confounders (e.g. gender, socioeconomic class) are implicitly controlled for which overcomes many of the issues encountered with traditional inter-person designs. Age or temporal variation can also be accounted for and under certain circumstances it retains high relative efficiency.⁷

The single most important limitation of the self-controlled case-series method is the requirement that events do not affect subsequent exposures;⁸ this limits the utility of the method in assessing already well-known adverse effects Confounding can also occur with factors that change within an individual over time and which are independently associated with both exposure and outcome. Such time-varying confounders could considerably bias a self-controlled case-series analysis and would need to be identified and factored into any study design. For example, the decision to prescribe inhaled anti-cholinergics may be associated with periods of chronic obstructive pulmonary disease exacerbation, and such periods may be independently associated with an increased risk of the outcome under investigation (e.g. stroke). Recognition of potential time-varying confounders and sensitivity analysis excluding individuals in whom this is a possibility is integral to the approach. Another potential source of bias is death due to the adverse effect ending an observation earlier than it would have otherwise; as the self-controlled case-series method requires the assumption that the occurrence of an event does not censor, or affect, the observation period.⁸ Such bias is difficult to quantify and could be in either direction. Also as timing is crucial in a self-controlled case-series evaluation one has to pay particular attention to any factor that may introduce error; this makes studies of late-onset effects more difficult. In addition, this method is not easily amenable to a significant proportion of chronic therapies, such as those requiring lifelong administration without change or interruption e.g. subcutaneous insulin in Type I diabetes. In addition, the method will only work when the event risk is small over the observation period.³ Moreover, the method requires variability in the time [or age] of the event.⁸ The self-controlled case-series technique is also only able to provide an estimate of the relative, and not more clinically relevant, absolute incidence [though external measures could be used and the self-controlled case-series generated rate ratio applied].

What is the future?

Improvements to post-marketing drug safety assessment are already occurring through innovations and advances in data mining, signal detection and signal validation methodology. Moreover other electronic monitoring systems are increasingly being combined with other electronic database sources which allow for further large-scale drug safety monitoring. An example is the EU-ADR computerized system which has recently being developed for signal detection that exploits clinical data across Europe. Researchers then apply a variety of text mining, epidemiological and computational techniques.

Interpretation of results from different methods for evaluation of drug safety should not be viewed in isolation because each approach is complementary as each method has differing aims, strengths and weaknesses. Indeed, clearer insight may be gained by comparing, where possible, the results from each type of analysis using principles recently outlined for understanding the causal relevance of biomarkers for disease risk.⁹ Importantly, the efficiency and versatility of the self-controlled case-series method suggest that [when correctly applied and conducted] this technique could develop as an important adjunct in drug safety assessment.

Post-marketing observational studies currently remain unregistered and can be analysed in myriad ways.¹⁰ Indeed, there is no way of knowing how many analyses are attempted before a particular result is reported Although regulatory authorities have increasingly required commitments to conduct post-marketing studies as a condition of approval, many of these commitments remain unmet.¹¹ Most new drugs will subsequently require safety labelling changes as experience grows and new information emerges. For example, one in ten drugs are ultimately withdrawn or become retrospectively subject to warnings about serious or life-threatening adverse effects.¹² We believe the evidence used in making these decisions could be further improved; a recent report showed most decisions to remove a drug from the market are based on data from animal studies and spontaneous reports rather than formal clinical studies.¹³ However as most pharmacoepidemiological studies require time, rescource and expertise, they may not always be considered necessary when the risk is considered high and safer alternatives are readily available. Utilisation of controlled case-only techniques (such as the self-controlled case series method) may facilitate an increase in the number of analyses that can be undertaken independently of the drug manufacturers. Finally, we also suggest that studies in relation to drug assessment should now all be registered in the same way as clinical trials.