An Introduction to the Fundamentals of Cohort and Case–Control Studies

INTRODUCTION

As pharmacotherapy experts, pharmacists are continually updating their knowledge about drug effects. In addition to being knowledge users of research findings, pharmacists increasingly play a larger role in observational studies of drug effects. Observational studies are inherently nonexperimental and, unlike randomized clinical trials (RCTs), do not involve any manipulation (such as randomization) of the treatment and control groups by the investigator.

This article reviews for the practising pharmacist the fundamental design elements and foundational methodologic knowledge for conducting cohort and case–control studies, 2 common and robust observational study designs for elucidating drug–outcome associations. Readers interested in learning about other observational study designs, such as cross-sectional studies, ecological studies, case series, case reports, within-person studies, and quasi-experimental designs, or the critical appraisal of such designs, are referred elsewhere.^1–6

WHY WE NEED COHORT AND CASE–CONTROL STUDIES

We need well-designed and rigorous cohort and case– control studies because their findings provide knowledge complementary to that garnered from RCTs (Table 1). The design properties of RCTs maximize their ability to estimate the potential causal effects of drugs under ideal circumstances and thereby to estimate the efficacy of those drugs. However, many RCTs involve a relatively limited number of highly selected patients and a limited duration. Indeed, RCTs typically follow patients for only a small fraction of the time that the drug would be used in clinical practice, especially when the medications are for chronic diseases. Moreover, RCTs typically exclude complex patients, they often use irrelevant comparators (e.g., placebo), and they frequently measure outcomes that are not patient-centred (i.e., surrogate end points).⁷ Although many of these limitations may be overcome by designing more pragmatic RCTs that do indeed measure effectiveness,⁸ cohort and case–control studies are 2 feasible study design alternatives that address the limitations of RCTs (Table 1) without the considerable financial and human resource costs of pragmatic RCTs.

Table 1.

Limitations of Randomized Clinical Trials (RCTs) Potentially Addressed by Cohort and Case–Control Studies

Limitation of RCT*	Complementary Aspect of Cohort and Case–Control Studies
Use of a strict study protocol that is often not representative of typical care	Usually representative of settings of routine medical care
Exclusion of key patient populations, such as children, pregnant women, and elderly people	May focus on vulnerable and under-represented populations
Limited sample size	May include large number of patients, especially if secondary data sources are used, thereby allowing rare events to be detected
Short duration	May follow patients for long periods of time (e.g., years)
Evaluation of irrelevant treatment comparisons	May compare several relevant therapies
Outcomes measured may not be important to the patient (e.g., surrogate end points)	May include any outcome that is measurable within the data source
High cost	Relatively low cost

^*These limitations apply to typical RCTs. Designing more pragmatic RCTs would also overcome many these limitations.

COHORT STUDIES

Origins

A cohort is a group of people who share a common experience or characteristic. The term “cohort” first appeared in the medical literature in the 1930s in an article by epidemiologist W H Frost.⁹ Interestingly, the word “cohort” has military roots, originating from the Latin word “cohors”.¹⁰ The term was first used in the Roman military, where a group of 300 to 600 soldiers constituted a cohort.¹¹

Design

A cohort study compares the experience of 2 or more groups of patients who are followed concurrently forward in time (Figure 1). This prospective tracking, from exposure to outcome, is in fact one of the defining features of a cohort study.¹¹ The temporal sequence involved in following a group of patients who are exposed to a certain factor (the treatment group) and a group of patients who are not exposed to that factor (the control group) is akin to that of a clinical trial, where instead of chance determining a patient’s exposure status (as occurs in an RCT), choice or happenstance determines exposure status.

Figure 1.

Schematic for the design of cohort and case–control studies.

Selecting the Study Cohort

For any cohort study, a source population must be defined, from which the eligible study cohort is derived through application of various inclusion and exclusion criteria. At a minimum, patients entering the study cohort must be free of the outcome of interest. For example, in a cohort study designed to measure the association between atypical antipsychotics and diabetes mellitus, patients with diabetes would have to be excluded from the study cohort because they are not at risk of the outcome. Often, other restrictions are put in place to minimize the risk of bias. For example, restriction to new users of a medication will ensure avoidance of multiple biases.¹² Inclusion of prevalent or current drug users can lead to significant bias because patients who experience early intolerance or adverse effects of a drug may discontinue the drug, and the remaining cohort will consist of a healthier and usually more adherent group.¹³ Risk that varies over time, whereby new users have a higher risk of an adverse event, has been observed for numerous associations, including those between nonsteroidal anti-inflammatory drugs and upper gastrointestinal bleeding,¹⁴ oral contraceptives and venous thromboembolism,¹⁵ benzodiazepines and falls,¹⁶ and angiotensin-converting enzyme inhibitors and angioedema.¹⁷

Defining Drug Exposure Groups

Once the study cohort has been created, 2 or more exposure groups must be clearly defined, 1 of which must serve as the control or reference group. The reference group should be clinically relevant. For example, in a comparative safety or effectiveness study, patients taking a drug within the same therapeutic class or receiving usual care may serve as the reference group. If clinically and scientifically relevant, a group with no therapeutic exposure may be the reference group. Drug exposure may be measured in terms of persons or person-time (the time for which a person is exposed to a particular drug). Drug exposure is often categorized in a binary fashion (i.e., yes or no), based on either a minimum number of prescription records (e.g., at least 3 records) or a specified duration of exposure (e.g., at least 90 days’ exposure), or a combination of these 2 factors (i.e., cumulative exposure). Irrespective of how exposure is defined, it is essential that follow-up time be properly categorized following entry into the cohort to avoid time-related bias.¹⁸ Furthermore, the definition of exposure should be coherent with the study hypothesis. For example, a certain amount of time or a certain dose of drug may be required to elicit an effect, or a drug may continue to have an effect once discontinued (e.g., bisphosphonates). Moreover, decisions about when to discontinue drug exposure must be made. There are 2 common approaches: “as treated”, whereby drug exposure is recorded as being stopped when a person no longer meets the definition of exposure; and “intention-to-treat”, whereby a person is considered exposed from the time of first meeting the study’s exposure definition until experiencing the outcome of interest or the end of the study, irrespective of changes in actual exposure status. There is no consensus on how to best define drug exposures, and hence the definitions of exposure often vary considerably among cohort studies assessing identical drug–outcome associations.

Measuring Occurrence of Outcomes

Complete and accurate measurement of the outcome of interest is essential to ensure the validity of study results. When subjective outcome data (e.g., diagnosis of pneumonia) are being collected during the study period, exposure status should be blinded for the outcome assessors and adjudicators, to prevent responder bias. When previously collected data (i.e., secondary data) are being used, investigators should ideally use outcome definitions that have been validated in previous studies. For example, Hux and others¹⁹ validated definitions of diabetes by comparing International Classification of Diseases codes obtained from administrative health care databases in Ontario with diagnostic data from primary care charts.

Quantifying the Drug–Outcome Association

For cohort studies, the drug–outcome association is usually expressed as a relative risk, a relative rate, or a hazard ratio. Advanced statistical techniques are used to account for factors other than the drug exposure of interest that might distort the drug–outcome association. These factors or potential confounders are often handled simultaneously with multivariable regression models. Because these statistical models account for measured variables, it is crucial that the data source capture as many potential confounding variables (or proxies of confounders) as possible. Potential confounders should usually be measured before entry into the cohort, to avoid adjustment for factors in the causal pathway.

Strengths and Weaknesses

One of the major strengths of a cohort study is that the temporal sequence—drug exposure preceding outcome—is explicit in the study design. The incidence of a particular outcome among persons exposed to a certain drug can be directly calculated using a cohort design. Cohort studies are also relatively efficient for studying rare exposures, and multiple outcomes may be assessed for a single exposure. However, cohort studies with long observation periods may be more susceptible to losses to follow-up and to inaccurate measurement of exposures and outcomes. Large numbers of patients may be required to precisely estimate meaningful drug–outcome associations, especially for rare outcomes or outcomes that take a long time to occur.

CASE-CONTROL STUDIES

Origins

The first case–control study using the design with which we are familiar today was published in 1926. However, the concept of case–control studies has its origins in the investigation of disease etiologies through detailed histories and examination of patients.²⁰

Design

In a case–control study, a number of cases and noncases (controls) are identified, and the occurrence of one or more prior exposures is compared between groups to evaluate drug–outcome associations (Figure 1). A case–control study runs in reverse relative to a cohort study.²¹ As such, study inception occurs when a patient experiences an outcome and is thus designated a “case”. A modern conceptual view holds that the case–control study can be thought of as an efficient cohort design. Essentially, patients who would have experienced the outcome of interest in a cohort study are the cases in a case–control study. Similarly, patients who were at risk but did not experience the outcome of interest in a cohort study are the controls in a case–control study. The potential data sources for a case–control study are identical with those for a cohort study, and the investigator may collect data after study inception or may use previously collected data. An extension of the case–control study is the nested case–control study, which is a case–control study conducted within a cohort. Details regarding this design are beyond the scope of this article and are reviewed elsewhere.^22,23

Selection of Cases

The first step in a case–control study is to identify the cases through application of explicitly defined inclusion and exclusion criteria. Ideally, cases should be directly sampled from the source population in a manner that is unrelated to the drug exposures of interest; however, the source population that gave rise to the cases is often unknown and difficult to identify (except in a nested-case control study, where the source population is known). The case-selection process and the data sources from which cases were selected should be described in detail, especially if cases are from a variety of sources, such as hospital and community-based sources. Selecting only hospital-based cases may lead to systematic error related to hospital admission practices, whereby exposed cases may be more likely to be admitted and therefore selected into the study (a phenomenon known as Berksonian bias). Furthermore, only new (incident) cases should be selected, as nonincident cases usually over-represent long-term survivors, and diagnostic practices may change over time, introducing potential bias. When cases are selected from a secondary data source, the case definitions should be supported by previous validation studies.

Selection of Controls

The selection of controls in a case–control study is fraught with difficulty and is often the source of significant bias. Essentially, the controls should be selected from the same source population as the cases.²⁴ In other words, controls should be at risk of becoming cases and should come from a population with the same exposure distribution as the cases. Multiple controls are usually selected for each case, to increase the statistical efficiency of the study; however, the gains are minimal beyond 3 or 4 controls per case. Nonetheless, modern case–control studies involving large databases often use much higher control–case ratios to maximize study precision. To control for potential confounding, cases and controls are often matched on one or more patient characteristics, such as age or sex (although it may not always be appropriate to match on these variables). The study investigator must be careful not to match on too many factors or on factors that are not confounders, as doing so might lead to overmatching and bias. Furthermore, matching should not involve variables that the investigator is interested in examining in association with an outcome. The selection of controls is one of the most difficult aspects of epidemiologic research, and readers are encouraged to consult additional resources.^24–28

Defining Drug Exposure Groups

Similar to the situation for a cohort study, the drug exposures of interest and their definitions should be clearly specified in the methods. Because exposure in a case–control study is determined after the cases have been identified, a period before occurrence of the case, called the “look-back period” or “look-back window”, must be defined. A comparable look-back period must be defined for the control group. Look-back periods should consider the study hypothesis and thus may vary considerably from one study to another. For example, Abdelmoneim and others²⁹ specified a 120-day look-back period before the date of their cases (patients with acute coronary syndrome) to assess recent exposure to glyburide and gliclazide. Azoulay and others³⁰ specified an exposure window of any time prior to a year before the date of cases in their study evaluating the association between pioglitazone and bladder cancer. If the investigators are collecting exposure data themselves, then outcome status should be blinded to study personnel.

Quantifying the Drug–Outcome Association

In a case–control study, the odds ratio is the usual measure of association reported. This measure is the ratio of the odds of an exposure between cases and controls and in most cases approximates the relative risk. As in a cohort study, the analytic plan for a case–control study typically involves advanced statistical methods to adjust for multiple potential confounders.

Strengths and Weaknesses

The major strengths of the case–control design are statistical efficiency (i.e., uses fewer data to quantify a drug–outcome association than would be required in a cohort study), efficiency for studying rare outcomes, efficiency for studying conditions with long latency periods, efficiency for handling the time-varying nature of drug exposures, and relatively low cost. The weaknesses of case–control studies include inefficiency for studying rare exposures, difficulty of selecting unbiased controls, and inability to directly calculate incidence rates of outcomes.

LIMITATIONS OF COHORT AND CASE–CONTROL STUDIES

Bias and Confounding

Observational studies are methodologically difficult, susceptible to bias and confounding, and difficult to interpret, given the many types of bias potentially at play. For these reasons, observational studies are limited to studying drug–outcome associations and cannot be used to measure the causal effects of drugs. Recent methodologic advances in design and analytic techniques in pharmacoepidemiology have helped to combat the various types of selection bias, information bias, and confounding at play in cohort and case–control studies (see Appendix 1, available online at www.cjhp-online.ca/index.php/cjhp/issue/view/104/showToc). Many of these techniques can account for multiple potential confounders simultaneously. A comprehensive review of these techniques is beyond the scope of this article, but such reviews may be found elsewhere,^25,31–33 Bias and confounding result in spurious drug–outcome associations and are introduced at both the design and analysis stages. Appendix 2 (available online at www.cjhp-online.ca/index.php/cjhp/issue/view/104/showToc) illustrates the origin of bias in relation to the cohort design, and Appendix 3 (available online at www.cjhp-online.ca/index.php/cjhp/issue/view/104/showToc) lists common types of bias that occur in cohort and case–control studies of drug effects.

Study of Intended Drug Effects

Cohort and case–control studies are powerful approaches for estimating the association between drugs and unintended outcomes³⁴; however, their use for studying the intended effects of drugs has spurred debate in the past and remains controversial today.^35–37 This controversy has arisen because the propensity for bias and confounding is much higher when estimating the intended effects of drugs (i.e., benefits).³⁷ This higher propensity for bias is in turn due to the nonrandom nature of prescribing practices and is often referred to as “confounding by the reason for the prescription” or simply “confounding by indication”. Confounding by indication is expected with these types of studies, as it is good medical practice to prescribe intentionally and rationally, as opposed to prescribing according to a random process.³⁸ Some authors strongly recommend against using observational studies to study intended effects, suggesting instead that we consider restricting our research questions to those of unintended effects because confounding by indication introduces uncontrollable bias.^31,34,39,40 The literature contains numerous examples of confounding by indication. A most striking example is the distorted 27-fold increased risk of thrombotic events associated with use of warfarin, when in fact warfarin prevents thrombotic events.³⁹ Another example of confounding by indication is the observed relationship between short-acting ß-receptor agonists (e.g., salbutamol) and increased risk of death from asthma.⁴¹ Of course confounding by indication is not verifiable, but it must be considered when studying the intended effects of drugs.

GENERAL CONSIDERATIONS IN CONDUCTING A COHORT OR CASE–CONTROL STUDY

Protocol and Study Team

Cohort and case–control studies aim to quantitatively estimate the association between a drug exposure and outcome. Before embarking on a cohort or case–control study, the investigators must develop a well-articulated and focused research question.⁴² Furthermore, the study protocol, including a detailed methodologic and analytic plan, should be consistent with international guidelines.^43,44 The study team should have appropriate clinical and methodologic expertise. Clinical expertise is essential for developing exposure and outcome definitions, as well as for understanding the overall clinical context of how the research question fits into the current body of knowledge. Methodologic expertise is critical for ensuring that robust methods are used, to minimize bias and confounding.

Data Sources

To estimate a drug–outcome association in a cohort or case– control study, accurate and comprehensive data must be collected on the drug exposures and outcomes of interest. Study investigators may collect data after study inception or may use previously collected data. The major advantage of prospectively collecting the data (primary data collection) is that the investigators have control over what information is collected; in contrast, when previously collected data are used (secondary data collection), the investigators are limited to the information already collected. Data may often be missing from or inaccurately recorded in secondary data sources, which creates challenges when the data are used for research purposes. Although previously collected data are considered retrospective to study inception, the data themselves are often collected prospectively; therefore, use of the terms “retrospective” and “prospective” may be misleading and usually does not provide any clarity in terms of important design characteristics.²⁵ There are 3 main sources of existing data: administrative data, medical records, and surveys. Special considerations and the advantages and disadvantages of these secondary data sources are discussed elsewhere.^45,46 For studying drug effects, secondary data sources are more commonly used than primary data collection, primarily because of gains in time, cost, and statistical efficiency. Furthermore, use of secondary data sources avoids the Hawthorne effect, whereby knowledge of participation in a study changes the behaviour of study participants and may lead to bias.

CONCLUSIONS

Pharmacists use knowledge from cohort and case–control studies to inform patients, clinicians, and the general public about drug effects. At a basic level, cohort and case–control studies quantitatively estimate the relation between exposures and outcomes. They represent rigorous study designs for answering drug safety and effectiveness questions, with case–control studies being more prone to bias. The methodologic rigour of cohort and case–control studies evaluating drug–outcome associations is advancing, and approaches are being developed and refined that limit the generation of misleading study results. Indeed, both RCTs and observational studies are necessary, and neither is sufficient to learn about the totality of drug effects in the population.

This article is the sixth in the CJHP Research Primer Series, an initiative of the CJHP Editorial Board and the CSHP Research Committee. The planned 2-year series is intended to appeal to relatively inexperienced researchers, with the goal of building research capacity among practising pharmacists. The articles, presenting simple but rigorous guidance to encourage and support novice researchers, are being solicited from authors with appropriate expertise.

Previous article in this series:

Bond CM. The research jigsaw: how to get started. Can J Hosp Pharm. 2014;67(1):28–30.

Tully MP. Research: articulating questions, generating hypotheses, and choosing study designs. Can J Hosp Pharm. 2014;67(1):31–4.

Loewen P. Ethical issues in pharmacy practice research: an introductory guide. Can J Hosp Pharm. 2014;67(2):133–7.

Tsuyuki RT. Designing pharmacy practice research trials. Can J Hosp Pharm. 2014;67(3):226–9.

Bresee LC. An introduction to developing surveys for pharmacy practice research. Can J Hosp Pharm. 2014;67(4):286–91.