“First-Wave” Bias When Conducting Active Safety Monitoring of Newly Marketed Medications with Outcome-Indexed Self-Controlled Designs

Abstract

Large health care databases are used extensively for pharmacoepidemiologic studies. Unique methodological issues arise when applying self-controlled designs (i.e., using within-person comparisons) for active surveillance of newly marketed drugs. We use 3 examples to illustrate bias related to population-level exposure time trends when using outcome-indexed self-controlled (i.e., case-crossover) designs for active surveillance and evaluate the ability of the case-time-control design to adjust for bias from population-level exposure time trends. We mimicked active surveillance by conducting sequential analyses after market entry for 3 medications and outcomes (valdecoxib for myocardial infarction (MI), aripiprazole for MI, and telithromycin for acute liver failure) using Medicaid Analytic eXtracts (from all 50 US states, 2000–2006). The case-crossover exposure odds ratio (EOR) in the months immediately following valdecoxib market entry implausibly suggested a 12-fold higher risk of MI during exposed time relative to unexposed time; among age-, sex-, and time-matched controls, the corresponding EOR of 4.5 indicated strong population-level exposure time trends. Over subsequent monitoring periods, case-crossover EORs rapidly dropped to 1.4. Adjustment for bias from population-level exposure time trends with the case-time-control analysis resulted in more consistent associations between valdecoxib and MI across sequential monitoring periods. Similar results were observed in each example. Strong population-level exposure time trends can bias case-crossover studies conducted among “first-wave” users of newly marketed medications. Suggested strategies can help assess and adjust for population-level exposure time trends.

Researchers and regulators around the world are developing collaborative networks to facilitate rapid-cycle, active surveillance of the safety of marketed medical products. Distributed data networks, such as the European Commission's EU-ADR project (1), the US Food and Drug Administration's Mini-Sentinel pilot program (2), and the Observational Medical Outcomes Partnership (3) have the capacity to leverage health care utilization records for many millions of covered individuals. Although large observational health care databases have been used extensively for pharmacoepidemiologic safety studies (4, 5), the additional methodological issues when using this type of data for active surveillance of newly marketed medications have not been fully explored.

Self-controlled approaches are a class of study designs that make comparisons within individuals over time rather than between individuals (e.g., as in cohort design). In a self-controlled analysis, only individuals with crossover in exposure contribute to the estimation of exposure effect (6). Self-controlled designs are generally applicable in studies of transient exposures and acute-onset outcomes and can be well suited for many active safety surveillance scenarios (6, 7). A key strength of this class of study designs is that, because the comparisons are within person, potential confounders that do not vary within individuals over time cannot bias the analysis, regardless of whether they are measured or unmeasured (6). However, time-varying confounders will bias analyses unless they are appropriately addressed (8–10). Self-controlled designs can be broadly categorized as exposure indexed or outcome indexed, with many variants and analytical approaches within each category (7).

Exposure-indexed self-controlled designs, such as the self-controlled case series, in which the frequency of events occurring after exposure is compared with the frequency of events at other times, have been used for many years in the Vaccine Safety Datalink, a collaborative effort between the US Centers for Disease Control and Prevention (Atlanta, Georgia) and 8 large health maintenance organizations, which focuses on active monitoring and evaluation of vaccine safety (11, 12). The exposure-indexed self-controlled approach can be biased when a preexposure reference window is used and the occurrence of the outcome affects subsequent drug use (event-dependent exposure) or censors follow-up (event-dependent censoring), or when a postexposure reference window is used and drug exposure confers a long-term effect on the risk of the outcome (6, 13). Outcome-indexed designs, such as the unidirectional case-crossover design, in which the frequency of exposure in an index window preceding the event is compared with the frequency of exposure in a reference window preceding the index window, may be more appropriate for drug exposures (6).

Outcome-indexed self-controlled designs assume that exposure probability is stationary over the sampled person-time in the absence of a causal relationship. When this assumption is met, the difference in the frequency of observed exposure during the index and reference windows reflects a causal relationship between the exposure and the outcome (14). In the context of a newly marketed medication, when there may be rapid uptake before reaching a steady state of medication initiation and discontinuation, this assumption could be violated, leading to biased results (9).

The case-time-control method was proposed to adjust for population-level trends in exposure over time (9). It estimates exposure trends at the population level by matching cases to individuals at risk for the outcome of interest on a few key variables, such as calendar time and, sometimes, age or sex. Each control person contributes person-time covering the same index and reference windows as the case to whom he or she is matched. A crossover analysis of the exposure of interest using person-time sampled from controls is intended to estimate the expected trend in exposure among the source population for the cases. However, the ability to estimate the expected trend is dependent on the identification of an appropriate source population from which to sample control person-time (10, 15).

Figure 1 depicts the index and reference windows for cases and matched controls for a fictional medication X, which does not have a triggering effect on fictional outcome Y. Period A represents the initial months after medication X enters the market, a time during which there is rapid uptake of the medication; period B represents the use of the medication after a steady state of starting and stopping has been reached in the population. Because of the rapid uptake of the new drug in the population during period A, the probability of exposure is higher during the index window than during the reference window, resulting in violation of the assumption of stationary probability of exposure over time in the population. Similar issues would arise if there were decreasing population-level trends in exposure (e.g., decreasing market share of medication of interest with entry of another drug in the same class). In the depicted example, the case-crossover estimate would be 3.0 during period A. The population-level exposure time trend estimated among age-, sex-, and calendar time–matched controls would also be 3.0, resulting in a case-time-control odds ratio of 1.0. In contrast, during period B, among age-, sex-, and calendar time–matched controls, the probability of exposure is equally likely during the index and reference windows, suggesting that there is not a population-level influence on exposure probability over time. During this period, the case-crossover and population-level exposure time trend estimates are both null, and the resulting case-time-control estimate is also null.

Figure 1.

Impact of population-level time trends in exposure on case-crossover and control-crossover estimates. Each row within periods A and B represents the exposure status during the index and reference windows for a unique case or control occurring during that period. Medicaid Analytic eXtracts, United States, 2000–2006.

In this paper, we focus on the use of the unidirectional case-crossover design in an active-surveillance setting for sequential monitoring of newly marketed medications. With the increasing focus around the world on conducting rapid-cycle, active surveillance of newly marketed medications, it becomes critically important to be aware of the strong potential for population-level time trends in exposure and the need to implement appropriate adjustment strategies. Our aim is to demonstrate the potential for bias from population-level exposure time trends within this context and to evaluate the ability of the case-time-control strategy to counter this bias when conducting rapid-cycle, sequential, self-controlled evaluations of the safety of newly marketed medications.

METHODS

Data source

We used national Medicaid Analytic eXtract data from 2000 to 2006, including claims data for patients from 50 states and Washington, DC. Medicaid is a state and federal health insurance program that provides coverage to low-income individuals in the United States (16). The Medicaid Analytic eXtract data include dates of inpatient and outpatient claims with associated diagnosis and procedure codes, as well as outpatient claims for dispensed prescription medications.

Examples

We selected the following 3 medications that were newly marketed between 2001 and 2004: valdecoxib (market entry in 2001), aripiprazole (market entry in 2002), and telithromycin (market entry in 2004). Valdecoxib, a cyclooxygenase-2 inhibitor, and aripiprazole, an atypical antipsychotic, were selected as examples in which the intended duration of therapy tends to be chronic; however, the actual duration of therapy can be quite variable. The outcome of interest for valdecoxib and aripiprazole was acute nonfatal myocardial infarction (MI), defined as the first record of a primary or secondary inpatient International Classification of Diseases, Ninth Revision, diagnosis code of 410.x0 or 410.x1 and a length of stay between 3 and 180 days after a washout period of at least 180 days. Telithromycin is an antibiotic and was chosen as an example in which the intended duration of exposure is brief (5–7 days). For telithromycin, the outcome of interest was acute hepatotoxicity, defined as the first record of an inpatient International Classification of Diseases, Ninth Revision, diagnosis code of 570, 573.3, or 782.4; an International Classification of Diseases, Ninth Revision, procedure code of 00.9x or 50.5x; or a Current Procedural Terminology code of 47135 or 47136 with a washout period of at least 180 days.

Of these 3 examples, 2 of the medications have known adverse effects, whereas the third has suspected but not demonstrated adverse effects. The relationship between telithromycin and acute liver toxicity has been described in adverse event reports, leading to a warning of hepatotoxicity on the drug label (17–19). Several observational studies have suggested small increases in the risk of hepatotoxicity associated with telithromycin use (ranging from 25% to 45% relative increases) but have all had very wide confidence intervals (20, 21). Concerns regarding valdecoxib and cardiovascular events resulted in the Food and Drug Administration removing the drug from the market (22). The literature regarding the relationship between aripiprazole and MI has been mixed (23–25). Evidence for the strength and direction of each of these drug-outcome pairs has come largely from case reports, observational cohort studies, and randomized controlled trials, in which the operational hypotheses and target estimands may be complementary to but not identical to those of a case-crossover study (e.g., “Why me?” vs. “Why now?”) (26).

The case-crossover and other self-controlled designs can be sensitive to decisions about the timing and spacing of index and reference windows (13, 27). Considerations of washout, latency, and induction periods, as well as the pharmacology of the drug under investigation are extremely important and should be used to inform these decisions for the specific exposure and outcome relationship under investigation. The primary aim of our study was to examine bias from population-level time trends across the 3 selected examples. To facilitate comparison, we applied uniform parameters defining exposure during the index and reference windows. The selected parameters do not take into account medication half-lives or expected duration of exposure effect and are likely not optimal for ascertaining causal effects. However, they are useful for illustrating the potential for bias from exposure time trends during surveillance of newly marketed medications, which can occur independently of the true underlying effect.

We also used the selected examples to explore how the intended duration of therapy can affect bias in measures of effect. We hypothesized that bias due to population-level exposure trends would be most pronounced in the early marketing period in each of our examples and would diminish more rapidly for the transiently used antibiotic, telithromycin, than for either valdecoxib or aripiprazole.

Study designs

Case-crossover

For each example, we conducted case-crossover studies that compared the exposure odds during an index window of 30 days prior to the date of outcome to the exposure odds during a reference window of 90–120 days prior to the date of the outcome.

Patients were included if they experienced the event of interest following a washout period of at least 180 days during which the patients were enrolled in the health care plan and did not have the outcome of interest (a 30-day gap in enrollment was allowed). For each of the newly marketed medications, exposure during the index or reference window was defined as a binary variable. We used the dates of dispensation and the number of days' supply dispensed (plus 7 days to allow for moderate nonadherence) to determine the number of days within the index or reference window that an individual had medication available. We then defined an individual as exposed during the index or reference window if there were, at minimum, 3 days' supply of medication available during that time window.

Case-time-control

For each example, we used 1:n variable ratio matching of cases to samples of person-time among controls. Individuals were eligible as controls for a case if they were at risk for the outcome on the case's index date and were enrolled in the health plan for at least 180 days prior to the index date (a 30-day gap in enrollment was allowed). The sampled control person-time was matched to the case on age (same year of birth), calendar time (same index date), and sex. The definitions of index windows, reference windows, and exposures were the same for cases and matched control person-time.

Sequential monitoring

To mimic active monitoring, we conducted sequential case-crossover and case-time-control analyses as if data were prospectively accruing in the database. The analyses included cases that occurred at least 120 days after the respective market entry dates for each example. We repeated analyses quarterly, using accumulated data until December 31, 2006. We required a lag of 120 days from market entry before cases were eligible to be included to ensure that it would be theoretically possible for individuals to be exposed to the newly marketed medication during the reference window (90–120 days prior to the date of the event).

Estimand

The case-crossover parameter of interest is the exposure odds ratio (EOR) for the newly marketed medication during the index period versus the reference period. The parameter of interest for the case-time-control estimate is the EOR for the newly marketed medication among cases after adjustment for the EOR for the newly marketed medication among matched control person-time.

RESULTS

In Table 1, we see the number of cases, number of matched control person-time samples, number ever exposed, and number transiently exposed to each newly marketed medication and selected reference medication. Roughly 66%, 48%, and 97% of cases who had any exposure to valdecoxib, aripiprazole, or telithromycin, respectively, during the index or reference window had crossover in exposure. Among matched control person-time, approximately 89%, 66%, and 98% of the matched control person-time with any exposure during the index or reference window had crossover in exposure.

Table 1.

Numbers of Cases and Matched Control Person-Time Samples Ever Exposed or Transiently Exposed to 3 Newly Marketed Medications of Interest During Index and Reference Windows, Medicaid Analytic eXtracts, 2000–2006

Monitoring Dates		Cases					Controls
Start	End	Total No.	No. of Ever Users	Transient Exposure, %a	Mean Age (SD), yearsb	Female, %b	Total No.c	No. of Ever Users	Transient Exposure, %a
Valdecoxib (Outcome: Acute Myocardial Infarction)
November 1, 2001	June 30, 2002	3,756	20	100	52.6 (9.1)	53.9	453,490	1,791	100
November 1, 2001	September 30, 2002	6,950	58	91	52.7 (9.0)	52.9	832,919	5,675	97
November 1, 2001	December 31, 2002	10,104	123	82	52.9 (8.9)	52.4	1,170,022	10,782	93
November 1, 2001	March 31, 2003	13,740	186	73	53.0 (8.9)	52	1,496,829	16,618	91
November 1, 2001	June 30, 2003	17,290	242	74	53.1 (9.0)	51.7	1,800,671	22,503	91
November 1, 2001	September 30, 2003	20,705	285	73	53.1 (9.0)	51.5	2,082,248	28,524	90
November 1, 2001	December 31, 2003	24,094	354	73	53.1 (9.0)	51.5	2,338,268	34,578	89
November 1, 2001	March 31, 2004	27,858	442	71	53.2 (9.0)	51.4	2,585,981	40,964	89
November 1, 2001	June 30, 2004	31,574	511	70	53.2 (9.0)	51.3	2,816,905	47,323	88
November 1, 2001	September, 30, 2004	35,140	594	66	53.2 (9.1)	51.4	3,038,481	53,860	88
November 1, 2001	December 31, 2004	38,417	667	66	53.2 (9.1)	51.4	3,240,629	60,598	88
November 1, 2001	March 31, 2005	40,221	691	65	53.2 (9.1)	51.4	3,374,886	63,754	88
November 1, 2001	June 30, 2005	42,169	711	66	53.3 (9.2)	51.5	3,556,321	65,877	88
November 1, 2001	September 30, 2005	44,141	720	66	53.3 (9.4)	51.6	3,726,586	66,665	89
November 1, 2001	December 31, 2005	46,263	720	66	53.4 (9.5)	51.6	3,878,446	66,675	89
Aripiprazole (Outcome: Acute Myocardial Infarction)
November 15, 2002	June 30, 2003	4,368	4	100	53.4 (9.1)	50.5	466,291	1,858	91
November 15, 2002	September 30, 2003	7,967	16	69	53.3 (9.1)	50.5	868,377	4,168	83
November 15, 2002	December 31, 2003	11,503	41	73	53.4 (9.1)	50.9	1,225,324	6,751	79
November 15, 2002	March 31, 2004	15,420	61	66	53.5 (9.1)	50.8	1,560,244	9,588	76
November 15, 2002	June 30, 2004	19,260	86	60	53.5 (9.0)	50.9	1,864,155	12,707	75
November 15, 2002	September 30, 2004	22,928	124	55	53.4 (9.1)	51.1	2,149,265	16,310	73
November 15, 2002	December 31, 2004	26,291	153	53	53.4 (9.1)	51.1	2,404,571	20,037	72
November 15, 2002	March 31, 2005	28,150	180	52	53.4 (9.1)	51.2	2,572,417	22,705	71
November 15, 2002	June 30, 2005	30,135	207	53	53.5 (9.3)	51.4	2,782,423	25,602	69
November 15, 2002	September 30, 2005	32,143	231	52	53.5 (9.5)	51.5	2,978,356	28,614	68
November 15, 2002	December 31, 2005	34,301	260	51	53.6 (9.7)	51.5	3,153,662	31,817	68
November 15, 2002	March 31, 2006	36,904	293	50	53.7 (9.8)	51.6	3,325,127	35,117	67
November 15, 2002	June 30, 2006	39,487	319	49	53.7 (9.9)	51.6	3,477,442	38,366	67
November 15, 2002	September 30, 2006	42,079	352	49	53.8 (10.0)	51.6	3,626,479	41,790	67
November 15, 2002	December 31, 2006	44,546	389	48	53.9 (10.1)	51.6	3,758,653	44,915	66
Telithromycin (Outcome: Acute Hepatotoxicity)
April 1, 2004	September 30, 2004	1,394	1	100	37.9 (17.9)	56.2	459,238	168	100
April 1, 2004	December 31, 2004	3,429	9	100	37.8 (18.0)	55.5	1,006,625	1,113	99
April 1, 2004	March 31, 2005	4,764	13	100	37.4 (18.1)	55.8	1,390,449	2,166	99
April 1, 2004	June 30, 2005	5,966	18	94	37.2 (18.2)	55.9	1,742,984	3,152	99
April 1, 2004	September 30, 2005	7,319	21	95	37.2 (18.3)	56.2	2,100,343	3,850	99
April 1, 2004	December 31, 2005	8,661	27	96	37.3 (18.3)	56.5	2,430,701	4,564	98
April 1, 2004	March 31, 2006	10,292	30	97	37.3 (18.3)	56.5	2,797,729	5,474	98
April 1, 2004	June 30, 2006	11,869	32	97	37.1 (18.4)	56.5	3,145,188	6,137	98
April 1, 2004	September 30, 2006	13,549	32	97	37.2 (18.5)	56.3	3,480,770	6,488	98
April 1, 2004	December 31, 2006	15,021	32	97	37.2 (18.5)	56.3	3,777,571	6,701	98

Abbreviation: SD, standard deviation.

^a Percent transient exposure among patients ever exposed during index or reference window.

^b Age and sex distribution among transiently exposed cases.

^c Total number reflects age-, sex-, and calendar time–matched person-time sampled from individuals at risk of the event. Control persons may be matched to more than 1 case if they meet eligibility criteria and are at risk for the event at the index date for the case(s) to whom they are matched.

The high proportion of cases and matched control person-time with crossover in exposure for telithromycin is not surprising because the intended duration of therapy with this antibiotic is brief. The relatively high proportion of persons who were exposed and had crossover in exposure to valdecoxib and aripiprazole over the 120-day span of the index and reference windows highlights the limited duration of treatment or lack of adherence to these medications in this population.

In the first 10 months after valdecoxib was approved and entered the market, there were 6,950 MI cases meeting eligibility criteria, of whom 58 were transiently exposed to valdecoxib during the index and reference periods prior to the MI event (Table 1). In the initial monitoring periods after market entry, the case-crossover analyses suggested more than 12-fold greater odds of MI during time exposed to valdecoxib relative to time unexposed. Although the confidence intervals were wide, the lower bound of 4.3 clearly did not include the null (Table 2). Over consecutive monitoring periods, the case-crossover EOR rapidly dropped. By the 15th monitoring period, the case-crossover estimate for valdecoxib was consistent with 1.4-times greater odds of MI during exposed time relative to unexposed time.

Table 2.

Case-Crossover, Control-Crossover, and Exposure Time Trend Adjusted Estimates, Medicaid Analytic eXtracts, 2000–2006

Monitoring Dates		Case-Crossover		Control-Crossovera		Case-Time-Control
Start	End	EOR	95% CI	EOR	95% CI	EOR	95% CI
Valdecoxib (Outcome: Acute Myocardial Infarction)
November 1, 2001	June 30, 2002	MNC		38.52	32.03, 46.31	MNC
November 1, 2001	September 30, 2002	12.00	4.33, 33.28	4.55	4.37, 4.74	2.64	0.95, 7.32
November 1, 2001	December 31, 2002	4.88	2.90, 8.23	2.65	2.59, 2.72	1.84	1.09, 3.10
November 1, 2001	March 31, 2003	3.32	2.22, 4.96	1.90	1.86, 1.94	1.75	1.17, 2.62
November 1, 2001	June 30, 2003	2.77	1.98, 3.86	1.69	1.66, 1.72	1.64	1.17, 2.29
November 1, 2001	September 30, 2003	2.65	1.95, 3.59	1.56	1.54, 1.58	1.70	1.25, 2.31
November 1, 2001	December 31, 2003	2.28	1.75, 2.98	1.53	1.51, 1.55	1.49	1.14, 1.95
November 1, 2001	March 31, 2004	1.96	1.55, 2.48	1.43	1.41, 1.45	1.37	1.08, 1.74
November 1, 2001	June 30, 2004	1.72	1.38, 2.13	1.40	1.38, 1.41	1.23	0.99, 1.52
November 1, 2001	September, 30, 2004	1.66	1.36, 2.04	1.37	1.36, 1.38	1.21	0.99, 1.49
November 1, 2001	December 31, 2004	1.62	1.34, 1.97	1.36	1.35, 1.38	1.19	0.98, 1.44
November 1, 2001	March 31, 2005	1.56	1.29, 1.88	1.27	1.26, 1.28	1.22	1.01, 1.48
November 1, 2001	June 30, 2005	1.47	1.22, 1.77	1.20	1.19, 1.21	1.23	1.02, 1.48
November 1, 2001	September 30, 2005	1.41	1.18, 1.69	1.16	1.15, 1.17	1.22	1.01, 1.46
November 1, 2001	December 31, 2005	1.41	1.18, 1.69	1.16	1.15, 1.17	1.22	1.01, 1.46
Aripiprazole (Outcome: Acute Myocardial Infarction)
November 15, 2002	June 30, 2003	MNC		5.31	4.92, 5.74	MNC
November 15, 2002	September 30, 2003	2.67	0.71, 10.05	3.14	3.00, 3.29	0.85	0.23, 3.20
November 15, 2002	December 31, 2003	3.29	1.41, 7.66	2.51	2.42, 2.60	1.31	0.56, 3.05
November 15, 2002	March 31, 2004	3.00	1.47, 6.14	2.19	2.12, 2.25	1.37	0.67, 2.81
November 15, 2002	June 30, 2004	3.33	1.75, 6.35	2.05	2.00, 2.10	1.63	0.85, 3.10
November 15, 2002	September 30, 2004	2.40	1.42, 4.04	2.02	1.98, 2.07	1.19	0.70, 2.00
November 15, 2002	December 31, 2004	2.52	1.56, 4.09	1.89	1.86, 1.93	1.33	0.82, 2.16
November 15, 2002	March 31, 2005	2.13	1.38, 3.29	1.84	1.80, 1.87	1.16	0.75, 1.79
November 15, 2002	June 30, 2005	2.33	1.55, 3.51	1.79	1.76, 1.82	1.30	0.87, 1.96
November 15, 2002	September 30, 2005	2.16	1.47, 3.17	1.74	1.71, 1.77	1.24	0.85, 1.83
November 15, 2002	December 31, 2005	2.09	1.46, 3.01	1.69	1.67, 1.72	1.24	0.86, 1.78
November 15, 2002	March 31, 2006	1.81	1.29, 2.54	1.64	1.62, 1.67	1.10	0.78, 1.55
November 15, 2002	June 30, 2006	1.80	1.30, 2.50	1.60	1.57, 1.62	1.13	0.81, 1.57
November 15, 2002	September 30, 2006	1.68	1.23, 2.28	1.55	1.52, 1.57	1.08	0.80, 1.47
November 15, 2002	December 31, 2006	1.54	1.15, 2.06	1.51	1.49, 1.53	1.02	0.76, 1.37
Telithromycin (Outcome: Acute Hepatotoxicity)
April 1, 2004	September 30, 2004	MNC		MNC		MNC
April 1, 2004	December 31, 2004	MNC		5.88	5.09, 6.80	MNC
April 1, 2004	March 31, 2005	12.00	1.56, 92.29	2.76	2.54, 3.00	4.35	0.56, 33.48
April 1, 2004	June 30, 2005	3.25	1.06, 9.97	1.75	1.65, 1.87	1.85	0.60, 5.69
April 1, 2004	September 30, 2005	1.86	0.74, 4.65	1.44	1.36, 1.53	1.29	0.51, 3.23
April 1, 2004	December 31, 2005	2.25	0.98, 5.17	1.47	1.40, 1.55	1.53	0.66, 3.52
April 1, 2004	March 31, 2006	1.90	0.88, 4.09	1.35	1.29, 1.41	1.41	0.65, 3.04
April 1, 2004	June 30, 2006	1.82	0.87, 3.79	1.21	1.16, 1.27	1.50	0.72, 3.13
April 1, 2004	September 30, 2006	1.82	0.87, 3.79	1.16	1.11, 1.21	1.56	0.75, 3.27
April 1, 2004	December 31, 2006	1.82	0.87, 3.79	1.18	1.13, 1.23	1.54	0.74, 3.22

Abbreviations: CI, confidence interval; EOR, exposure odds ratio; MNC, model did not converge.

^a Matched on age, sex, and index date of matched case.

Over the first 10 months following market entry, we selected 832,919 samples of age, sex, and calendar time from control patients at risk of acute MI matched to cases on year of birth, sex, and calendar time. Control patients could contribute to the estimate of population-level time trends for more than 1 case; however, the index/reference window sampled from the control is matched to the calendar time for the index/reference window for the respective cases to whom the control patient is matched. Among the control person-time samples with transient exposure to valdecoxib, the odds of exposure were 4.6 times greater during the index period than the reference period, with the lower bound of the 95% confidence interval at 4.4, indicating a strong population-level trend in exposure. By the 15th sequential monitoring period, the crossover estimates among the control person-time samples dropped to 1.2.

Adjustment of the case-crossover estimates for population-level trends in exposure over time (case-time-control) greatly reduced the strength of the association between brief exposure to valdecoxib and acute MI and produced estimates that were relatively consistent across the sequential monitoring periods, with confidence intervals tightening as data accrued. Overall, the case-time-control analyses suggest that the magnitude of the association between brief (<30 days) transient exposure to valdecoxib and occurrence of acute MI is relatively small (Table 2, Figure 2).

Figure 2.

Sequential estimates: case-crossover, control-crossover, and case-time-control for A) valdecoxib, B) aripiprazole, and C) telithromycin. Medicaid Analytic eXtracts, United States, 2000–2006.

Qualitatively similar results were obtained for sequential case-crossover, control-crossover, and case-time-control analyses of aripiprazole and telithromycin (Table 2, Figure 2). Although the exposure time trends observed among matched control person-time dropped precipitously for each of the selected examples, this drop was more pronounced for telithromycin than for valdecoxib and aripiprazole (Figure 3).

Figure 3.

Rapid decline of the exposure log odds ratio in a control-crossover analysis after market entry (estimating the population-level time trend in exposure). Medicaid Analytic eXtracts, United States, 2000–2006.

DISCUSSION

Self-controlled designs are useful for identifying associations between transient exposures and abrupt outcomes. Outcome-indexed self-controlled designs are particularly useful for studying outcomes related to drug exposure. However, these designs involve assumptions (such as stable probability of exposure) that must be met for unbiased estimation; these assumptions may not be met for many real-world studies conducted using observational data (9, 10, 13, 14, 28, 29). In particular, we observed that uptake of a newly marketed drug can violate the stationary probability assumption and lead to severely biased effect estimates, especially in the early marketing period.

The control-crossover estimates represent the average magnitude of bias due to population-level exposure time trends over the same period of time as the case patients to whom they are matched. When the probability of exposure increases over time in the population, adjustment of the case-crossover estimate for the exposure time trend would reduce the magnitude of a case-crossover estimate above the null and increase the magnitude for an estimate below the null. The converse would occur when the probability of exposure decreases over time in the population. In our example, the confidence intervals for the control-crossover estimates of population-level exposure time trends were tight, reflecting the large number of controls that we allowed to be matched to each case (up to 500).

The case-time-control design attempts to adjust the case-crossover estimate for population-level changes in exposure probability over time (9, 10). In 3 examples, we found that the case-time-control approach was able to reduce the bias due to the population-level exposure trend in the case-crossover estimates. Across monitoring periods, case-time-control estimates were more stable than the case-crossover estimates and closer to the null.

The spuriously high crossover estimates seen among cases and controls following market entry of each of the selected examples are due to the selection of reference windows prior to the index window and the fact that only patients with crossover in exposure contribute to the estimation of the relationship between exposure to the newly marketed agent and the outcome of interest (6, 14). Immediately after a medical product enters the market, the only people who have crossover in exposure are those who initiate use. This violates the assumption of stationary exposure probability, and unadjusted crossover estimates will be biased until the population reaches a steady state of starting and stopping (6, 14). Although the case-time-control design has been discussed extensively in the methods literature, there have been few applied studies using this design in pharmacoepidemiology (9, 15, 30, 31). However, issues related to population-level time trends in exposure are particularly pertinent and likely to arise as self-controlled study designs are implemented in settings of active surveillance of newly marketed medications where strong population-level time trends in exposure may be anticipated.

Telithromycin is an antibiotic with an intended short duration of use; the impact of this can be seen in the population-level time trends for telithromycin compared with the trends seen for valdecoxib and aripiprazole. Exposure to telithromycin was almost universally intermittent among both cases and controls over the index and reference periods. For newly marketed medical products with a brief duration of exposure or point exposures (e.g., vaccines), a steady state of starting and stopping can be reached much more quickly than for medical products that are used more chronically or with more variable duration of exposure (6).

Although the case-time control design can help estimate and adjust for population-level time trends, this approach relies on the selection of an appropriate control group in whom to estimate this trend (15, 31). One can imagine that various subpopulations prescribed a newly marketed medication might have very different trends in exposure over time than the general population. Selection of an inappropriate control group could incompletely adjust or increase the magnitude of bias.

In these examples, control person-time samples were matched on age, sex, and calendar time. These person-time samples could be matched more closely to cases on a variety of factors or on a summary disease risk score calculated prior to the reference window. However, adding further restrictions to better match the sampled person-time at risk of the outcome to the characteristics of the cases comes with a cost; even with very large data sources, these restrictions can severely limit the number of matches that can be identified.

The diagnostics produced when mimicking active surveillance of our 3 applied examples of newly marketed medications were designed to check for potential population-level exposure trends over time. In our examples, these diagnostics were able to clearly show the presence of such trends when conducting active surveillance of newly marketed medications with an outcome-indexed self-controlled design.

There are many design choices and analytical strategies available when conducting rapid-cycle active surveillance. Each may be more or less susceptible to various sources of bias. With a classic cohort study or exposure-indexed self-controlled approach, there is little need to consider population-level time trends in exposure as a source of bias. However, cohort studies use between-person comparisons rather than within-person comparisons and do not implicitly account for unmeasured, time-invariant confounders. Furthermore, cohort studies may be biased when an appropriate between-person comparison group cannot be identified. Exposure-indexed self-controlled methods can be biased when the probability of exposure within an individual is altered by occurrence of the event of interest, as might be the case for many exposure-outcome pairs. For example, the probability of receiving nonsteroidal anti-inflammatory medication is likely to differ within an individual before and after a hospitalization for gastrointestinal bleeding. Additionally, exposure-indexed self-controlled methods are generally not appropriate for outcomes that censor follow-up, such as death. Although some methods have been developed for dealing with biases from event-dependent exposures and event-dependent censoring in exposure-indexed self-controlled designs, these are complex and can involve many assumptions, as well as imputation of counterfactual exposures (8, 32).

Recognizing the assumptions, strengths, and weaknesses of self-controlled designs is essential when performing and interpreting study results. When conducting prospective, rapid-cycle, sequential surveillance, it may not be prudent to wait for a steady state to be achieved before conducting a case-crossover study if the magnitude of population-level exposure time trends can be assessed and appropriate adjustment implemented during sequential monitoring. Crossover estimates in matched control person-time during sequential monitoring can be used to help check assumptions and assess the potential magnitude of bias from population-level exposure trends and to adjust for that bias.