Core Concepts in Pharmacoepidemiology: Confounding by Indication and the Role of Active Comparators

Abstract

Confounding by indication poses a significant threat to the validity of non-experimental studies assessing effectiveness and safety of medical interventions. While no different from other forms of confounding in theory, confounding by indication often requires specific methods to address the bias it creates in addition to common epidemiological adjustment or restriction methods. Clinical indication influencing treatment prescription is patient-specific and complex, making it challenging to measure within non-experimental research. Restriction of the study population to patients with the indication for treatment would effectively mitigate confounding by indication and bring about comparability between exposure and comparator populations with respect to probability of the exposure. Active comparators are often an effective practical solution to restrict the study population in this manner when indication cannot be measured accurately. This article discusses various forms of confounding by indication, the utility of active comparators for non-experimental studies of treatment effects, and the active comparator, new user (ACNU) study design to implicitly condition on indication. Considerations for selecting active comparators and conducting an ACNU study design are discussed to enable increased adoption of these methods, improve quality of non-experimental studies, and ultimately strengthen our evidence base for intended and unintended treatment effects in relevant target populations.

Introduction

Confounding by indication, a term describing confounding in the setting of non-experimental studies of medical intervention effects, poses a significant threat to study validity.^{1 2} Treatment use is often directly driven by the anticipated risk for the outcome, severely hindering our ability to uncover true causal effects of treatments on health outcomes.^1,3 Without properly controlling confounding by indication, effect estimates will be distorted, often leading to incorrect conclusions, unnecessary or underpowered randomized trials designed based off of biased observational findings, and poor-quality evidence for regulatory and treatment decision-making.⁴

By characterizing this pervasive and challenging bias, we aim to foster understanding and detection, as well as increased application of effective mitigation strategies. To do so, we highlight the importance of active comparators to reduce potential for confounding by indication through study design, and present considerations for successful implementation of active comparators and the active comparator, new user (ACNU) study design in pharmacoepidemiology.⁵

Defining the Problem

Confounding by indication is a form of confounding in which the clinical indication for receiving the study treatment is a risk factor for the outcome of interest (Figure 1A).^1,6,7 Conceptually and mathematically, it follows the same rules as any other type of confounding.^8,9 Confounding by indication typically results from:

FIGURE 1.

Confounding by indication as depicted through causal diagrams^a,b

^a 1A) The above conceptual example illustrates the mechanism of confounding due to disease presence as indication for treatment. For further example, consider a hypothetical study comparing risk of venous thromboembolism (VTE)-related hospitalization among patients treated with direct oral anticoagulants (DOACs) vs. non-treated patients. Indications for DOAC use that are risk factors for VTE hospitalization (e.g., a prior, recent VTE diagnosis) introduce a higher risk of the outcome specifically within the treated patients; those without a prior VTE may not be treated. Confounding by indication could distort results, likely inflating the apparent risk of VTE-related hospitalization due to DOAC treatment.

^b 1B) Illustrated is a classic training example of confounding due to disease severity as indication for treatment. Asthma severity can influence prescription with beta-agonists such that patients with milder cases may not be treated at all or as frequently as severe patients, depending on the medication. Similarly, severe asthma patients are at increased risk for asthma-related adverse outcomes. Confounding by indication would therefore be introduced if asthma severity is not sufficiently controlled. (Example based off of the real-world evaluation of possible confounding by indication due to increased asthma severity among Fenoterol users. ^10,32,33)

• Presence of disease in the treated

  If the disease is a risk factor for the study outcome and the disease is treated with the study drug, then treated patients are more likely than non-treated to have the disease, and therefore are at a higher risk for adverse health outcomes associated with the disease, leading to confounding by indication (Figure 1A).

• Disease severity

  Even after conditioning on the presence of disease, differences in treatment by disease severity or subtype can introduce confounding by indication. Increased severity can increase a patient’s risk for the study outcome and also increase likelihood of treatment; less severe patients may be at a lower risk of the outcome, and correspondingly may have a lower likelihood of treatment (at all, as frequently, or at comparable dosages as patients with more severe disease).^6,10 This is also often termed ‘confounding by severity’ (Figure 1B).⁶ Similarly, distinct subtypes of a condition may require different treatment patterns for the same disease (e.g., multiple sclerosis phenotypes; systolic vs. diastolic hypertension).

• Comorbidities and related clinical factors

  Clinical factors aside from the underlying disease presence, severity level or subtype can introduce confounding by indication, depending on the exposure and outcome under study. Comorbidities, concomitant medication, or other patient-specific factors (e.g., body mass index [BMI], smoking) that influence indication for treatment should be considered to determine if they are also risk factors for the study outcome. Alternatively, clinical considerations including absolute or relative contraindications (e.g., renal disease, heart failure) should be carefully considered for confounding by (contra-)indication.

Unique challenges of confounding by indication

So why is confounding by indication a particularly challenging bias?¹¹ If we could adequately measure all reasons for treatment choices we could control for them using standard epidemiologic study design or analysis methods. Unfortunately, indication for treatment is often difficult or impossible to accurately capture due to complexity of clinical judgement for treatment decisions. Within common pharmacoepidemiology data sources such as administrative claims or other healthcare databases measuring indication is particularly challenging. Existing data sources do not typically capture reason for treatment in a structured or standardized manner, and severity of disease is difficult to assess for many conditions.² While it may be possible to measure disease, or approximate severity and comorbidity presence through use of clinical codes (e.g., International Classification of Disease codes, dispensed prescriptions), residual confounding can remain.¹² Without adequately measuring indication, controlling for confounding by indication through adjustment or restriction methods is not feasible or sufficient.^2,11 Further, even if one could sufficiently measure reasons for treatment, the indication may be too closely related to a specific medication to separate out distinct treatment effects from the underlying risk for the outcome. Lack of variability in exposure conditioning on a strong risk factor for the outcome (here: the indication) is not unique to pharmacoepidemiology and has been termed “structural confounding”.¹³

Active Comparators, and the Active Comparator, New User Study Design - a Solution

Ideally, a non-experimental study emulates a randomized trial.¹⁴ After meeting inclusion and exclusion criteria patients are randomized to receive the treatment of interest or a placebo (or control). But how is “placebo” defined for a non-experimental study design? As noted above, including patients not treated with the study drug would introduce confounding by indication because many will not have the indication for treatment, or those who do may have contraindications or mild cases compared to treated patients. Furthermore, there is no clear timeline when to begin longitudinal follow-up among the non-treated without a treatment start date. Cohort entry defined according to disease diagnosis date (a commonly proposed alternative since both cohorts have a clear date for this measure) is usually implausible, as data prior to treatment start is then included in follow-up outcome ascertainment for the treated population, resulting in immortal time bias. Differential disease duration prior to treatment or cohort entry, often an important covariate, also becomes challenging to quantify for the non-treated. These issues can be addressed through application of active comparators and the ACNU study design.^15,16

Active comparators restrict by indication

Active comparators are treatment alternatives to the study drug that are indicated for the same disease and disease severity. They enable us to indirectly restrict our population to patients with a comparable indication to the study drug.^5,15 If alternative treatments for the same indication can be identified and are assumed to be used interchangeably, i.e., prescribed with clinical equipoise, then we can address confounding by indication. Instead of comparing treated vs. non-treated, we compare patients initiating the study drug vs. the active comparator.¹⁷ Study inclusion criteria then implicitly include the indication and restrict the population to patients with the same indication for treatment, even when reason for treatment cannot be measured.¹⁷ Patients on the study treatment and the comparator are then followed longitudinally from start date of each treatment to ascertain outcomes. Including a “wash-out” period prior to treatment start excludes patients previously treated with either the study drug or active comparator (i.e., prevalent users) in order to approximate randomized clinical trial (RCT) conditions focusing on an intervention. Taken together, these components generate the established ACNU study design.^5,16

Active comparator use in non-experimental studies and the ACNU design change the research question from “Should I treat patients of indication X with the treatment of interest or not?” to “Given that a patient with indication X needs treatment, should I initiate treatment with the treatment of interest or the active comparator?”. Both questions are relevant, but only the latter can be answered in settings where indication cannot be measured precisely. This is particularly relevant for comparative effectiveness research often not covered by RCTs.¹⁸

Positivity assumption and restriction in confounding by indication

Restricting to patients with the indication for study treatment also satisfies positivity, a key assumption for causal inference in non-experimental research. Positivity requires that all study participants have a non-zero probability of being included in either exposure group.⁷ This criterion is clearly violated without restriction on indication, as non-diseased patients do not typically receive treatment.⁷

Clinical equipoise for active comparators

An ideal active comparator should be in clinical equipoise with the study treatment, meaning no risk factors for the outcome affect prescribing decisions between treatments. Under this assumption of exchangeability, treatment effect estimates will not be confounded by indication. Strictly speaking, equipoise means every physician could use the study treatment or the active comparator for any given patient. More loosely defined, equipoise means some physicians would prescribe the treatment of interest and some would prescribe the comparator to the same patient with the same indication. For the latter scenario we do need to assume, however, no systematic differences in outcomes exist between physicians prescribing the treatment of interest vs. those prescribing the comparator. The validity of this assumption may be a source of residual confounding. Note that this assumption also needs to be made when using the physicians’ prescribing preference as an instrumental variable for confounding control.¹⁹

Active comparator, new user study design

In addition to mitigation of confounding by indication, the ACNU design mitigates additional sources of bias such prevalent-user bias¹⁶, time varying hazards, timing in assessment of covariates, and immortal time bias¹⁸, although a full discussion of these biases is beyond the scope of this article. The ACNU design also inherently reduces the potential for confounding by frailty, another challenging type of confounding in non-experimental studies of treatment effects.²⁰ For these reasons, the ACNU study design has become the standard in pharmacoepidemiology.

Considerations for Active Comparator Selection and Implementation

Effectiveness of active comparators to control confounding by indication is dependent on selection of good comparators to the study treatment. While ‘good’ comparators can be subjective and challenging to identify, several considerations below can aid in the selection process (Figure 2).^5,21

FIGURE 2.

Considerations for active comparator selection within active comparator, new user designs

CKD: chronic kidney disease; HF: heart failure; PS: propensity scores.

^a The process shown is iterative such that balances in patient covariates must be re-assessed after excluding patients with observed imbalances from the population to identify whether restriction affected any prior balance or imbalance.

^b If a good active comparator is unable to be identified and alternative options are not suitable for confounding control, it must be considered whether the study should be performed given likelihood of confounding by indication.

^c Note, in addition to confounding by indication control, additional sources of bias and residual confounding in the study must be properly addressed.

Comparators must be in clinical equipoise with the study treatment (on average, not necessarily within patient subgroups treated by the same physician, for example). Necessary steps when assessing potential differences between treatments include:

1. Review treatment guidelines and solicit clinician input to understand indication, prescribing differences and additional treatment considerations, including possible site-specific, regional or calendar time differences.

   Drug utilization studies are also a useful tool when evaluating active comparator options to understand real-world treatment patterns and assess potential deviations in prescribing from the specified treatment guidelines.

2. Review of ‘Table 1’ patient characteristics. The assumption of clinical equipoise cannot be tested. However, a similar (balanced) distribution of key measured characteristics between treatment populations makes equipoise more plausible, even in the absence of information on balance of unmeasured risk factors (e.g., smoking or BMI, which are not typically available in administrative claims datasets). Standardized mean differences (SMD) or propensity scores (PS) can be used to quantitatively assess balance across numerous covariates.

   • Standardized mean differences are often used to statistically assess individual covariate balance. To assess balance over a number of covariates, we need to average the absolute values of SMDs across all variables, resulting in an average standardized absolute mean difference (ASAMD) for the population. Typical convention uses a cut-point on or around 0.1 to determine whether populations are ‘different’, for the individual variables (i.e., absolute values of SMDs) and for the ASAMD²². However, 0.1 is arbitrary and confounding is dependent on the expected impact of the variable on the outcome. Strong risk factors for the outcome may need a stricter threshold than 0.1, whereas imbalances >0.1 may be tolerated for covariates that have minimal impact on the outcome.
     Additionally, SMDs do not meaningfully help identify patient characteristics that should remain imbalanced, i.e., instrumental variables not affecting the risk for the outcome.^23,24 Any such variables would require manual review to confirm irrelevance.
     In considering comparisons, it’s useful to evaluate large differences in conditions such as chronic kidney disease or heart failure that can be indicative of disease severity differentials or contraindications. Consider excluding patients with these conditions rather than analytically adjusting for them since residual confounding by severity after adjustment is likely. While exclusion will limit generalizability, it will improve validity.
   • Discrimination statistics, including the c-statistic, can inform balance between treatments. If the prediction of treatment based on measured covariates is good (e.g., PS c-statistics over 0.85), this suggests strong imbalances exist across treatments. Conversely, a c-statistic of under 0.7 would suggest small imbalances of measured covariates. Alternatively, Walker et al. proposes a PS overlap measure to help decide whether treatments are in equipoise.²⁵

It may be difficult to identify a single active comparator to use, either because there is no alternative option (such as for statins or insulin as a drug class), or because there are multiple options (e.g., second line treatments for type 2 diabetes mellitus [T2DM]). For the former, it would be difficult to estimate a drug class effect, but may be relevant to compare treatments within-class (e.g., atorvastatin vs. rosuvastatin²⁶; insulin glargine vs. Neutral Protamine Hagedorn [NPH] insulin). For the latter, multiple active comparators could be used by implementing separate ACNU study designs. Differences in baseline characteristics and treatment effect estimates across active comparators could then be discussed and would provide a broader view and, likely, clinical applicability. It is not advised to group multiple active comparators together within a single ACNU study. Doing so often results in increasingly stringent assumptions applied to the comparators and limits generalizability because the comparator cohort is defined by a specific “mix” of comparator drug classes. Such an approach may be plausible for specific research questions in which multiple active comparators are confirmed to have similar benefit-risk profiles, and therefore the collective could represent a treatment ‘standard of care’ for comparison. However, interpretation of findings from this approach for drug selection during prescribing will be limited.

The above steps should be used iteratively. If a new active comparator is considered because of crude covariate imbalances with a previous option, “Table 1” balance should be assessed again. If a condition that strongly predicts treatment choice shows imbalance between cohorts (e.g., renal disease, heart failure), patients with that condition should be excluded, and “Table 1” balance re-assessed among the restricted population. If an active comparator with clinical equipoise can be identified using the above iterative process, PS are an efficient way to remove remaining imbalances in a large number of covariates to increase exchangeability.²⁷ A small number of important risk factors for the outcome can also be balanced by individual or frequency matching.

Additional Considerations

Non-clinical factors

Clinical indication is not the only driver of treatment variation – myriad other factors can also influence decision-making for treatment use or exposure assignment, including but not limited to frailty, insurance (including differences in access or level of coverage), variations in drug formularies within and between countries, out of pocket cost for patients, and treatment availability during the study period (e.g., market fluctuations, insurance reimbursement changes, regulatory restrictions or label changes). All issues potentially impacting variability in treatment use should be evaluated to consider whether inability to measure any such factors would bias or invalidate study findings, in which case the study may not be possible.

Data quality and availability

Data sources for study use must be evaluated to confirm they are fit for purpose to research the exposures and outcomes in question. While this is true for all non-experimental studies using existing data, it is mentioned here specifically to consider whether critical variables related to the study treatment or active comparator are limited in the data source. Are patient clinical characteristics available in the dataset to assess equipoise of comparators? Are data sufficient to define treatment wash out periods for an ACNU design? Relevant criteria to consider will differ across each unique study. It’s also important to consider sample size of study treatment and active comparators within the planned datasets. If limited data are available for the comparator (e.g., due to the drug being newly available on the market) it may not be possible to precisely estimate treatment effects. Alternatively, patients switching from the comparator drug(s) to the drug of interest can severely limit cohort size of new users of the drug of interest.

Administrative claims-based datasets are particularly useful for active comparator designs overall, and ACNU designs in particular.^2,28 Key characteristics beneficial for ACNU research include large sample sizes, longitudinal completeness, availability of clinical diagnoses, accessible prescription data, and standardized data collection and availability. Other common data sources such health register databases within Scandinavian countries²⁹ enable collection of the patient’s full treatment history (best case scenario for a new user design). Implementing an ACNU design in primary data sources or electronic health record (EHR)-based studies can be more challenging, although such studies can provide important data on confounders otherwise commonly missing from claims (e.g. BMI, laboratory values in EHR data, where available).

A Real-World Example of Active Comparator Implementation for Confounding by Indication

The below real-world example illustrates the importance and necessity of active comparators and the ACNU design to control confounding by indication, and discusses a method to evaluate equipoise amidst limitations in data availability using external datasets.

TS was asked to participate in a study assessing a potential effect of insulin glargine, a human insulin analogue, on cancer risk.³⁰ Obesity was determined a priori as a main factor influencing indication for long-acting insulin among patients with T2DM given its important role in impaired glucose control. As obesity is also a known risk factor for several cancers, confounding by indication was likely. Early assessment of data sources revealed no feasible dataset was available that was large enough to conduct a new user study design (a necessary design to control other biases in the study), and which also included BMI data for confounder adjustment. With the inability to measure BMI, a study design employing a “no insulin treatment” comparator group would be invalid and biased. Study researchers instead proposed an ACNU design using human NPH insulin as the active comparator following a comparator selection process - assuming BMI did not influence physician choice of insulin glargine vs. NPH insulin (i.e., treatments were in equipoise) (Figure 3).

FIGURE 3.

Active comparators to mitigate confounding by indication due to BMI

A1c: blood glucose; BMI: body mass index; NPH insulin: Neutral Protamine Hagedorn insulin.

^a The non-active comparator study design depicted above illustrates confounding by indication resulting from differential prescribing based on BMI. Without ability to measure and control for BMI within the dataset, the non-active comparator study comparing insulin glargine use to non-use would be confounded.

^b The active comparator study design implicitly conditions on the indication for insulin treatment. Validation of insulin prescription patterns in an external dataset containing BMI measurements confirmed that the active comparator, NPH insulin, was not prescribed differently than insulin glargine based on BMI status, thereby controlling confounding by indication due to BMI³⁰.

To evaluate this equipoise assumption, study investigators assessed BMI distribution within two external EHR validation studies to understand effect of BMI on insulin treatment choice. Use of smaller, cross-sectional EHR data that contained BMI thus enabled balance assessment of possible clinical factors influencing treatment probability. Results of these validation studies confirmed BMI did not affect treatment choice. Assuming transportability of these findings to claims data, investigators felt confident using the larger claims dataset (without BMI) to implement the ACNU study design. The use of two separate EHR datasets increased confidence in the transportability of findings. From these results, researchers were able to implicitly condition on the indication for insulin through use of the active comparator drug, in order to control for a strong driver of insulin prescription (BMI) without measuring it (Figure 3).

Limitations

Despite the utility of the ACNU design, major challenges include the availability and appropriateness of active comparators, and the possibility of residual unmeasured confounding even if all measured characteristics are balanced between treatment groups. When active comparator options are unavailable, ‘inactive comparators’ – treatments prescribed for a different indication that is not a risk factors for the study outcome and that do not affect the risk for the outcome²¹ – could be considered to help address biases in treatment comparisons. Inactive comparators effectively synchronize treatment start dates for patient follow up, although these do not address confounding by indication.²¹ Inactive comparator implementation considerations are discussed in depth in D’Arcy et al.²¹ Alternative strategies to control confounding by indication when an ACNU design is not feasible, such as instrumental variable analysis, are available and well-described, but are not the focus of this article.³¹ Similarly, numerous other forms of confounding and bias are also problematic for non-experimental studies of medical interventions and must be addressed during study design and/or analysis. As previously noted, the ACNU study design helps to control several of these additional possible biases (e.g., prevalent-user bias, confounding by frailty), although it does not account for all.

Conclusion

Confounding by indication is not conceptually different from any other form of confounding, although it can be particularly strong and difficult to address. Indication, particularly severity of disease, can be challenging or impossible to measure in data sources widely used for pharmacoepidemiologic research and therefore cannot usually be controlled for analytically. Use of active comparators and the ACNU study design can help to address confounding by indication by implicitly conditioning on the indication, including disease severity, in specific settings. If a good active comparator can be identified, the ACNU study design allows us to assess effects of medical interventions on intended outcomes, thus dramatically widening the scope of pharmacoepidemiologic research. This will not always be possible, nor work, and so potential for remaining confounding by indication will need to be carefully considered, and alternative methods explored if appropriate for the study question.¹⁸

KEY POINTS:

• Confounding by indication is a type of confounding where clinical indication influences treatment and is an independent risk factor for the study outcome.

• Indication for treatment is complex and unique to each patient. It can be challenging to measure in existing data, limiting the utility of traditional confounding control methods (e.g., adjustment, restriction).

• Active comparators that are in clinical equipoise with the study treatment effectively mitigate confounding by indication by restricting the study population to those with a similar indication for treatment.

• The active comparator, new user (ACNU) study design is a powerful method to control confounding by indication and other forms of bias in non-experimental research of treatment and safety effects.

• Residual and unmeasured confounding require careful consideration within all non-experimental studies, including ACNU designs.