Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2023 Mar 1.
Published in final edited form as: Clin Trials. 2022 Nov 14;20(1):47–58. doi: 10.1177/17407745221131820

Randomized controlled trials of biomarker targets

Margret Erlendsdottir 1,2,*, Forrest W Crawford 1,3,4,5
PMCID: PMC9974557  NIHMSID: NIHMS1838524  PMID: 36373783

Abstract

Introduction:

Randomized controlled trials are used to estimate the causal effect of a treatment on a health outcome of interest in a patient population. Often the specified treatment in a randomized controlled trial is a medical intervention – such as a drug or procedure – experienced directly by the patient. Sometimes the “treatment” in an randomized controlled trial is a target – such as a goal biomarker measurement – that the patient’s physician attempts to reach using available medications or procedures. Large randomized controlled trials of biomarker targets are common in clinical research, and trials have been conducted to compare targets in the management of hypertension, diabetes, anemia, and acute respiratory distress syndrome. However, different randomized controlled trials intended to evaluate the same biomarker targets have produced conflicting recommendations, and meta-analyses that aggregate results of trials of biomarker targets have been inconclusive.

Methods:

We use causal reasoning to explain why randomized controlled rials of biomarker targets can arrive at conflicting or misleading conclusions. We describe four key threats to the validity of trials of targets: 1) intention-to-treat analysis can be misleading when a direct effect of target assignment on the outcome exists due to lack of blinding; 2) incomparability in results across trials of targets; 3) time-varying adaptive treatment strategies; and 4) Goodhart’s law, “when a measure becomes a target, it ceases to be a good measure.”

Results:

We illustrate these findings using evidence from fifteen randomized controlled trials of blood pressure targets for management of hypertension. Randomized trials of blood pressure targets exhibit substantial variation in the trial patient populations and anti-hypertensives used to achieve the blood pressure targets assigned in the trials. The trials did not compare or account for time-varying treatment strategies used to reach the randomized targets. Possible “off-target” effects of anti-hypertensive medications needed to reach lower blood pressure targets may explain the absence of a clear benefit from intensive blood pressure control.

Discussion:

Researchers should critically assess meta-analyses of trials of targets for variation in the types, distributions and off-target effects of therapies studied. Trial investigators should release detailed information about the biomarker targets compared in new randomized trials, as well as confounders, treatments delivered, and outcomes. New randomized controlled trials should experimentally compare treatment strategies incorporating biomarker results, rather than targets alone. Causal inference methodology used to address time-varying confounding should be used to compare time-varying treatment strategies in observational settings.

Keywords: Intention-to-treat, randomized controlled trial, causal inference, meta-analysis, hypertension

Introduction

Randomized controlled trials (RCTs) reveal the effect of an intervention on health outcomes of patients who receive it, compared to those who do not. Random assignment of patients to treatments ensures that observed differences in patient outcomes following receipt of different treatments can be attributed to the effect of treatment.1 Often, treatments compared in RCTs are well-defined clinical interventions, such as drugs or procedures, that are received directly by the patient. Other RCTs compare therapeutic targets – such as biomarker measurements – that the patient’s physician attempts to reach by administering drugs or procedures that may or may not be specified in the trial protocol. Randomized trials of biomarker targets measure the causal effect of assignment to a target, not the effect of any particular intervention.2

Trials of clinical targets are common in medicine and have been used to establish management strategies in chronic diseases like hypertension, diabetes, and post-operative anemia.3–5 RCTs of blood pressure targets have reported that lower systolic blood pressure targets decrease risk of poor cardiovascular outcomes, such as stroke and myocardial infarction.6–13 RCTs comparing hemoglobin A1c targets for type 2 diabetes mellitus management have shown that targets below 7% delay the development of complications due to type 2 diabetes mellitus.14–17

Guidelines based on trials of targets have generated controversy due to their conflicting recommendations.4,18 After analyzing RCTs of blood pressure targets, the American College of Cardiology and the American Heart Association recommended that physicians target a systolic blood pressure less than 130 mmHg in patients over the age of 60 with cardiovascular risk factors.6,18 The American College of Physicians and American Association of Family Physicians instead recommend targeting a systolic blood pressure of at least 140 mmHg.7 Concerns regarding the lower target arise from the possible side effects of aggressively lowering blood pressure, including increased risk of falls in vulnerable elderly patients.18,19

Established principles of causal reasoning provide a framework for understanding and analyzing clinical trials of biomarker targets. The ICH E9 “Statistical Principles for Clinical Trials”20 presents industry guidance for the design, conduct, analysis, and evaluation of clinical trials. These guidelines were clarified in an addendum “Estimands and sensitivity analysis in clinical trials.”21 The guidelines emphasize the importance of rigorously defining the “treatment of interest” in a clinical trial. The “estimand” in a trial is the treatment effect to be estimated, “how the outcome of treatment compares to what would have happened to the same subjects under alternative treatment (i.e., had they not received the treatment, or had they received a different treatment).”21 The Addendum guidance was further clarified using formal causal language by Lipkovich et al. 22.

Accordingly, in a trial of biomarker targets, the “treatment of interest” is assignment of a patient to a particular biomarker target, and the estimand is the difference in outcome when subjects are assigned to one biomarker target versus another target. Figure 1 shows a causal directed acyclic graph representing relationships between the randomized assigned biomarker target Z, treatment X, realized biomarker value B, outcome Y, and the physician and patient features L.23,24 In a trial of biomarker targets, the treatment of interest is the assigned biomarker target Z, and the estimand of interest is the difference in outcome Y under different values of target Z. The actual interventions X (e.g. medications) received by the patient are chosen at the discretion of the physician given the assignment Z and the patient and physician features L. Furthermore, treatment X can be time-varying and be modified during the trial according to post-randomization intermediate outcomes, such as new observations of the biomarker being targeted. The physician and patient features, treatments, and possibly the randomized target together determine the observed outcome. In a trial of targets, the randomized target may exert a “direct” effect on the outcome that is not mediated by the particular treatment received by the patient.

Figure 1:

Figure 1:

Open in a new tab

Causal structure of randomized trial of a biomarker assignment. The randomly assigned biomarker target is Z, which affects the treatment(s) X chosen by the physician. The health outcome Y is a consequence of treatment X and the achieved value of the biomarker target B. Baseline patient and physician features are represented by L. Because trials of biomarker targets require the treating physician to be unblinded to the target assignment Z, a possible direct effect of Z on Y is denoted by the dashed arrow.

In this article, we use principles of causal reasoning to formalize the estimands of interest in trials of biomarker targets and to demonstrate the reasons that trials of biomarker targets can produce contradictory or misleading results. First, standard “intention-to-treat” analyses of trials of biomarker targets may not measure any meaningful effect of target assignment or treatment. Second, effect estimates from randomized trials of biomarker targets may not be comparable across studies even when the biomarker targets, patient population, and outcomes are exactly the same. Third, treatment strategies used by physicians to achieve biomarker targets are adaptive and change over time, so trials of biomarker targets may not be useful for learning about treatment effects even under randomization. Fourth, intervening to alter a clinical biomarker as a therapeutic target may compromise the prognostic value of that biomarker, including the achieved target value. We present a case study of evidence from fifteen randomized trials of blood pressure targets to illustrate the pitfalls of evaluating clinical guidelines using trials of targets.

Methods

Trials of targets are analyzed using the intention-to-treat principle

The intention-to-treat (ITT) principle posits that pragmatic trials should be analyzed using the randomized assignment to treatment as the exposure of interest.25,26 The ICH E9 guidelines state that subjects in trials analyzed according to the ITT principle “should be followed-up and assessed regardless of adherence to the planned course of treatment.”21 Trials of biomarker targets are usually conducted and analyzed according to the ITT principle, but randomized assignment is to a biomarker target rather than a specific treatment. Specifically, trials of biomarker targets evaluate a “treatment policy” as described in the addendum to the ICH E9 guidelines, according to which changes in treatment following randomization are “considered irrelevant in defining the treatment of interest.”21 For example, the Systolic Blood Pressure Intervention Trial (SPRINT), an influential trial of blood pressure targets, randomly assigned patients to systolic blood pressure targets of 120 mmHg or 140 mmHg and was analyzed using an ITT approach.12 Physicians targeted these blood pressure goals by prescribing a combination of any major class of antihypertensive agents. The outcome of interest was the time to first adverse cardiovascular event. SPRINT compared patients according to their randomized target assignment without regard for factors that could have affected the outcome or effect estimate, like compliance, loss to follow up, or medication prescribed to achieve the target. SPRINT reported an estimate of the effect of assignment to a blood pressure goal on adverse cardiovascular events.

In placebo-controlled double-blind RCTs, ITT analysis has the advantage of recovering a conservative estimate of treatment effect under realistic conditions for clinical practice.2,27–30 ITT analysis estimates the effect of the treatment assignment on the outcome; practical complications like non-compliance and protocol violations are built into the estimate.27–29 Because assignment is randomized, the ITT effect is free of confounding by design, and ITT analysis does not require collection of detailed information about actual treatment received by patients. In placebo-controlled double-blind RCTs, the ITT analysis recovers the null effect if the treatment has no effect regardless of compliance, or generates a conservative estimate of effect if the treatment does have an effect due to the inability of the placebo group to experience non-compliance with the treatment.2,30 Trials of biomarker targets compare two groups receiving intervention, rather than assigning one arm to placebo. In the case of trials of targets, the reported ITT effect from a trial of targets reflects implementation of target-based guidelines in real clinical practice, in which physicians choose the medications used to reach the target for each patient.21 The estimated ITT effect of the target Z on outcome Y is unbiased due to randomization.

ITT analysis of a trial of biomarker targets may suffer from the weaknesses of ITT analysis of placebo-controlled randomized trials in general - for example, through violations of the “consistency” assumption necessary for causal inference.2,25 These violations can occur when there is hidden variation in treatment, often termed “multiple versions of treatment.”22 In the case of trials of targets, hidden variation may occur in the intervention variable X between trial arms and across trials due to differences in physician and patient features. Trials of targets may not necessarily specify treatment regimens that physicians should use to reach the assigned target.9,12,31,32 Therefore, even though randomized trials of targets return an unbiased estimate of the effect of assigned target Z, they may not provide information about the effect of particular interventions X used to reach the assigned target.

Furthermore, ITT analysis of trials of targets may not enjoy the same conservative properties as ITT analysis of placebo-controlled double-blind RCTs.33,34 Problematic scenarios arise when the assigned target exerts a direct effect, shown as a dashed arrow between assignment Z and the outcome Y in Figure 1. The direct effect Z → Y may exist in trials of therapeutic targets because physicians cannot be blinded to the assigned target; they must use their knowledge of the assigned target to specify one of several possible treatment regimens. ITT analysis amounts to comparison of the outcome Y under different assigned targets Z, ignoring the treatment regime X received by the patient and the realized biomarker value B. When the target Z has an effect on the outcome Y and the actual treatment X delivered to the patient has no effect whatsoever on the outcome, the ITT effect from a trial of targets can nevertheless be non-null. Likewise, when biomarker target assignment directly affects the outcome but not the delivered treatment, the ITT effect can still be non-null. In contrast, when the assigned target exerts no direct effect on the outcome (the Z → Y arrow is absent in Figure 1), the ITT effect from a trial of targets remains conservative and recovers the null when the treatment has no effect on the outcome. Formal proofs of these facts are given in the Supplement.

Evidence from SPRINT suggests that differential direct effects of biomarker target on outcome can arise in trials conducted to compare blood pressure targets.18 In SPRINT, the group assigned to the “intensive” treatment arm had 30% more clinic visits during the study than the control arm. Increased frequency of clinic visits may affect health outcomes through mechanisms other than management of blood pressure. For example, increased frequency of clinic visits have been associated with improved screening for malignancy, including colorectal cancer screening35, melanoma36, and hepatocellular carcinoma37, leading to identification of disease at earlier and more treatable stages. In addition, many patients increase compliance with prescribed treatments around the time of follow-up clinic visits and decrease compliance between visits.38 While this phenomenon has been observed in compliance to anti-hypertensives39, it broadly affects compliance to interventions prescribed for other conditions as well, including adherence to pre-exposure prophylaxis in patients with HIV40, continuous positive airway pressure in patients with obstructive sleep apnea41, and glucose control in diabetes.42

Trials of biomarker targets in different populations of physicians may not be comparable

Consider two hypothetical trials of systolic blood pressure targets 120 mmHg and 140 mmHg on a cardiovascular outcome. Assume that in both trials: 1) the same target biomarker levels are being compared, 2) the trials include patients of the same features, 3) randomization imposes an identical distribution on assignment to a target biomarker level, 4) the same outcome measure is used, and 5) the biological mechanism of the treatment effect on the outcome is the same within patient strata, so the outcome given treatment and stratum has the same distribution in both trials. In each trial, the investigators estimate the effect of the assigned target on the cardiovascular outcome. Each trial is internally valid: it produces an unbiased estimate of the causal effect of the systolic blood pressure target assignment on the outcome, for the particular patient and physician group under study.

Unfortunately, these two trials of the same targets may not be comparable because of differences in the physician population, even when the biological effect of active treatment is the same.2,43 While any two randomized trials may be difficult to compare due to differences in inclusion criteria and study protocols, trials of targets involve another, and often, hidden source of variation – physician behavior. The prescribing behavior of physicians in a given trial can differ across trials because trials of targets often allow physicians to use their judgment to tailor treatments to patients. For example, ACCORD and SPRINT compared the same systolic blood pressure targets: 120 mmHg vs. 140 mmHg. However, in ACCORD, 41% of patients in the intensive treatment group and 16% in the standard treatment group were taking 4 or more anti-hypertensives by the end of the trial period.9 At the end of SPRINT, 24.3% of patients in the intensive treatment group and 6.9% in the standard treatment group were taking that number.12 In other words, the same biomarker target assignment resulted in different distributions of actual treatments delivered to patients in the two trials.

Because two trials of the same targets may be incomparable, aggregating or averaging results of trials of targets via meta-analysis may be misleading. Figure 2 illustrates two hypothetical trials of the same systolic blood pressure targets in which the baseline distribution of age is the same but the distribution of treatments delivered by physicians differ across arms and between trials. Here, the treatment is binary and represents intensive versus standard regimes of anti-hypertensives. The patient populations in the trials have the same age distribution (top row), and the physicians in the two trials employ different treatment strategies by age, given the assigned systolic blood pressure target (second row). Physicians in Trial 1 are much more likely to treat patients assigned to a systolic blood pressure target of 140 mmHg using the intensive strategy than the standard strategy. The expected outcome functions (third row), representing the biological effect of interventions and risk factors, are the same in both trials. Due to the differences in patient age and the prescribing behavior of the physicians, the trial results, represented as risk differences (fourth row) are starkly divergent: Trial 1 shows a clear beneficial effect, but Trial 2 shows a null result.

Figure 2:

Figure 2:

Open in a new tab

Illustration of two hypothetical randomized trials of the same blood pressure biomarker targets in the same patient population producing different effect estimates. In both trials, the same target biomarker levels Z are compared, randomization imposes an identical distribution on the assignment Z, the same outcome measure Y is used, the patient characteristics L in both trials are the same (top row), and the outcome Y given X and L has the same distribution in both trials (third row). In each trial, the investigators estimate the effect of the assignment Z on the cardiovascular outcome Y. The only difference is the distribution of physicians’ treatments under different values of the biomarker target. The trials give starkly different results, shown as risk differences: Trial 1 shows a benefit of assignment to 140 mmHg versus 120 mmHg, while trial 2 is inconclusive.

Trials of biomarker targets employ adaptive time-varying treatments

In trials of biomarker targets, physicians deliver treatments whose types and doses are adjusted over time based on patient features, response, and biomarker target goals. Trials of biomarker targets almost always involve time-varying adaptive treatments because the target measurement can only be reached through treatments whose effects on the patient’s biomarker measurements take time to accrue. Most trials of biomarker targets do not collect detailed information on the time-varying adaptive treatment strategies that physicians use to reach an assigned target and are not useful for learning about time-varying sequences of treatment, despite randomization.

Figure 3 shows an illustration of the causal structure of a trial of biomarker targets in a simplified setting with two longitudinal time points. The randomized target Z influences treatments X1 and X2 delivered at time points 1 and 2. Measurements of the biomarker being targeted, such as systolic blood pressure, are taken at baseline (B0) and before the second treatment (B1). Together, all the biomarker measurements and treatments affect the outcome Y.

Figure 3:

Figure 3:

Open in a new tab

Illustration of the causal structure of time-varying treatments in a simplified trial of biomarker targets with two longitudinal time points. Baseline patient and physician characteristics L are omitted for simplicity. Biomarker measurements B0 and B1 are taken and treatments X1 and X2 prescribed at the two time points. Treatments are affected by prior biomarker measurements, prior treatments, and the target assignment Z. The outcome Y is measured following administration of X2, and is a function of all prior biomarker measurements and treatments. Because B1 is simultaneously a causal consequence of X1 and a cause of X2, special adjustment approaches are required to estimate the effect of (X1, X2) on Y.

Different estimands may be considered to evaluate the effects of the randomized target and the time-varying adaptive treatment strategies used to reach the target.22 First, the ITT effect of the randomized target on the outcome is still unconfounded and estimable, even if the biomarker measurements and treatments are unobserved. However, we may wish to learn about the joint effect of the randomized target Z and the treatment strategy (X1, X2) on the outcome Y to determine the optimal strategy for achieving the target. Estimating the effect of the treatment strategy (X1, X2) would require observing both the treatment strategy (X1, X2) and the biomarker measurements (B0, B1) at each time point and estimating the effect of the treatments on the outcome. Because biomarker B1 is both a cause of the outcome (a confounder), and a consequence of prior treatments and biomarker values (a post-treatment variable), B1 is a “time-varying confounder” and causes simple adjustment approaches to fail to recover the causal effect of the sequence of treatments (X1, X2) on the outcome.44,45

Goodhart’s law may explain contradictory findings in trials of biomarker targets

The economist Charles Goodhart observed that “any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.”46 The adage was later reformulated in its now-famous form, “when a measure becomes a target, it ceases to be a good measure.”47 In the context of clinical medicine, we might rephrase Goodhart’s law as follows:

When the population distribution of a biomarker is forced to collapse onto a target value, the biomarker may cease to be a satisfactory predictor of the clinical endpoint of interest.

In observational (i.e. routine clinical) settings, lower (or higher) biomarker measures may be associated with better prognosis in a particular group of patients, and so the biomarker naturally becomes a target to be controlled. But the interventional means by which a target biomarker measurement is reached need not always improve prognosis, because the causal mechanism by which the intervention acts on the outcome need not pass solely through the biomarker.48

Figure 4 shows an illustrative example. At left, observational evidence suggests that the biomarker B (such as systolic blood pressure) is associated with an outcome Y (such as a cardiovascular event) because the risk of the adverse outcome appears to increase with the value of the biomarker. At right, a randomized trial is constructed to compare intensive and standard biomarker target (Z) values for control. Suppose that the risk of the outcome is lower under the intensive target when treatment X is held constant. However, reaching the intensive target requires delivering a different level of treatment. This level of treatment is itself associated with a higher risk of adverse outcomes. Thus, the trial reports the erroneous finding that “assignment to the intensive target results in higher risk than assignment to the standard target.” This misconception arises because the risk of the adverse outcome under intensive treatment x1 is higher than under standard treatment x0, even though for any fixed treatment x, patients assigned to b0 fare worse than those assigned to b1. A formal characterization of Goodhart’s law in trials of biomarker targets is given in the Supplement.

Figure 4:

Figure 4:

Open in a new tab

Illustration of Goodhart’s law when the value of a hypothetical biomarker B is used as a target for control. In the observational setting at left, larger values of B are associated with worse outcome Y. Based on this evidence, biomarker target values B = b0 and B = b1 are chosen to be tested in an RCT. At right, a trial of the randomized biomarker target assignment Z on the same outcome Y now involves the physician’s choice of treatment X and the realized biomarker value B. Successful targeting means that physicians choose treatments X = x0 and X = x1 to achieve B = b0 and B = b1 respectively. The trial reports the risks evaluated at X = x0, B = b0 versus X = x1, B = b1, leading to the seemingly contradictory result that lower values of the biomarker cause worse outcome. This conclusion is false because the risk under B = b0 dominates the risk under B = b1 for every treatment X = x.

The direct effect of the intervention X on the outcome Y may exist in trials of targets because of off-target (possibly adverse) effects of X. For example, thiazide diuretics and angiotensin converting enzyme inhibitors are first-line anti-hypertensives commonly used to manage hypertension requiring pharmacologic control.6 Thiazide diuretics are known to have a risk profile that includes dose-dependent metabolic derangements including hyperglycemia, hyperglycemia, hypokalemia and hyponatremia.49 Angiotensin converting enzyme inhibitors often lead to a reduction in kidney function, measured by reductions in glomerular filtration rate, and may also cause hyperkalemia and worsened renal artery stenosis in those with risk factors.50

Results

Case study: Trials of blood pressure targets

These four threats to the validity of trials of biomarker targets may apply to recent trials of blood pressure targets. Clinical guidelines for management of hypertension published by the ACC/AHA in 2017 are based on meta-analysis of fifteen randomized trials of blood pressure targets. Based on this meta-analysis, the ACC/AHA guidelines recommend physicians target a systolic blood pressure of 130 mmHg in patients over 60 years old with cardiovascular risk factors. However, these guidelines have remained controversial due to concerns about increased risk of falls due to hypotension and adverse renal events due aggressive anti-hypertensive regimens.18,51–53

We constructed a dataset based on the fifteen trials included in the meta-analysis of blood pressure targets by Reboussin et al. 3. The trials, key inclusion and exclusion criteria, the targets compared, the primary outcome of interest, and other trial features are summarized in the Supplement. We calculated pooled estimates of the mean and standard deviation of the age and baseline systolic and diastolic blood pressure for each trial population. We also calculated relative risks and 95% confidence intervals for each outcome using the number of events reported by each study, summarized in Table 4 of the Appendix.3 Figure 5 shows the relative risk of all-cause mortality, major adverse cardiovascular events, myocardial infarction, and stroke reported by each of the trials between the trial arms assigned to the “intensive” versus “standard” blood pressure targets. Although the meta-analysis by Reboussin et al. 3 concluded that intensive blood pressure control was beneficial, most of the individual trials reported null results.

Figure 5:

Figure 5:

Open in a new tab

Relative risks (on a logarithmic scale) of four major outcomes for trials included in the ACC/AHA meta-analysis between the trial arms assigned to “intensive” versus “standard” blood pressure targets3: all-cause mortality (black), stroke (red), major adverse cardiovascular events (orange), and myocardial infarction (blue). All trials compared higher versus lower blood pressure targets. The trials marked with an asterisk (*) included at least one trial arm targeted to a systolic blood pressure lower than 130 mmHg. The aggregated results reported by the ACC/AHA meta-analyses are included in the last two entries. Trials are sorted by the magnitude of their average relative risk across the four outcomes (smallest to largest).

The trials exhibit substantial variation in the trial patient populations. Three of the trials – ABCD-N, ACCORD, and UKPDS – included only subjects with diabetes (ABCDN, ACCORD, UKPDS). Two trials, Cardio-Sis and SPRINT, excluded all diabetics. Three trials – AASK, MDRD, and REIN-2 – included only subjects with nephropathies or chronic kidney disease, and HALT-PKD included only subjects with autosomal dominant polycystic kidney disease. The average age of subjects included in the trials ranged from 36.6 to 76.6 years. The average baseline systolic blood pressure of subjects ranged from 126.7 to 169.5 mmHg. Only four of the fifteen studies were composed of predominantly women. Eight trials reported the percentage of non-white subjects included in the trial, which ranged from all white subjects (Cardio-Sis54) to all African-American subjects (AASK55). We conducted principal components analysis to investigate the trial features - the mean age, baseline systolic blood pressure, percentage of women, and the systolic blood pressure targets compared – and the relative risks of each endpoint. The results, shown in the Supplement, illustrate substantial heterogeneity in patient and trial features. ITT analysis of individual trials summarizes the effect of assignment to the intensive blood pressure target on the outcome, but the trials may not be comparable due to differences in patient populations. Aggregation of trial results via meta-analysis is likewise problematic because the trial patient and physician populations may be incomparable.

The trials did not compare or account for the time-varying treatment strategies used to reach the randomized targets. The limited information about medications delivered revealed variety across trials in the anti-hypertensives used by physicians. The frequency of usage of different classes of anti-hypertensives are summarized in the Supplement. The percentage of patients on an angiotensin converting enzyme inhibitor, diuretic, beta-blocker, or calcium channel blocker ranged from 2.1–99.6%, 8.9–54.9%, 5–92%, and 7–79% respectively. Furthermore, treatment choices were often left to the discretion of treating physicians. ACCORD, MDRD, SPS3, and SPRINT encouraged treatment according to previously established guidelines but otherwise did not specify either first-line drugs or randomize use of particular drugs9,12,31,32. AASK, HOMED-BP, and Wei et al. randomized both blood pressure target and use of a combination of angiotensin converting enzyme inhibitor, angiotensin receptor blocker, beta-blocker, diuretics, or calcium channel blocker as the first-line anti-hypertensive.11,55,56

Possible “off-target” effects of anti-hypertensive medications needed to reach a lower blood pressure target may explain the absence of a clear benefit from intensive blood pressure control. In most trials, the intensive treatment group targeted to a lower blood pressure required higher doses of anti-hypertensives to achieve the blood pressure goal. SPRINT reported an increased frequency of acute kidney injury, hypotension, syncope, electrolyte abnormalities and kidney failure in the treatment arm, targeted to a systolic blood pressure of 120 mmHg.12 ACCORD reported a decrease in estimated glomerular filtration rate and an increase in macroalbuminuria in the group targeted to a systolic blood pressure of 120 mmHg, interpreted as “signals of possible harm [. . . ] but the implications of these changes [. . . ] are uncertain.”9 The limited data reported by the trials on the frequency of adverse events caused by intensive blood pressure lowering makes it difficult to investigate the possible side effects of aggressive blood pressure management. The estimate of overall effect reported by each trial of targets incorporates both the potential benefit of reaching a lower blood pressure target and the potential harm arising from off-target effects of larger doses of anti-hypertensives. These differing effects have not been sufficiently disentangled by trials of targets, or a meta-analysis of these trials.

Discussion

Trials of biomarker targets, analyzed using the intention to treat principle, are commonly used in medicine to generate clinical guidelines. We have demonstrated that trials of biomarker targets have several possible weaknesses, and guidelines based on evidence from trials of targets should be interpreted carefully. We present several recommendations for recognizing, designing, and analyzing trials of biomarker targets.

Identifying and interpreting trials of targets

Trials of biomarker targets have several identifying features. First, trials of targets randomize assignment to a target, such as blood pressure, heart rate, blood oxygenation saturation, hemoglobin levels, or hemoglobin A1c levels, that is reached through assignment to a variety of different treatment mechanisms. Trials that assess the effects of therapies that affect a particular biomarker, but in which randomization is of the therapy itself rather than a biomarker target, are not trials of targets. For example, randomized trials of non-high-intensity versus high-intensity statin therapy for lowering blood cholesterol are not trials of targets. Consumers of evidence generated by trials of targets should be alert to the possibility that assignment to a biomarker target may exert a direct causal effect on the outcome (as shown by the dashed Z → Y arrow in Figure 1) because these trials may not return a null result even when the treatments used in the trial have no effect whatsoever on the outcome (Supplementary Materials Section 1.2). A positive result from such a trial could lead to unnecessary increases in medication burden and harms due to polypharmacy.

Second, we suggest that researchers critically assess meta-analyses of trials of targets for variation among included trials in the types and distributions of therapies studied. Because trials of targets may not report detailed information about treatments used to reach the target biomarker values, variation between the trials due to differing distributions of medication regimens may not be explicitly discussed or appropriately modeled using standard methods in meta-analysis (Figure 2).

Third, we recommend that researchers consider differential profiles of off-target effects for medications that are used to reach a biomarker target. As Goodhart’s law suggests, off-target effects of medications can create paradoxical relationships between targets and outcomes (Figure 4 and Supplementary Materials Section 2), where the apparent results of randomized trials of targets can be at odds with the results of observational studies that provided the original motivation for the trials.

Improving the design of trials of targets

Generating better evidence to guide clinical practice is possible through more granular trial data reporting, better trial design, and use of modern statistical methods for causal inference. Differences in the distribution of baseline patient and physician behavior can interfere with aggregating trials of targets into coherent clinical guidelines. Though trial investigators may not be able to release patient-level records due to privacy considerations, they should aim to report as much information as possible about the joint distribution of the target, confounders, treatments delivered, and outcomes. Summaries for each stratum of patients should be standardized when possible across trials, where stratification occurs on patient features and the types of interventions utilized in the trial. Stratified estimates of effect could be aggregated across trials if the strata were defined in the same way. If the data reported by the trial is sufficient to characterize the joint distribution of treatments, patient features, and outcomes in each arm of the trial, meta-analytic regression-based methods can adjust for relevant differences between trials by conditioning on treatments and patient features.

Treatment strategies incorporating biomarker results, rather than biomarker targets alone, should be compared experimentally whenever possible. Comparison of high-intensity and low-intensity interventions on a biomarker is possible in clinical trials if the treatment algorithm is fully defined at the beginning of the study. This algorithm could take into account measurements of the biomarker that arise during the course of the trial. For example, rather than randomizing patients to a specific blood pressure target, researchers could generate two treatment algorithms representing high-intensity and low-intensity interventions on blood pressure. Such an algorithm would prescribe in advance the class and dose of medication that should be used in the trial in enough detail that physicians could reasonably implement the algorithm in the trial population. The intensity of treatment would be reflected in the class, dose, and number of medications prescribed rather than the target being reached. Furthermore, the algorithm should specify the strategy that physicians should use to adjust treatment if the prescribed class or dose of medication is not well-tolerated or is insufficient to manage a patient’s hypertension. In the case of hypertension, there exist multiple trials of first-line anti-hypertensive medications, and current guidelines recommend initiation of thiazide diuretics, angiotensin-converting enzyme inhibitors, and calcium channel blockers.6 Observational studies can also provide information that guides treatment choices. A large observational study of 4.9 million patients, LEGEND-HTN, compared monotherapies for incident hypertension, concluding that thiazide and thiazide-like diuretics were superior to other therapies.58 Future studies could build on this work by additionally specifying adjustment strategies after initiation of treatment for comparison.

Improving the analysis of trials of targets

Causal inference methodology can be used to address time-varying confounding (Figure 3), allowing for comparison of the time-varying strategies used to adjust anti-hypertensive medications following initiation of treatment. It is sometimes possible to estimate the effect of a time-varying treatment even when the sequence of treatments is not sequentially randomized. Under certain assumptions, the g-formula and marginal structural models can be used to adjust for time-varying covariates to evaluate the effects of time-varying treatment strategies when detailed information about intermediate clinical measurements, outcomes, and treatments delivered to patients is available.59 The g-formula decomposes the average potential outcome under a given treatment regimen into a weighted average of stratum-specific expected treatment effects, conditional on both the treatment and covariate history of the patient.60 Marginal structural models use inverse probability of treatment weights to construct a “pseudo-population” in which the treatment of interest is randomly assigned at every time point.61 The g-formula and marginal structural models use adjustment and weighting strategies to emulate randomization of the sequence of treatments, so that groups assigned different treatments are comparable.59

The accumulation of large electronic health record databases raises the possibility of using statistical methods to adjust for time-varying confounding in observational settings to inform clinical guidelines. Inverse probability weighting61 and g-formula62–64 approaches have been used to estimate the effect of adaptive time-varying treatments in large observational datasets. For example, Zhang et al. 65 used electronic health records to compare erythropoietin dosing strategies targeting low, middle, and high hematocrit targets in patients with end-stage renal disease. A comparison of four systolic blood pressure targets between 120mmHg and 150 mmHg using the g-formula by Johnson et al. 66 found an association between targeting a systolic blood pressure of 120 mmHg and a decrease in adverse cardiovascular events. However, further work is necessary to compare the time-varying treatment strategies used to achieve these targets.

Supplementary Material

suppl_material

Acknowledgements:

This work was supported by National Institute of Health grant NICHD DP2 HD091799–01. We are grateful to P. M. Aronow, Gregg S. Gonsalves, Amy Justice, Fan Li, Winston Lin, Haidong Lu, Aldo Peixoto, Janet Tate, and Thomas A. Thornhill for helpful comments.

Footnotes

Conflicts of interest: We have no conflicts of interests to report.

All data are taken from previously published manuscripts. Data and code are available to be shared upon request.

References

  • [1].Morgan Kari Lock and Rubin Donald B. Rerandomization to improve covariate balance in experiments. The Annals of Statistics, 40(2):1263–1282, 04 2012. [Google Scholar]
  • [2].Hernán Miguel A and Hernández-Díaz Sonia. Beyond the intention-to-treat in comparative effectiveness research. Clinical Trials, 9(1):48–55, 2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [3].Reboussin David M., Allen Norrina B., Griswold Michael E., Guallar Eliseo, Hong Yuling, Lackland Daniel T., Miller Edgar (Pete) R., Polonsky Tamar, Thompson-Paul Angela M., and Vupputuri Suma. Systematic review for the 2017 ACC/AHA/AAPA/ABC/ACPM/AGS/APhA/ASH/ASPC/NMA/PCNA guideline for the prevention, detection, evaluation, and management of high blood pressure in adults: A report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Hypertension, 71(6):e116–e135, 2018. [DOI] [PubMed] [Google Scholar]
  • [4].Ikeda Masayuki and Shimazawa Rumiko. Challenges to hemoglobin A1C as a therapeutic target for type 2 diabetes mellitus. Journal of General and Family Medicine, 20(4):129–138, 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [5].Jeffrey L Carson Frederick Sieber, Donald Richard Cook Donalrd Hoover, Noveck Helaine, Chaitman Bernard, Fleisher Lee, Beaupre Lauren, Macaulay william, Rhoads George G, Paris Barbara, Zagorin Aleksandra, Sanders David W, Zakriya Khwaja J, and Magaziner Jay. Liberal versus restrictive blood transfusion strategy: 3-year survival and cause of death results from the FOCUS randomised controlled trial. The Lancet, 385(9974):1183–1189, 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [6].Whelton Paul K., Carey Robert M., Aronow Wilbert S., Casey Donald E., Collins Karen J., Cheryl Dennison Himmelfarb Sondra M. DePalma, Gidding Samuel, Jamerson Kenneth A., Jones Daniel W., MacLaughlin Eric J., Muntner Paul, Ovbiagele Bruce, Smith Sidney C., Spencer Crystal C., Stafford Randall S., Taler Sandra J., Thomas Randal J., Williams Kim A., Williamson Jeff D., and Wright Jackson T.. 2017 ACC/AHA/AAPA/ABC/ACPM/AGS/APhA/ASH/ASPC/NMA/PCNA guideline for the prevention, detection, evaluation, and management of high blood pressure in adults: A report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Hypertension, 71(6):e13–e115, 2018. [DOI] [PubMed] [Google Scholar]
  • [7].Qaseem A, Wilt TJ, Rich R, Humphrey LL, Frost J, Forciea MA, and the Clinical Guidelines Committee of the American College of Physicians and the Commission on Health of the Public and Science of the American Academy of Family Physicians. Pharmacologic treatment of hypertension in adults aged 60 years or older to higher versus lower blood pressure targets: A Clinical Practice Guideline From the American College of Physicians and the American Academy of Family Physicians. Annals of Internal Medicine, 166:430–437, 2018. [DOI] [PubMed] [Google Scholar]
  • [8].Schrier Robert W., Estacio Raymond O., Esler Anne, and Mehler Philip. Effects of aggressive blood pressure control in normotensive type 2 diabetic patients on albuminuria, retinopathy and strokes. Kidney International, 61(3):1086–1097, 2002. [DOI] [PubMed] [Google Scholar]
  • [9].ACCORD. Effects of intensive blood-pressure control in type 2 diabetes mellitus. New England Journal of Medicine, 362(17):1575–1585, 2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [10].Schrier Robert W., Abebe Kaleab Z., Perrone Ronald D., Torres Vicente E., Braun William E., Steinman Theodore I., Winklhofer Franz T., Brosnahan Godela, Czarnecki Peter G., Hogan Marie C., Miskulin Dana C., Rahbari-Oskoui Frederic F., Grantham Jared J., Harris Peter C., Flessner Michael F., Bae Kyongtae T., Moore Charity G., and Chapman Arlene B.. Blood pressure in early autosomal dominant polycystic kidney disease. New England Journal of Medicine, 371(24):2255–2266, 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [11].Asayama Kei, Ohkubo Takayoshi, Metoki Hirohito, Obara Taku, Inoue Ryusuke, Kikuya Masahiro, Thijs Lutgarde, Staessen Jan A, and Imai Yutaka. Cardiovascular outcomes in the first trial of anti-hypertensive therapy guided by self-measured home blood pressure. Hypertension Research, 35: 1102–1110, 2012. [DOI] [PubMed] [Google Scholar]
  • [12].SPRINT. A randomized trial of intensive versus standard blood-pressure control. New England Journal of Medicine, 373(22):2103–2116, 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [13].UKPDS. Tight blood pressure control and risk of macrovascular and microvascular complications in type 2 diabetes: UKPDS 38. BMJ, 317(7160):703–713, 1998. [PMC free article] [PubMed] [Google Scholar]
  • [14].Seidu S, Achana FA, Gray LJ, Davies MJ, and Khunti K. Effects of glucose-lowering and multifactorial interventions on cardiovascular and mortality outcomes: A meta-analysis of randomized control trials. Diabetic Medicine, 33(3):280–289, 2016. [DOI] [PubMed] [Google Scholar]
  • [15].REICHARD P, BERGLUND B, BRITZ A, CARS I, NILSSON BY, and ROSENQVIST U. Intensified conventional insulin treatment retards the microvascular complications of insulin-dependent diabetes mellitus (IDDM): the Stockholm Diabetes Intervention Study (SDIS) after 5 years. Journal of Internal Medicine, 230(2):101–108, 1991. [DOI] [PubMed] [Google Scholar]
  • [16].The Diabetes Control and Complications Trial Research Group. The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. New England Journal of Medicine, 329(14):977–986, 1993. [DOI] [PubMed] [Google Scholar]
  • [17].Ohkubo Yasuo, Kishikawa Hikedi, Araki Eiichi, Miyata Takao, Isami Satoshi, Motoyoshi Sadatoshi, Kojima Yujiro, furuyoshi Naohiko, and Shichiri Motoaki. Intensive insulin therapy prevents the progression of diabetic microvascular complications in Japanese patients with non-insulin-dependent diabetes mellitus: a randomized prospective 6-year study. Diabetes Research and Clinical Practice, pages 103–117, April 1995. [DOI] [PubMed] [Google Scholar]
  • [18].Kaul Sanjay. Are guidelines for treatment of hypertension trustworthy? Journal of the American College of Cardiology, 73(23):3027–3030, 2019. [DOI] [PubMed] [Google Scholar]
  • [19].Kaul Fabio, Reboldi Gianpaolo, and Verdecchia Paolo. Hypertension and the J-curve phenomenon: Implications for tight blood pressure control. Hypertension Research, 36:109–111, 2013. [DOI] [PubMed] [Google Scholar]
  • [20].E9 statistical principles for clinical trials. U.S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research, Center for Biologics Evaluation and Research, September 2018. [Google Scholar]
  • [21].E9(R1) Statistical Principles for Clinical Trials: Addendum: Estimands and Sensitivity Analysis in Clinical Trials. U.S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research, Center for Biologics Evaluation and Research, May 2021. [Google Scholar]
  • [22].Lipkovich Ilya, Ratitch Bohdana, and Mallinckrodt Craig H.. Causal Inference and Estimands in Clinical Trials. Statistics in Biopharmaceutical Research, 12:54–67, January 2020. [Google Scholar]
  • [23].Pearl Judea. Causal inference in statistics: An overview. Statistics Surveys, 3:96–146, 2009. [Google Scholar]
  • [24].VanderWeele Tyler J. and Robins James M.. Signed directed acyclic graphs for causal inference. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 72(1):111–127, 2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [25].Sheiner Lewis B and Rubin Donald B. Intention-to-treat analysis and the goals of clinical trials. Clinical Pharmacology & Therapeutics, 57(1):6–15, 1995. [DOI] [PubMed] [Google Scholar]
  • [26].Gupta Sandeep K. Intention-to-treat concept: A review. Perspectives in Clinical Research, 2(3): 109, 2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [27].Wertz Robert T. Intention to treat: Once randomized, always analyzed. Clinical Aphasiology, 23: 57–64, 1995. [Google Scholar]
  • [28].Heritier Stephane R, Gebski Val J, and Keech Anthony C. Inclusion of patients in clinical trial analysis: The intention-to-treat principle. Medical Journal of Australia, 179(8):438–440, 2003. [DOI] [PubMed] [Google Scholar]
  • [29].Montori Victor M and Guyatt Gordon H. Intention-to-treat principle. Cmaj, 165(10):1339–1341, 2001. [PMC free article] [PubMed] [Google Scholar]
  • [30].Rosenberger William F and Lachin John M. Randomization in Clinical Trials: Theory and Practice. John Wiley & Sons, 2015. [Google Scholar]
  • [31].Klahr Saulo, Levey Andrew S., Beck Gerald J., Caggiula Arlene W., Hunsicker Lawrence, Kusek John W., and Striker Gary. The effects of dietary protein restriction and blood-pressure control on the progression of chronic renal disease. New England Journal of Medicine, 330(13):877–884, 1994. [DOI] [PubMed] [Google Scholar]
  • [32].SPS3. Blood-pressure targets in patients with recent lacunar stroke: The SPS3 randomised trial. The Lancet, 382(9891):507–515, 2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [33].Goetghebeur Els and Loeys Tom. Beyond intention to treat. Epidemiologic Reviews, 24(1):85–90, 2002. [DOI] [PubMed] [Google Scholar]
  • [34].Shrier Ian, Steele Russell J, Verhagen Evert, Herbert Rob, Riddell Corinne A, and Kaufman Jay S. Beyond intention to treat: What is the right question? Clinical Trials, 11(1):28–37, 2014. [DOI] [PubMed] [Google Scholar]
  • [35].Halm Ethan A, Beaber Elisabeth F, McLerran Dale, Chubak Jessica, Corley Douglas, Rutter Carolyn M, Doubeni Chyke A, Haas Jennifer S, and Balasrubramanian Bijal A. Association between primary care visits and colorectal cancer screening outcomes in the era of population health outreach. Journal of General Internal Medicine, 31(10):1190–1197, 06 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [36].Roetzheim Richard G., Lee Ji-Hyun, Ferrante Jeanne M., Gonzalez Eduardo C., Chen Ren, Fisher Kate J., Love-Jackson Kymia, and McCarthy Ellen P.. The Influence of Dermatologist and Primary Care Physician Visits on Melanoma Outcomes Among Medicare Beneficiaries. The Journal of the American Board of Family Medicine, 26(6):637–647, 2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [37].Wong Carrie R., Garcia Ruel T., Trinh Huy N., Lam Khoa D., Ha Nghi B., Nguyen Huy A., Nguyen Khanh K., Levitt Brian S., and Nguyen Mindie H.. Adherence to Screening for Hepatocellular Carcinoma Among Patients with Cirrhosis or Chronic Hepatitis B in a Community Setting. Digestive Diseases and Sciences, 54:2712–2721, October 2009. [DOI] [PubMed] [Google Scholar]
  • [38].Cramer Joyce A., Scheyer Richard D., and Mattson Richard H.. Compliance Declines Between Clinic Visits. Archives of Internal Medicine, 150(7):1509–1510, 07 1990. [PubMed] [Google Scholar]
  • [39].Brixner Diana I., Carrie McAdam-Marx Xiangyang Ye, Lau Helen, and Munger Mark A. Assessment of time to follow-up visits in newly-treated hypertensive patients using an electronic medical record database. Current Medical Research and Opinion, 26(8):1881–1891, 2010. [DOI] [PubMed] [Google Scholar]
  • [40].Blair Cheré S, Beymer Matthew R, Kofron Ryan M, Bolan Robert K, Jordan Wilbert C, Haubrich Richard H, Wohl Amy R, and Landovitz Raphael J. PrEP Nonadherence, White Coat Dosing, and HIV Risk Among a Cohort of MSM. Open Forum Infectious Diseases, 7:1–4, August 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [41].Budhiraja Rohit, Kushida Clete A., Nichols Deborah A., Walsh James K., Simon Richard D., Gottlieb Daniel J., and Quan Stuart F.. Impact of Randomization, Clinic Visits, and Medical and Psychiatric Cormorbidities on Continuous Positive Airway Pressure Adherence in Obstructive Sleep Apnea. Journal of Clinical Sleep Medicine, 12:333–341, March 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [42].Zueger Thomas, Gloor Manuel, Lehmann Vera, Melmer Andreas, Kraus Mathias, Feuerriegel Stefan, Laimer Markus, and Stettler Christoph. White coat adherence effect on glucose control in adult individuals with diabetes. Diabetes Research and Clinical Practice, 168:108392, October 2020. [DOI] [PubMed] [Google Scholar]
  • [43].Hernán Miguel A and Robins James M. Per-protocol analyses of pragmatic trials. New England Journal of Medicine, 377(14):1391–1398, 2017. [DOI] [PubMed] [Google Scholar]
  • [44].Rosenbaum Paul R. The consequences of adjustment for a concomitant variable that has been affected by the treatment. Journal of the Royal Statistical Society: Series A, 147(5):656–666, 1984. [Google Scholar]
  • [45].Robins James. A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect. Mathematical Modelling, 7 (9–12):1393–1512, 1986. [Google Scholar]
  • [46].Goodhart Charles AE. Problems of monetary management: The UK experience. In Monetary Theory and Practice, pages 91–121. Springer, 1984. [Google Scholar]
  • [47].Strathern Marilyn. ‘Improving ratings’: Audit in the British University system. European review, 5 (3):305–321, 1997. [Google Scholar]
  • [48].Manheim David and Garrabrant Scott. Categorizing variants of Goodhart’s Law. arXiv preprint arXiv:1803.04585, 2018. [Google Scholar]
  • [49].Carlsen JE Køber L, Torp-Pedersen C, and Johansen P. Relation between dose of bendrofluazide, antihypertensive effect, and adverse biochemical effects. BMJ, 300:975–978, April 1990. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [50].Bakris George L. and Weir Matthew R.. Angiotensin-Converting Enzyme Inhibitor–Associated Elevations in Serum Creatinine: Is This a Cause for Concern? Archives of Internal Medicine, 160: 685–693, March 2000. [DOI] [PubMed] [Google Scholar]
  • [51].Nguyen Quang T., Anderson Scott R., Sanders Lindsay, and Nguyen Loida. Managing Hypertension in the Elderly: A Common Chronic Disease with Increasing Age. American Health and Drug Benefits, 5:146–153, 05 2012. [PMC free article] [PubMed] [Google Scholar]
  • [52].Mancia Giuseppe and Grassi Guido. Aggressive Blood Pressure Lowering Is Dangerous: The J-Curve. Hypertension, 64:29–36, 01 2014. [DOI] [PubMed] [Google Scholar]
  • [53].Messerli Franz H., Mancia Giuseppe, Conti Richard, Hewkin Ann, Kupfer Stuart, Champion Annette, Kolloch Rainer, Benetos Athanase, and Pepine Carl J.. Dogma Disputed: Can Aggressively Lowering Blood Pressure in Hypertensive Patients with Coronary Artery Disease Be Dangerous. Annals of Internal Medicine, 144:884–893, 2006. [DOI] [PubMed] [Google Scholar]
  • [54].Verdecchia Paolo, Jan A Staessen Fabio Angeli, Giovanni de Simone Augusto Achilli, Ganau Antonello, Mureddu Gianfrancesco, Pede Sergio, Aldo P Maggioni Donato Lucci, Reboldi Gianpaolo, and Cardio-Sis investigators. Usual versus tight control of systolic blood pressure in non-diabetic patients with hypertension (Cardio-Sis): An open-label randomised trial. The Lancet, 374(9689): 525–533, 2009. [DOI] [PubMed] [Google Scholar]
  • [55].Wright JT, Bakris G, and Green T. Effect of blood pressure lowering and antihypertensive drug class on progression of hypertensive kidney disease: Results from the AASK trial. Journal of the American Medical Association, 288(19):2421–2431, 2002. [DOI] [PubMed] [Google Scholar]
  • [56].Wei Yong, Jin Zhimin, Shen Guoying, Zhao Xiaowei, Yang Wanhua, Zhong Ye, and Wang Jiguang. Effects of intensive antihypertensive treatment on Chinese hypertensive patients older than 70 years. The Journal of Clinical Hypertension, 15(6):420–427, 2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [57].Cruickshank John M., Thorp Jeffrey M., and Zacharias James F.. Benefits and Potential Harm of Lowering High Blood Pressure. Lancet, 329:581–584, 03 1987. [DOI] [PubMed] [Google Scholar]
  • [58].Suchard Marc A, Schuemie Martijn J, Krumholz Harlan M, You Seng Chan, Chen RuiJun, Pratt Nicole, Reich Christian G, Duke Jon, Madigan David, Hripcsak George, and Ryan Patrick B. Comprehensive comparative effectiveness and safety of first-line antihypertensive drug classes: A systemtatic, multinational, large-scale analysis. The Lancet, 394:1816–1826, 11 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [59].Hernán MA and Robins JM. Causal inference: What if. Chapman & Hall/CRC, 2020. [Google Scholar]
  • [60].Robins James. A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect. Mathematical Modelling, 7(9): 1393–1512, 1986. [Google Scholar]
  • [61].Hernán MA, Brumback B, and Robins JM. Marginal structural models to estimate the causal effect of zidovudine on the survival of HIV-positive men. Epidemiology, 11(5):561–570, 2000. [DOI] [PubMed] [Google Scholar]
  • [62].Lin Victoria, Sean McGrath Zilu Zhang, Petito Lucia C., Logan Roger W., Hernán Miguel A., and Young Jessica G.. gformula: An R package for estimating effects of general time-varying treatment interventions via the parametric g-formula. arXiv, 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [63].van der Laan Mark and Luedtke Alexander. Targeted learning of the mean outcome under an optimal dynamic treatment rule. Journal of Causal Inference, 3(1):61–95, 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [64].Mohammad Ali Mansournia Mahyar Etminan, Danaei Goodarz, Kaufman Jay S, and Collins Gary. Handling time varying confounding in observational research. BMJ, 359, 08 2017. [DOI] [PubMed] [Google Scholar]
  • [65].Zhang Yi, Jessica G Young Mae Thamer, and Hernán Miguel A. Comparing the Effectiveness of Dynamic Treatment Strategies Using Electronic Health Records: An Application of the Parametric g-Formula to Anemia Management Strategies. Health Services Research, 53:1900–1918, June 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [66].Johnson Kipp W., Glicksberg Benjamin S., Hodos Rachel A., Shameer Khader, and Dudley Joel T.. Causal inference on electronic health records to assess blood pressure treatment targets: an application of the parametric g formula. Pacific Symposium on Biocomputing, pages 180–191, 2018. [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

suppl_material
NIHMS1838524-supplement-suppl_material.pdf (217.6KB, pdf)

RESOURCES