Abstract
The Diabetes Prevention Program (DPP) was a 3-year randomized clinical trial (RCT) with evaluation of lifestyle and metformin interventions compared with placebo for diabetes prevention in high-risk adults. Both interventions significantly reduced diabetes incidence, prompting the long-term Diabetes Prevention Program Outcomes Study (DPPOS) to assess the progression of diabetes and its complications over 22 years. During follow-up, departures from the original metformin or placebo assignment occurred primarily because of development of diabetes that, by protocol, was managed by clinicians outside the study, after participants developed diabetes with HbA1c ≥7.0%. Diabetes development led to changes in metformin treatment and addition of other glucose-lowering therapies. Using statistical methods designed to estimate intervention effects despite these deviations, we consistently found that metformin reduced diabetes incidence. However, using these methods to evaluate whether use of metformin for prediabetes confers continued benefits after diabetes diagnosis did not substantially change the conclusions from those of the simpler intention-to-treat analysis that did not account for treatment changes. All of the analytic methods used resulted in similar metformin effect estimates with 95% CIs for hazard ratios including 1.0 (no effect) for all outcomes except for diabetes incidence. Elucidating metformin’s long-term role in mitigating diabetes-related complications beyond its effects on diabetes prevention is challenging.
Graphical Abstract
Introduction
The effects of interventions, including drugs, can be assessed in interventional or noninterventional studies. In noninterventional studies (“observational” or “epidemiological” studies, including postmarketing drug surveillance studies), clinical decisions are made not by the investigator but by the participant and clinician. The outcome(s) of interest are compared among participants receiving or not receiving treatments under study. A primary difficulty is confounding by variables that are associated both with treatments and with outcomes; i.e., those receiving or not receiving a treatment differ with respect to characteristics other than the treatment, and these characteristics influence the outcomes. Confounding variables can create an association between the treatment and outcome that is partly or entirely due to the associations with the confounding variable. Common confounders include age, sex, race and ethnicity, socioeconomic conditions, and health conditions. When all confounders are known and measured accurately, adjustments for confounding can be made using statistical methods enabling correct causal inferences about treatment effects. There may be very many confounders, however, not all of which are known or measured. Thus, noninterventional studies cannot answer causal questions with certainty and can be especially misleading when unadjusted confounders exist.
In an interventional study, by contrast, the treatment assignment is determined by study design, preferably including randomization, in which case the study is a randomized clinical trial (RCT). Randomization controls for measured, unmeasured, and unknown confounders at baseline in sufficiently large studies; i.e., the expected distributions of all potential confounders are equal among the treatment groups. When all variables other than the treatment are equally distributed among the groups, group differences in the outcome are presumed to be caused by the intervention.
Although appropriately sized RCTs should be free of major confounding by baseline factors, problems may arise postrandomization. Inferences from an RCT would be straightforward if each participant followed the assigned treatment exactly and remained in the trial with all specified outcomes measured for the duration of the study. This rarely happens, but is most likely in a small, single-center, short-duration trial of highly acceptable interventions in individuals who are unlikely to change treatments or discontinue the study early. Longer RCTs more often have such problems, which are compounded when intervening events, such as development of new diseases or treatment contraindications, lead to departure from the assigned intervention. In some RCTs, the end of the randomized treatment phase is followed by a noninterventional phase where the investigator no longer assigns treatments but assessments of outcomes are continued or new outcome measurements are introduced. RCTs may be converted to observational studies or the intensity of the interventions may be decreased for economic reasons or the inability of participants to continue the original treatments because of intercurrent events, treatment fatigue, or death. Follow-up may also be extended to ascertain long-term results, such as mortality, even when initial study objectives, such as disease remission, control, or cure, have been accomplished. Analysis of such studies is challenging because of potentially huge biases; i.e., metformin may be discontinued or added by nonstudy clinicians because of development of chronic kidney disease, heart disease, or need for insulin therapy. All of the factors related to changes in the interventions may not be known or measured, and therefore may not be adequately controlled, even through the alternative methods used in this article—except for the instrumental variable (IV) method, designed to adjust for unmeasured confounders. Attempts to deal with these problems may involve a combination of statistical methods used in RCTs and noninterventional studies (1–3).
This article reports the approach of the Diabetes Prevention Program (DPP) (4), an RCT, and its long-term extension, the Diabetes Prevention Program Outcomes Study (DPPOS) (5) for analyzing metformin effects on diabetes, cancer, nephropathy, cardiovascular diseases (CVD), and mortality in the face of such challenges. We assessed lasting effects of the original interventions on diabetes incidence and complications as seen in the Da Qing RCT (6).
Motivating Example: The DPP and DPPOS
The DPP/DPPOS is the longest RCT in the U.S. targeting adults with elevated risk of type 2 diabetes, i.e., prediabetes and overweight or obesity. Participants were randomized to one of three interventions: intensive lifestyle, metformin, and placebo. To illustrate methods to estimate long-term treatment effects in the follow-up of RCTs with deviations from the original treatment assignment, we consider only the metformin and placebo groups. Randomization produced treatment groups that were roughly equal in the distribution of baseline characteristics, including demographic and health variables (Table 1). The primary outcome was diabetes incidence assessed with an annual oral glucose tolerance test and semiannual fasting plasma glucose measurements (4). After confirmed diagnoses of diabetes, the participants and their physicians were informed, annual oral glucose tolerance tests were discontinued, fasting plasma glucose was measured semiannually, and HbA1c was measured annually. Study metformin or placebo (still masked) was continued if fasting plasma glucose was <140 mg/dL. If fasting plasma glucose reached ≥140 mg/dL, the study drug was discontinued and participants were referred to their physicians for all diabetes management. They could prescribe or discontinue metformin, and other glucose-lowering drugs were often instituted.
Table 1.
Participant characteristics at baseline by randomization assignment
| Total, n = 2,155 | Placebo, n = 1,082 | Metformin, n = 1,073 | |
|---|---|---|---|
| Sex (n) | |||
| Men | 698 | 335 | 363 |
| Women | 1,457 | 747 | 710 |
| Race and ethnicity (n) | |||
| Non-Hispanic White | 1,188 | 586 | 602 |
| African American | 441 | 220 | 221 |
| Hispanic | 330 | 168 | 162 |
| American Indian | 111 | 59 | 52 |
| Asian American | 85 | 49 | 36 |
| Age (years) | 50.6 (10.4) | 50.3 (10.4) | 50.9 (10.3) |
| BMI (kg/m2) | 34.0 (6.6) | 34.1 (6.7) | 33.9 (6.6) |
| Fasting glucose (mg/dL) | 107 (8) | 107 (8) | 107 (8) |
| Fasting glucose (mmol/L) | 5.92 (0.47) | 5.92 (0.47) | 5.91 (0.47) |
| 2-h glucose (mg/dL) | 165 (17) | 164 (17) | 165 (17) |
| 2-h glucose (mmol/L) | 9.15 (0.95) | 9.13 (0.95) | 9.16 (0.96) |
| HbA1c (%) | 5.91 (0.50) | 5.91 (0.51) | 5.91 (0.50) |
| HbA1c (mmol/mol) | 41.1 (5.5) | 41.1 (5.5) | 41.1 (5.5) |
Data are means (SD) unless otherwise indicated.
After significant reductions in diabetes incidence were observed in the intensive lifestyle (by 58%) and metformin (by 31%) groups compared with the placebo group during an average of 2.8 years of follow-up, the masked treatment phase of DPP was ended in 2001 (4). After a 1-year bridge period, DPPOS was started to assess long-term intervention effects on diabetes and its complications (5). Placebo was discontinued, and unmasked metformin was provided to the original metformin group, unless contraindicated. Until 2014, group-implemented quarterly lifestyle classes were offered to all participants. Diabetes was assessed and managed as in DPP, except that study metformin (now unmasked) was continued until HbA1c was ≥7.0%, after which clinicians outside of the study managed the diabetes. For uniformity of calculations throughout follow-up, we defined the “diabetes management change” as the first DPP or DPPOS research visit with a confirmed diabetes diagnosis and HbA1c ≥7.0%. Analyses were stratified by this critical intercurrent event (7). We refer to the group originally assigned to placebo as the placebo group, even after the placebo was discontinued.
With longer follow-up some of the effects of the original interventions may be attenuated because of departures from the assigned treatments. Percentage of participants taking metformin at annual examinations before the diabetes management change, including among those who never developed diabetes during follow-up, is shown in Fig. 1, upper panel. All metformin use among placebo participants was prescribed outside the study. At the first DPP annual examination after randomization, 91% of the metformin group and none of the placebo group reported taking metformin. Subsequently, the prevalence of taking metformin declined in the metformin group and increased in the placebo group (largely due to the diagnosis of diabetes) but remained substantially higher in the metformin group (Fig. 1, lower panel). In addition, other glucose-lowering drugs were often instituted. Figure 2 shows mean percentages of annual examinations at which participants reported taking metformin, other glucose-lowering drugs, neither, or both. The use of other drugs, as expected, was much higher after the diabetes management change but was similar between the two groups. A key feature of this RCT is that the study drug, metformin, tested for diabetes prevention is also the first-line drug for diabetes treatment, so many participants in the placebo group who developed diabetes were then treated with metformin. This complicates the assessment of its effects on diabetes complications. Similar problems occur in other RCTs in which a drug tested for prevention is also used for treatment of an intermediate outcome. In the Physicians’ Health Study, for example, aspirin was tested in primary prevention of CVD (8). Aspirin reduced the incidence of nonfatal CVD, which in turn predicts CVD death, but most placebo group participants who experienced nonfatal CVD were then treated with aspirin. This complicated the assessment of aspirin’s effect for fatal CVD, requiring alternative methods of statistical analysis.
Figure 1.
Percentage of participants taking metformin (in study or out of study) by year and treatment group. The upper panel includes only exams in participants without diabetes or with diabetes from the time period before the diabetes management change (i.e., before the first examination with diabetes and HbA1c ≥7.0%). The lower panel includes all examinations in all participants, with or without diabetes. The x-axis labels denote the DPP follow-up years (DPP Y01, etc.) and DPPOS follow-up years (OS Y1, etc.). MET, metformin; PLAC, placebo.
Figure 2.
Fractions of annual examinations at which participants reported taking placebo (or no drug), metformin, other glucose-lowering drug, or combinations, according to time before or after the diabetes management change, i.e., the first examination with diabetes and HbA1c ≥7.0%. MET, metformin; PLAC, placebo.
Research Questions
Effects in the DPPOS of metformin on incidence of diabetes, diabetic kidney disease, retinopathy, neuropathy, CVD, cancer, and mortality have previously been reported (4,5,9–12). We now wish to estimate the counterfactual effects or “biological efficacy” (1) of metformin, i.e., the effects that would have pertained between two groups perfectly balanced at baseline if all assigned treatments had been implemented and maintained throughout the study. The counterfactual metformin effects will differ depending on the participants compared. For example, we can study the preventive effects of metformin among participants with prediabetes when we restrict follow-up to the time before diabetes diagnosis. We may also study the effects of metformin on diabetes complications among participants with diabetes. Because most participants are prescribed metformin after diabetes diagnosis, we cannot study the effects of a fixed binary metformin indicator throughout the course of DPP and DPPOS, but we can study the effects of cumulative metformin exposure.
Here we describe four commonly used statistical methods to account for departures from assigned interventions. They provide different approaches to understanding metformin effects in this study, ultimately answering two related questions:
How did randomization to the metformin group compared with the placebo group affect the incidence of an outcome? This question is about the effect of the policy of assigning metformin regardless of adherence to that policy, i.e., intention to treat (ITT). This is the default and often only treatment question addressed in RCTs.
How is actual metformin use associated with the outcome? Use was determined by random assignment, development of diabetes, or other reasons for starting or stopping metformin. Isolating this effect is difficult because of the various reasons for starting or discontinuing metformin and the many reference conditions and drugs with which metformin is compared. The goal is estimating the counterfactual situation in which all participants assigned to metformin took it continuously, those not assigned never took it, and there was no informative censoring. Randomization of the original treatment assignments does not control for postrandomization treatment changes, however, so adjustment for confounding is necessary.
A further issue is that the condition with which metformin use is compared, i.e., the comparand, changes for many participants during the study. When metformin is compared with placebo, the differences in outcomes are attributable to the causal effects of metformin. When other drugs or interventions are used, however, in either intervention group, metformin’s comparand changes from placebo to the other intervention(s). In this case, differences in outcomes cannot be attributed to metformin’s effect relative to placebo; rather, they reflect metformin’s effect relative to the other interventions or comparands.
Review of Statistical Methods for Estimation of Treatment Effects
For simplicity, this section describes RCTs with comparison of one study drug with a placebo, but the discussion is generalizable to other situations including three or more treatment arms, no placebo arm, or nondrug interventions. Detailed description of the methods described here can be found in the Supplementary Material. RCTs are viewed as the gold standard for inferring causality because randomization controls for unmeasured and unknown baseline confounders and because of their rigor and high standards in design, implementation, and analyses but can give biased estimates of drug effects in the face of departures from the assigned interventions (3,7). We characterize methodological approaches to the above questions in four groups (Table 2). The common ITT analysis compares groups of participants according to the initial randomization regardless of adherence to assigned treatments. The ITT treatment effect estimates reflect real-life situations where treatment assignments are not always followed. By contrast, as treated (AT) analyses estimate the effects of actual drug exposure despite possible systematic differences between participants who did or did not take the study drug. AT analyses include all person-time of all participants in the study, with classification of each interval of follow-up according to taking assigned intervention or not. AT analyses are often confounded by known and unknown variables associated with taking the drug, so such variables should be measured, when possible, and controlled for in analyses. In the hypothetical situation in which there are no departures from the assigned intervention, the ITT and AT approaches will give identical results, and the two questions (policy and actual use) will have the same answer. Otherwise, the two approaches address different questions.
Table 2.
Brief description of analytic methods
| ITT = intention to treat | Compares randomized groups regardless of adherence |
| AT = as treated | Compares those taking or not taking metformin at each time point (or the cumulative metformin exposures) regardless of random assignment. Requires adjustment for confounders because randomization of treatment assignments does not control for confounders of treatment changes. |
| IV = instrumental variable | Uses randomization as an IV to account for confounding in examination of effect of metformin use. |
| IPCW = inverse-probability- of-censoring weighting | Creates an analytic subset of person-time prior to censoring for treatment departure, with weighting of each participant’s data by the inverse of the probability of censoring. |
Another approach is estimating effects of the drug exposure when the exposed and unexposed groups have comparable risk factor distributions, such as in a counterfactual situation where two individuals are identical except that only one takes the drug. The inverse probability of censoring weighting (IPCW) method maps the composition of the drug and placebo samples to two balanced samples, and adjustment methods account for observed and unobserved confounders directly or indirectly. The IPCW analysis creates pseudo-exposed and pseudo-unexposed groups that represent the initial randomized drug and placebo arms by weighting participants’ data by the inverse of the probability of not being censored at the time of discontinuing the original intervention (13–14). Therefore, the IPCW analysis estimates the drug effect where distributions of confounders are balanced between the exposed and unexposed groups.
The IV method comprises a two-step procedure to summarize confounders of drug use (15). It does not require measuring confounders. It adjusts for two drug-related covariates in assessing the relationship between the treatment and the outcome of interests: residuals and the observed drug exposure. In the first stage, the IV (the original assignment) is used to predict actual study drug use. In linear regression, residuals are independent of the covariates (i.e., the IV); therefore they capture the effects of other factors on the outcome (actual drug exposure) beyond the IV (the original assignment), which can be viewed as confounders of the effects of drug assignment on the outcome. Therefore, the coefficient of the observed study drug use in the second-stage regression represents the counterfactual effects of taking the drug, accounting for confounders.
We illustrate four of these methods in analyzing the DPPOS. ITT is the default method in RCTs, and AT is also often used. We also used the IPCW and IV methods to adjust for confounding factors related to departures from assigned interventions. All analyses include adjustment for baseline covariates to improve precision of the estimates of the effects of the randomized interventions (16). Other related methods, such as AT analyses that do not include adjustment for confounders and “per protocol” analyses that include only participants with total adherence to assigned interventions, are not discussed here. They are generally biased (3,7) because of confounding by the variables associated with adherence that can also be risk factors for the outcomes under study.
Application to the DPP/DPPOS: Results and Discussion
Analyses covered follow-up from each participant’s randomization (1996–1999) until the last visit before the closing date of 23 February 2020. Table 3 defines the six outcomes analyzed: diabetes, all cancers, obesity-related cancers, nephropathy, major adverse cardiovascular events (i.e., nonfatal myocardial infarction, nonfatal stroke, or cardiovascular death), and all-cause mortality (4,9–12). Supplementary Table 1, upper panel, shows the numbers of first events, person-years of follow-up, and incidence rates for each outcome by treatment group.
Table 3.
Select DPP/DPPOS outcomes evaluated in this analysis
| Outcome | Brief definition | Reference no. for detailed description |
|---|---|---|
| Diabetes | Fasting glucose ≥126 mg/dL or 2-h glucose ≥200 mg/dL, confirmed | 4 |
| Cancer | Cancer of any type except for non-melanoma skin cancer | 9 |
| Obesity-related cancer | Cancer of the following types: meningioma, multiple myeloma, or cancer of the esophagus, kidneys, uterus, ovaries, thyroid, breast (postmenopausal), liver, gallbladder, upper stomach, pancreas, colon, or rectum | 9 |
| Nephropathy | Urine albumin–to–creatinine ratio ≥30 mg/g or eGFR <45 mL/min/1.72 m2 confirmed at next visit or end-stage kidney disease | 10 |
| MACE | Major adverse cardiovascular event: nonfatal myocardial infarction, nonfatal stroke, or cardiovascular death | 11 |
| Mortality | Death from any cause | 12 |
eGFR, estimated glomerular filtration rate.
Many treatment changes were protocol driven rather than due to “non-adherence.” Because many participants were prescribed metformin after diabetes diagnosis, and treatments could be changed from metformin to other glucose-lowering drugs by nonstudy clinicians (Fig. 2), the conditions with which metformin is compared changed with disease progression; i.e., for analysis of the first part of the study, the metformin effect is estimated in comparison with placebo. After some participants developed diabetes and were treated with other medicines—with or without metformin—the metformin effect is estimated in comparison with all the other diabetes treatments in use. This is true for both ITT and AT analyses. Therefore, the estimated metformin effects could differ, even in opposite directions, in comparisons with placebo or with other medicines that may reduce or increase the risk of an outcome of interest. We therefore stratified results by the time before or after the diabetes management change, i.e., the first observation of confirmed diabetes with HbA1c ≥7.0%, because after this point 1) incidence rates for all the outcomes, especially nephropathy, are much higher and 2) metformin’s comparands are more likely to change. Crude incidence rates for each of the five outcomes are stratified by this time point and treatment group (Supplementary Table 1 and Fig. 3). Owing to eligibility criteria, all participants started before the diabetes management change and some, but not all, later moved into the “after” stratum. For most outcomes, especially nephropathy, the difference in incidence rates before or after the diabetes management change is much greater than the difference between treatment groups, and the crude incidence rates are higher in the metformin group, especially for the nephropathy and mortality outcomes.
Figure 3.
Incidence rates (events/1,000 person-years at risk) for five outcomes by treatment group. Narrow bars show rates stratified by time before (B) and after (A) the diabetes management change, i.e., the first occurrence in each participant of diabetes with HbA1c ≥7.0% (after which diabetes was managed outside of the study). Wide bars show incidence rates during the total (T) follow-up period. The diabetes outcome is not shown because, by definition, all time at risk occurred before the diagnosis of diabetes. Further, the diabetes incidence rates were much higher (66.7 events/1,000 person-years for the placebo group and 55.1 for the metformin group) and would be outside the scale of this figure. MACE, major adverse cardiovascular events; Obesity-Ca, obesity-related cancer.
Hazard ratios (HRs) were computed from proportional hazards models under each of the four analytic methods (ITT, AT, IV, and IPCW) for each outcome. Covariates and C-statistics for each model are shown in Supplementary Table 2. Effect estimates as HRs for metformin versus placebo are shown in Supplementary Table 3 and Fig. 4. Figure 4 and Supplementary Table 3, upper panel, show results over the entire follow-up period. The HRs for metformin effect showed significantly reduced diabetes incidence and varied little by analytic methods, ranging from 0.70 to 0.82. The lowest HR, i.e., greatest preventive effect, was estimated with the IV method. The small differences in these estimates follow from the limited use of metformin in placebo group participants at visits prior to the development of diabetes. This was not the case for the other outcomes that could have occurred before or after development of diabetes. Supplementary Table 3, lower panels, shows results stratified by time before or after the diabetes management change. CIs are wider than for the total period, especially for the later period, because the events and person-years were divided among the two periods.
Figure 4.
HRs with 95% CIs for estimated metformin effects for six outcomes by four methods. ITT: intention to treat; AT: each examination scored as any metformin taken or not (dichotomous); IV: instrumental variable; IPCW: inverse-probability-of-censoring weighting. Outcome definitions and numbers of events are shown in Table 3 and Supplementary Table 1. Details can be found in Supplementary Table 3.
In the ITT analysis comparisons are by assignment to the metformin versus placebo group, regardless of subsequent changes in drug treatment. The AT comparison is for taking versus not taking metformin, for whatever reason(s), analyzed as a binary variable at each examination. The IV analysis expresses the HR per 10 cumulative years of metformin use with adjustment for the residuals from a linear regression using the original metformin assignment (i.e., the IV) to predict the cumulative years of metformin exposure. Unlike for observed metformin exposure, the original metformin assignment is independent of confounders and the residuals represent metformin exposure due to confounding. The IPCW method differs in that the number of events counted is much lower because follow-up time is censored the first time the assigned treatment is switched; thus, the number of events is lower because those occurring later are not informative. Results were largely consistent across methods for all outcomes except mortality, for which the HR was highest (1.21) in the AT analysis, which might, in part, reflect diabetes in placebo group participants treated with metformin. For outcomes other than diabetes, for which metformin was significantly beneficial by all methods, the HRs had wide CIs that included 1.0.
Although the absolute numbers of outcome events were higher prior to than after the diabetes management change (Supplementary Table 3), the outcome incidence rates were higher after the diabetes management change, especially for nephropathy and mortality (Fig. 3). Of note, these were the outcomes for which higher percentages occurred after this point: 30% of the nephropathy outcomes and 39% of the deaths, compared with 18%–22% of the other outcomes, occurred after the diabetes management change. For these two outcomes, the incidence rates over the total time period were, therefore, higher than the rates before the diabetes management change (Fig. 3). For outcomes other than diabetes, all 95% CIs of the HRs included 1.0 for the period before the diabetes management change and for total follow-up. Examination of the point estimates suggests an interesting pattern. The HRs were ≥1.0 by the ITT, AT, and IV methods for all of the outcomes after the diabetes management change (Supplementary Table 3, lower panel), although this cannot be interpreted as evidence for harm from metformin. Data after the diabetes management change no longer allow for a valid comparison of metformin versus placebo or no drug. Although participants in the two treatment groups were similar at randomization, except for the study assignment (Table 1), this was not true at the time of the diabetes management change. In the metformin group diabetes diagnosis was delayed, and metformin participants are systematically different from placebo participants at the time of diagnosis (4,5). Further, metformin’s comparands are a variety of different drugs, some of which may have beneficial effects on some outcomes. Therefore, we show results for this time period not to ascertain metformin effects but to illustrate how analyses that include time after protocol-driven changes in diabetes management complicate assessment of metformin’s effects. Our analyses do not resolve the conflicting findings suggesting either beneficial or harmful effects of metformin on kidney disease, as discussed previously (10). The differences between the before and total period results (Supplementary Table 3) would presumably be greater if a greater portion of follow-up time had occurred after the diabetes management change.
The differences in metformin’s associations with complications rates before and after the diabetes management change are only suggestions, in that for all methods the CIs are wide and overlap. Results from IPCW are extremely erratic and based on few events, in the later period.
During DPPOS, annual diabetes incidence rates declined in the metformin group to approximate those in the placebo group; therefore the metformin effects declined over time (17). The reasons for this are complex, but the change in rates indicates that our estimates of metformin effects, at least for diabetes incidence, would be greater if the analyses were restricted to shorter follow-up durations.
Summary and Conclusions
Assessment of metformin effects in the DPP/DPPOS is difficult because departures from random assignment to metformin or placebo were inevitable owing to the very long follow-up and protocol-specified treatment changes. Therefore, metformin’s comparand, i.e., the condition with which taking metformin was compared, differed by person and time during the study. At study outset, the comparand was placebo. After diabetes diagnosis, worsening hyperglycemia, and development and treatment for diabetes-related complications and their risk factors, comparands differed, with variance. The reasons for out-of-study prescription of metformin and other drugs were often risk factors or preventive factors for the various outcomes. Change in comparands may have led to the surprising findings that metformin appeared to be beneficial or neutral before the diabetes management change when the comparand for metformin was usually placebo or no drug, but metformin appeared to be associated with worse outcomes beyond that point. The suggestion of harm, however, cannot be supported because after the diabetes management change the two treatment groups were no longer comparable and metformin’s comparands were variable and often included drugs with beneficial effects for some complications, e.g., sodium–glucose cotransporter 2 inhibitor effects on renal function.
Despite the large sample (>1,000 per intervention group), the 95% CIs for the HRs were wide and included 1.0 for all outcomes other than diabetes (Fig. 4). The wide CIs also make it difficult to compare the different analytic approaches. These approaches did not provide effect estimates that differed substantially from the standard ITT estimates, despite the many treatment changes over time. Although IPCW and IV methods may provide point estimates of metformin effects closer to its causal effects, they pay the price of greater conceptual and computational complexity and wider CIs. IPCW also suffers from not including some outcomes, especially those occurring later in the course of disease. IPCW is, therefore, subject to bias, as previously noted (18), if the metformin effects differ in early and later follow-up. The IPCW and IV analyses do not argue convincingly for conclusions different from those already published for most of these outcomes in the DPP/DPPOS with use of ITT analyses (9–12).
In conclusion, for this RCT with long-term follow-up, we have compared four options for estimating metformin’s effects in the face of the treatment departures, many of which were protocol specified. The problem was minor for the diabetes outcome that occurred before the protocol-specified change in diabetes management, and all four procedures gave similar results. The problem was potentially more serious later, when a variety of glucose-lowering drugs were given and more diabetes complications developed. This is especially problematic for DPPOS as the major interest is treatment effects for long-term health outcomes. The ITT and AT methods may not correctly account for changes in assigned intervention because some of the causes of changes, such as diabetes diagnoses, may mediate metformin’s effects on other outcomes. Adjustment for mediators can bias effect estimates. The IV method may, in theory, better account for such changes and can adjust for unmeasured confounders. These conclusions are limited, however, by the wide CIs around all the effect estimates. These methods, therefore, may be more suitable in much larger studies, those with stronger intervention effects, or those with fewer alternate treatments when the study-assigned treatment is changed. In DPP/DPPOS, the ITT method remains, however, the best evaluation of the policy of starting treatment with metformin, while the aim of the other methods is to allow unconfounded estimation of the biological effects of actually taking metformin. Finally, the analytic approaches discussed here will be applicable to assessment of secondary outcomes in RCTs in which the study intervention is often used in those not assigned to it after the occurrence of the primary outcome.
This article contains supplementary material online at https://doi.org/10.2337/figshare.29591417.
Article Information
Acknowledgments. The Diabetes Prevention Program Research Group gratefully acknowledges the commitment and dedication of the participants of the DPP and DPPOS. The authors thank Melinda C. Power, Department of Epidemiology, Milken School of Public Health, George Washington University, for advice on this article.
S.E.K. is an editor of Diabetes Care but was not involved in any of the decisions regarding review of the manuscript or its acceptance.
The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. The opinions expressed are those of the study group and do not necessarily reflect the views of the funding agencies.
Duality of Interest. McKesson BioServices and Matthews Media Group provided support services under subcontract with the Coordinating Center. No other potential conflicts of interest relevant to this article were reported.
Data and Resource Availability. In accordance with the NIH Public Access Policy, we continue to provide all manuscripts to PubMed Central including this manuscript. For DPP/DPPOS the protocols and lifestyle and medication intervention manuals have been provided to the public through the public website https://dppos.bsc.gwu.edu/. The DPPOS abides by the NIDDK data sharing policy and implementation guidance as required by the NIDDK/NIH (https://repository.niddk.nih.gov/home).
Handling Editors. The journal editor responsible for overseeing the review of the manuscript was Mark A. Atkinson.
Funding Statement
Research reported in this publication was supported by the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) of the National Institutes of Health (NIH) under award nos. U01 DK048489, U01 DK048339, U01 DK048377, U01 DK048349, U01 DK048381, U01 DK048468, U01 DK048434, U01 DK048485, U01 DK048375, U01 DK048514, U01 DK048437, U01 DK048413, U01 DK048411, U01 DK048406, U01 DK048380, U01 DK048397, U01 DK048412, U01 DK048404, U01 DK048387, U01 DK048407, U01 DK048443, and U01 DK048400, and in part by the National Institute on Aging of the NIH under award 5 U19 AG078558, through providing funding during DPP and DPPOS to the clinical centers and the Coordinating Center for the design and conduct of the study and collection, management, analysis, and interpretation of data. Funding was also provided by the Eunice Kennedy Shriver National Institute of Child Health and Human Development, the National Eye Institute, the National Heart, Lung, and Blood Institute, the National Cancer Institute, the Office of Research on Women’s Health, the National Institute on Minority Health and Health Disparities, the Centers for Disease Control and Prevention, and the American Diabetes Association. The Southwestern American Indian Centers were supported directly by the NIDDK, including its Intramural Research Program, and the Indian Health Service. The General Clinical Research Center Program, National Center for Research Resources, and U.S. Department of Veterans Affairs supported data collection at many of the clinical centers. Merck KGaA provided medication for DPPOS. DPP/DPPOS have also received donated materials, equipment, or medicines for concomitant conditions from Bristol-Myers Squibb, Parke-Davis, LifeScan, Health o meter, Hoechst Marion Roussel, Merck-Medco Managed Care, Merck & Co., Nike Sports Marketing, SlimFast Foods, and Quaker Oats. The Henry M. Jackson Foundation provided support services under subcontract with the Coordinating Center. The sponsor of this study was represented on the Steering Committee and played a part in study design, how the study was done, and publication.
Footnotes
Clinical trial reg. nos. NCT00004992 and NCT00038727, clinicaltrials.gov
A complete list of members of the Diabetes Prevention Program Research Group can be found in the supplementary material online.
This article is part of a special article collection available at https://diabetesjournals.org/collection/2292/DPP-and-DPPOS-Article-Collection.
Contributor Information
William C. Knowler, Email: dppmail@bsc.gwu.edu.
Diabetes Prevention Program Research Group:
George A. Bray, Kishore M. Gadde, Iris W. Culbert, Jennifer Arceneaux, Annie Chatellier, Amber Dragg, Catherine M. Champagne, Crystal Duncan, Barbara Eberhardt, Frank Greenway, Fonda G. Guillory, April A. Herbert, Michael L. Jeffirs, Betty M. Kennedy, Erma Levy, Monica Lockett, Jennifer C. Lovejoy, Laura H. Morris, Lee E. Melancon, Donna H. Ryan, Deborah A. Sanford, Kenneth G. Smith, Lisa L. Smith, Julia A. St.Amant, Richard T. Tulley, Paula C. Vicknair, Donald Williamson, Jeffery J. Zachwieja, Kenneth S. Polonsky, Janet Tobian, David A. Ehrmann, Margaret J. Matulik, Karla A. Temple, Bart Clark, Kirsten Czech, Catherine DeSandre, Brittnie Dotson, Ruthanne Hilbrich, Wylie McNabb, Ann R. Semenske, Celeste C. Thomas, Jose F. Caro, Kevin Furlong, Barry J. Goldstein, Pamela G. Watson, Kellie A. Smith, Jewel Mendoza, Marsha Simmons, Wendi Wildman, Renee Liberoni, John Spandorfer, Constance Pepe, Richard P. Donahue, Ronald B. Goldberg, Ronald Prineas, Jeanette Calles, Anna Giannella, Patricia Rowe, Juliet Sanguily, Paul Cassanova-Romero, Sumaya Castillo-Florez, Hermes J. Florez, Rajesh Garg, Lascelles Kirby, Olga Lara, Carmen Larreal, Valerie McLymont, Jadell Mendez, Arlette Perry, Patrice Saab, Bertha Veciana, Steven M. Haffner, Helen P. Hazuda, Maria G. Montez, Kathy Hattaway, Juan Isaac, Carlos Lorenzo, Arlene Martinez, Monica Salazar, Tatiana Walker, Dana Dabelea, Richard F. Hamman, Patricia V. Nash, Sheila C. Steinke, Lisa Testaverde, Jennifer Truong, Denise R. Anderson, Larry B. Ballonoff, Alexis Bouffard, Brian Bucca, B. Ned Calonge, Lynne Delve, Martha Farago, James O. Hill, Shelley R. Hoyer, Tonya Jenkins, Bonnie T. Jortberg, Dione Lenz, Marsha Miller, Thomas Nilan, Leigh Perreault, David W. Price, Judith G. Regensteiner, Emily B. Schroeder, Helen Seagle, Carissa M. Smith, Brent VanDorsten, Edward S. Horton, Medha Munshi, Kathleen E. Lawton, Sharon D. Jackson, Catherine S. Poirier, Kati Swift, Ronald A. Arky, Marybeth Bryant, Jacqueline P. Burke, Enrique Caballero, Karen M. Callaphan, Barbara Fargnoli, Therese Franklin, Om P. Ganda, Ashley Guidi, Mathew Guido, Alan M. Jacobsen, Lyn M. Kula, Margaret Kocal, Lori Lambert, Kathleen E. Lawton, Sarah Ledbury, Maureen A. Malloy, Roeland J.W. Middelbeek, Maryanne Nicosia, Cathryn F. Oldmixon, Jocelyn Pan, Marizel Quitingon, Riley Rainville, Stacy Rubtchinsky, Ellen W. Seely, Jessica Sansoucy, Dana Schweizer, Donald Simonson, Fannie Smith, Caren G. Solomon, Jeanne Spellman, James Warram, Steven E. Kahn, Brenda K. Montgomery, Basma Fattaleh, Celeste Colegrove, Wilfred Fujimoto, Robert H. Knopp, Edward W. Lipkin, Michelle Marr, Ivy Morgan-Taggart, Anne Murillo, Kayla O’Neal, Dace Trence, Lonnese Taylor, April Thomas, Elaine C. Tsai, Samuel Dagogo-Jack, Abbas E. Kitabchi, Mary E. Murphy, Laura Taylor, Jennifer Dolgoff, William B. Applegate, Michael Bryer-Ash, Debra Clark, Sandra L. Frieson, Uzoma Ibebuogu, Raed Imseis, Helen Lambeth, Lynne C. Lichtermann, Hooman Oktaei, Harriet Ricks, Lily M.K. Rutledge, Amy R. Sherman, Clara M. Smith, Judith E. Soberman, Beverly Williams-Cleaves, Avnisha Patel, Ebenezer A. Nyenwe, Ethel Faye Hampton, Boyd E. Metzger, Mark E. Molitch, Amisha Wallia, Mariana K. Johnson, Daphne T. Adelman, Catherine Behrends, Michelle Cook, Marian Fitzgibbon, Mimi M. Giles, Deloris Heard, Cheryl K.H. Johnson, Diane Larsen, Anne Lowe, Megan Lyman, David McPherson, Samsam C. Penn, Thomas Pitts, Renee Reinhart, Susan Roston, Pamela A. Schinleber, Matthew O’Brien, Monica Hartmuller, David M. Nathan, Charles McKitrick, Heather Turgeon, Mary Larkin, Marielle Mugford, Kathy Abbott, Ellen Anderson, Laurie Bissett, Kristy Bondi, Enrico Cagliero, Jose C. Florez, Linda Delahanty, Valerie Goldman, Elaine Grassa, Lindsery Gurry, Kali D’Anna, Fernelle Leandre, Peter Lou, Alexandra Poulos Elyse Raymond, Valerie Ripley, Christine Stevens, Beverly Tseng, Kathy Chu, Nopporn Thangthaeng, Jerrold M. Olefsky, Elizabeth Barrett-Connor, Sunder Mudaliar, Maria Rosario Araneta, Mary Lou Carrion-Petersen, Karen Vejvoda, Sarah Bassiouni, Madeline Beltran, Lauren N. Claravall, Jonalle M. Dowden, Steven V. Edelman, Pranav Garimella, Robert R. Henry, Javiva Horne, Marycie Lamkin, Simona Szerdi Janesch, Diana Leos, William Polonsky, Rosa Ruiz, Jean Smith, Jennifer Torio-Hurley, F. Xavier Pi-Sunyer, Blandine Laferrere, Jane E. Lee, Susan Hagamen, David B. Allison, Nnenna Agharanya, Nancy J. Aronoff, Maria Baldo, Jill P. Crandall, Sandra T. Foo, Kim Kelly-Dinham, Carmen Pal, Kathy Parkes, Mary Beth Pena, Ellen S. Rooney, Gretchen E.H. Van Wye, Kristine A. Viscovich, Mary de Groot, David G. Marrero, Kieren J. Mather, Melvin J. Prince, Susie M. Kelly, Marcia A. Jackson, Gina McAtee, Paula Putenney, Ronald T. Ackermann, Carolyn M. Cantrell, Yolanda F. Dotson, Edwin S. Fineberg, Megan Fultz, John C. Guare, Angela Hadden, James M. Ignaut, Marion S. Kirkman, Erin O’Kelly Phillips, Kisha L. Pinner, Beverly D. Porter, Paris J. Roach, Nancy D. Rowland, Madelyn L. Wheeler, Vanita Aroda, Michelle Magee, Robert E. Ratner, Michelle Magee, Gretchen Youssef, Sue Shapiro, Natalie Andon, Catherine Bavido-Arrage, Geraldine Boggs, Marjorie Bronsord, Ernestine Brown, Holly Love Burkott, Wayman W. Cheatham, Susan Cola, Cindy Evans, Peggy Gibbs, Tracy Kellum, Lilia Leon, Milvia Lagarda, Claresa Levatan, Milajurine Lindsay, Asha K. Nair, Jean Park, Maureen Passaro, Angela Silverman, Gabriel Uwaifo, Debra Wells-Thayer, Renee Wiggins, Mohammed F. Saad, Karol Watson, Christine Darwin, Preethi Srikanthan, Tamara Horwich, Adrian Casillas, Arleen Brown, Maria Budget, Sujata Jinagouda, Medhat Botrous, Anthony Sosa, Sameh Tadros, Khan Akbar, Claudia Conzues, Perpetua Magpuri, Carmen Muro, Noemi Neira, Kathy Ngo, Michelle Chan, Veronica Villarreal, Amer Rassam, Debra Waters, Kathy Xapthalamous, Julio V. Santiago, Samuel Dagogo-Jack, Neil H. White, Angela L. Brown, Samia Das, Prajakta Khare-Ranade, Tamara Stich, Ana Santiago, Edwin Fisher, Emma Hurt, Tracy Jones, Michelle Kerr, Lucy Ryder, Cormarie Wernimont, Sherita Hill Golden, Christopher D. Saudek, Vanessa Bradley, Emily Sullivan, Tracy Whittington, Caroline Abbas, Adrienne Allen, Frederick L. Brancati, Sharon Cappelli, Jeanne M. Clark, Jeanne B. Charleston, Janice Freel, Katherine Horak, Alicia Greene, Dawn Jiggetts, Deloris Johnson, Hope Joseph, Kimberly Loman, Nestoras Mathioudakis, Henry Mosley, John Reusing, Richard R. Rubin, Alafia Samuels, Thomas Shields, Shawne Stephens, Kerry J. Stewart, LeeLana Thomas, Evonne Utsey, Paula Williamson, David S. Schade, Karwyn S. Adams, Janene L. Canady, Carolyn Johannes, Claire Hemphill, Penny Hyde, Leslie F. Atler, Patrick J. Boyle, Mark R. Burge, Lisa Chai, Kathleen Colleran, Ateka Fondino, Ysela Gonzales, Doris A. Hernandez-McGinnis, Patricia Katz, Carolyn King, Julia Middendorf, Amer Rassam, Sofya Rubinchik, Willette Senter, Debra Waters, Jill Crandall, Harry Shamoon, Janet O. Brown, Gilda Trandafirescu, Danielle Powell, Norica Tomuta, Elsie Adorno, Liane Cox, Helena Duffy, Samuel Engel, Allison Friedler, Angela Goldstein, Crystal J. Howard-Century, Jennifer Lukin, Stacey Kloiber, Nadege Longchamp, Helen Martinez, Dorothy Pompi, Jonathan Scheindlin, Elissa Violino, Elizabeth A. Walker, Judith Wylie-Rosett, Elise Zimmerman, Joel Zonszein, Trevor Orchard, Elizabeth Venditti, Rena R. Wing, Susan Jeffries, Gaye Koenning, M. Kaye Kramer, Marie Smith, Susan Barr, Catherine Benchoff, Miriam Boraz, Lisa Clifford, Rebecca Culyba, Marlene Frazier, Ryan Gilligan, Stephanie Guimond, Susan Harrier, Louann Harris, Andrea Kriska, Qurashia Manjoo, Monica Mullen, Alicia Noel, Amy Otto, Jessica Pettigrew, Bonny Rockette-Wagner, Debra Rubinstein, Linda Semler, Cheryl F. Smith, Valarie Weinzierl, Katherine V. Williams, Tara Wilson, Bonnie Gillis, Marjorie K. Mau, Narleen K. Baker-Ladao, John S. Melish, Richard F. Arakaki, Renee W. Latimer, Mae K. Isonaga, Ralph Beddow, Nina E. Bermudez, Lorna Dias, Jillian Inouye, Kathy Mikami, Pharis Mohideen, Sharon K. Odom, Raynette U. Perry, Robin E. Yamamoto, William C. Knowler, Robert L. Hanson, Harelda Anderson, Norman Cooeyate, Charlotte Dodge, Mary A. Hoskin, Carol A. Percy, Alvera Enote, Camille Natewa, Kelly J. Acton, Vickie L. Andre, Rosalyn Barber, Shandiin Begay, Peter H. Bennett, Mary Beth Benson, Evelyn C. Bird, Brenda A. Broussard, Brian C. Bucca, Marcella Chavez, Sherron Cook, Jeff Curtis, Tara Dacawyma, Matthew S. Doughty, Roberta Duncan, Cyndy Edgerton, Jacqueline M. Ghahate, Justin Glass, Martia Glass, Dorothy Gohdes, Wendy Grant, Ellie Horse, Louise E. Ingraham, Merry Jackson, Priscilla Jay, Roylen S. Kaskalla, Karen Kavena, David Kessler, Kathleen M. Kobus, Jonathan Krakoff, Jason Kurland, Catherine Manus, Cherie McCabe, Sara Michaels, Tina Morgan, Yolanda Nashboo, Julie A. Nelson, Steven Poirier, Evette Polczynski, Christopher Piromalli, Mike Reidy, Jeanine Roumain, Debra Rowse, Robert J. Roy, Sandra Sangster, Janet Sewenemewa, Miranda Smart, Chelsea Spencer, Darryl Tonemah, Rachel Williams, Charlton Wilson, Michelle Yazzie, Raymond Bain, Sarah Fowler, Marinella Temprosa, Michael D. Larsen, Kathleen Jablonski, Tina Brenneman, Sharon L. Edelstein, Solome Abebe, Julie Bamdad, Melanie Barkalow, Joel Bethepu, Tsedenia Bezabeh, Anna Bowers, Nicole Butler, Jackie Callaghan, Caitlin E. Carter, Costas Christophi, Gregory M. Dwyer, Mary Foulkes, Yuping Gao, Robert Gooding, Adrienne Gottlieb, Kristina L. Grimes, Nisha Grover-Fairchild, Lori Haffner, Heather Hoffman, Steve Jones, Tara L. Jones, Richard Katz, Preethy Kolinjivadi, John M. Lachin, Yong Ma, Pamela Mucik, Robert Orlosky, Qing Pan, Susan Reamer, James Rochon, Alla Sapozhnikova, Hanna Sherif, Charlotte Stimpson, Ashley Hogan Tjaden, Fredricka Walker-Murray, Audrey McMaster, Rhea Mundra, Hannah Rapoport, Nolan Kuenster, Elizabeth M. Venditti, Andrea M. Kriska, Linda Semler, Valerie Weinzierl, Santica Marcovina, F. Alan Aldrich, Jessica Harting, John Albers, Greg Strylewicz, Robert Janicek, Anthony Killeen, Deanna Gabrielson, R. Eastman, Judith Fradkin, Sanford Garfield, Christine Lee, Edward Gregg, Ping Zhang, Dan O’Leary, Gregory Evans, Matthew Budoff, Chris Dailing, Elizabeth Stamm, Ann Schwartz, Caroline Navy, Lisa Palermo, Pentti Rautaharju, Ronald J. Prineas, Teresa Alexander, Charles Campbell, Sharon Hall, Yabing Li, Margaret Mills, Nancy Pemberton, Farida Rautaharju, Zhuming Zhang, Elsayed Z. Soliman, Julie Hu, Susan Hensley, Lisa Keasler, Tonya Taylor, Barbara Blodi, Ronald Danis, Matthew Davis, Larry Hubbard, Ryan Endres, Deborah Elsas, Samantha Johnson, Dawn Myers, Nancy Barrett, Heather Baumhauer, Wendy Benz, Holly Cohn, Ellie Corkery, Kristi Dohm, Amitha Domalpally, Vonnie Gama, Anne Goulding, Andy Ewen, Cynthia Hurtenbach, Daniel Lawrence, Kyle McDaniel, Jeong Pak, James Reimers, Ruth Shaw, Maria Swift, Pamela Vargo, Sheila Watson, Jose A. Luchsinger, Jennifer Manly, Elizabeth Mayer-Davis, Robert R. Moran, Ted Ganiats, Kristin David, Andrew J. Sarkin, Erik Groessl, Naomi Katzir, Helen Chong, William H. Herman, Michael Brändle, Morton B. Brown, David Altshuler, Liana K. Billings, Ling Chen, Maegan Harden, Toni I. Pollin, Alan R. Shuldiner, Paul W. Franks, and Marie-France Hivert
Supporting information
References
- 1. Sommer A, Zeger SL.. On estimating efficacy from clinical trials. Stat Med 1991;10:45–52 [DOI] [PubMed] [Google Scholar]
- 2. Toh S, Hernán MA.. Causal inference from longitudinal studies with baseline randomization. Int J Biostat 2008;4:Article 22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Hernán MA, Hernández-Díaz S.. Beyond the intention-to-treat in comparative effectiveness research. Clin Trials 2012;9:48–55 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Knowler WC, Barrett-Connor E, Fowler SE, et al. ; Diabetes Prevention Program Research Group . Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. New Engl J Med 2002;346:393–403 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Diabetes Prevention Program Research Grou p; Knowler WC, Fowler SE, Hamman RF, et al. 10-year follow-up of diabetes incidence and weight loss in the Diabetes Prevention Program Outcomes Study. Lancet 2009;374:1677–1686 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Gong Q, Zhang P, Wang J, et al. ; Da Qing Diabetes Prevention Study Group . Morbidity and mortality after lifestyle intervention for people with impaired glucose tolerance: 30-year results of the Da Qing Diabetes Prevention Outcome Study. Lancet Diabetes Endocrinol 2019;7:452–461 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. U.S. Food and Drug Administration. E9(R1) Statistical Principles for Clinical Trials: Addendum: Estimands and Sensitivity Analysis in Clinical Trials: Guidance for Industry. Revision 1. U.S. Department of Health and Human Services, Silver Spring, MD, 2021 [Google Scholar]
- 8. Cook NR, Cole SR, Hennekens CH.. Use of a marginal structural model to determine the effect of aspirin on cardiovascular mortality in the Physicians' Health Study. Am J Epidemiol 2002;155:1045–1053 [DOI] [PubMed] [Google Scholar]
- 9. Heckman-Stoddard BM, Crandall JP, Edelstein SL, et al. ; DPP Research Group . Randomized study of metformin and intensive lifestyle intervention on cancer incidence over 21 years of follow-up in the Diabetes Prevention Program. Cancer Prev Res 2025;18:401–411 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Molitch ME, Tripputi M, Levey AS, et al. ; Diabetes Prevention Program Research Group . Effects of metformin and intensive lifestyle interventions on the incidence of kidney disease in adults in the DPP/DPPOS. J Diabetes Complications 2023;37:108556. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Goldberg RB, Orchard TJ, Crandall JP, et al. ; Diabetes Prevention Program Research Group . Effects of long-term metformin and lifestyle interventions on cardiovascular events in the Diabetes Prevention Program and its outcome study. Circulation 2022;145:1632–1641 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Lee CG, Heckman-Stoddard B, Dabelea D, et al. ; Diabetes Prevention Program Research Group . Effect of metformin and lifestyle interventions on mortality in the Diabetes Prevention Program and Diabetes Prevention Program Outcomes Study. Diabetes Care 2021;44:2775–2782 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Robins JM, Finkelstein DM.. Correcting for noncompliance and dependent censoring in an AIDS Clinical Trial with inverse probability of censoring weighted (IPCW) log-rank tests. Biometrics 2000;56:779–788 [DOI] [PubMed] [Google Scholar]
- 14. Grafféo N, Latouche A, Le Tourneau C, Chevret S.. ipcwswitch: an R package for inverse probability of censoring weighting with an application to switches in clinical trials. Comput Biol Med 2019;111:103339. [DOI] [PubMed] [Google Scholar]
- 15. Martínez-Camblor P, Mackenzie T, Staiger DO, Goodney PP, O’Malley AJ.. Adjusting for bias introduced by instrumental variable estimation in the Cox proportional hazards model. Biostatistics 2019;20:80–96 [DOI] [PubMed] [Google Scholar]
- 16. Hauck WW, Anderson S, Marcus SM.. Should we adjust for covariates in nonlinear regression analyses of randomized trials? Control Clin Trials 1998;19:249–256 [DOI] [PubMed] [Google Scholar]
- 17. Knowler WC, Doherty L, Edelstein SL, et al. ; DPP/DPPOS Research Group . Long-term effects and effect heterogeneity of lifestyle and metformin interventions on type 2 diabetes incidence over 21 years in the US Diabetes Prevention Program randomised clinical trial. Lancet Diabetes Endocrinol 2025;13:469–481 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Howe CJ, Cole SR, Chmiel JS, Muñoz A.. Limitation of inverse probability-of-censoring weights in estimating survival in the presence of strong selection bias. Am J Epidemiol 2011;173:569–577 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.