Abstract
Background
The use of oral glucose-lowering therapies with insulin is common, but the cardiovascular effects are largely unknown. Among users of long-acting insulin, we conducted a population-based case-control study to evaluate the incident myocardial infarction (MI) and incident stroke risks associated with the use of sulfonylureas and the use of metformin.
Methods
Cases were Group Health Cooperative enrollees with type 2 diabetes who used long-acting insulin at the time of diagnosis with a first MI (n=413) or first stroke (n=247) from 1995-2010. Controls (n=443) with type 2 diabetes who used long-acting insulin were matched to cases on age, sex, and calendar year. Sulfonylurea and metformin use was classified as current, past, or never using electronic pharmacy records. MI and stroke diagnoses were validated by medical record review. Analyses were adjusted for potential confounders.
Results
Current use of sulfonylureas compared with never use was associated with a higher risk of MI (OR 1.67; 95% CI, 1.10-2.55) but not stroke (OR 1.22; 95% CI, 0.74-2.00). Current use of metformin compared with never use was associated with a lower risk of stroke (OR 0.54; 95% CI, 0.31-0.95) but not MI (OR 0.77; 95% CI, 0.44-1.33). Past use of sulfonylureas and past use of metformin were not associated with either outcome.
Conclusions
Sulfonylureas in combination with long-acting insulin may increase the risk of MI compared with the use of insulin alone. Metformin may be an important cardiovascular disease prevention therapy for patients on insulin therapy.
Keywords: sulfonylureas, metformin, insulin, myocardial infarction, stroke, case-control study
INTRODUCTION
An estimated 380 million adults worldwide have diabetes mellitus,1 which doubles the risk of heart disease and stroke.2 For type 2 diabetes, there is little evidence from randomized controlled trials (RCTs) that cardiovascular complications are prevented by glucose-lowering therapies, including the most commonly used oral drugs, sulfonylureas and metformin.3,4 Instead, there have been concerns about the cardiovascular safety of sulfonylureas since 1970, when the University Group Diabetes Program trial found that the first-generation sulfonylurea tolbutamide increased the risk of cardiovascular death compared with either insulin or diet.5 Several observational studies since then have reported that newer sulfonylureas are associated with an increased risk of cardiovascular disease compared with metformin, the current first-line treatment.6,7 Whether these findings reflect harm from sulfonylureas, benefit from metformin,8 or both, remains unclear.
Glucose-lowering treatment for type 2 diabetes is usually initiated with oral drugs, but many patients eventually require insulin therapy to achieve recommended levels of glycemic control, typically with a long-acting insulin.9 About half of patients on insulin therapy continue to use oral drugs,10 mostly sulfonylureas and metformin,11 but evidence from well-controlled studies about clinical benefits or harm is lacking 12,13 and current treatment guidelines do not provide clear recommendations about the use of oral drugs with insulin.14 Evaluating oral drugs in combination with insulin, compared with insulin alone as the active control treatment, can address the unanswered clinical question of whether they should be used together with insulin therapy.
Using information from electronic health databases and medical record reviews, we conducted a population-based case-control study among users of long-acting insulin to assess the incident myocardial infarction (MI) and incident stroke risks associated with the use of sulfonylureas and the use of metformin, compared with the use of long-acting insulin alone. For both outcomes, we hypothesized that sulfonylurea use would be associated with elevated risk and metformin use with decreased risk.
METHODS
Study design and participants
Subjects were enrolled in previous case-control studies of incident MI and incident stroke conducted at Group Health Cooperative (GHC), an integrated healthcare delivery system in Washington State.15-19 Cases were GHC enrollees, aged 30-79, who experienced a first (incident) MI or first (incident) stroke from January 1995 through December 2010. Hospitalized cases were identified from hospitalization and claims databases (International Classification of Disease, Ninth Revision codes 410, 411, 427.4, 427.5 for MI, and 430, 431, 432.9, and 434-436 for stroke); out-of-hospital fatal cases were identified from a computerized match between GHC enrollment files and the Washington state death registry. Trained abstractors reviewed medical records to confirm case diagnoses and identify uncertain events, which were reviewed by physicians. Events that were a complication of a surgery or procedure were ineligible. We have used these methods in previous studies.15-19 In a blinded validation study of MI events,16 the completeness of case ascertainment was 95%, and 97% of eligible case subjects met standard criteria for definite or probable MI.
Controls were randomly selected GHC enrollees, aged 30-79, who were frequency matched to the subjects in the largest case group (MI case group) by age, sex, hypertension status, and calendar year. All subjects were assigned an index date. For hospitalized cases, the index date was the date of admission for the MI or stroke event. For out-of-hospital fatal cases, the index date was the date of death. For controls, the index date was a random date within the calendar year they were sampled. Cases and controls with less than one year of enrollment or fewer than four GHC clinic visits before their index date were excluded.
Case and control subjects with diabetes who used long-acting insulin on the index date, with or without oral diabetes drugs, were eligible for this study. Twenty-nine subjects whose onset of diabetes treatment occurred before age 21, a proxy for type 1 diabetes, were excluded. One subject who used a diabetes drug (pioglitazone) other than sulfonylureas, metformin, or insulin on the index date was excluded. Based on the estimated sample size and estimated prevalence of sulfonylureas use, prior to conducting analyses, we estimated there would be 86% power to detect a relative risk of 1.5 for the sulfonylurea-MI comparison.
Information from enrollment, pharmacy, and laboratory databases and from reviews of the entire medical record up to the index date was used to determine eligibility and to assess drug exposures and risk factors for cardiovascular disease, including current smoking status (yes/no) and the duration of diabetes. For subjects consenting to telephone interviews, additional information was obtained on smoking and race.
Drug exposure ascertainment
Insulins were categorized into long-acting insulin (half-life longer than regular insulin), regular insulin, and rapid-acting insulin (half-life shorter than regular insulin). In a pilot study on patterns of medication use in this study population, dosing instructions and days supply variables were found to be unreliable for insulin prescriptions, but 95% of the time intervals between long-acting insulin prescriptions occurred within 180 days. Also, the current use of long-acting insulin at the index date was confirmed by medical record review for 99% of subjects with a filled prescription in the past 180 days. In this study, insulin use was defined as having a filled prescription within 180 days before the index date.
The use of oral diabetes medications was categorized into current use, past use, and never use. Current use was assessed by algorithms that incorporated information on dose, prescribing instructions, and days supply for the most recent filled prescription, assuming 80% compliance.15 If the most recent filled prescription lasted until the index date, that subject was a current user. Past use was defined as having a filled prescription after the first recorded long-acting insulin prescription, but not meeting criteria for current use. Never use was defined as having no filled prescription after the first recoded long-acting insulin prescription. Use of hypertension medications and statins was assessed using similar algorithms.
Statistical analysis
All subjects were users of long-acting insulin. Incident MI analyses excluded case and control subjects with a prior MI; incident stroke analyses excluded case and control subjects with a prior stroke. Logistic regression was used to estimate odds ratios (OR) and 95% confidence intervals (CI) for the incident MI and incident stroke risks among the various sulfonylurea and metformin drug exposure groups, which were evaluated together in the same models. Because of the incidence density sampling of case and control subjects, the odds ratio is an unbiased estimator of the rate ratio.20 In primary analyses, current and past users of sulfonylureas were compared with never users, and current and past users of metformin were compared with never users. Analyses were adjusted for matching variables (age, sex, hypertension status, index year), duration of diabetes, presence of cardiovascular risk factors (current smoking, diastolic blood pressure, serum total cholesterol, serum creatinine, body mass index [BMI], and nephrotic syndrome), prior cardiovascular disease (angina, previous revascularization, peripheral arterial disease, atrial fibrillation, congestive heart failure), use of other insulins (rapid-acting and regular), and use of cardiovascular therapies (statins, ACE inhibitors). Continuous variables were modeled with linear terms in regression models. HbA1c was not adjusted for in primary analyses because this marker of glycemic control could mediate short-term cardiovascular effects of diabetes therapies. For variables with missing data in > 1% of subjects (diabetes duration and HbA1c), multiple imputation was conducted, using all variables from primary analyses to perform 25 imputations with the assumption of a multivariate normal distribution.21
In secondary analyses, as an approach to address residual confounding, current users of sulfonylureas and metformin were compared with past users. Also, current users of sulfonylureas were compared with current users of metformin. To evaluate dose-related effects, current users of sulfonylureas and metformin were divided at the median daily dose into low- and high-dose groups. For the most commonly used sulfonylureas, drug-specific MI and stroke associations were assessed. In exploratory analyses, effect modification was assessed by including binary interaction terms in regression models. Drug-drug interactions with ACE inhibitors and beta blockers were evaluated, because ACE inhibitors may increase the risk of sulfonylurea-related hypoglycemia and beta blockers may mitigate the physiologic consequences of hypoglycemia, a potential mediator of adverse cardiovascular effects.22,23
Sensitivity analyses were conducted to evaluate assumptions about drug exposure ascertainment, adjustment for potential confounders that could be mediators (HbA1c, BMI), and exclusion criteria. Using a previously described approach,24 we evaluated the expected impact of a hypothetical unmeasured binary confounder on the sulfonylurea-MI association across an array of confounder prevalences among current and never users of sulfonylureas. Relative risks of 1.25 and 2 were assumed for the hypothetical confounder-MI association, based on another study that used a similar approach to assess unmeasured confounding.25 All analyses were conducting with STATA version 11.
RESULTS
413 incident MI cases, 247 incident stroke cases (213 ischemic, 30 hemorrhagic, 4 unknown type), and 443 control subjects on long-acting insulin therapy were eligible for analysis. The mean duration of GHC enrollment was 21 years, the mean age was 67 years, and 47% of subjects were female. Compared with controls, MI and stroke cases had a longer duration of diabetes, more cardiovascular risk factors, and used fewer cardiovascular therapies (Table 1). Among controls, current users of sulfonylureas and metformin had a shorter duration of diabetes, fewer cardiovascular risk factors, and used more cardiovascular therapies than did past users and never users (Table 2).
Table 1.
Characteristics of cases and controls.
Characteristics | MI cases (n=413) | Stroke cases (n=247) | Controls (n=443) |
---|---|---|---|
Age, mean (SD), y | 66 (9) | 68 (8) | 66 (9) |
Female, No. (%) | 221 (54%) | 130 (53%) | 164 (37%) |
Race, No. (%) | |||
White | 346 (84%) | 205 (83%) | 362 (82%) |
Black | 22 (5%) | 25 (10%) | 46 (10%) |
Other | 44 (11%) | 17 (7%) | 35 (8%) |
Enrolled at GHC, mean (SD), y | 21 (12) | 21 (12) | 22 (12) |
Duration of diabetes, mean (SD), y | 15 (8) | 15 (8) | 13 (7) |
Body mass index, mean (SD), kg/m2 | 33 (7) | 32 (7) | 33 (7) |
Current tobacco use, No. (%) | 44 (11%) | 16 (6%) | 38 (9%) |
Prior myocardial infarction, No. (%) | 0 (0%) | 52 (21%) | 84 (19%) |
Prior stroke, No. (%) | 63 (15%) | 0 (0%) | 67 (15%) |
Angina, No. (%) | 135 (33%) | 101 (41%) | 143 (32%) |
Peripheral arterial disease, No. (%) | 57 (14%) | 28 (11%) | 38 (9%) |
Prior revascularization, No. (%) | 56 (14%) | 68 (28%) | 109 (25%) |
Atrial fibrillation, No. (%) | 37 (9%) | 55 (22%) | 50 (11%) |
Congestive heart failure, No. (%) | 81 (20%) | 66 (27%) | 77 (17%) |
Nephrotic syndrome, No. (%) | 17 (4%) | 9 (4%) | 6 (1%) |
Treated hypertension, No. (%) | 370 (90%) | 226 (92%) | 406 (92%) |
Systolic BP, mean (SD), mmHg | 142 (23) | 145 (23) | 139 (20) |
Diastolic BP, mean (SD), mmHg | 75 (13) | 77 (12) | 75 (11) |
Cholesterol, mean (SD), mg/dL | 206 (53) | 202 (55) | 195 (50) |
Creatinine, mean (SD), mg/dL | 1.6 (1.4) | 1.5 (1.1) | 1.3 (0.8) |
Hemoglobin A1c, mean (SD), % | 8.4 (1.7) | 8.6 (1.7) | 8.2 (1.5) |
Statin use, No. (%) | 188 (46%) | 92 (37%) | 248 (56%) |
ACE inhibitor use, No. (%) | 205 (50%) | 120 (49%) | 277 (63%) |
Beta blocker use, No. (%) | 138 (33%) | 91 (37%) | 159 (36%) |
Regular insulin use, No. (%) | 198 (48%) | 115 (47%) | 188 (42%) |
Rapid-acting insulin, No. (%) | 51 (12%) | 30 (12%) | 42 (9%) |
Abbreviations: ACE = angiotensin converting enzyme, BP = blood pressure, GHC = Group Health Cooperative, MI = myocardial infarction. Data were missing in <=1% of subjects except for duration of diabetes (10%) and hemoglobin A1c (6%).
Table 2.
Characteristics of sulfonylurea and metformin users among control subjects.
Sulfonylurea use | Metformin use | |||||
---|---|---|---|---|---|---|
Characteristics | Never (n=216) | Past (n=104) | Current (n=123) | Never (n=236) | Past (n=75) | Current (n=132) |
Age, mean (SD), y | 66 (9) | 69 (8) | 66 (10) | 68 (9) | 67 (9) | 64 (9) |
Female, No. (%) | 76 (35%) | 50 (48%) | 38 (31%) | 87 (37%) | 25 (33%) | 52 (39%) |
Race, No. (%) | ||||||
White | 169 (78%) | 87 (84%) | 106 (87%) | 191 (81%) | 61 (81%) | 110 (83%) |
Black | 26 (12%) | 10 (10%) | 10 (8%) | 26 (11%) | 6 (8%) | 14 (11%) |
Other | 21 (10%) | 7 (7%) | 7 (6%) | 19 (8%) | 8 (11%) | 8 (6%) |
Enrolled at GHC, mean (SD), y | 22 (12) | 24 (11) | 20 (11) | 22 (12) | 23 (11) | 22 (12) |
Duration of diabetes, mean (SD), y | 14 (8) | 15 (6) | 11 (5) | 14 (8) | 15 (6) | 12 (6) |
Body mass index, mean (SD), kg/m2 | 33 (7) | 33 (7) | 34 (6) | 32 (6) | 35 (7) | 35 (7) |
Current tobacco use, No. (%) | 17 (8%) | 10 (10%) | 11 (9%) | 20 (8%) | 5 (7%) | 13 (10%) |
Prior myocardial infarction, No. (%) | 43 (20%) | 20 (19%) | 21 (17%) | 50 (21%) | 16 (21%) | 18 (14%) |
Prior stroke, No. (%) | 39 (18%) | 13 (13%) | 15 (12%) | 42 (18%) | 12 (16%) | 13 (10%) |
Angina, No. (%) | 66 (31%) | 39 (38%) | 38 (31%) | 77 (33%) | 27 (36%) | 39 (30%) |
Peripheral arterial disease, No. (%) | 21 (10%) | 11 (11%) | 6 (5%) | 29 (12%) | 6 (8%) | 3 (2%) |
Prior revascularization, No. (%) | 54 (25%) | 24 (23%) | 31 (25%) | 58 (25%) | 23 (31%) | 28 (21%) |
Atrial fibrillation, No. (%) | 24 (11%) | 11 (11%) | 15 (12%) | 28 (12%) | 10 (13%) | 12 (9%) |
Congestive heart failure, No. (%) | 42 (19%) | 19 (18%) | 16 (13%) | 53 (22%) | 15 (20%) | 9 (7%) |
Nephrotic syndrome, No. (%) | 4 (2%) | 2 (2%) | 0 (0%) | 5 (2%) | 1 (1%) | 0 (0%) |
Treated hypertension, No. (%) | 192 (89%) | 98 (94%) | 116 (94%) | 212 (90%) | 72 (96%) | 122 (92%) |
Systolic BP, mean (SD), mmHg | 138 (19) | 140 (20) | 142 (20) | 140 (21) | 139 (20) | 138 (17) |
Diastolic BP, mean (SD), mmHg | 75 (11) | 75 (11) | 76 (10) | 75 (12) | 74 (10) | 76 (10) |
Cholesterol, mean (SD), mg/dL | 201 (50) | 190 (47) | 188 (51) | 199 (49) | 193 (53) | 189 (49) |
Creatinine, mean (SD), mg/dL | 1.3 (0.8) | 1.3 (1.0) | 1.2 (0.6) | 1.4 (0.9) | 1.4 (0.9) | 1.0 (0.2) |
Hemoglobin A1c, mean (SD), % | 8.0 (1.5) | 8.1 (1.5) | 8.5 (1.5) | 8.0 (1.5) | 8.4 (1.8) | 8.4 (1.5) |
Statin use, No. (%) | 107 (50%) | 57 (55%) | 84 (68%) | 117 (50%) | 45 (60%) | 86 (65%) |
ACE inhibitor use, No. (%) | 133 (62%) | 61 (59%) | 83 (67%) | 137 (58%) | 43 (57%) | 97 (73%) |
Beta blocker use, No. (%) | 70 (32%) | 36 (35%) | 53 (43%) | 79 (33%) | 30 (40%) | 50 (38%) |
Regular insulin use, No. (%) | 112 (52%) | 50 (48%) | 26 (21%) | 123 (52%) | 28 (37%) | 37 (28%) |
Rapid-acting insulin, No. (%) | 21 (10%) | 10 (10%) | 11 (9%) | 11 (5%) | 16 (21%) | 15 (11%) |
Abbreviations: ACE = angiotensin converting enzyme, BP = blood pressure, GHC = Group Health Cooperative, MI = myocardial infarction. Data were missing in <=1% of subjects except for duration of diabetes (10%) and hemoglobin A1c (6%).
88% of long-acting insulin use was with NPH insulin and 8% with insulin glargine. The use of sulfonylureas, metformin, and rapid-acting insulin on the index date gradually increased over time while regular insulin use decreased (Figure 1). 36% of sulfonylurea users also used metformin, and 43% of metformin users also used sulfonylureas. The mean (SD) duration of sulfonylurea use after the first long-acting insulin prescription was 2.8 (2.8) and 2.6 (2.9) years among current and past users; the mean duration of metformin use was 2.9 (2.5) and 2.0 (2.2) years among current and past users. Risk factors that are often poorly measured with claims data were associated with cardiovascular disease risk; smoking was associated with a higher MI risk (OR 1.88; 95% CI, 1.07-3.30) and each additional year of diabetes duration was associated with a 3% higher risk of both MI and stroke (95% CI, 1-6%).
Figure 1.
Secular trends in the use of glucose-lowering drugs with long-acting insulin among control subjects on the index date.
Current use of sulfonylureas compared with never use was associated with a significantly higher MI risk (OR 1.67; 95% CI, 1.10-2.55) and a non-significantly higher stroke risk (OR 1.22; 95% CI, 0.74-2.00; Table 3). Current use of metformin compared with never use was associated with a significantly lower stroke risk (OR 0.54; 95% CI, 0.31-0.95) and a non-significantly lower MI risk (OR 0.85; 95% CI, 0.55-1.30). For both outcomes, there was no evidence of an interaction between current sulfonylurea use and current metformin use. Neither outcome was significantly associated with the past use of sulfonylureas or the past use of metformin.
Table 3.
Sulfonylurea and metformin associations with myocardial infarction and stroke risks among users of long-acting insulin.
Myocardial Infarction | Stroke | |||||||
---|---|---|---|---|---|---|---|---|
Cases | Controls | OR | 95% CI | Cases | Controls | OR | 95% CI | |
Sulfonylureas (SU) | ||||||||
Current use | 109 | 98 | 1.67 | 1.10-2.55 | 60 | 104 | 1.22 | 0.74-2.00 |
Past use | 109 | 83 | 1.12 | 0.75-1.69 | 66 | 90 | 0.94 | 0.57-1.53 |
Never use | 187 | 170 | 1 (ref) | 118 | 175 | 1 (ref) | ||
Metformin (MET) | ||||||||
Current use | 83 | 111 | 0.85 | 0.55-1.30 | 40 | 117 | 0.54 | 0.32-0.92 |
Past use | 94 | 59 | 1.20 | 0.76-1.90 | 45 | 63 | 0.77 | 0.44-1.32 |
Never use | 228 | 181 | 1 (ref) | 159 | 189 | 1 (ref) | ||
Other comparisons | ||||||||
Current SU vs past SU | 1.49 | 0.93-2.39 | 1.30 | 0.74-2.26 | ||||
Current MET vs past MET | 0.71 | 0.42-1.17 | 0.71 | 0.38-1.34 | ||||
Current SU vs current MET | 1.97 | 1.05-3.70 | 2.24 | 1.05-4.76 |
CI = confidence interval, OR = odds ratio, SU = sulfonylurea, MET = metformin. All analyses were adjusted for index year, age, sex, hypertension status, smoking, prior cardiovascular disease, atrial fibrillation, nephrotic syndrome, diastolic blood pressure, body mass index, total cholesterol, serum creatinine, duration of diabetes, and the use of statins, ACE inhibitors, and regular and rapid-acting insulin.
Although comparisons of current users with past users were limited by small numbers, the odds ratios for the sulfonylurea comparisons were similar for MI (1.49) and stroke (1.30), as were the odds ratios for the metformin comparisons for MI (0.71) and stroke (0.71). Compared with the current use of metformin, the current use of sulfonylureas was associated with higher risks of both MI (OR 1.97; 95% CI, 1.05-3.70) and stroke (OR 2.24; 95% CI, 1.05-4.76).
There was no evidence for a dose-response trend for either sulfonylureas or metformin (Appendix Table 1). The most commonly used sulfonylureas were glyburide (66%) and glipizide (28%); the MI and stroke associations for these two drugs were similar to the overall sulfonylurea associations (Appendix Table 2).
Overall, the statistical evidence for interactions was weak (Figure 2). The sulfonylurea-MI association was greater with age ≥ 70 vs. < 70 (OR 2.61 vs 1.25, Pinteraction = 0.056) and with HbA1c < 8% vs. ≥ 8% (OR 2.47 vs. 1.13, Pinteraction = 0.044). The metformin-stroke association was weaker with age >=70 vs. <70 (OR 0.98 vs. 0.37, Pinteraction = 0.054). Use of statins, ACE inhibitors, and beta blockers did not result in significant drug-drug interactions. Multiple testing corrections were not made for these analyses of effect modification.
Figure 2. Interaction and subgroup analyses of sulfonylurea and metformin associations with myocardial infarction and stroke risks among users of long-acting insulin.
CI = confidence interval, DM = diabetes mellitus, OR = odds ratio, P (int) = P value for interaction. Subgroups based on continuous variables were dichotomized at the median. All analyses were adjusted for index year, age, sex, hypertension status, smoking, prior cardiovascular disease, atrial fibrillation, nephrotic syndrome, diastolic blood pressure, body mass index, total cholesterol, serum creatinine, duration of diabetes, and the use of statins, ACE inhibitors, and regular and rapid-acting insulin. Dotted lines indicate the odds ratio for the comparison of current users with never users from primary analyses.
Results from sensitivity analyses were similar to the primary findings, with sulfonylurea-MI ORs ranging from 1.58 to 1.73 and metformin-stroke ORs ranging from 0.49 to 0.62 (Appendix Table 3). Adjusting for HbA1c did not attenuate the sulfonylurea-MI (OR 1.62; 95% CI, 1.06-2.46) or the metformin-stroke associations (OR 0.54; 95% CI, 0.32-0.91); neither did excluding subjects with creatinine > 1.5 mg/dL or with congestive heart failure. An unmeasured confounder associated with a 2-fold increased MI risk would have to be present in 70% more current users than never users of sulfonylureas to render the sulfonylurea-MI association null (Appendix Table 4).
DISCUSSION
In this case-control study of patients with type 2 diabetes using long-acting insulin, we evaluated separately the associations of sulfonylurea and metformin therapy with the risks of incident MI and incident stroke. Compared with the use of long-acting insulin alone, the use of sulfonylureas with long-acting insulin was associated with a 67% higher risk of MI and the use of metformin with long-acting insulin was associated with a 46% lower risk of stroke. It is possible but difficult to explain these drug-specific associations on the basis of confounding. Detailed information from medical records reviews was used to validate MI and stroke diagnoses and to assess potential confounding factors, including smoking and the duration of diabetes. Our findings were robust to a number of sensitivity analyses that evaluated assumptions about medication adherence, potential confounding and mediating variables, exclusion criteria, and the presence of unmeasured confounding.
Prior evidence about the cardiovascular safety of sulfonylureas has been conflicting. In a comprehensive systematic review of the cardiovascular safety of sulfonylureas that included 112 RCTs, sulfonylureas were associated with slightly increased the risks of stroke and all-cause mortality, but most trials were short in duration and reported few if any cardiovascular events.4 In the UK Prospective Diabetes Study (UKPDS), which contributed more events than any other trial, sulfonylureas did not increase the risk of cardiovascular events compared with diet control.26 However, a more recent RCT conducted in China27 and several observational studies6,7 have reported increased cardiovascular risks with sulfonylureas compared with metformin. One previous study used claims data from the national health system in Denmark to compare sulfonylureas with metformin among patients on any type of insulin therapy.28 The increased risk of cardiovascular events in that study (RR 1.26; 95% CI, 1.10-1.43) was lower than but broadly consistent with the sulfonylurea-metformin comparisons for MI (OR 2.0; 95% CI, 1.05-3.70) and stroke (OR 2.2; 95% CI, 1.05-4.76) in our study. The lower relative risk in the Danish study might be explained by the relatively short duration of sulfonylurea use (mean 92 days), the lack of validation of outcomes and confounders in that study, or perhaps differences in underlying risk between the Danish and GHC populations.
Most recent observational studies of the cardiovascular safety of sulfonylureas have included metformin as an active control treatment to address confounding by indication,25,28,29 but this approach has limitations.30 First, since metformin has become the first-line therapy for type 2 diabetes based on cardiovascular benefits observed in the UKPDS trial,8 patients using a sulfonylurea instead of metformin are now likely to have a contraindication to metformin or to be intolerant of its side effects.31 Comparisons of sulfonylureas with other non-metformin therapies as second-line treatment strategies are of greater interest, providing the motivation for the ongoing Glycemic Reduction Approaches in Diabetes comparative effectiveness trial.32 Second, the sulfonylurea-metformin comparison does not distinguish between harm from sulfonylureas, benefit form metformin, or the presence of both. Third, among patients on insulin therapy, the comparison of sulfonylureas with metformin does not reflect a clinical decision; oral drugs are not typically used as add-on therapies after insulin therapy has been initiated, but rather the oral drugs that had been used previously are either continued or discontinued.
We selected long-acting insulin without oral drugs as the active control treatment in our study, because insulin can be titrated to the desired degree of glycemic control for most patients and because the use of long-acting insulin marks a similar point in the progression of diabetes, whether or not oral drugs are used in combination. This approach offered some protection against confounding by indication while isolating the potential harms related to sulfonylureas from the effects of metformin. Moreover, between treatment groups, there were only small difference in the distribution of some measured risk factors, which were assessed carefully using information from electronic health databases, detailed medical record reviews, and interviews, and then adjusted for.
Several potential biologic mechanisms for the cardiovascular toxicity of sulfonylureas have been proposed. Sulfonylureas lower serum glucose levels by inhibiting ATP-sensitive potassium channels in pancreatic beta cells, stimulating insulin secretion.33 By inhibiting similar potassium channels in myocardial cells, sulfonylureas can prevent protective cellular responses that limit infarction after myocardial ischemia, including ischemic preconditioning.34 Experimental studies in animal models of ischemia have demonstrated this for some sulfonylureas including glyburide,35,36 but not others.37 In our study, the similar risks associated with glyburide and glipizide and the lack of a dose-related trend do not support the impairment of ischemic preconditioning as the primary explanation for the potential cardiovascular toxicity of sulfonylureas. The small difference in glycemic control observed between sulfonylurea users and nonusers, measured by HbA1c, is also an unlikely explanation.
A more plausible mechanism might involve hypoglycemia, which occurs more often with sulfonylureas than with other oral therapies.4 Hypoglycemia increases sympathetic activity and triggers the release of counterregulatory hormones, which results in increased myocardial work, platelet aggregation and activation, inflammation, and vessel wall stiffness.38 These physiologic effects of hypoglycemia have been hypothesized to precipitate myocardial and cerebral ischemia.39 In clinical and epidemiologic studies, severe hypoglycemia has been associated with electrocardiographic markers of ischemia40 and a 2-fold increase in cardiovascular events,41 but there is debate over whether this relationship is causal.42 Results from subgroup analyses in our study are consistent with hypoglycemia as a mediator of adverse cardiovascular effects; the sulfonylurea-MI association was greater among patients with tighter glycemic control and the elderly, populations that may be particularly susceptible to hypoglycemia and its consequences.43,44
In contrast with sulfonylureas, evidence is accumulating that metformin may reduce the risk of cardiovascular complications when used in combination with insulin. A systematic review of 26 RCTs found that combination therapy with metformin and insulin resulted in lower HbA1c and weight loss compared with insulin alone, but findings for cardiovascular events were inconclusive because of low event rates.13 In a small placebo-controlled RCT of type 2 diabetics on insulin therapy, metformin appeared to reduce the risk of cardiovascular events by 39% (95% CI, 6-60%), but substantial imbalances in prior cardiovascular disease between treatment groups limit the interpretation of this study.45 Previous RCTs and observational studies have not reported a reduction in stroke risk with metformin, either as monotherapy or in combination with insulin; the novel stroke finding in our study requires replication.
Metformin lowers serum glucose primarily by decreasing hepatic glucose production. The precise molecular mechanisms are not well characterized,46 but recent studies suggest that metformin inhibits glucagon signaling pathways and alters the hepatocellular redox state.47,48 In contrast with sulfonylureas, metformin reduces infarct size and impairment of cardiac function in animal models of ischemia,49,50 perhaps by improving energy metabolism51 or by limiting apoptosis.52 The mechanism by which metformin protects against cardiovascular disease appears to be independent of its glucose-lowering effects, and remains an active area of investigation.
Our study had several strengths. Drug exposures, potential confounding factors, and outcomes were assessed using detailed information from medical record reviews and from electronic health databases. Complete information on care received was available because subjects were enrolled in a closed health system. The choice of the active control group minimized the potential for confounding by indication; the use of long-acting insulin marks a similar point in the progression of diabetes, whether or not oral drugs are continued.
There were also limitations. Findings from this primary prevention population may not generalize to patients who have had prior cardiovascular events or patients using newer forms of long-acting insulin. Power to detect interactions was limited. We excluded subjects who were diagnosed with diabetes before age 21, but it is possible that some of the users of insulin who were never users of oral drugs may have had type 1 diabetes. As with all observational studies, there may have been residual confounding, which is the most important threat to the validity of our findings. For example, in our study sulfonylurea users and metformin users had a slightly lower prevalence of cardiovascular risk factors than did nonusers; if the assessment and adjustment for cardiovascular risk factors was incomplete, our results may have underestimated the harm from sulfonylureas and overestimated the benefit from metformin.
Despite the development of several new classes of glucose-lowering drugs in recent years, metformin and sulfonylureas remain the first- and second-most prescribed oral therapies.11 Current American and European treatment guidelines suggest that continuing sulfonylureas after initiating basal insulin therapy can prevent the deterioration of glycemic control, and the combination of sulfonylureas with insulin is one of several recommended second-line treatment strategies.14 In light of the potential increased risk of cardiovascular events, this recommendation may require reevaluation. Our study also contributes to a growing body of evidence that metformin may be an effective cardiovascular disease prevention therapy for patients with type 2 diabetes on insulin therapy, supporting current guidelines that recommend this combination.
Supplementary Material
KEY POINTS.
-About half of diabetic patients on insulin therapy continue to use oral glucose-lowering drugs, primarily metformin and sulfonylureas. The potential cardiovascular risks and benefits of this practice are largely unknown.
-In our study, the use of sulfonylureas with long-acting insulin was associated with an increased risk of myocardial infarction compared with the use of long-acting insulin alone. Guidelines that permit the use of sulfonylureas with insulin may require re-examination.
-The use of metformin with long-acting insulin was associated with a decreased risk of stroke compared with long-acting insulin alone, supporting guidelines that recommend metformin use for all patients with type 2 diabetes.
ACKNOWLEDGEMENTS
None
FUNDING
This research was supported by grants HL085251, HL073410, HL085251, and HL068986 from the National Heart, Lung, and Blood Institute (NHLBI). JSF was supported by grant K08HL116640 from the NHLBI. Funding agencies did not influence the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.
Footnotes
CONFLICTS OF INTEREST
BMP serves on the DSMB of a clinical trial of a device funded by the manufacturer (Zoll LifeCor) and on the Steering Committee of the Yale Open Data Access Project funded by Johnson & Johnson. No other potential conflicts of interest relevant to this article were reported.
CONTRIBUTIONS
JSF, WTL, BM, SRH, NSW, and BMP contributed to the conception and the design of the study. JSF, NLS, SRH, and BMP obtained funding. JSF conducted the analyses, drafted the manuscript, had full access to all of the study data, and takes responsibility for the integrity of the data and the accuracy of the data analysis. All authors contributed to the interpretation of the results, critically revised the manuscript for important intellectual content, and approved the final version of the manuscript. JSF is the guarantor of this work.
REFERENCES
- 1.Guariguata L, Whiting DR, Hambleton I, Beagley J, Linnenkamp U, Shaw JE. Global estimates of diabetes prevalence for 2013 and projections for 2035. Diabetes Res Clin Pract. 2014 Feb;103(2):137–149. doi: 10.1016/j.diabres.2013.11.002. [DOI] [PubMed] [Google Scholar]
- 2.Sarwar N, Gao P, Seshasai SR, et al. Diabetes mellitus, fasting blood glucose concentration, and risk of vascular disease: a collaborative meta-analysis of 102 prospective studies. Lancet. 2010 Jun 26;375(9733):2215–2222. doi: 10.1016/S0140-6736(10)60484-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Boussageon R, Supper I, Bejan-Angoulvant T, et al. Reappraisal of metformin efficacy in the treatment of type 2 diabetes: a meta-analysis of randomised controlled trials. PLoS Med. 2012;9(4):e1001204. doi: 10.1371/journal.pmed.1001204. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Monami M, Genovese S, Mannucci E. Cardiovascular safety of sulfonylureas: a meta- analysis of randomized clinical trials. Diabetes Obes Metab. 2013 Oct;15(10):938–953. doi: 10.1111/dom.12116. [DOI] [PubMed] [Google Scholar]
- 5.Meinert CL, Knatterud GL, Prout TE, Klimt CR. A study of the effects of hypoglycemic agents on vascular complications in patients with adult-onset diabetes. II. Mortality results. Diabetes. 1970;19(Suppl):789–830. [PubMed] [Google Scholar]
- 6.Phung OJ, Schwartzman E, Allen RW, Engel SS, Rajpathak SN. Sulphonylureas and risk of cardiovascular disease: systematic review and meta-analysis. Diabetic medicine : a journal of the British Diabetic Association. 2013 Oct;30(10):1160–1171. doi: 10.1111/dme.12232. [DOI] [PubMed] [Google Scholar]
- 7.Abdelmoneim AS, Eurich DT, Light PE, et al. Cardiovascular safety of sulphonylureas: over 40 years of continuous controversy without an answer. Diabetes Obes Metab. 2015 Jun;17(6):523–532. doi: 10.1111/dom.12456. [DOI] [PubMed] [Google Scholar]
- 8.Effect of intensive blood-glucose control with metformin on complications in overweight patients with type 2 diabetes (UKPDS 34). UK Prospective Diabetes Study (UKPDS) Group. Lancet. 1998 Sep 12;352(9131):854–865. [PubMed] [Google Scholar]
- 9.Wallia A, Molitch ME. Insulin therapy for type 2 diabetes mellitus. JAMA. 2014 Jun 11;311(22):2315–2325. doi: 10.1001/jama.2014.5951. [DOI] [PubMed] [Google Scholar]
- 10.Centers for Disease Control National Diabetes Statistics Report. 2014 [Google Scholar]
- 11.Hampp C, Borders-Hemphill V, Moeny DG, Wysowski DK. Use of antidiabetic drugs in the u.s., 2003-2012. Diabetes Care. 2014 May;37(5):1367–1374. doi: 10.2337/dc13-2289. [DOI] [PubMed] [Google Scholar]
- 12.Goudswaard AN, Furlong NJ, Rutten GE, Stolk RP, Valk GD. Insulin monotherapy versus combinations of insulin with oral hypoglycaemic agents in patients with type 2 diabetes mellitus. Cochrane database of systematic reviews. 2004;(4):CD003418. doi: 10.1002/14651858.CD003418.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Hemmingsen B, Christensen LL, Wetterslev J, et al. Comparison of metformin and insulin versus insulin alone for type 2 diabetes: systematic review of randomised clinical trials with meta-analyses and trial sequential analyses. BMJ. 2012;344:e1771. doi: 10.1136/bmj.e1771. [DOI] [PubMed] [Google Scholar]
- 14.Inzucchi SE, Bergenstal RM, Buse JB, et al. Management of hyperglycemia in type 2 diabetes: a patient-centered approach: position statement of the American Diabetes Association (ADA) and the European Association for the Study of Diabetes (EASD). Diabetes care. 2012 Jun;35(6):1364–1379. doi: 10.2337/dc12-0413. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Boger-Megiddo I, Heckbert SR, Weiss NS, et al. Myocardial infarction and stroke associated with diuretic based two drug antihypertensive regimens: population based case-control study. BMJ. 2010;340:c103. doi: 10.1136/bmj.c103. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Psaty BM, Heckbert SR, Atkins D, et al. The risk of myocardial infarction associated with the combined use of estrogens and progestins in postmenopausal women. Arch. Intern. Med. 1994 Jun 27;154(12):1333–1339. [PubMed] [Google Scholar]
- 17.Psaty BM, Heckbert SR, Koepsell TD, et al. The risk of myocardial infarction associated with antihypertensive drug therapies. JAMA : the journal of the American Medical Association. 1995 Aug 23-30;274(8):620–625. [PubMed] [Google Scholar]
- 18.Heckbert SR, Weiss NS, Koepsell TD, et al. Duration of estrogen replacement therapy in relation to the risk of incident myocardial infarction in postmenopausal women. Arch. Intern. Med. 1997 Jun 23;157(12):1330–1336. [PubMed] [Google Scholar]
- 19.Klungel OH, Heckbert SR, Longstreth WT, Jr., et al. Antihypertensive drug therapies and the risk of ischemic stroke. Arch. Intern. Med. 2001 Jan 8;161(1):37–43. doi: 10.1001/archinte.161.1.37. [DOI] [PubMed] [Google Scholar]
- 20.Miettinen O. Estimability and estimation in case-referent studies. Am J Epidemiol. 1976 Feb;103(2):226–235. doi: 10.1093/oxfordjournals.aje.a112220. [DOI] [PubMed] [Google Scholar]
- 21.Schafer JL. Analysis of Incomplete Multivariate Data. Chapman & Hall/CRC; Boca Raton, FL: 1997. [Google Scholar]
- 22.Kerr D, MacDonald IA, Heller SR, Tattersall RB. Beta-adrenoceptor blockade and hypoglycaemia. A randomised, double-blind, placebo controlled comparison of metoprolol CR, atenolol and propranolol LA in normal subjects. Br. J. Clin. Pharmacol. 1990 Jun;29(6):685–693. doi: 10.1111/j.1365-2125.1990.tb03689.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Herings RM, de Boer A, Stricker BH, Leufkens HG, Porsius A. Hypoglycaemia associated with use of inhibitors of angiotensin converting enzyme. Lancet. 1995 May 13;345(8959):1195–1198. doi: 10.1016/s0140-6736(95)91988-0. [DOI] [PubMed] [Google Scholar]
- 24.Lin DY, Psaty BM, Kronmal RA. Assessing the sensitivity of regression results to unmeasured confounders in observational studies. Biometrics. 1998 Sep;54(3):948–963. [PubMed] [Google Scholar]
- 25.Roumie CL, Hung AM, Greevy RA, et al. Comparative effectiveness of sulfonylurea and metformin monotherapy on cardiovascular events in type 2 diabetes mellitus: a cohort study. Ann. Intern. Med. 2012 Nov 6;157(9):601–610. doi: 10.7326/0003-4819-157-9-201211060-00003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33). UK Prospective Diabetes Study (UKPDS) Group. Lancet. 1998 Sep 12;352(9131):837–853. [PubMed] [Google Scholar]
- 27.Hong J, Zhang Y, Lai S, et al. Effects of metformin versus glipizide on cardiovascular outcomes in patients with type 2 diabetes and coronary artery disease. Diabetes Care. 2013 May;36(5):1304–1311. doi: 10.2337/dc12-0719. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Mogensen UM, Andersson C, Fosbol EL, et al. Sulfonylurea in combination with insulin is associated with increased mortality compared with a combination of insulin and metformin in a retrospective Danish nationwide study. Diabetologia. 2015 Jan;58(1):50–58. doi: 10.1007/s00125-014-3372-z. [DOI] [PubMed] [Google Scholar]
- 29.Berkowitz SA, Krumme AA, Avorn J, et al. Initial choice of oral glucose-lowering medication for diabetes mellitus: a patient-centered comparative effectiveness study. JAMA Intern Med. 2014 Dec 1;174(12):1955–1962. doi: 10.1001/jamainternmed.2014.5294. [DOI] [PubMed] [Google Scholar]
- 30.Patorno E, Patrick AR, Garry EM, et al. Observational studies of the association between glucose-lowering medications and cardiovascular outcomes: addressing methodological limitations. Diabetologia. 2014 Nov;57(11):2237–2250. doi: 10.1007/s00125-014-3364-z. [DOI] [PubMed] [Google Scholar]
- 31.Flory JH, Hennessy S. Metformin Use Reduction in Mild to Moderate Renal Impairment: Possible Inappropriate Curbing of Use Based on Food and Drug Administration Contraindications. JAMA Intern Med. 2015 Jan 5; doi: 10.1001/jamainternmed.2014.6936. [DOI] [PubMed] [Google Scholar]
- 32.Nathan DM, Buse JB, Kahn SE, et al. Rationale and design of the glycemia reduction approaches in diabetes: a comparative effectiveness study (GRADE). Diabetes Care. 2013 Aug;36(8):2254–2261. doi: 10.2337/dc13-0356. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Proks P, Reimann F, Green N, Gribble F, Ashcroft F. Sulfonylurea stimulation of insulin secretion. Diabetes. 2002 Dec;51(Suppl 3):S368–376. doi: 10.2337/diabetes.51.2007.s368. [DOI] [PubMed] [Google Scholar]
- 34.Meier JJ, Gallwitz B, Schmidt WE, Mugge A, Nauck MA. Is impairment of ischaemic preconditioning by sulfonylurea drugs clinically important? Heart. 2004 Jan;90(1):9–12. doi: 10.1136/heart.90.1.9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Grover GJ, Sleph PG, Dzwonczyk S. Role of myocardial ATP-sensitive potassium channels in mediating preconditioning in the dog heart and their possible interaction with adenosine A1-receptors. Circulation. 1992 Oct;86(4):1310–1316. doi: 10.1161/01.cir.86.4.1310. [DOI] [PubMed] [Google Scholar]
- 36.Toombs CF, Moore TL, Shebuski RJ. Limitation of infarct size in the rabbit by ischaemic preconditioning is reversible with glibenclamide. Cardiovasc. Res. 1993 Apr;27(4):617–622. doi: 10.1093/cvr/27.4.617. [DOI] [PubMed] [Google Scholar]
- 37.Mocanu MM, Maddock HL, Baxter GF, Lawrence CL, Standen NB, Yellon DM. Glimepiride, a novel sulfonylurea, does not abolish myocardial protection afforded by either ischemic preconditioning or diazoxide. Circulation. 2001 Jun 26;103(25):3111–3116. doi: 10.1161/01.cir.103.25.3111. [DOI] [PubMed] [Google Scholar]
- 38.Desouza CV, Bolli GB, Fonseca V. Hypoglycemia, diabetes, and cardiovascular events. Diabetes Care. 2010 Jun;33(6):1389–1394. doi: 10.2337/dc09-2082. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Wright RJ, Frier BM. Vascular disease and diabetes: is hypoglycaemia an aggravating factor? Diabetes. Metab. Res. Rev. 2008 Jul-Aug;24(5):353–363. doi: 10.1002/dmrr.865. [DOI] [PubMed] [Google Scholar]
- 40.Desouza C, Salazar H, Cheong B, Murgo J, Fonseca V. Association of hypoglycemia and cardiac ischemia: a study based on continuous monitoring. Diabetes care. 2003 May;26(5):1485–1489. doi: 10.2337/diacare.26.5.1485. [DOI] [PubMed] [Google Scholar]
- 41.Goto A, Arah OA, Goto M, Terauchi Y, Noda M. Severe hypoglycaemia and cardiovascular disease: systematic review and meta-analysis with bias analysis. BMJ. 2013;347:f4533. doi: 10.1136/bmj.f4533. [DOI] [PubMed] [Google Scholar]
- 42.Zoungas S, Patel A, Chalmers J, et al. Severe hypoglycemia and risks of vascular events and death. The New England journal of medicine. 2010 Oct 7;363(15):1410–1418. doi: 10.1056/NEJMoa1003795. [DOI] [PubMed] [Google Scholar]
- 43.Zammitt NN, Frier BM. Hypoglycemia in type 2 diabetes: pathophysiology, frequency, and effects of different treatment modalities. Diabetes care. 2005 Dec;28(12):2948–2961. doi: 10.2337/diacare.28.12.2948. [DOI] [PubMed] [Google Scholar]
- 44.McAulay V, Deary IJ, Frier BM. Symptoms of hypoglycaemia in people with diabetes. Diabetic medicine : a journal of the British Diabetic Association. 2001 Sep;18(9):690–705. doi: 10.1046/j.1464-5491.2001.00620.x. [DOI] [PubMed] [Google Scholar]
- 45.Kooy A, de Jager J, Lehert P, et al. Long-term effects of metformin on metabolism and microvascular and macrovascular disease in patients with type 2 diabetes mellitus. Arch. Intern. Med. 2009 Mar 23;169(6):616–625. doi: 10.1001/archinternmed.2009.20. [DOI] [PubMed] [Google Scholar]
- 46.Rena G, Pearson ER, Sakamoto K. Molecular mechanism of action of metformin: old or new insights? Diabetologia. 2013 Sep;56(9):1898–1906. doi: 10.1007/s00125-013-2991-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Miller RA, Chu Q, Xie J, Foretz M, Viollet B, Birnbaum MJ. Biguanides suppress hepatic glucagon signalling by decreasing production of cyclic AMP. Nature. 2013 Feb 14;494(7436):256–260. doi: 10.1038/nature11808. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Madiraju AK, Erion DM, Rahimi Y, et al. Metformin suppresses gluconeogenesis by inhibiting mitochondrial glycerophosphate dehydrogenase. Nature. 2014 Jun 26;510(7506):542–546. doi: 10.1038/nature13270. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Calvert JW, Gundewar S, Jha S, et al. Acute metformin therapy confers cardioprotection against myocardial infarction via AMPK-eNOS-mediated signaling. Diabetes. 2008 Mar;57(3):696–705. doi: 10.2337/db07-1098. [DOI] [PubMed] [Google Scholar]
- 50.Legtenberg RJ, Houston RJ, Oeseburg B, Smits P. Metformin improves cardiac functional recovery after ischemia in rats. Hormone and metabolic research = Hormon-und Stoffwechselforschung = Hormones et metabolisme. 2002 Apr;34(4):182–185. doi: 10.1055/s-2002-26705. [DOI] [PubMed] [Google Scholar]
- 51.Kawabata H, Ishikawa K. Cardioprotection by metformin is abolished by a nitric oxide synthase inhibitor in ischemic rabbit hearts. Hypertension research : official journal of the Japanese Society of Hypertension. 2003 Jan;26(1):107–110. doi: 10.1291/hypres.26.107. [DOI] [PubMed] [Google Scholar]
- 52.Elmadhun NY, Sabe AA, Lassaletta AD, Chu LM, Sellke FW. Metformin mitigates apoptosis in ischemic myocardium. The Journal of surgical research. 2014 Nov;192(1):50–58. doi: 10.1016/j.jss.2014.05.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.