Abstract
Objective
To evaluate the effectiveness and safety of metformin use in clinical practice in a large sample of pharmacologically treated patients with type 2 diabetes and different levels of renal function.
Design
Observational study between July 2004 and December 2010, mean follow-up 3.9 years.
Setting
Hospital outpatient clinics and primary care in Sweden.
Participants
51 675 men and women with type 2 diabetes, registered in the Swedish National Diabetes Register, and on continuous glucose-lowering treatment with oral hypoglycaemic agents (OHAs) or insulin.
Main outcome measures
Risks of cardiovascular disease (CVD), all-cause mortality and acidosis/serious infection, associated with each treatment regimens, were analysed in all patients and in subgroups with different estimated glomerular filtration rate (eGFR) intervals. Covariance adjustment and propensity scores were used to adjust for several baseline risk factors and characteristics at Cox regression.
Results
Compared with metformin in monotherapy, HRs for fatal/non-fatal CVD and all-cause mortality with all other OHAs combined (approximately 80% sulphonylureas) in monotherapy were 1.02 (95% CI 0.93 to 1.12) and 1.13 (1.01 to 1.27), while 1.18 (1.07 to 1.29) and 1.34 (1.19 to 1.50) with insulin in monotherapy, adjusting using propensity scores. Metformin, compared with any other treatment, showed reduced risks of acidosis/serious infection (adjusted HR 0.85, 95% CI 0.74 to 0.97) and all-cause mortality (HR 0.87, 95% CI 0.77 to 0.99), in patients with eGFR 45–60 ml/min/1.73 m2, and no increased risks of all-cause mortality, acidosis/serious infection or CVD were found in patients with eGFR 30–45 ml/min/1.73 m2.
Conclusions
Metformin showed lower risk than insulin for CVD and all-cause mortality and slightly lower risk for all-cause mortality compared with other OHA, in these 51 675 patients followed for 4 years. Patients with renal impairment showed no increased risk of CVD, all-cause mortality or acidosis/serious infection. In clinical practice, the benefits of metformin use clearly outbalance the risk of severe side effects.
Article summary
Article focus
To evaluate the risks of CVD, acidosis/serious infection and mortality associated with metformin and other glucose-lowering treatments, in a cohort of 51 675 type 2 diabetes patients and in subgroups with different degrees of renal impairment.
Key messages
Metformin was associated with reduced risk of CVD, acidosis/serious infection and all-cause mortality compared with insulin and a reduced risk of all-cause mortality compared with other OHAs.
The effects were consistent in patients with renal impairment (eGFR 45–60 ml/min/1.73 m2), and there were no increased risk of acidosis/serious infection even in patients with low renal function (eGFR 30–45 ml/min/1.73 m2).
Strengths and limitations of this study
A large cohort with comprehensive data on patient characteristics was studied.
A composite end point including diagnosis of acidosis, shock, acute renal failure and serious infections was used to evaluate the occurrence of lactic acidosis.
Introduction
Type 2 diabetes mellitus (DM2) is a common disease, which causes major morbidity and mortality due to micro- and macrovascular complications.1 A range of glucose-lowering agents with different properties aims at preventing these complications. The UK Prospective Diabetes Study (UKPDS) demonstrated a reduced risk of all-cause mortality in the subgroup of obese DM2 patients treated with metformin compared with sulphonylureas, insulin or diet alone.2 3 Further beneficial effects with metformin have been recognised.4 5 Thus, international treatment guidelines recommend metformin as first-line pharmacological treatment in DM2 patients primarily based on the results from the UKPDS substudy including 342 patients on metformin.6–9
Metformin have been considered causing increased risk of lactic acidosis. Consequently, metformin treatments have been contraindicated in patients at risk of developing lactic acidosis, for example, patients with cardiovascular and renal disease.10 Given the high prevalence of micro- and macrovascular disease in the DM2 population,11 a relatively large proportion was comprehended by the contraindications. However, several studies have suggested this concern to be exaggerated.12–14
In the light of these findings, most guidelines have become less strict towards metformin treatment in these patients.6 8 However, there is still a great need for clinical and epidemiological studies investigating the overall effects of metformin in patients considered vulnerable to such treatment. Therefore, the aim of this survey was to investigate benefits and risks associated with different glucose-lowering medications, in a cohort of 51 675 DM2 patients in clinical practice and in subgroups of patients with different degrees of renal impairment.
Material and methods
In this population-based longitudinal study information was linked from four national registers in Sweden: the National Diabetes Register (NDR), the prescribed drug register,15 the patient register and the cause of death register.16 17 The NDR is based and administered in Gothenburg, Sweden. Since the establishment in 1996, NDR has been working with systematic quality improvement, research and development in the field of diabetes mellitus. In 2009, NDR covered 262 333 patients with type 1 diabetes mellitus and DM2.18–21 Physicians and nurses in hospital outpatient clinics and primary healthcare clinics report to the NDR at least annually via the internet or via direct transfer of data from medical record databases. All included patients have agreed to be registered before inclusion.
Study population
This study, approved by the central ethical review board at the University of Gothenburg, comprises 51 675 DM2 patients. All pharmacologically treated DM2 patients aged ≥40 to <85 years and registered in the NDR between 1 July 2004 and 31 December 2007 were eligible for inclusion in the study (online appendix figure 1). DM2 was defined as treatment with diet only, oral hypoglycaemic agents (OHAs) only or onset age of diabetes ≥40 years and treatment with insulin only or combined with OHA. Patients had to be registered in the NDR 1 year prior to and 1 year following their first prescription of glucose-lowering treatment. In order to achieve adequate length of the follow-up period, they had to be initiated on glucose-lowering treatment before 2007 to be included.
In each patient, baseline was defined as occurring after 12 months of continuous use of the prescribed glucose-lowering medication. Only patients who had filled at least three prescriptions or 18 fills of multidose dispensed drugs during this 12-month period were included in the study. Patients who had collected both ordinary prescriptions and multidose dispensed drugs were excluded. Thus, 12 months of continuous glucose-lowering medication at baseline was ensured. The patients were classified according to glucose-lowering treatment, and clinical characteristics were analysed at baseline. All patients with data available for the analysed variables were included. Other OHA consisted of patients treated with all OHAs other than metformin. The majority of this group (approximately 80%) was treated with sulphonylureas (SU).
Baseline
Variables assessed at baseline are presented in table 1. History of congestive heart failure (CHF) and cardiovascular disease (CVD) were defined as at least one event of CHF or CVD, respectively, anytime between the year of 1987 and the start of the study. History of serious infections was defined as at least one severe infection within 6 months prior to baseline and the variable previous hospitalisation as hospitalisation for at least three consecutive days within 6 months prior to baseline.
Table 1.
Baseline characteristics in all 51 675 type 2 diabetes patients and in groups based on glucose-lowering treatments
| Metformin only | Metformin + other OHA | Metformin + insulin | Insulin only | Other OHA only | Insulin + other OHA | Metformin + insulin + other OHA | Total | |
| N | 14 697 (28%) | 8807 (17%) | 7109 (14%) | 12 291 (24%) | 5171 (10%) | 1365 (2.6%) | 2235 (4.3%) | 51 675 (100%) |
| Age (years) | 63.8 (9.7) | 65.4 (9.7) | 64.6 (8.8) | 65.2 (10.5) | 69.7 (9.5) | 69.6 (9.1) | 64.7 (8.5) | 65.3 (9.8) |
| HbA1c (%) | 6.9 (3.1) | 7.3 (3.2) | 7.7 (3.4) | 7.6 (3.4) | 6.9 (3.0) | 7.7 (3.3) | 7.9 (3.4) | 7.3 (3.3) |
| <6.9 | 8131 (55.3%) | 3296 (37.4%) | 1710 (24.1%) | 3433 (27.9%) | 2898 (56.0%) | 326 (23.9%) | 361 (16.2%) | 20 155 (39.0%) |
| 6.9–8.8 | 6037 (41.1%) | 4982 (56.6%) | 4240 (59.6%) | 7045 (57.3%) | 2139 (41.4%) | 845 (61.9%) | 1478 (66.1%) | 26 766 (51.8%) |
| >8.8 | 529 (3.6%) | 529 (6.0%) | 1159 (16.3%) | 1813 (14.8%) | 134 (2.6%) | 194 (14.2%) | 396 (17.7%) | 4754 (9.2%) |
| Systolic blood pressure (mm Hg) | 139.4 (16.6) | 141.2 (16.9) | 140.8 (16.8) | 138.9 (18.0) | 141.4 (17.5) | 141.9 (18.2) | 141.9 (16.4) | 140.1 (17.2) |
| Diabetes duration (years) | 4.6 (4.3) | 8.9 (5.9) | 11.6 (7.0) | 14.3 (10.4) | 7.5 (6.2) | 11.6 (6.8) | 12.0 (6.2) | 9.4 (8.0) |
| eGFR (ml/min/1.73 m2) | 82.0 (20.2) | 80.4 (21.1) | 79.2 (21.5) | 73.6 (23.8) | 73.8 (21.1) | 68.9 (22.4) | 79.5 (21.2) | 78.1 (21.9) |
| <45 | 231 (1.6%) | 222 (2.5%) | 238 (3.3%) | 1370 (11.1%) | 404 (7.8%) | 214 (15.7%) | 63 (2.8%) | 2742 (5.3%) |
| 45–60 | 1572 (10.7%) | 1167 (13.3%) | 1024 (14.4%) | 1955 (15.9%) | 888 (17.2%) | 255 (18.7%) | 316 (14.1%) | 7177 (13.9%) |
| >60 | 12 894 (87.7%) | 7418 (84.2%) | 5847 (82.2%) | 8966 (72.9%) | 3879 (75.0%) | 896 (65.6%) | 1856 (83.0%) | 41 756 (80.8%) |
| BMI (kg/m2) | 30.7 (4.9) | 29.9 (4.9) | 31.6 (5.0) | 27.4 (4.8) | 27.2 (4.5) | 28.3 (4.9) | 31.1 (5.1) | 29.5 (5.1) |
| Non-HDL-C (mmol/l) | 3.64 (1.00) | 3.53 (0.98) | 3.39 (0.97) | 3.35 (0.98) | 3.60 (0.99) | 3.52 (1.04) | 3.39 (0.94) | 3.50 (0.99) |
| Microalbuminuria | 21.0% | 25.9% | 33.8% | 30.8% | 24.2% | 34.4% | 34.5% | 27.2% |
| Previous hospitalisation | 11.5% | 11.6% | 18.1% | 23.1% | 15.5% | 22.9% | 15.2% | 16.1% |
| Female sex | 44.7% | 40.0% | 43.5% | 40.2% | 39.2% | 39.8% | 41.4% | 41.9% |
| History of CVD | 15.9% | 17.6% | 25.9% | 26.4% | 21.4% | 30.2% | 24.6% | 21.4% |
| History of CHF | 3.5% | 3.9% | 7.3% | 8.9% | 6.4% | 10.1% | 5.5% | 5.9% |
| History of serious infections | 1.8% | 1.6% | 2.9% | 4.9% | 2.8% | 5.2% | 2.7% | 2.9% |
| Cardiac glycosides | 2.6% | 2.7% | 4.1% | 3.0% | 4.3% | 6.2% | 4.3% | 3.3% |
| Organic nitrates | 6.3% | 7.4% | 9.7% | 8.5% | 8.8% | 13.0% | 10.1% | 8.1% |
| Acetylsalicylic acid | 45.4% | 48.8% | 57.3% | 45.7% | 45.9% | 55.4% | 58.6% | 48.6% |
| Lipid-modifying agents | 49.9% | 54.0% | 61.7% | 44.8% | 44.1% | 51.7% | 63.6% | 51.1% |
| Antihypertensive agents | 71.8% | 74.0% | 81.9% | 66.2% | 71.3% | 79.5% | 83.0% | 72.9% |
| Multidose dispensation | 1.4% | 1.6% | 2.7% | 3.0% | 1.5% | 3.7% | 2.6% | 2.1% |
| Smoker | 14.8% | 14.0% | 12.8% | 14.9% | 12.3% | 12.2% | 13.6% | 14.0% |
Means ± 1 SD and frequencies (%) are given. There were statistically significant differences (p<0.001) in all variables between the groups.
BMI, body mass index; CHF, congestive heart failure; CVD, cardiovascular disease; eGFR, estimated glomerular filtration rate; HDL-C, high-density lipoprotein-cholesterol; OHA, oral hypoglycaemic agent.
The patients were screened using methods applied at each local centre, but guidelines were available to ensure the use of similar methodology at all centres. Body mass index (BMI) was calculated as weight in kilograms divided by height in metres squared. A smoker was a patient smoking one or more cigarettes per day or a pipe daily or who had stopped smoking within the past 3 months. Cumulative microalbuminuria was defined as urine albumin excretion >20 μg/min in two of the three consecutive tests. Laboratory analyses, including total cholesterol and high-density lipoprotein-cholesterol (HDL-C), were carried out at local laboratories. HbA1c analyses are quality assured in Sweden by regular calibration with Mono-S, a high-performance liquid chromatography method. HbA1c values were converted to the National Glycohemoglobin Standardization Program standard levels.22 Non-HDL-C was calculated by subtracting HDL-C from total cholesterol. Estimated glomerular filtration rate (eGFR) was calculated using the Modification of Diet in Renal Disease equation.23
Follow-up
All patients were followed from baseline until the occurrence of an end point event, or otherwise, until censor date of 31 December 2010. Patients who experienced an end point event between first prescription and baseline were excluded from the analysis of that specific end point. Patients changing treatment during the study were not censored, and end point events were attributed the initial treatment. Mean follow-up was 3.9 years. The five end points analysed were any CVD, fatal CVD, any acidosis/serious infection, fatal acidosis/serious infection and all-cause mortality. CVD was defined as diagnosis of myocardial infarction, angina pectoris, intracerebral haemorrhage, cerebral infarction, unspecified stroke, peripheral vascular disease, or intervention with percutaneous coronary intervention or coronary artery bypass grafting, whichever occurred first. Acidosis/serious infection was defined as diagnosis of acidosis, serious infection, shock or acute renal failure, which are frequently associated with lactic acidosis. Serious infections requiring hospitalisation for anti-infectious treatment as well as acidosis, shock and acute renal failure requiring treatment in hospital, usually intensive care thus, registered in the inpatient register, were included in the composite end point. The International Classifications of Diseases-10 codes for all end points are given in online appendix materials. A fatal event was defined as an event followed by death in the subsequent 28 days.
Statistical methods
Baseline characteristics were compared, unadjusted, using analysis of variance and logistic regression (table 1). Propensity scores were estimated using boosted CART24 since logistic regression did not achieve good balance. Baseline characteristics were then compared using logistic regression or OLS regression, adjusted for octiles of the propensity score (online appendix table 1). Unadjusted survival of the end points by treatment groups in all patients was estimated with the Kaplan–Meier estimator (figure 1).
Figure 1.
Time (months) to event of all-cause mortality (A), any cardiovascular disease (CVD) (B) and any acidosis/serious infection (C) in each treatment group, unadjusted. OHA, oral hypoglycaemic agent.
Cox regression models were used to estimate HRs for all end points in groups of patients with different glucose-lowering treatments and metformin only as reference. Adjustments were made for age, sex, diabetes duration, HbA1c, non-HDL-C, BMI, smoking, eGFR, multidose dispensation, previous hospitalisation, history of CVD and CHF, microalbuminuria, and treatment with antihypertensive agents, lipid-lowering agents and cardiac glycosides (table 2). HRs were also estimated in patients with insulin only or other OHA only compared with metformin only, adjusted by stratification for octiles of propensity scores as described above (table 3). HRs were estimated in subgroups with different eGFR intervals (table 4), for metformin, insulin or other OHA, with any other glucose-lowering treatment as reference. Adjustment was made for same covariates as in table 2.
Table 2.
Adjusted HRs with 95% CIs for any CVD, fatal CVD, any acidosis/serious infection, fatal acidosis/serious infection and all-cause mortality in all patients, in each treatment group, and with metformin only as reference
| Metformin only | Insulin only | Other OHA only | Insulin + other OHA | Metformin + other OHA | Metformin + insulin | Metformin + insulin + other OHA | |
| Any CVD | Reference | 1.28 (1.19 to 1.37)*** | 1.13 (1.04 to 1.23)** | 1.40 (1.24 to 1.58)*** | 1.11 (1.03 to 1.20)** | 1.28 (1.19 to 1.38)*** | 1.33 (1.19 to 1.49)*** |
| Fatal CVD | Reference | 1.41 (1.18 to 1.68)*** | 1.30 (1.08 to 1.56)** | 1.17 (0.91 to 1.51) | † | † | 1.21 (0.92 to 1.58) |
| Any acidosis/serious infection | Reference | 1.37 (1.26 to 1.50)*** | 1.16 (1.04 to 1.28)** | 1.31 (1.13 to 1.51)*** | 1.04 (0.95 to 1.14) | 1.20 (1.09 to 1.32)*** | 1.15 (1.00 to 1.32)* |
| Fatal acidosis/serious infection | Reference | 1.63 (1.29 to 2.07)*** | 1.28 (0.98 to 1.67) | 1.32 (0.91 to 1.89) | 0.94 (0.72 to 1.23) | 1.41 (1.08 to 1.83)* | 1.12 (0.73 to 1.67) |
| All-cause mortality | Reference | 1.47 (1.35 to 1.61)*** | 1.30 (1.18 to 1.44)*** | 1.30 (1.12 to 1.50)*** | 1.15 (1.05 to 1.27)** | 1.25 (1.13 to 1.38)*** | 1.31 (1.14 to 1.52)*** |
Adjustments were made for age, sex, diabetes duration, HbA1c, non-high-density lipoprotein-cholesterol, body mass index, smoking, estimated glomerular filtration rate, multidose dispensation, previous hospitalisation, history of CVD and congestive heart failure, microalbuminuria, and treatment with antihypertensive agents, lipid-lowering agents and cardiac glycosides.
*p<0.05; **p<0.01; ***p<0.001.
Non-proportional hazards, group excluded from analysis.
CVD, cardiovascular disease; OHA, oral hypoglycaemic agent.
Table 3.
Adjusted HRs with 95% CIs for any CVD, fatal CVD, any acidosis/serious infection, fatal acidosis/serious infection and all-cause mortality in patients with insulin only and patients with metformin only as reference or in patients with other OHA only and patients with metformin only as reference
| Events/patients (N/N) | Events/patients (N/N) | HR (95% CI) | p Value | |
| Insulin only | Metformin only | Insulin only vs metformin only | ||
| Any CVD | 2389/11 427 | 1734/14 317 | 1.18 (1.07 to 1.29) | <0.001 |
| Fatal CVD | 681/12 285 | 264/14 696 | 1.12 (0.91 to 1.40) | 0.29 |
| Any acidosis/serious infection | 1867/11 860 | 1154/14 517 | 1.28 (1.14 to 1.43) | <0.001 |
| Fatal acidosis/serious infection | 325/12 284 | 127/14 697 | 1.45 (1.07 to 1.97) | 0.019 |
| All-cause mortality | 2002/12 291 | 971/14 697 | 1.34 (1.19 to 1.50) | <0.001 |
| Other OHA only | Metformin only | Other OHA only vs metformin only | ||
| Any CVD | 929/4964 | 1734/14 317 | 1.02 (0.93 to 1.12) | 0.71 |
| Fatal CVD | 237/5171 | 264/14 696 | 1.03 (0.84 to 1.26) | 0.80 |
| Any acidosis/serious infection | 623/5062 | 1154/14 517 | 1.05 (0.94 to 1.18) | 0.41 |
| Fatal acidosis/serious infection | 109/5171 | 127/14 697 | 1.13 (0.83 to 1.53) | 0.44 |
| All-cause mortality | 745/5171 | 971/14 697 | 1.13 (1.01 to 1.27) | 0.032 |
Each comparison was adjusted by stratification with octiles of propensity scores.
CVD, cardiovascular disease; OHA, oral hypoglycaemic agent.
Table 4.
Adjusted HRs with 95% CIs for any CVD, any acidosis/serious infection and all-cause mortality in subgroups of patients with different eGFR intervals
| 30≤ eGFR <45 |
45≤ eGFR <60 |
eGFR ≥60 |
All patients |
||||||||
| N (% of total) | Events (% of total) | HR (95 % CI) | N (% of total) | Events (% of total) | HR (95 % CI) | N (% of total) | Events (% of total) | HR (95 % CI) | N | Events | |
| Any CVD | |||||||||||
| Metformin | 670 (35.4) | 210 (30.7) | 1.00 (0.83 to 1.19) | 3839 (57.7) | 849 (51.2) | 0.94 (0.84 to 1.05) | 27 083 (67.3) | 3698 (63.4) | 0.98 (0.92 to 1.05) | 31 628 | 4774 |
| Insulin | 1180 (62.3) | 474 (69.2) | 1.30 (1.02 to 1.64)* | 3201 (48.1) | 930 (56.1) | 1.24 (1.09 to 1.42)** | 16 718 (41.5) | 2853 (48.9) | 1.19 (1.11 to 1.27)*** | 21 503 | 4476 |
| Other OHA | 702 (37.1) | 241 (35.2) | 1.03 (0.85 to 1.26) | 2450 (36.8) | 608 (36.7) | 1.05 (0.93 to 1.18) | 13 552 (33.7) | 2065 (35.4) | 1.03 (0.97 to 1.09) | 16 817 | 2965 |
| Total in group | 1894 | 685 | 6655 | 1657 | 40 239 | 5829 | |||||
| Any acidosis/serious infection | |||||||||||
| Metformin | 692 (33.9) | 143 (28.4) | 0.98 (0.79 to 1.21) | 4000 (57.5) | 557 (49.4) | 0.85 (0.74 to 0.97)* | 27 618 (67.3) | 2444 (60.6) | 0.91 (0.84 to 0.98)* | 32 345 | 3155 |
| Insulin | 1302 (63.7) | 366 (72.6) | 1.34 (1.02 to 1.76)* | 3406 (48.9) | 652 (57.9) | 1.07 (0.91 to 1.26) | 17 152 (41.8) | 2057 (51) | 1.22 (1.12 to 1.32)*** | 22 310 | 3260 |
| Other OHA | 738 (36.1) | 166 (32.9) | † | 2555 (36.7) | 379 (33.6) | 0.87 (0.75 to 1.00) | 13 852 (33.7) | 1375 (34.1) | 1.02 (0.95 to 1.09) | 17 265 | 1960 |
| Total in group | 2044 | 504 | 6960 | 1127 | 41 048 | 4034 | |||||
| All-cause mortality | |||||||||||
| Metformin | 715 (33.3) | 179 (27) | 1.02 (0.84 to 1.24) | 4079 (56.8) | 558 (46.5) | 0.87 (0.77 to 0.99)* | 28 015 (67.1) | 2120 (56.9) | 0.87 (0.81 to 0.94)*** | 32 848 | 2873 |
| Insulin | 1386 (64.6) | 468 (70.5) | 1.16 (0.91 to 1.47) | 3550 (49.5) | 701 (58.4) | 1.12 (0.95 to 1.31) | 17 565 (42.1) | 1921 (51.5) | 1.29 (1.19 to 1.41)*** | 23 000 | 3328 |
| Other OHA | 766 (35.7) | 222 (33.4) | 0.97 (0.79 to 1.19) | 2626 (36.6) | 429 (35.7) | 0.97 (0.84 to 1.11) | 14 049 (33.6) | 1375 (36.9) | 1.10 (1.02 to 1.19)* | 17 578 | 2087 |
| Total in group | 2146 | 664 | 7177 | 1201 | 41 756 | 3729 | |||||
HR associated with the examined agent in any combination is given with any other glucose-lowering treatment as reference. Adjustments were made for age, sex, diabetes duration, HbA1c, non-high-density lipoprotein-cholesterol, body mass index, smoking, eGFR, multidose dispensation, previous hospitalisation, history of CVD and congestive heart failure, microalbuminuria, and treatment with antihypertensive agents, lipid-lowering agents and cardiac glycosides.
*p<0.05; **p<0.01; ***p<0.001.
Non-proportional hazards, group excluded from analysis.
CVD, cardiovascular disease; eGFR, estimated glomerular filtration rate; OHA, oral hypoglycaemic agents.
Functional form of continuous covariates was checked using a Kolmogorov-type supremum test,25 and in some models, it was found suitable to add a quadratic term for age or diabetes duration. The proportional hazards assumption was checked by including the interaction between covariates and log of follow-up time. Violations of the proportional hazards assumption were handled by stratifying on the violating covariate or by modelling the effect as time dependent.26 Additional checks of the form of the time dependence were made using plots of scaled Schoenfeld residuals.27
All statistical analyses were performed with SAS V.9.2 (SAS Institute), except Kaplan–Meier curves produced in SPSS V.18 (SPSS Inc.) and propensity scores estimated using the package TWANG in R (R Foundation for Statistical Computing). A two-sided p value <0.05 was considered statistically significant.
Results
Patients
Table 1 gives the distribution between treatments and clinical characteristics at baseline, unadjusted. The total population presented a mean (±SD) age of 65.3±9.8 years with mean diabetes duration of 9.4±8.0 years and a mean HbA1c of 7.3±3.3%. The proportion of women was 41.9%, and 14.0% of the population were smokers. Mean BMI was 29.5±5.1 kg/m2, mean systolic blood pressure 140 mm Hg, mean non-HDL 3.5 mmol/l and mean eGFR 78.1±21.9 ml/min/1.73 m2. History of CVD was found in 21.4% of population, history of CHF in 5.9% and history of serious infections in 2.9% . There were statistically significant differences between the groups defined for all variables. Patients on insulin-based treatments presented longer diabetes duration, higher mean HbA1c, more often microalbuminuria and history of CVD, CHF and serious infections than the population in general.
Patients treated with metformin generally presented high eGFR and BMI. Patients on metformin in monotherapy were the youngest participants, with the shortest diabetes duration, and had a low mean HbA1c. They also relatively seldom had history of CVD, CHF or serious infections. Patients treated with other OHA in monotherapy presented the highest mean age, the lowest mean HbA1c and the lowest mean BMI. After adjustment with propensity score, all differences in baseline characteristics except for history of CHF and BMI were erased (online appendix table 1). CHF and BMI were further adjusted for with stratification and as a covariate, respectively. Median daily doses of metformin were approximately 1100 mg in the metformin monotherapy group, 1700 mg in metformin + other OHA, 1700 mg in metformin + insulin and 1900 mg in metformin + insulin + other OHA.
Online appendix table 2 gives time of exposure to the glucose-lowering agents and proportions of patients changing treatment, in each group. The proportions changing treatment ranged between 56.5% and 93.9%. Comparison of baseline characteristics in patients who changed treatment and patients who did not change treatment showed significant differences, with, for example, more frequent history of CVD, CHF and serious infections in patients who did not change treatment (p<0.05).
Outcomes
Figure 1 shows unadjusted time to an event of all-cause mortality, any CVD and any acidosis/serious infection in each treatment group. The steepest decreases of curves were seen with insulin only and insulin in combination with other OHA. Table 2 gives HR with 95% CIs for all end points, adjusted for covariates as given in the table. All treatments were associated with significantly increased risks of all-cause mortality and any CVD compared with metformin only, with HR ranging from 1.47 (95% CI 1.35 to 1.61) to 1.15 (95% CI 1.05 to 1.27) for all-cause mortality and from 1.40 (95% CI 1.24 to 1.58) to 1.11 (95% CI 1.03 to 1.20) for any CVD. Insulin only and other OHA only also showed a significantly increased risk of fatal CVD. All treatments except metformin in combination with other OHA were associated with a significantly increased risk of any acidosis/serious infection, and insulin only or in combination with metformin showed an increased risk of fatal acidosis/serious infection. Relatively few fatal events occurred during follow-up (online appendix table 3), contributing to the wider CIs for these risk estimates. Similar results were seen when using other OHA only as reference group instead of metformin only (online appendix table 4). This analysis also showed significantly increased risks of all end points except fatal CVD associated with insulin only compared with other OHA only. Furthermore, insulin only or in combination with other glucose-lowering agents was constantly associated with increased risk of any CVD compared with other OHA only.
Table 3 gives HR with 95% CI for all end points with insulin only or other OHA only compared with metformin only, adjusted for propensity score. Insulin was associated with significantly increased risks of any CVD (HR 1.18, 95% CI 1.07 to 1.29), all-cause mortality (HR 1.34, 95% CI 1.19 to 1.50) and also any acidosis/serious infection and fatal acidosis/serious infection (HR 1.28, 95% CI 1.14 to 1.43 and HR 1.45, 95% CI 1.07 to 1.97, respectively). When comparing other OHA only to metformin only, a borderline significantly increased risk was seen for all-cause mortality (HR 1.13, 95% CI 1.01 to 1.27). The results were identical when analysing SU only instead of other OHA only and metformin only as reference, with significant HR for all-cause mortality. As shown in online appendix table 5, HR was 0.99 (95% CI 0.89 to 1.09) for any CVD, 1.01 (95% CI 0.82 to 1.25) for fatal CVD, 1.15 (95% CI 1.02 to 1.30) for all-cause mortality, 1.00 (95% CI 0.89 to 1.14) for any acidosis/serious infection and 1.17 (95% CI 0.85 to 1.60) for fatal acidosis/serious infection.
As shown in online appendix table 6, insulin in combination with metformin was associated with a reduced risk for all-cause mortality (HR 0.84, 95% CI 0.76 to 0.91) and any acidosis/serious infection (HR 0.86, 95% CI 0.79 to 0.94) when compared with insulin only. No reduced risk for CVD was seen. Insulin in combination with other OHA was not associated with reduced risk for any of the end points compared with insulin only.
Table 4 gives HR with 95% CI for any CVD, any acidosis/serious infection and all-cause mortality in subgroups of patients with different eGFR intervals, adjusted for covariates as given in the table. Treatments with metformin, insulin or other OHA in any combination were compared with any other treatment. Metformin-based treatments were associated with reduced risks of any acidosis/serious infection (HR 0.85, 95% CI 0.74 to 0.97) and all-cause mortality (HR 0.87, 95% CI 0.77 to 0.99) in the subgroup of patients with eGFR 45–60 ml/min/1.73 m2. Similar results were seen in the subgroup with eGFR >60 ml/min/1.73 m2 (HR 0.91, 95% CI 0.84 to 0.98) for any acidosis/serious infection and (HR 0.87, 95% CI 0.81 to 0.94) for all-cause mortality. Both insulin and other OHA were associated with increased risk of all-cause mortality in patients with eGFR >60 ml/min/1.73 m2. Insulin was also associated with increased risk of any acidosis/serious infection in patients with eGFR 30–45 or >60 ml/min/1.73 m2 and increased risk of any CVD in patients with eGFR 30–45, 45–60 or >60 ml/min/1.73 m2.
Discussion
This population-based observational study demonstrates beneficial effects of metformin in clinical practice. As expected, there were significant differences in clinical characteristics between the groups. These differences, however, disappeared after adjustment with propensity scores. Still, metformin in monotherapy showed a significantly reduced risk of any CVD, all-cause mortality, any acidosis/serious infection and fatal acidosis/serious infection compared with insulin in monotherapy. A borderline significant risk reduction of all-cause mortality was also shown compared with other OHA in monotherapy. Furthermore, there was no increased risk of severe outcomes in patients with impaired renal function.
The beneficial effects of metformin, shown in this survey, are generally consistent with previous findings. The UKPDS demonstrated a reduction in all-cause mortality in the subgroup of obese DM2 patients treated with metformin compared with diet, SU or insulin,2 also confirmed in a 10-year post-interventional follow-up.3 Furthermore, several recent observational studies have reported reduced risk with metformin compared with all other hypoglycaemic agents for coronary heart disease28 29 and for total mortality in patients with previous coronary heart disease,14 as also seen in this study for total mortality in patients with normal or slightly reduced renal function. In the present study, a reduced risk of total mortality was also found when comparing insulin in combination with metformin to insulin only, although not verifying a finding of reduced risk of macrovascular events in a small clinical trial comparing the addition of metformin to placebo in insulin-treated DM2 patients.4
Interestingly, we found somewhat reduced risk for total mortality, but not for CVD, with metformin only compared with SU only or other OHA only. These findings were also clearly demonstrated regarding total mortality in two large recent observational studies on DM2 patients from Denmark and Cleveland, USA.30 31 However, the Danish study also found reduced risk for CVD with metformin compared with SU,31 using a propensity score, including age, sex, comorbidities, income and cardiac drugs as covariates, and used the score for matching limiting included patients. Possibly our study using propensity score for stratification of all included patients, and also including traditional cardiovascular risk factors as covariates, may better reflect the risk difference between metformin and SU, an important matter for clinical practice with many patients still given SU.
The increased risk for CVD and total mortality with insulin found in the present study could be due to these patients presenting a more severe disease. However, adjustment was made for diabetes duration and HbA1c among other covariates, and an increased risk caused by insulin per se cannot be ruled out, as also has been underlined in other recent observational studies.28 29 32–35 Lifestyle measures with weight reduction may be of value for obese patients with insulin included as treatment. The on-going ORIGIN randomised trial evaluating the risk of CVD with insulin glargine compared with standard glucose-lowering treatment in patients with diabetes at high risk for vascular disease will give further information.36
Subgroup analyses with patients presenting different degrees of renal impairment were conducted and did not show any increased risk of CVD, acidosis/serious infection or all-cause mortality associated with metformin-based treatments in patients with eGFR 30–45, 45–60 or >60 ml/min/1.73 m2. Rather, metformin-based treatments were associated with reduced risks of all-cause mortality and acidosis/serious infection in patients with eGFR 45–60 or >60 ml/min/1.73 m2. The prevalence of renal impairment differed between the groups, with patients presenting an eGFR <45 ml/min/1.73 m2 being rare in metformin-based treatments. However, the prevalence of eGFR 45–60 ml/min/1.73 m2 ranged between 10.7% and 14.4% in these patients and did not differ much from other patients. Consequently, the subgroup with eGFR 45–60 ml/min/1.73 m2 was based on a surprisingly large material, while the subgroup with eGFR 30–45 ml/min/1.73 m2 constituted relatively few patients.
A recently published observational study examined the effects of metformin in 19 691 patients with diabetes and advanced CVD, thus considered vulnerable to metformin.14 The results indicated significantly reduced risk of all-cause mortality in patients treated with metformin compared with other glucose-lowering treatments. Results were consistent in a subgroup of patients with renal impairment (eGFR 30–60 ml/min/1.73 m2). The survey, however, only analysed all-cause mortality and could therefore not detect potential cases of lactic acidosis. Furthermore, the follow-up was short (2 years) and did not analyse patients with eGFR 30–45 and 45–60 ml/min/1.73 m2, separately. Furthermore, glucose-lowering treatments were only specified as metformin use or not, and no adjustments for crucial covariates such as HbA1c or diabetes duration were made. Several studies have failed to demonstrate increased incidence rate of lactic acidosis in DM2 patients treated with metformin. Thus, DM2 with its comorbidities or any glucose-lowering treatment rather than metformin use per se have been suggested to be risk factors for lactic acidosis.12
The large population, 51 675 DM2 patients, and extensive adjustments for many important covariates are apparent strengths of the present survey. Risk calculations were made, adjusted for covariates with propensity scores and in Cox regression models. The propensity score achieved perfectly well-balanced groups regarding baseline characteristics indicating the robustness of this statistical analysis. The Cox regression models enabled more comprehensive comparisons between several glucose-lowering regimens. Data are collected from the NDR database with a currently estimated coverage of more than 90% of all patients in hospital outpatient clinics and almost 80% of all patients in primary care in Sweden, suggesting it to be highly representative of clinical practice. Furthermore, we presented exposure time for the different glucose-lowering treatments and median daily doses of metformin in each treatment group, which showed clinically relevant doses.
Despite extensive adjustments, covariates of possible importance could have been missed. Thus, the presence of confounding by indication may not be fully avoided. Furthermore, patients who changed glucose-lowering treatment during the study were not censored. It could be that patients with advancing disease more frequently changed to a specific glucose-lowering medication, diluting the results observed. Comparison of baseline characteristics indicated higher proportions of history of CVD, CHF and serious infections in patients changing treatment. This could have affected the results, even though the proportions of patients changing treatment were high in all groups. Only eight cases of diagnosed lactic acidosis were reported during the follow-up (four cases in metformin only, two cases in metformin + insulin and two cases in insulin only) and thus analyses with lactic acidosis as an end point would not provide desirable strength. Therefore, a composite end point (acidosis/serious infections), including diagnosis of serious infections (n=4782), acute renal failure (n=914), acidosis (n=167) and shock (n=17), was used. The six most common diagnoses patients were hospitalised for in this composite end point were pneumonia (unspecified), bacterial pneumonia (unspecified), acute renal failure (unspecified), acute tubulointerstitial nephritis, sepsis (unspecified) and gastroenteritis and colitis of unspecified origin. Altogether, this complicates the evaluation of lactic acidosis per se, although this diagnosis in practice only occurs in combination with severe infections or CVD. Furthermore, lactic acidosis reported with use of biguanides mostly involve phenformine, which was early withdrawn from the market, as lactic acidosis was 20 times more frequent than with metformin.12 In cases of lactic acidosis, plasma metformin concentration has also not proved to be of any prognostic significance.37 The patient group treated with other OHA were mainly treated with SU and to a very limited degree with glitazones, acarbose or DPP-4 inhibitors during the study period. Investigation of the individual effectiveness of these agents would, however, be of interest in the future.
In conclusion, this nationwide observational study of 51 675 DM2 patients supports the previously observed effectiveness of metformin. Metformin was associated with reduced risk of all-cause mortality compared with both insulin and other OHA and for several additional end points compared with insulin. The results were consistent in a subgroup of patients with renal impairment, and no increased risk of acidosis/serious infection was seen. Together with previous findings, this constitutes evident support to the less strict approach to metformin treatment in patients with renal impairment, advocated in most guidelines. Thus, considerably more DM2 patients may be considered for treatment with metformin.
Supplementary Material
Acknowledgments
We thank the patients, regional NDR coordinators and all participating nurses and physicians.
Footnotes
To cite: Ekström N, Schiöler L, Svensson A-M, et al. Effectiveness and safety of metformin in 51 675 patients with type 2 diabetes and different levels of renal function: a cohort study from the Swedish National Diabetes Register. BMJ Open 2012;2:e001076. doi:10.1136/bmjopen-2012-001076
Contributors: NE, LS, BZ, JC, A-MS, JMJ, KE-O, SG and BE contributed to the conception and design. LS, JC and A-MS contributed to the acquisition of data and performed the calculations. LS, NE, JC, BZ, SG, A-MS and BE contributed to the analysis and interpretation of data. NE and BE contributed to drafting the article. NE, BE, BZ, JC, A-MS, LS, KE-O, SG and JMJ contributed to revising the article critically for important intellectual content and final approval of the version to be submitted.
Funding: The Region Västra Götaland and the Swedish Association of Local Authorities and Regions fund the National Diabetes Register (NDR). The NDR is supported by the Swedish Society for Diabetology, and the Swedish Diabetes Association. LS, A-MS, JMJ, and SG were also supported by an unrestricted research grant from BMS, France. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.
Disclaimer: BZ is employed by the Medical Products Agency (MPA), Uppsala, Sweden. BZ has not received any financial support or other benefits from BMS, France, or any commercial sponsor. Results and views of the present study represent the authors and not necessarily any official views of the MPA where BZ is employed.
Competing interests: BE has served as a lecturer for all pharmacological companies manufacturing glucose-lowering agents and participated in advisory boards for Eli Lilly Sweden AB and Eli Lilly & Co, Sanofi-aventis, Sweden, Boehringer Ingelheim AB, Sweden, and MSD, Sweden.
Ethics approval: Ethics approval was provided by the central ethical review board at the University of Gothenburg.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data sharing statement: No additional data available.
References
- 1.Zimmet P, Alberti KG, Shaw J. Global and societal implications of the diabetes epidemic. Nature 2001;414:782–7 [DOI] [PubMed] [Google Scholar]
- 2.Anonymous. Effect of intensive blood-glucose control with metformin on complications in overweight patients with type 2 diabetes (UKPDS 34). UK prospective diabetes study (UKPDS) group. Lancet 1998;352:854–65 [PubMed] [Google Scholar]
- 3.Holman RR, Paul SK, Bethel MA, et al. 10-year follow-up of intensive glucose control in type 2 diabetes. N Engl J Med 2008;359:1577–89 [DOI] [PubMed] [Google Scholar]
- 4.Kooy A, de Jager J, Lehert P, et al. Long-term effects of metformin on metabolism and microvascular and macrovascular disease in patients with type 2 diabetes mellitus. Arch Intern Med 2009;169:616–25 [DOI] [PubMed] [Google Scholar]
- 5.Kahn SE, Haffner SM, Heise MA, et al. Glycemic durability of rosiglitazone, metformin, or glyburide monotherapy. N Engl J Med 2006;355:2427–43 [DOI] [PubMed] [Google Scholar]
- 6.Nathan DM, Buse JB, Davidson MB, et al. Medical management of hyperglycemia in type 2 diabetes: a consensus algorithm for the initiation and adjustment of therapy: a consensus statement of the American Diabetes Association and the European Association for the Study of Diabetes. Diabetes Care 2009;32:193–203 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Handelsman Y, Mechanick JI, Blonde L, et al. American Association of Clinical Endocrinologists Medical Guidelines for clinical practice for developing a diabetes mellitus comprehensive care plan. Endocr Pract 2011;17(Suppl 2):1–53 [DOI] [PubMed] [Google Scholar]
- 8.Type 2 Diabetes: National Clinical Guideline for Management in Primary and Secondary Care (Update). London: Royal College of Physicians of London, 2008 [PubMed] [Google Scholar]
- 9.National Guidelines For Diabetes Care. 2010. http://www.socialstyrelsen.se/nationalguidelines/nationalguidelinesfordiabetescare (accessed 17 Jun 2011). [Google Scholar]
- 10.Bailey CJ, Turner RC. Metformin. N Engl J Med 1996;334:574–9 [DOI] [PubMed] [Google Scholar]
- 11.Isomaa B, Almgren P, Tuomi T, et al. Cardiovascular morbidity and mortality associated with the metabolic syndrome. Diabetes Care 2001;24:683–9 [DOI] [PubMed] [Google Scholar]
- 12.Salpeter SR, Greyber E, Pasternak GA, et al. Risk of fatal and nonfatal lactic acidosis with metformin use in type 2 diabetes mellitus. Cochrane Database Syst Rev 2010;(4):CD002967. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Eurich DT, McAlister FA, Blackburn DF, et al. Benefits and harms of antidiabetic agents in patients with diabetes and heart failure: systematic review. BMJ 2007;335:497. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Roussel R, Travert F, Pasquet B, et al. Metformin use and mortality among patients with diabetes and atherothrombosis. Arch Intern Med 2010;170:1892–9 [DOI] [PubMed] [Google Scholar]
- 15.Wettermark B, Hammar N, Fored CM, et al. The new Swedish Prescribed Drug Register–opportunities for pharmacoepidemiological research and experience from the first six months. Pharmacoepidemiol Drug Saf 2007;16:726–35 [DOI] [PubMed] [Google Scholar]
- 16.Merlo J, Lindblad U, Pessah-Rasmussen H, et al. Comparison of different procedures to identify probable cases of myocardial infarction and stroke in two Swedish prospective cohort studies using local and national routine registers. Eur J Epidemiol 2000;16:235–43 [DOI] [PubMed] [Google Scholar]
- 17.Tunstall-Pedoe H, Kuulasmaa K, Amouyel P, et al. Myocardial infarction and coronary deaths in the World Health Organization MONICA Project. Registration procedures, event rates, and case-fatality rates in 38 populations from 21 countries in four continents. Circulation 1994;90:583–612 [DOI] [PubMed] [Google Scholar]
- 18.Eeg-Olofsson K, Cederholm J, Nilsson PM, et al. Risk of cardiovascular disease and mortality in overweight and obese patients with type 2 diabetes: an observational study in 13,087 patients. Diabetologia 2009;52:65–73 [DOI] [PubMed] [Google Scholar]
- 19.Eeg-Olofsson K, Cederholm J, Nilsson PM, et al. Glycemic control and cardiovascular disease in 7,454 patients with type 1 diabetes: an observational study from the Swedish National Diabetes Register (NDR). Diabetes Care 2010;33:1640–6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Eliasson B, Svensson AM, Miftaraj M, et al. Clinical use and effectiveness of lipid lowering therapies in diabetes mellitus-an observational study from the Swedish national diabetes register. PLoS One 2011;6:e18744. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Lind M, Bounias I, Olsson M, et al. Glycaemic control and incidence of heart failure in 20 985 patients with type 1 diabetes: an observational study. Lancet 2011;378:140–6 [DOI] [PubMed] [Google Scholar]
- 22.Hoelzel W, Weykamp C, Jeppsson JO, et al. IFCC reference system for measurement of hemoglobin A1c in human blood and the national standardization schemes in the United States, Japan, and Sweden: a method-comparison study. Clin Chem 2004;50:166–74 [DOI] [PubMed] [Google Scholar]
- 23.Levey AS, Bosch JP, Lewis JB, et al. A more accurate method to estimate glomerular filtration rate from serum creatinine: a new prediction equation. Modification of diet in renal disease study group. Ann Intern Med 1999;130:461–70 [DOI] [PubMed] [Google Scholar]
- 24.Lee BK, Lessler J, Stuart EA. Improving propensity score weighting using machine learning. Stat Med 2010;29:337–46 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Lin DY, Wei LJ, Ying Z. Checking the Cox model with cumulative sums of martingale-based residuals. Biometrika 1993;80:557–72 [Google Scholar]
- 26.Collett D. Modelling Survival Data in Medical Research. 2 edn Boca Raton: CRC press, 2003 [Google Scholar]
- 27.Grambsch PM, Therneau TM. Proportional hazards tests and diagnostics based on weighted residuals. Biometrika 1994;81:515–26 [Google Scholar]
- 28.Margolis DJ, Hoffstad O, Strom BL. Association between serious ischemic cardiac outcomes and medications used to treat diabetes. Pharmacoepidemiol Drug Saf 2008;17:753–9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Mellbin LG, Malmberg K, Norhammar A, et al. The impact of glucose lowering treatment on long-term prognosis in patients with type 2 diabetes and myocardial infarction: a report from the DIGAMI 2 trial. Eur Heart J 2008;29:166–76 [DOI] [PubMed] [Google Scholar]
- 30.Pantalone KM, Kattan MW, Yu C, et al. Increase in overall mortality risk in patients with type 2 diabetes receiving glipizide, glyburide, or glimepiride monotherapy versus metformin a retrospective analysis. Diabetes Obes Metab. Published Online First: 6 April 2012. doi:10.1111/j.1463-1326.2012.01604.x [DOI] [PubMed] [Google Scholar]
- 31.Schramm TK, Gislason GH, Vaag A, et al. Mortality and cardiovascular risk associated with different insulin secretagogues compared with metformin in type 2 diabetes, with or without a previous myocardial infarction: a nationwide study. Eur Heart J 2011;32:1900–8 [DOI] [PubMed] [Google Scholar]
- 32.Currie CJ, Peters JR, Tynan A, et al. Survival as a function of HbA(1c) in people with type 2 diabetes: a retrospective cohort study. Lancet 2010;375:481–9 [DOI] [PubMed] [Google Scholar]
- 33.Gamble JM, Simpson SH, Eurich DT, et al. Insulin use and increased risk of mortality in type 2 diabetes: a cohort study. Diabetes Obes Metab 2010;12:47–53 [DOI] [PubMed] [Google Scholar]
- 34.Eeg-Olofsson K, Cederholm J, Nilsson PM, et al. New aspects of HbA1c as a risk factor for cardiovascular diseases in type 2 diabetes: an observational study from the Swedish National Diabetes Register (NDR). J Intern Med 2010;268:471–82 [DOI] [PubMed] [Google Scholar]
- 35.Mellbin LG, Malmberg K, Norhammar A, et al. Prognostic implications of glucose-lowering treatment in patients with acute myocardial infarction and diabetes: experiences from an extended follow-up of the Diabetes Mellitus Insulin-Glucose Infusion in Acute Myocardial Infarction (DIGAMI) 2 Study. Diabetologia 2011;54:1308–17 [DOI] [PubMed] [Google Scholar]
- 36.Gerstein H, Yusuf S, Riddle MC, et al. Rationale, design, and baseline characteristics for a large international trial of cardiovascular disease prevention in people with dysglycemia: the ORIGIN Trial (Outcome Reduction with an Initial Glargine Intervention). Am Heart J 2008;155:26–32, e1–6. [DOI] [PubMed] [Google Scholar]
- 37.Lalau JD, Race JM. Lactic acidosis in metformin-treated patients. Prognostic value of arterial lactate levels and plasma metformin concentrations. Drug Saf 1999;20:377–84 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.