Perils of Observational Data Analyses

Short abstract

See Article Ko et al

Prevention guidelines around the world recommend statin therapy for the secondary and primary prevention of atherosclerotic cardiovascular disease (ASCVD).1, 2, 3 Recommendations are based on data from >185 000 participants enrolled in over 28 statin cardiovascular outcomes trials.4 However, the vast majority of statin trials were performed in white populations in the United States and Europe, with few trials including a majority of Asian ancestry participants.

In this issue of the Journal of the American Heart Association (JAHA), Park et al attempt to address an important safety issue that could influence the potential for net benefit from statin therapy in Korean primary prevention patients.5 They used the Korean national health insurance healthcare database to identify over 2 million adults with total cholesterol levels ≥240 mg/dL who were eligible for primary prevention statin therapy. With a new user design of this database, Park et al identified >500 000 propensity‐matched pairs of statin users and nonusers. They found that ever‐statin users were more likely to have been diagnosed with diabetes mellitus over a 3.9‐year period (hazard ratio [HR] 1.88, 95% CI 1.85–1.93) than never‐statin users. They also found that the risk of diabetes mellitus was higher with increasing duration of statin use (1‐year HR 1.26, 1–2 years HR 2.22, and >2 years HR 2.62, all compared with <1 year). Higher intensity statin therapy and higher daily dose were also associated with higher risk of new‐onset diabetes mellitus.

How does this compare with the randomized trial data? An analysis of individual data found that the statin‐treated group had an 11% excess risk of new‐onset diabetes mellitus compared with placebo/control (HR 1.11, 95% CI 1.03–1.20), and that high‐intensity statins increased the risk by an additional 12% compared with moderate‐intensity statins (HR 1.12, 95% CI 1.04–1.22).6 These rates are substantially less than those observed in the Korean observational analysis for low‐moderate‐intensity statin (HR 1.75) and high‐intensity statin therapy (HR 2.31).

Unfortunately, inferring causality and magnitude of effect from observational data sets is not possible. There are inherent biases in who receives statins, who takes statins, and how often they are taken, and occurrence and assessment of outcomes. The authors did use some methods to address bias, including a new user design to decrease survivor bias, prospective follow‐up to address temporality of exposure, and propensity scoring to address bias by indication. In the case of statins, characteristics that influence statin use (treatment by indication) include age, sex, low‐density lipoprotein cholesterol (LDL‐C) level, the presence of ASCVD, or risk factors for ASCVD. Risk factors for ASCVD also overlap the risk factors for type 2 diabetes mellitus, further confounding the association between statin use and diabetes mellitus prevalence. Although some of these characteristics were balanced after propensity score adjustment, further potential for bias arises from missing variables, which reduced the propensity‐matched analytic cohort to 25% of the eligible cohort.

Propensity matching is unable to adjust for other sources of bias, including detection bias and unmeasured confounders. Detection bias may largely explain the findings in the study, which suggest a greater risk of new‐onset diabetes mellitus in Korean patients than has been observed in randomized statin trials. Analytic strategies that could have been used to adjust for detection bias include sensitivity analyses, conditioning on outcome detectability, and use of negative controls. A previous evaluation of detection bias in a study of the statin–diabetes mellitus association found that diabetes mellitus was detected more often in statin users and in more adherent users. Accounting for 20% more diabetes mellitus detection in adherent statin users reduced the HR for new‐onset diabetes mellitus from 1.53 (95% CI 1.44–1.64) to 1.28.7 Use of negative outcomes or negative controls completely eliminated the excess risk because of higher adherence. Unmeasured confounders are always an issue, but may have been less of an issue in this data set since it captured 98% of the Korean population, all of whom have medication coverage.

The best evidence for causality comes from randomized, controlled clinical trials. The most informative trial for primary prevention in East Asian populations comes from the MEGA (Management of Elevated Cholesterol in the Primary Prevention Group of Adult Japanese) trial.8 MEGA enrolled 7832 Japanese patients without coronary heart disease and total cholesterol 5.69 to 6.98 mmol/L (220–270 mg/dL). Participants (mean age 58 years, 68% were women and 21% had diabetes mellitus) were randomized to diet or diet plus pravastatin 10 to 20 mg. Mean LDL‐C was reduced by 15% (23 mg/dL) with pravastatin compared with control. The relative risk of coronary heart disease and stroke was reduced by 30% (95% CI 10–46%), and total cardiovascular events (including revascularization) were reduced by 26% (95% CI 6–41%). Safety appeared similar in both groups. There were 172 cases of diabetes mellitus reported in the statin group and 164 cases in the control group (HR 1.07, 95% CI 0.86–1.34).6 Subgroup analysis according to diabetes mellitus, impaired fasting glucose, and normal fasting glucose levels at baseline found a stepwise increasing rate of cardiovascular events with worsening glucose status, without evidence of heterogeneity of treatment effect.9 Interestingly, the 3 subgroups had similar body mass index of 24 kg/m². Subgroup analysis by sex revealed that women had lower rates of coronary heart disease/stroke than men: 4.6% in women and 12.9% in men in the placebo group over a mean follow‐up of 5.3 years.10

The only other primary prevention trial with a large proportion of Asian ancestry participants was the HOPE (Heart Outcomes Prevention Evaluation‐3) trial.11 HOPE‐3 evaluated a moderate intensity statin, rosuvastatin 10 mg, compared with placebo in 12 705 primary prevention patients regardless of baseline LDL‐C levels. The mean age was 66 years, and included a primarily non‐European ancestry population: 29% Chinese, 15% South Asian, other Asian 6%, and 27% Hispanic participants; self‐declared Asians n=6241. Although Asians had a lesser reduction in LDL‐C, they had a similar relative risk reduction in ASCVD from rosuvastatin (HR 0.75, 95% CI 0.64–0.88).12 No differences in muscle symptoms or treatment discontinuation were observed for Asians. There was no increase in diabetes mellitus risk in Asians (3.02% rosuvastatin versus 4.04% placebo; P=0.0342), although diabetes mellitus risk was increased in non‐Asians (4.70% rosuvastatin versus 3.52% placebo, P=0.025; Pinteraction=0021). Of note, the mean body mass index in the HOPE‐3 Asian participants was 25.7 and 28.5 kg/m² in the non‐Asian participants.

High‐intensity statin therapy has not been evaluated in long‐term randomized trials in majority Asian populations, in part because of concerns regarding safety. Altered pharmacokinetics in Asian individuals has led to recommendations to adjust the dosage of rosuvastatin13; other statins, including atorvastatin, do not require dose adjustment.

The mechanisms whereby statins slightly increase the excess risk of diabetes mellitus are not fully elucidated.14 Mendelian randomization studies have found that selected 3‐hydroxy‐3‐methylglutaryl‐coenzyme reductase polymorphisms associated with lower LDL‐C levels are also associated with a slight excess of diabetes mellitus, commensurate with the level observed in the clinical trials.6 In the statin trials, statin users had gained an additional pound during the trial, which may have tipped them across the diabetes mellitus diagnostic threshold about 2 months before those in the placebo group.6, 15, 16 Indeed, the excess of diabetes mellitus in the trials occurred only in those with diabetes risk factors. This is not consistent with the observational analysis of the Korean population, which had a mean age of 55 years, were mostly women (62%), were not obese (mean body mass index 24.4 kg/m², had relatively low rates of hypertension [30%], and most were normoglycemic [66%] and nonsmokers [72%]). This further supports the conclusion that confounding substantially influenced the association observed in the Korean study.

The potential for net benefit should inform the decision to initiate statin therapy according to the recent 2018 American Heart Association/American College of Cardiology cholesterol guideline. The population of statin‐treated patients in the Park et al study had LDL‐C levels ≥240 mg/dL, with a mean level of ≈260 mg/dL. LDL‐C levels were not reported, but likely constituted 60% to 70% of total cholesterol. LDL‐C levels of 155 to 175 mg/dL are below the level requiring mandatory statin therapy according to the 2018 American Heart Association/ American College of Cardiology cholesterol guideline. However, they may inform the treatment decision in patients with 5% to <20% 10‐year ASCVD risk. The majority of the Korean cohort of statin‐treated patients likely had a <5% 10‐year ASCVD risk based on both the level of risk factor levels reported, and the potentially lower 10‐year ASCVD risk of nonobese Asian populations.17 Therefore, even in the absence of harm, the potential for benefit may have been relatively low over a 10‐year treatment period for many of the patients in the Korean cohort.

In the end, the benefits and harms of drug therapy need to be assessed in well‐conducted randomized, controlled trials, preferably in the population under treatment. The data available from randomized trials to date suggest no excess risk of diabetes mellitus from statin therapy used in lean Asian populations. Even the most rigorous methodology is unable to compensate for serious biases inherent in observational data sets. The results of observational analyses can at best be hypothesis generating, and at worst result in inappropriate therapeutic approaches that can harm patients.

Disclosures

Dr Robinson reports research grants to Institution: Acasti, Amarin, Amgen, Astra‐Zeneca, Esai, Esperion, Merck, Regeneron, Sanofi, Takeda and consulting for Amgen, Merck, Novartis, Novo‐Nordisk, Pfizer, Regeneron, and Sanofi.

J Am Heart Assoc. 2019;8:e012490 DOI: 10.1161/JAHA.119.012490.