Abstract
Background and objectives
Patients with CKD are more likely than others to have abnormalities in serum potassium (K+). Aside from severe hyperkalemia, the clinical significance of K+ abnormalities is not known. We sought to examine the association of serum K+ with mortality and hospitalization rates within narrow eGFR strata to understand how the burden of hyperkalemia varies by CKD severity. Associations were examined between serum K+ and discontinuation of medications that block the renin-angiotensin-aldosterone system (RAAS), which are known to increase serum K+.
Design, setting, participants, & measurements
A cohort of patients with CKD (eGFR<60 ml/min per 1.73 m2) with serum K+ data were studied (n=55,266) between January 1, 2009, and June 30, 2013 (study end). Serum K+, eGFR, and covariates were considered on a time-updated basis. Mortality, major adverse cardiovascular events (MACE), hospitalization, and discontinuation of RAAS blockers were considered per time at risk.
Results
During the study, serum K+ levels of 5.5–5.9 and ≥6.0 mEq/L were most prevalent at lower eGFR: they were present, respectively, in 1.7% and 0.2% of patient-time for eGFR of 50–59 ml/min per 1.73 m2 versus 7.6% and 1.8% of patient-time for eGFR<30 ml/min per 1.73 m2. Serum K+ level <3.5 mEq/L was present in 1.2%–1.4% of patient-time across eGFR strata. The median follow-up time was 2.76 years. There was a U-shaped association between serum K+ and mortality; pooled adjusted incidence rate ratios were 3.05 (95% confidence interval, 2.53 to 3.68) and 3.31 (95% confidence interval, 2.52 to 4.34) for K+ levels <3.5 mEq/L and ≥6.0 mEq/L, respectively. Within eGFR strata, there were U-shaped associations of serum K+ with rates of MACE, hospitalization, and discontinuation of RAAS blockers.
Conclusions
Both hyperkalemia and hypokalemia were independently associated with higher rates of death, MACE, hospitalization, and discontinuation of RAAS blockers in patients with CKD who were not undergoing dialysis. Future studies are needed to determine whether interventions targeted at maintaining normal serum K+ improve outcomes in this population.
Keywords: chronic kidney disease, glomerular filtration rate, hospitalization, mortality, ACE inhibitors, follow-up studies, humans, hyperkalemia, hypokalemia, potassium
Introduction
In states of health, serum potassium (K+) is tightly regulated, typically between 3.5 and 5.0 mEq/L (1). Homeostatic maintenance of serum K+ is important for many physiologic processes, such as cardiac conduction and inotropy, smooth muscle tone, and neuronal signaling (2–4). In CKD, K+ excretory capacity is diminished and patients with CKD are predisposed to hyperkalemia. Moreover, patients with CKD are commonly treated with medications that decrease (e.g., diuretics) and increase (e.g., angiotensin-converting enzyme inhibitors) serum K+, making serum K+ abnormalities more frequent among patients with CKD than other populations.
Very high serum K+ levels predisposes patients to ventricular arrhythmia and sudden death (5). To date, no studies with rigorous data on patients with CKD have examined whether modest deviations in serum K+ from normal are clinically significant, and if so, at what thresholds these effects manifest. Moreover, no data show whether the effects of serum K+ vary according to the severity of underlying CKD.
Patients with CKD are at high risk for cardiovascular events and progression to ESRD. Antagonists of the renin-angiotensin-aldosterone system (RAAS), such as angiotensin-converting enzyme inhibitors, angiotensin-receptor blockers, and mineralocorticoid antagonists, are protective in both regards. However, because these drugs increase serum K, they may be discontinued in the setting of high ambient serum K+ levels, thereby abrogating their potential cardiorenal benefits (6–8). To date, the influence of serum K+ on discontinuation of RAAS blockers has not been well characterized.
To add clarity to the literature, the objective of this retrospective study of patients with CKD was to measure the independent associations of serum K+ with rates of mortality, hospitalization, major adverse cardiovascular events (MACE), and discontinuation of RAAS blockers. In addition, to determine the significance of serum K+ by CKD severity, the analyses were stratified according to eGFR.
Materials and Methods
Study Data
Study data were obtained from electronic health records (EHRs) of HealthCare Partners, a managed care organization caring for more than 1 million members. Most members are located in California, where records are maintained on a single, integrated EHR platform containing demographic, comorbidity, laboratory, clinical, and vital status data collected during outpatient primary care and specialist visits. Because HealthCare Partners assumes financial risk for members—serving as the de facto payer—it receives and maintains comprehensive medical and pharmacy claims data, enabling visibility of hospitalizations and the events driving them (e.g., myocardial infarction, stroke) as well as prescription drug use.
Patients
The study examined patients aged ≥18 years who, between January 1, 2009, and June 30, 2013, had eGFR<60 ml/min per 1.73 m2 and subsequently, or concurrently, had serum K+ measured. The earliest date on which these criteria were met was considered the index date. Patients receiving a kidney transplant or dialysis before their serum K+ index date were excluded.
This retrospective observational study used preexisting, deidentified data and was exempt from institutional review board or ethics committee approval according to U.S. Department of Health and Human Services (45 CFR part 46). We adhered to the Declaration of Helsinki.
Study Design
Patients were studied beginning on the index date until the earliest of the following: death, dialysis initiation, kidney transplant, loss to follow-up, or end of study (June 30, 2013). Given its varying nature, serum K+ (exposure) was time-updated during the study. In the primary analysis, serum K+ and covariates were updated at the time of each successive K+ measurement, creating a series of patient intervals: at-risk time accrued and events occurring during each interval were attributed to the serum K+ measured immediately preceding the interval start and adjusted for covariates as of the interval start. To account for bias that may be introduced according to frequency of serum K+ measurement, a sensitivity analysis was conducted in which exposure status and covariates were updated at fixed-quarterly intervals but were otherwise analyzed in a manner identical to that used in the primary analysis. In the fixed-interval quarterly analysis, when no K+ was measured in the preceding quarter, the most recent antecedent serum K+ was pulled forward for analysis from a prior quarter. Serum K+ and eGFR values were sourced from the laboratory component of the EHR, consisting exclusively of measurements made in the ambulatory setting; K+ values did not include serum K+ measurements made during hospitalizations.
Covariates considered were those imbalanced across categories of serum K+ and known or presumed to be associated with the outcomes studied: age, sex, race/ethnicity, diabetes, congestive heart failure, coronary artery disease, cerebrovascular disease, β-blocker use, RAAS blocker use, nondihydropyridine calcium channel blocker use, thiazide diuretic use, loop diuretic use, and eGFR. Demographic data were sourced from EHRs. Comorbidity data were sourced from combined EHR and medical claims. Medication use was sourced from pharmacy claims files; patients taking prescription drugs containing one or more drug classes (e.g., thiazide and β-blocker) were credited for receiving corresponding medications.
Studied outcomes were all-cause death and hospitalizations, MACE, and RAAS blocker discontinuation. MACE was defined as a composite of arrhythmia, myocardial infarction, stroke, and heart failure exacerbation. Operational definitions were based on primary International Classification of Diseases, Ninth Revision, codes for inpatient hospitalizations: 427–427.99, 410–410.99, 430–438.99, and 428–428.99, respectively. RAAS blocker discontinuation was considered only among RAAS blocker users and were operationally defined as a failure to fill RAAS blocker prescriptions within 14 days of exhausting the prior supply.
Statistical Analyses
Patient characteristics were described as means, SDs, counts, and proportions as dictated by data type and were compared across serum K+ categories using one-way ANOVA models and chi-squared tests. Outcomes were considered as rates: number of events divided by time at risk. Patient-years, number of events, and rates (both crude and adjusted) were reported by serum K+ exposure groups within eGFR strata (<30, 30–39, 40–49, and 50–59 ml/min per 1.73 m2).
Generalized additive models were used to analyze the nonlinear association between serum K+ and MACE, hospitalization, and mortality. Results of these analyses were used to guide the discrete categorization of serum K+ for subsequent analyses. On the basis of these findings, serum K+ was categorized as <3.5, 3.5–3.9, 4.0–4.4, 4.5–4.9, 5.0–5.4, 5.5–5.9, and ≥6.0 mEq/L. Formal association estimates of serum K+ with mortality, MACE, and hospitalization were estimated using generalized estimating equations with independent or exchangeable working correlation structure; Poisson/negative binomial links were used on the basis of the empirical distribution of outcomes in the study cohort. Candidate covariates (as noted earlier in this section) were screened univariately and retained when P value for the generalized score test was ≤0.10. In several instances, covariates were withheld to enable model convergence.
Associations between serum K+ and RAAS blocker discontinuation were limited to patient intervals for which the patient was taking a RAAS blocker at interval start (i.e., during patient intervals that were at risk for discontinuation). Patient-time was handled analogously as described for serum K+ time, with the exception that at-risk time was truncated at the time of RAAS blocker discontinuation. Associations were estimated using mixed-effects Poisson models, with random intercept terms representing individual patients and fixed-effects terms as described. In all instances, models were first fit to assess for K+-by-eGFR interaction (through inclusion of two-way crossproduct terms); when interaction was not observed, reduced models that omitted the interaction term were fit to provide pooled estimates. All analyses were performed using SAS 9.3 (SAS Institute, Inc., Cary, NC)
Results
A total of 55,266 patients qualified for study. Baseline patient characteristics are presented in Table 1. Higher serum K+ was incrementally associated with diabetes prevalence, coronary artery disease prevalence, and the use of RAAS blockers. Lower serum K+ was incrementally associated with female sex and thiazide diuretic use. Time spent with serum K+ levels <3.5 mEq/L and ≥6.0 mEq/L was associated with younger age, prevalent congestive heart failure, and loop diuretic use.
Table 1.
Characteristic | Serum K+ Level (n=55,266) | Omnibus P Value | ||||||
---|---|---|---|---|---|---|---|---|
<3.5 mEq/L (n=897) | 3.5–3.9 mEq/L (n=5664) | 4.0–4.4 mEq/L (n=18,712) | 4.5–4.9 mEq/L (n=18,947) | 5.0–5.4 mEq/L (n=8270) | 5.5–5.9 mEq/L (n=2187) | ≥6.0 mEq/L (n=589) | ||
Mean age ±SD (yr) | 72.2±11.9 | 73.8±11.5 | 74.6±11.3 | 74.8±11.1 | 74.7±11.3 | 74.3±11.5 | 73.9±12.3 | <0.001 |
Race | <0.001 | |||||||
American Indian/Alaskan Native | 1 (0.1) | 9 (0.2) | 40 (0.2) | 34 (0.2) | 29 (0.4) | 3 (0.1) | 3 (0.5) | |
Asian | 31 (3.5) | 194 (3.4) | 667 (3.6) | 573 (3.0) | 246 (3.0) | 65 (3.0) | 17 (2.9) | |
Black | 86 (9.6) | 464 (8.2) | 994 (5.3) | 702 (3.7) | 266 (3.2) | 50 (2.3) | 14 (2.4) | |
Declined to state | 62 (6.9) | 461 (8.1) | 1403 (7.5) | 1494 (7.89) | 634 (7.67) | 200 (9.1) | 64 (10.9) | |
Native Hawaiian/Pacific Islander | 5 (0.6) | 20 (0.4) | 61 (0.3) | 55 (0.3) | 21 (0.3) | 12 (0.6) | 1 (0.2) | |
Unknown | 517 (57.6) | 3326 (58.7) | 11,566 (61.8) | 11,818 (62.4) | 5179 (62.6) | 1368 (62.6) | 368 (62.5) | |
White | 195 (21.7) | 1190 (21.0) | 3981 (21.3) | 4271 (22.5) | 1895 (22.9) | 489 (22.4) | 122 (20.7) | |
Women | 597 (66.6) | 3830 (67.6) | 11,632 (62.2) | 10,590 (55.9) | 4319 (52.2) | 1116 (51.0) | 269 (45.7) | <0.001 |
Diabetes | 183 (20.4) | 1252 (22.1) | 4217 (22.5) | 5003 (26.4) | 2413 (29.2) | 675 (30.9) | 203 (34.5) | <0.001 |
CHF | 101 (11.3) | 504 (8.9) | 1379 (7.4) | 1469 (7.8) | 690 (8.3) | 193 (8.8) | 68 (11.5) | <0.001 |
CAD | 98 (10.9) | 697 (12.3) | 2274 (12.2) | 2531 (13.4) | 1147 (13.9) | 329 (15.0) | 92 (15.6) | <0.001 |
CVA | 62 (6.9) | 369 (6.5) | 1237 (6.6) | 1178 (6.2) | 502 (6.1) | 151 (6.9) | 46 (7.8) | 0.29 |
Loop diuretics | 121 (13.5) | 487 (8.6) | 1266 (6.8) | 1330 (7.0) | 581 (7.0) | 182 (8.3) | 64 (10.9) | <0.001 |
Thiazide diuretics | 292 (32.6) | 1514 (26.7) | 3001 (16.0) | 2066 (10.9) | 779 (9.4) | 208 (9.5) | 64 (10.9) | <0.001 |
β-blockers | 128 (14.3) | 1002 (17.7) | 2934 (15.7) | 2952 (15.6) | 1284 (15.5) | 355 (16.3) | 90 (15.3) | 0.005 |
Nondihydropyridine calcium channel blockers | 61 (6.8) | 324 (5.7) | 875 (4.7) | 730 (3.9) | 292 (3.5) | 78 (3.6) | 25 (4.2) | <0.001 |
RAAS blockers | 202 (22.5) | 1405 (24.8) | 5103 (27.3) | 5721 (30.2) | 2631 (31.8) | 754 (34.5) | 220 (37.4) | <0.001 |
Unless otherwise noted, values are the number (percentage) of patients. K+, potassium; CHF, congestive heart failure; CAD, coronary artery disease; CVA, cerebrovascular disease; RAAS, renin-angiotensin-aldosterone system;.
During the study, higher serum K+ levels were more prevalent at lower eGFR (Table 2). Of time spent with eGFR<30 ml/min per 1.73 m2, serum K+ was ≥6.0 mEq/L for 1.8% of patient-time, 5.5–5.9 mEq/L for 7.6% of patient-time, 5.0–5.4 mEq/L for 23.0% of patient-time, and <3.5 mEq/L for 1.4% of patient-time. Time spent at serum K+ <3.5 mEq/L was constant across eGFRs (1.2%–1.4%). For all eGFRs, time spent with hyperkalemia was greater than that for hypokalemia. Frequency of K+ measurement was similar across eGFR strata: for patient-time spent at eGFR<30 ml/min per 1.73 m2 stratum, serum K+ was measured 1.37–2.48 times over the study compared with 1.21–2.42 times for eGFR 50–59 ml/min per 1.73 m2. The median follow-up time was 2.76 years (25th quartile, 1.62 years; 75th quartile, 4.03 years).
Table 2.
Variable | Serum K+ Level | ||||||
---|---|---|---|---|---|---|---|
<3.5 mEq/L | 3.5–3.9 mEq/L | 4.0–4.4 mEq/L | 4.5–4.9 mEq/L | 5.0–5.4 mEq/L | 5.5–5.9 mEq/L | ≥6.0 mEq/L | |
eGFR<30 ml/min per 1.73 m2 | |||||||
No. of patient-years (total, 9207) | 126 | 726 | 2211 | 3160 | 2121 | 699 | 164 |
Patient-time (%) | 1.4 | 7.9 | 24.0 | 34.3 | 23.0 | 7.6 | 1.8 |
Mean frequency of K+ testing per year | 1.65±1.35 | 1.95±1.96 | 2.27±2.26 | 2.48±2.39 | 2.23±2.17 | 1.69±1.35 | 1.37±0.98 |
Median frequency of K+ testing per year (p25, p75) | 1 (1, 2) | 1 (1, 2) | 1 (1, 3) | 2 (1, 3) | 1 (1, 3) | 1 (1, 2) | 1 (1, 1) |
Mean patient-days | 77±117 | 138±173 | 178±203 | 199±214 | 172±201 | 106±138 | 57±99 |
Median patient-days (p25, p75) | 38 (13, 95) | 80 (31, 170) | 106 (42, 237) | 122 (48, 275) | 99 (38, 225) | 60 (19, 128) | 20 (6, 63) |
eGFR 30–39 ml/min per 1.73 m2 | |||||||
No. of patient-years (total 19,230) | 225 | 1681 | 5694 | 6984 | 3636 | 873 | 137 |
Patient-time (%) | 1.2 | 8.7 | 29.6 | 36.3 | 18.9 | 4.5 | 0.7 |
Mean frequency of K+ testing per year | 1.46±1.03 | 1.67±1.39 | 2.00±1.78 | 2.06±1.73 | 1.80±1.46 | 1.43±0.91 | 1.23±0.59 |
Median frequency of K+ testing per year (p25, p75) | 1 (1, 2) | 1 (1, 2) | 1 (1, 2) | 1 (1, 2) | 1 (1, 2) | 1 (1, 2) | 1 (1, 1) |
Mean patient-days | 97±129 | 172±189 | 226±220 | 235±224 | 190±190 | 116±144 | 62±108 |
Median patient-days (p25, p75) | 52 (18, 122) | 111 (48, 225) | 153 (71, 313) | 165 (74, 328) | 130 (59, 260) | 70 (21, 158) | 18 (5, 75) |
eGFR 40–49 ml/min per 1.73 m2 | |||||||
No. of patient-years (total 37,339) | 486 | 3845 | 12,796 | 13,612 | 5455 | 1018 | 127 |
Patient-time (%) | 1.3 | 10.3 | 34.3 | 36.5 | 14.6 | 2.7 | 0.3 |
Mean frequency of K+ testing per year | 1.38±0.91 | 1.60±1.21 | 1.91±1.51 | 1.94±1.51 | 1.61±1.16 | 1.32±0.73 | 1.17±0.49 |
Median frequency of K+ testing per year (p25, p75) | 1 (1, 1) | 1 (1, 2) | 1 (1, 2) | 1 (1, 2) | 1 (1, 2) | 1 (1, 1) | 1 (1, 1) |
Mean patient-days | 129±165 | 211±207 | 271±241 | 273±239 | 204±193 | 121±147 | 62±99 |
Median patient-days (p25, p75) | 70 (27, 170) | 145 (67, 289) | 196 (94, 377) | 201 (97, 377) | 146 (69, 280) | 76 (23, 165) | 19 (5, 84) |
eGFR 50–59 ml/min per 1.73 m2 | |||||||
No. of patient-years (total 77,241) | 974 | 9554 | 30,349 | 26,230 | 8701 | 1316 | 117 |
Patient-time (%) | 1.3 | 12.4 | 39.3 | 34.0 | 11.3 | 1.7 | 0.2 |
Mean frequency of K+ testing per year | 1.61±1.35 | 1.92±1.63 | 2.42±1.92 | 2.26±1.73 | 1.71±1.26 | 1.34±0.92 | 1.21±0.73 |
Median frequency of K+ testing per year (p25, p75) | 1 (1, 2) | 1 (1, 2) | 2 (1, 3) | 2 (1, 3) | 1 (1, 2) | 1 (1, 1) | 1 (1, 1) |
Mean patient-days | 160±182 | 283±254 | 387±297 | 358±277 | 252±217 | 144±158 | 67±107 |
Median patient-days (p25, p75) | 97 (34, 216) | 205 (93, 393) | 319 (152, 556) | 292 (141, 509) | 191 (92, 355) | 95 (28, 201) | 20 (4, 90) |
Mean values are expressed with SDs. K+, potassium; p25, 25th percentile; p75, 75th percentile.
Within eGFR strata, there were statistically significant U-shaped associations between serum K+ and mortality (Table 3). For eGFR<30 ml/min per 1.73 m2, adjusted mortality rate was significantly higher for K+ levels ≥6.0 mEq/L, 3.5–3.9 mEq/L, and <3.5 mEq/L versus the referent K+ level of 4.5–4.9 mEq/L (omnibus P<0.001). For eGFR of 30–39 ml/min per 1.73 m2, adjusted mortality rate was significantly higher for K+ levels ≥6.0 mEq/L and <3.5 mEq/L versus the referent K+ levels of 4.5–4.9 mEq/L (omnibus P=0.04). For eGFR of 40–49 ml/min per 1.73 m2, significant differences versus the referent K+ levels of 4.5–4.9 mEq/L were observed for K+ levels <3.5 mEq/L, 3.5–3.9 mEq/L, 4.0–4.4 mEq/L, and 5.5–5.9 mEq/L (omnibus P=0.001); point estimates for K+ levels of 5.0–5.4 and ≥6.0 mEq/L were consistent with a U-shaped association but did not statistically significantly differ from referent on a pairwise basis. Within the eGFR stratum of 50–59 ml/min per 1.73 m2, adjusted mortality rates were significantly higher for K+ levels ≥6.0 mEq/L and <3.5 mEq/L versus the referent K+ levels of 4.5–4.9 mEq/L (omnibus P<0.001). Because the test for interaction of eGFR on the association between serum K+ and mortality was not statistically significant (K=0.72), estimates were pooled across eGFR strata (Figure 1). A U-shaped association pattern was observed. With the exception of time spent at serum K+ levels of 4.0–4.4 mEq/L, pooled mortality adjusted incidence rate ratios (aIRRs) were statistically significant for all serum K+ categories versus the referent K+ levels of 4.5–4.9 mEq/L. Similar results were obtained in fixed-time interval analysis; mortality aIRRs for all strata differed significantly from the referent K+ levels of 4.5–4.9 mEq/L except for K+ levels of 5.0–5.4 mEq/L (Supplemental Figure 1).
Table 3.
Variable | Serum K+ Level | Omnibus P Value | ||||||
---|---|---|---|---|---|---|---|---|
<3.5 mEq/L | 3.5–3.9 mEq/L | 4.0–4.4 mEq/L | 4.5–4.9 mEq/L | 5.0–5.4 mEq/L | 5.5–5.9 mEq/L | ≥6.0 mEq/L | ||
eGFR<30 ml/min per 1.73 m2 | ||||||||
Deaths, n | 32 | 94 | 212 | 277 | 175 | 63 | 39 | |
Patient-years | 126 | 726 | 2211 | 3160 | 2121 | 699 | 164 | |
Crude ratea (95% CI) | 0.26 (0.18 to 0.36) | 0.13 (0.11 to 0.16) | 0.10 (0.08 to 0.11) | 0.09 (0.08 to 0.10) | 0.08 (0.07 to 0.10) | 0.09 (0.07 to 0.12) | 0.24 (0.18 to 0.32) | |
Adjusted IRRb,c (95% CI) | 2.77 (1.89 to 4.06) | 1.30 (1.02 to 1.66) | 1.05 (0.87 to 1.26) | Ref | 1.01 (0.83 to 1.22) | 1.11 (0.84 to 1.47) | 3.08 (2.17 to 4.37) | <0.001 |
eGFR 30–39 ml/min per 1.73 m2 | ||||||||
Deaths, n | 35 | 130 | 297 | 310 | 152 | 50 | 8 | |
Patient-years | 225 | 1681 | 5694 | 6984 | 3636 | 873 | 137 | |
Crude ratea (95% CI) | 0.16 (0.11 to 0.22) | 0.08 (0.07 to 0.09) | 0.05 (0.05 to 0.06) | 0.04 (0.04 to 0.05) | 0.04 (0.04 to 0.05) | 0.06 (0.04 to 0.08) | 0.06 (0.03 to 0.12) | |
Adjusted IRRb,d (95% CI) | 4.31 (2.36 to 7.89) | 1.08 (0.68 to 1.69) | 0.90 (0.64 to 1.25) | Ref | 0.73 (0.47 to 1.11) | 0.98 (0.52 to 1.88) | 2.74 (1.13 to 6.74) | 0.04 |
eGFR 40–49 ml/min per 1.73 m2 | ||||||||
Deaths, n | 29 | 142 | 391 | 354 | 171 | 44 | 6 | |
Patient-years | 486 | 3845 | 12,796 | 13,612 | 5455 | 1018 | 127 | |
Crude ratea (95% CI) | 0.06 (0.04 to 0.09) | 0.04 (0.03 to 0.04) | 0.03 (0.03 to 0.03) | 0.03 (0.02 to 0.03) | 0.03 (0.03 to 0.04) | 0.04 (0.03 to 0.06) | 0.05 (0.02 to 0.11) | |
Adjusted IRRb,c (95% CI) | 2.52 (1.71 to 3.71) | 1.46 (1.19 to 1.78) | 1.16 (1.00 to 1.34) | Ref | 1.18 (0.99 to 1.42) | 1.68 (1.23 to 2.30) | 1.72 (0.76 to 3.86) | 0.001 |
eGFR 50–59 ml/min per 1.73 m2 | ||||||||
Deaths, n | 57 | 277 | 612 | 469 | 170 | 36 | 7 | |
Patient-years | 974 | 9554 | 30,349 | 26,230 | 8701 | 1316 | 117 | |
Crude ratea (95% CI) | 0.06 (0.05 to 0.08) | 0.03 (0.03 to 0.03) | 0.02 (0.02 to 0.02) | 0.02 (0.02 to 0.02) | 0.02 (0.02 to 0.02) | 0.03 (0.02 to 0.04) | 0.06 (0.03 to 0.13) | |
Adjusted IRRb,d (95% CI) | 4.15 (2.66 to 6.51) | 1.74 (1.33 to 2.28) | 1.14 (0.92 to 1.42) | Ref | 1.02 (0.74 to 1.42) | 0.99 (0.49 to 2.01) | 3.90 (1.23 to 12.32) | <0.001 |
Reference group for comparison is 4.5–4.9 mEq/L. P value interaction between eGFR category and K=0.72. K+, potassium; 95% CI, 95% confidence interval; IRR, incidence rate ratio; Ref, reference.
No adjustments were taken for crude rate calculations, which are expressed as events per patient-year. A Poisson distribution was used to construct the 95% CI of the crude rate for each eGFR stratum.
IRRs were adjusted for prior hospitalization, age, sex, race/ethnicity, diabetes, congestive heart failure, coronary artery disease, cerebrovascular accident, and the use of four types of medications (loop diuretics, thiazide diuretics, renin-angiotensin-aldosterone system blockers, and nondihydropyridine calcium channel blockers). Poisson distributions were used.
Model was not adjusted for race/ethnicity.
Model was not adjusted for diabetes.
There was a statistically significant U-shaped association between serum K+ and MACE (Table 4). Referent to serum K+ levels of 4.5–4.9 mEq/L, aIRRs were significantly increased for K+ levels ≥6.0 mEq/L and <3.5 mEq/L at each eGFR stratum. Because the test for interaction of eGFR on the association between serum K+ and MACE was not statistically significant (K=0.94), estimates were pooled across eGFR strata (Figure 2). In the overall cohort, a U-shaped association pattern was observed. With the exception of serum K+ category of 5.0–5.4 mEq/L, adjusted rates of MACE were significantly higher for each category versus the referent K+ levels of 4.5–4.9 mEq/L, with the greatest associations seen for K+ levels ≥6.0 mEq/L and <3.5 mEq/L.
Table 4.
Variable | Serum K+ Level | P Value | ||||||
---|---|---|---|---|---|---|---|---|
<3.5 mEq/L | 3.5–3.9 mEq/L | 4.0–4.4 mEq/L | 4.5–4.9 mEq/L | 5.0–5.4 mEq/L | 5.5–5.9 mEq/L | ≥6.0 mEq/L | ||
eGFR<30 ml/min per 1.73 m2 | ||||||||
Crude ratea (95% CI) | 12.75 (12.14 to 13.39) | 8.45 (8.24 to 8.66) | 6.08 (5.98 to 6.18) | 5.65 (5.57 to 5.74) | 5.16 (5.06 to 5.26) | 5.49 (5.31 to 5.67) | 9.02 (8.57 to 9.49) | |
Adjusted IRRb (95% CI) | 1.63 (1.25 to 2.14) | 1.50 (1.25 to 1.80) | 1.12 (1.00 to 1.26) | Ref | 1.01 (0.88 to 1.15) | 1.14 (0.98 to 1.33) | 2.11 (1.68 to 2.65) | <0.001 |
eGFR 30–39 ml/min per 1.73 m2 | ||||||||
Crude ratea (95% CI) | 9.65 (9.25 to 10.07) | 5.91 (5.79 to 6.03) | 4.56 (4.50 to 4.61) | 4.20 (4.15 to 4.25) | 4.07 (4.01 to 4.14) | 4.51 (4.38 to 4.66) | 6.85 (6.42 to 7.30) | |
Adjusted IRRb (95% CI) | 1.69 (1.29 to 2.22) | 1.25 (1.08 to 1.44) | 1.06 (0.97 to 1.16) | Ref | 1.02 (0.93 to 1.12) | 1.16 (1.00 to 1.35) | 1.44 (1.12 to 1.84) | <0.001 |
eGFR 40–49 ml/min per 1.73 m2 | ||||||||
Crude ratea (95% CI) | 5.30 (5.09 to 5.50) | 4.19 (4.12 to 4.25) | 3.49 (3.46 to 3.52) | 3.10 (3.07 to 3.13) | 3.27 (3.22 to 3.32) | 3.54 (3.43 to 3.66) | 4.09 (3.76 to 4.46) | |
Adjusted IRRb (95% CI) | 1.73 (1.30 to 2.29) | 1.19 (1.07 to 1.32) | 1.06 (0.99 to 1.14) | Ref | 1.07 (0.98 to 1.17) | 1.14 (0.98 to 1.33) | 1.56 (1.11 to 2.17) | <0.001 |
eGFR 50–59 ml/min per 1.73 m2 | ||||||||
Crude ratea (95% CI) | 3.99 (3.86 to 4.11) | 3.07 (3.03 to 3.10) | 2.50 (2.49 to 2.52) | 2.53 (2.52 to 2.55) | 2.54 (2.50 to 2.57) | 2.56 (2.47 to 2.65) | 3.27 (2.96 to 3.61) | |
Adjusted IRRb (95% CI) | 1.44 (1.19 to 1.75) | 1.13 (1.03 to 1.23) | 1.00 (0.94 to 1.07) | Ref | 0.97 (0.88 to 1.06) | 1.12 (0.96 to 1.30) | 2.11 (1.53 to 2.89) | <0.001 |
Reference group for comparison is 4.5–4.9 mEq/L. P value interaction between eGFR stratum and K=0.94 in the pooled study population. K+, potassium; 95% CI, 95% confidence interval; IRR, incidence rate ratio; Ref, reference.
The 95% CIs for crude rates are based on a Poisson model that has no additional covariates except serum K+ levels for each of the eGFR categories. Crude rates are expressed as events per patient-year.
Rates were adjusted for hospitalization within the preceding 30 days, age, sex, race, diabetes, congestive heart failure, coronary artery disease, cerebrovascular accident, loop diuretic use, thiazide diuretic use, renin-angiotensin-aldosterone system blocker use, β-blocker use, and nondihydropyridine calcium channel blocker use based on the generalized estimating equation with a negative binomial distribution and a log link specified.
Within each eGFR stratum, there was a statistically significant U-shaped association between serum K+ and hospitalizations (Figure 3, Supplemental Table 1). For eGFR<30 ml/min per 1.73 m2, adjusted hospitalization rates were significantly higher for K+ levels ≥6.0 mEq/L, 5.5–5.9 mEq/L, 3.5–3.9 mEq/L, and <3.5 mEq/L versus the referent K+ levels of 4.5–4.9 mEq/L (omnibus P<0.001). Within the eGFR strata of 30–39 and 40–49 ml/min per 1.73 m2, adjusted hospitalization rates were significantly higher for K+ levels ≥6.0 mEq/L, 3.5–3.9 mEq/L, and <3.5 mEq/L versus the referent K+ levels of 4.5–4.9 mEq/L (omnibus P=0.02 and 0.007, respectively). Within the eGFR stratum of 50–59 ml/min per 1.73 m2, adjusted hospitalization rate was higher for K+ level <3.5 mEq/L versus the referent K+ levels of 4.5–4.9 mEq/L (omnibus P=0.001). Because interaction between serum K+ and eGFR was observed (P interaction K=0.02), estimates were not pooled across eGFR strata.
Within each eGFR stratum, there was a U-shaped association between serum K+ and discontinuation of RAAS blocker (Table 5). Within the eGFR stratum of <30 ml/min per 1.73 m2, adjusted discontinuation rates were greatest for K+ levels <3.5 mEq/L and ≥6.0 mEq/L versus the referent K+ levels of 4.5–4.9 mEq/L. Similar patterns were observed in each eGFR stratum, with the greatest adjusted rates measured within the K+ ≥6.0 mEq/L category even in patients with the highest kidney function (omnibus P=0.001).
Table 5.
Variable | Serum K+ Level | Omnibus P Value | ||||||
---|---|---|---|---|---|---|---|---|
<3.5 mEq/L | 3.5–3.9 mEq/L | 4.0–4.4 mEq/L | 4.5–4.9 mEq/L | 5.0–5.4 mEq/L | 5.5–5.9 mEq/L | ≥6.0 mEq/L | ||
eGFR<30 ml/min per 1.73 m2 | ||||||||
Patient-years | 24 | 168 | 620 | 1016 | 706 | 243 | 51 | |
Stops, n | 44 | 176 | 587 | 893 | 636 | 290 | 86 | |
Crude rate/100 patient-years | 186.51 | 104.61 | 94.75 | 87.88 | 90.08 | 119.29 | 168.09 | |
Adjusted IRRa (95% CI) | 2.07 (1.52 to 2.81) | 1.19 (1.01 to 1.41) | 1.10 (0.99 to 1.22) | Ref | 1.00 (0.90 to 1.11) | 1.31 (1.15 to 1.49) | 1.81 (1.45 to 2.26) | <0.001 |
eGFR 30–39 ml/min per 1.73 m2 | ||||||||
Patient-years | 48 | 485 | 1877 | 2488 | 1409 | 367 | 60 | |
Stops, n | 48 | 354 | 1146 | 1440 | 894 | 303 | 63 | |
Crude rate/100 patient-years | 99.46 | 73.02 | 61.07 | 57.89 | 63.43 | 82.46 | 103.90 | |
Adjusted IRRa (95% CI) | 1.69 (1.26 to 2.26) | 1.28 (1.14 to 1.44) | 1.07 (0.99 to 1.16) | Ref | 1.08 (0.99 to 1.17) | 1.38 (1.22 to 1.57) | 1.71 (1.33 to 2.20) | <0.001 |
eGFR 40–49 ml/min per 1.73 m2 | ||||||||
Patient-years | 101 | 1071 | 3900 | 4544 | 2027 | 369 | 47 | |
Stops, n | 72 | 563 | 1916 | 2190 | 1058 | 269 | 53 | |
Crude rate/100 patient-years | 71.06 | 52.58 | 49.12 | 48.19 | 52.19 | 73.04 | 113.10 | |
Adjusted IRRa (95% CI) | 1.50 (1.19 to 1.91) | 1.13 (1.03 to 1.25) | 1.05 (0.98 to 1.11) | Ref | 1.06 (0.99 to 1.15) | 1.45 (1.28 to 1.65) | 2.21 (1.68 to 2.90) | <0.001 |
eGFR 50–59 ml/min per 1.73 m2 | ||||||||
Patient-years | 129 | 1441 | 5103 | 5171 | 1911 | 313 | 28 | |
Stops, n | 78 | 712 | 2254 | 2339 | 902 | 203 | 23 | |
Crude rate/100 patient-years | 60.48 | 49.40 | 44.17 | 45.24 | 47.21 | 64.96 | 83.37 | |
Adjusted IRRa (95% CI) | 1.37 (1.09 to 1.72) | 1.13 (1.03 to 1.23) | 1.00 (0.94 to 1.06) | Ref | 1.03 (0.95 to 1.11) | 1.39 (1.20 to 1.61) | 1.70 (1.13 to 2.56) | <0.001 |
Reference group is 4.5–4.9 mEq/L. P value interaction between eGFR category and K=0.66. K+, potassium; IRR, incidence rate ratio; 95% CI, 95% confidence interval; Ref, reference.
IRRs were adjusted for age, sex, race, diabetes, congestive heart failure, coronary artery disease, cerebral vascular accident, β-blocker use, and nondihydropyridine calcium channel blocker use, loop diuretic use, and thiazide diuretic use. Associations were estimated using mixed-effects Poisson models, with random intercept terms representing individual patients and fixed-effects terms as described.
Discussion
This retrospective study of a large, generalizable CKD population yielded several findings. First, serum K+ abnormalities were common among patients with CKD; hyperkalemia was more prevalent than hypokalemia, particularly at lower eGFR. Second, there was a strong, independent, and statistically significant U-shaped association between serum K+ and mortality among patients with CKD; adjusted death risk was approximately three times higher for K+ levels ≥6.0 mEq/L and <3.5 mEq/L and 1.5 times higher for K+ levels of 5.5–5.9 mEq/L and 3.5–3.9 mEq/L compared with K+ 4.5–4.9 mEq/L. Third, there was a potent U-shaped association between serum K+ and MACE among patients with CKD. Compared with serum K+ levels of 4.5–4.9 mEq/L, the adjusted MACE rate was nearly two-fold higher for K+ levels ≥6.0 mEq/L and <3.5 mEq/L. Fourth, within each eGFR stratum studied, there was a strong, independent, and statistically significant U-shaped association between serum K+ and hospitalization; notably, in the setting of eGFR<30 ml/min per 1.73 m2, the aIRR for K+ levels ≥6.0 mEq/L was three times greater than the referent. Fifth, extremes of serum K+ were associated with greater rates of discontinuation of RAAS blockers.
Prior studies have reported the prevalence of hyperkalemia and hypokalemia (9–11). However, the generalizability of these studies to the CKD population is limited by small sample sizes and a disproportionate representation of patients with very advanced CKD (10,11); the largest of these studies focused on clinical trial participants with concomitant heart failure, further limiting generalizability (11). In another study, a U-shaped association between serum K+ and mortality was observed among patients with CKD who had a mean eGFR<25 ml/min per 1.73 m2 (9); however, risk estimates for higher K+ categories did not achieve statistical significance, possibly because of a small sample size: Only 65 hyperkalemic patients were included in the analysis.
Perhaps as a result of a larger sample size and a greater ability to adjust for potential confounders, our study demonstrated a statistically significant association between hyperkalemia and death. Furthermore, the more representative distribution of eGFR in our study offers greater generalizability across the spectrum of CKD. In particular, we present a large, representative cohort, which demonstrated an incidence of hypokalemia (K+ level <3.5 mEq/L) approximately 1.4% of the time, regardless of eGFR, and an incidence of hyperkalemia (K+ level >5.0 mEq/L) 13%–32% of the time, with greater prevalence at lower eGFR. More significant hyperkalemia (K+ level >5.5 mEq/L) was present 2%–9% of the time, again with greater prevalence at lower eGFR. These findings further support the association of serum K+ and mortality and have widespread generalizability.
Limited published data examine the association between serum K+ and hospitalization. One Canadian study reported 6274 hospitalizations observed with a primary diagnosis code for hyperkalemia among 239,000 patients with CKD followed for 7 years (12). Our results demonstrate nearly equivalent risk of MACE with patient-time spent with serum K+ levels ≥6.0 mEq/L and <3.5 mEq/L compared with K+ levels of 4.5–4.9 mEq/L; however, because more patient-time was spent in hyperkalemia than hypokalemia, the burden of hyperkalemia-associated MACE may be most apparent. Furthermore, likely because of a lack of laboratory data, graded associations between serum K+ and outcomes were not reported.
The results of our study suggest that high and low serum K+ levels are associated with a higher risk for hospitalization, MACE, and mortality. Therefore, it is advisable that serum K+ be maintained within the normal range (3.5–5.0 mEq/L) among patients with CKD. For hypokalemia, management is reasonably straightforward because most patients can be corrected with oral supplements (13). Clinicians should be particularly vigilant in monitoring for hypokalemia, a potent precipitant in the setting of diuretic use, as this and other studies have shown (14).
However, compared with hypokalemia, hyperkalemia therapy is more challenging. Short-term treatment with insulin, calcium, and alkali is effective but short-lived (15), and options for long-term maintenance therapy are limited. For example, diuretics are effective (16) but may be inappropriate for all patients because of volume control: in the present study, <25% of patients with hyperkalemia at baseline were treated with loop or thiazide diuretics, suggesting that contraindications may exist for many patients. Likewise, sodium polystyrene sulfonate was a mainstay of management (17); however, its efficacy is questionable given the emerging safety concerns with its persistent use (18).
Furthermore, our study results indicate that one frequent response to hyperkalemia was discontinuation of RAAS blockers. Although this treatment strategy is effective at reducing serum K+, doing so may result in the loss of the cardiorenal protective effects of these drugs (7). Novel therapies are needed to ameliorate hyperkalemia among patients with CKD and permit ongoing therapy with RAAS blockers.
Of note, there is no agreed-upon standard for defining drug continuation using data from prescription-fill claims. Operationally we chose to define RAAS blocker discontinuation according to a ≥14-day gap in drug supply. Of patients who stopped RAAS blocker per this definition, 50.3% stayed off for the remainder of study; among those who resumed RAAS blocker use, only 1.1% did so within 30 days of the ascribed discontinuation (mean time to resumption, 296±204 days). This reassures that the discontinuation events considered in this study were not artifacts of an overly permissive operational definition.
Limitations of our study should be noted. Although efforts were taken to adjust for many plausible influential variables, confounding by unconsidered factors is possible. Associations between low serum K+ and outcomes may be confounded by nutritional status (as observed in other contexts) (19); availability of such data as height (for body mass index), proteinuria, and nutritional status prohibited their consideration in the present study. As stated, we studied the association between outcomes and serum K+ according to time intervals established by outpatient serum K+ measurements; to control for frequency of serum K+ measures, we also generated estimates using fixed intervals (calendar quarters) with similar results. It is noteworthy, however, that patients with midrange serum K+ have the lowest frequency of serum K+ measurement. These patients are least likely to experience changes in serum K+ because they are the least likely to receive interventions aimed at modifying serum K+ and can maintain a degree of K+ homeostasis. The corollary, therefore, is that the residual bias would be nondifferential, rendering the reported effect estimates conservative.
Another limitation is that this analysis of outpatient laboratory data were amnestic to K+ values during hospitalizations. Therefore, results provide insights regarding management of serum K+ in the ambulatory setting but should not be extrapolated to the acute management of serum K+ among hospitalized patients. Although the population was large and representative of the general CKD population, all patients were treated within a single health care–providing organization; therefore, findings should be confirmed externally. In addition, the study design could not consider over-the-counter treatments, such as potassium-containing supplements. Although the design permitted adjustment for several drug classes, other medications, such as those used infrequently and obtained over the counter (nonsteroidal anti-inflammatory medications), could not be considered. Finally, because this is an observational study, associations do not necessarily allow for causal interpretation.
In conclusion, in a large, generalizable cohort of patients with CKD, abnormalities in serum K+ were common, with hyperkalemia being more prevalent than hypokalemia. Serum K+ had a potent and statistically significant U-shaped association with mortality, MACE, and hospitalization, and hyperkalemia was associated with an increased likelihood of RAAS blocker discontinuation. Clinicians are advised to monitor and correct serum K+ abnormalities among patients with CKD. New strategies and therapies are needed for the treatment of chronic hyperkalemia.
Disclosures
A.Y. is employed by ZS Pharma, Inc. J.L, S.M.B., and D.J. are employed by DaVita Clinical Research. S.M.B.’s spouse is employed by AstraZeneca, Waltham, MA.
Supplementary Material
Acknowledgments
The authors thank DaVita Clinical Research for providing technical support for this research project.
This study and the corresponding analyses were supported by ZS Pharma, Inc., San Mateo, CA.
Some of the data described in this manuscript were presented at the 2014 Annual Meeting of the American Society for Nephrology and the 2014 American Heart Association Scientific Sessions.
Footnotes
Published online ahead of print. Publication date available at www.cjasn.org.
This article contains supplemental material online at http://cjasn.asnjournals.org/lookup/suppl/doi:10.2215/CJN.01730215/-/DCSupplemental.
References
- 1.Hoskote SS, Joshi SR, Ghosh AK: Disorders of potassium homeostasis: Pathophysiology and management. J Assoc Physicians India 56: 685–693, 2008 [PubMed] [Google Scholar]
- 2.Ishii K, Norota I, Obara Y: Endocytic regulation of voltage-dependent potassium channels in the heart. J Pharmacol Sci 120: 264–269, 2012 [DOI] [PubMed] [Google Scholar]
- 3.Petkov GV: Role of potassium ion channels in detrusor smooth muscle function and dysfunction. Nat Rev Urol 9: 30–40, 2012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Poolos NP, Johnston D: Dendritic ion channelopathy in acquired epilepsy. Epilepsia 53[Suppl 9]: 32–40, 2012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.An JN, Lee JP, Jeon HJ, Kim H, Oh YK, Kim YS, Lim CS: Severe hyperkalemia requiring hospitalization: predictors of mortality. Crit Care 16: R225, 2012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Einhorn LM, Zhan M, Hsu VD, Walker LD, Moen MF, Seliger SL, Weir MR, Fink JC: The frequency of hyperkalemia and its significance in chronic kidney disease. Arch Intern Med 169: 1156–1162, 2009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.KDIGO 2012 clinical practice guideline for the evaluation and management of chronic kidney disease. Kidney Int Suppl 3: 1–150, 2013 [DOI] [PubMed] [Google Scholar]
- 8.Weir MR, Rolfe M: Potassium homeostasis and renin-angiotensin-aldosterone system inhibitors. Clin J Am Soc Nephrol 5: 531–548, 2010 [DOI] [PubMed] [Google Scholar]
- 9.Sarafidis PA, Blacklock R, Wood E, Rumjon A, Simmonds S, Fletcher-Rogers J, Ariyanayagam R, Al-Yassin A, Sharpe C, Vinen K: Prevalence and factors associated with hyperkalemia in predialysis patients followed in a low-clearance clinic. Clin J Am Soc Nephrol 7: 1234–1241, 2012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Korgaonkar S, Tilea A, Gillespie BW, Kiser M, Eisele G, Finkelstein F, Kotanko P, Pitt B, Saran R: Serum potassium and outcomes in CKD: Insights from the RRI-CKD cohort study. Clin J Am Soc Nephrol 5: 762–769, 2010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Bowling CB, Pitt B, Ahmed MI, Aban IB, Sanders PW, Mujib M, Campbell RC, Love TE, Aronow WS, Allman RM, Bakris GL, Ahmed A: Hypokalemia and outcomes in patients with chronic heart failure and chronic kidney disease: Findings from propensity-matched studies. Circ Heart Fail 3: 253–260, 2010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Wiebe N, Klarenbach SW, Allan GM, Manns BJ, Pelletier R, James MT, Bello A, Hemmelgarn BR, Tonelli M, Alberta Kidney Disease Network : Potentially preventable hospitalization as a complication of CKD: A cohort study. Am J Kidney Dis 64: 230–238, 2014 [DOI] [PubMed] [Google Scholar]
- 13.Elliott MJ, Ronksley PE, Clase CM, Ahmed SB, Hemmelgarn BR: Management of patients with acute hyperkalemia. CMAJ 182: 1631–1635, 2010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Wang HH, Hung CC, Hwang DY, Kuo MC, Chiu YW, Chang JM, Tsai JC, Hwang SJ, Seifter JL, Chen HC: Hypokalemia, its contributing factors and renal outcomes in patients with chronic kidney disease. PLoS One 8: e67140, 2013 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Acker CG, Johnson JP, Palevsky PM, Greenberg A: Hyperkalemia in hospitalized patients: Causes, adequacy of treatment, and results of an attempt to improve physician compliance with published therapy guidelines. Arch Intern Med 158: 917–924, 1998 [DOI] [PubMed] [Google Scholar]
- 16.Ahmed A, Husain A, Love TE, Gambassi G, Dell’Italia LJ, Francis GS, Gheorghiade M, Allman RM, Meleth S, Bourge RC: Heart failure, chronic diuretic use, and increase in mortality and hospitalization: An observational study using propensity score methods. Eur Heart J 27: 1431–1439, 2006 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Watson M, Abbott KC, Yuan CM: Damned if you do, damned if you don’t: Potassium binding resins in hyperkalemia. Clin J Am Soc Nephrol 5: 1723–1726, 2010 [DOI] [PubMed] [Google Scholar]
- 18.Sterns RH, Rojas M, Bernstein P, Chennupati S: Ion-exchange resins for the treatment of hyperkalemia: Are they safe and effective? J Am Soc Nephrol 21: 733–735, 2010 [DOI] [PubMed] [Google Scholar]
- 19.Kovesdy CP, Regidor DL, Mehrotra R, Jing J, McAllister CJ, Greenland S, Kopple JD, Kalantar-Zadeh K: Serum and dialysate potassium concentrations and survival in hemodialysis patients. Clin J Am Soc Nephrol 2: 999–1007, 2007 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.