Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Sep 2.
Published in final edited form as: Circ Cardiovasc Qual Outcomes. 2020 Sep 2;13(9):e006415. doi: 10.1161/CIRCOUTCOMES.119.006415

Kidney function and potassium monitoring after initiation of renin-angiotensin-aldosterone system blockade therapy and outcomes in two North American populations.

Rishi V Parikh 1,*, Danielle M Nash 2,3,*, K Scott Brimble 4,5, Maureen Markle-Reid 3,7, Thida C Tan 1, Eric McArthur 2, Farzien Khoshniat-Rad 1, Manish M Sood 2,11, Sijie Zheng 6,10, Leonid Pravoverov 6,10, Gihad E Nesrallah 5,8,9, Amit X Garg 2,3,5,12,*, Alan S Go 1,13,14,*
PMCID: PMC7717925  NIHMSID: NIHMS1610570  PMID: 32873054

Abstract

Background:

Clinical practice guidelines recommend routine kidney function and serum potassium testing within 30 days of initiating angiotensin-converting enzyme inhibitor (ACEi) or angiotensin II receptor blocker (ARB) therapy. However, evidence is lacking about whether follow-up testing reduces therapy-related adverse outcomes.

Methods and Results:

We conducted two population-based retrospective cohort studies in Kaiser Permanente Northern California (KPNC), United States of America and Ontario, Canada, Patients with outpatient serum creatinine (SCr) and potassium tests in the 30 days after starting ACEi or ARB therapy were matched 1:1 to patients without follow-up tests. We evaluated the association of follow-up testing with 30-day all-cause mortality and hospitalization with acute kidney injury or hyperkalemia using Cox regression. We also developed and externally validated a risk score to identify patients at risk of having abnormally high SCr and potassium values in follow-up. We identified 75,251 matched pairs initiating ACEi or ARB therapy between January 1, 2007 and December 31, 2017 in KPNC. Follow-up testing was not significantly associated with 30-day all-cause mortality in KPNC (hazard ratio [HR] 0.75, 95% confidence interval [CI]: 0.54-1.06), and was associated with higher mortality in 84,905 matched pairs in Ontario (HR: 1.32, 95% CI: 1.07-1.62). In KPNC, follow-up testing was significantly associated with higher rates of hospitalization with acute kidney injury (HR: 1.66, 95% CI: 1.10-2.22) and hyperkalemia (HR: 3.36, 95% CI: 1.08-10.41), as was observed in Ontario. The risk score for abnormal potassium provided good discrimination (AUC=0.75) and excellent calibration of predicted risks, while the risk score for abnormal SCr provided moderate discrimination (AUC=0.62) but excellent calibration.

Conclusions:

Routine laboratory monitoring after ACEi or ARB initiation was not associated with a lower risk of 30-day mortality. We identified patient subgroups in which targeted testing may be effective in identifying therapy-related changes in serum potassium or kidney function.

Angiotensin converting enzyme inhibitor (ACEi) and angiotensin II receptor blocker (ARB) therapy are a central part of therapeutic strategies to treat hypertension, as well as to preserve kidney function for patients with or at risk for chronic kidney disease (CKD).1 However, initiating ACEi or ARB therapy can cause acute kidney injury or hyperkalemia,2-5 which may increase the risk of subsequent cardiovascular events, kidney failure, and death.6-8

Several consensus-based clinical guidelines recommend routine laboratory monitoring after ACEi/ARB initiation in order to identify asymptomatic kidney injury or hyperkalemia early, so that these abnormalities can be corrected to prevent adverse outcomes.9-11 However, adherence to these guidelines is variable, and estimates of the proportion of patients who receive a follow-up test within a year of initiating ACEi/ARB therapy range from 10-60%.12-14 In addition, evidence is lacking as to whether follow-up testing systematically reduces ACEi/ARB therapy-related adverse outcomes.15, 16

To bridge this knowledge gap, we studied two large, population-based North American cohorts to determine whether routine laboratory monitoring after ACEi/ARB initiation is associated with lower risks of 30-day all-cause death and hospitalizations involving acute kidney injury or hyperkalemia. To help identify high-risk subgroups that may benefit from targeted laboratory monitoring we also created a score to stratify patients on their risk of having elevations in their concentration of serum creatinine or potassium after initiating ACEi/ARB therapy.

METHODS

Source Populations and Study Samples

The two source populations were from Kaiser Permanente Northern California (KPNC) and Ontario, Canada. KPNC is a large, integrated healthcare delivery system that provides comprehensive care for over 4.4 million members across 21 medical centers and over 245 outpatient clinics in Northern California. The KPNC membership is highly representative of the surrounding local and statewide population in terms of age, sex and race/ethnicity.17 Care is captured through KPNC’s comprehensive electronic medical record system that is integrated across all practice settings. The province of Ontario, Canada has >13 million residents, 17% of whom are 65 years or older.18 Healthcare services in Ontario are funded through the Ontario Health Insurance Plan program with the exception of outpatient medications, which are only funded for segments of the population, including all people aged 65 years and older.19 These healthcare encounters are recorded in administrative databases, which use unique, encoded identifiers and are held at ICES – a not-for-profit research institute. Unfortunately, due to the nature of protected health information used in this study, data cannot be made available for public access.

We identified all adult members (≥18 years) initiating ACEi or ARB therapy at KPNC between January 1, 2007 and December 31, 2017. We made the following exclusions: 1) patients with <1 year of continuous health plan membership and pharmacy benefit before the index ACEi/ARB prescription date to ensure complete capture of baseline comorbidities and laboratory values, 2) patients who received ACEi or ARB therapy in the previous 180 days to ensure new users, 3) patients with both an ACEi and ARB on the prescription date since co-prescription increases risk of adverse outcomes, 4) patients without at least one outpatient value for both serum creatinine and serum potassium in the prior year to assess baseline levels, 5) patients with baseline potassium values >5.5 mEq/L or a prescription for a potassium binder in the previous 180 days, 6) patients with baseline estimated glomerular filtration rate (eGFR) >150 mL/min/1.73 m2 using the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) equation20, as these values are known to be unreliable estimates of kidney function, and 7) patients with a history of end-stage kidney disease (ESKD) defined as the receipt of chronic dialysis or kidney transplant (Figure 1). The Ontario cohort was assembled using similar inclusion and exclusion criteria in patients aged 66 years or older (Supplementary Figure 1).

Figure 1. Cohort assembly diagram for patients at KPNC, January 1, 2007- December 31, 2017.

Figure 1.

Open in a new tab

Abbreviations: ACEi, Angiotensin-converting enzyme inhibitor; ARB, angiotensin II receptor blocker.

For routine monitoring, people typically receive a laboratory requisition for SCr and electrolyte testing at the time of their initial ACEi or ARB prescription, where the blood sample is drawn in the following weeks. We defined our study exposure of outpatient laboratory monitoring as completing both a serum creatinine and serum potassium test within 30 days following the ACEi or ARB dispensing date. The date of testing was considered the index date for outcome ascertainment. We excluded patients if they did not have both tests done at the same time. We also excluded patients if they had evidence of certain healthcare encounters (hospitalization, emergency department visit, any other laboratory tests completed, or visit with a primary care physician, nephrologist, internist, endocrinologist, or cardiologist) between the dispensing date and day before index date, as this may have suggested that testing was done in response to new symptoms or an adverse event rather than routine laboratory monitoring.

To reduce potential bias in outcome ascertainment, we randomly assigned an index date to matched patients in the untested group based on the distribution of time between the dispensing and index dates in the follow-up test group. For the untested group, we repeated the exclusions described above. We also excluded patients in the untested group who died in the period between the dispensing date and assigned index date to avoid the potential for immortal time bias, since patients in the tested group (by definition) could not die during this interim period.

This study was approved by the KPNC institutional review board. Waiver of informed consent was obtained due to the nature of the study. The use of ICES data in this project was authorized under section 45 of Ontario’s Personal Health Information Protection Act, which does not require review by a Research Ethics Board. No informed consent from patients was required.

Follow-up and Outcomes

We assessed all clinical outcomes within 30 days of the index date. We used a 30-day follow-up since clinically significant increases in serum creatinine and potassium following ACEi or ARB initiation generally present within this time period.21-23 As a sensitivity analysis, we also assessed outcomes at 60 days. Our primary outcome was all-cause mortality, and secondary outcomes included hospitalization with acute kidney injury and hospitalization with hyperkalemia. We defined acute kidney injury events based on Kidney Disease Improving Global Outcomes (KDIGO) serum creatinine-based criteria of an increase of ≥50% or 0.3 mg/dL above a baseline value.24 We selected the highest serum creatinine value either within the first three days of a hospitalization or during an emergency department visit which resulted in a hospitalization. We identified hyperkalemia events using the highest serum potassium value within the same time window, and defined it using a serum potassium value of ≥5.5 mEq/L. In the Ontario cohort, we only assessed the outcomes of hospitalization with acute kidney injury and hyperkalemia among patients residing in areas with linked inpatient laboratory data capture (~28% of the matched cohort). Additional a priori outcomes included all-cause hospitalization and a hospital or emergency department encounter with ventricular arrhythmia or sudden cardiac death.

Covariates

In the KPNC cohort, demographic characteristics (age, sex, self-reported race/ethnicity) were obtained from health plan databases. Relevant comorbidities were defined by diagnostic or procedure ICD-9/ICD-10 codes (codes available upon request) and supplemented with laboratory test results, outpatient vital signs, and/or prescribed medications using electronic medical record-based data that was cleaned and linked at the individual-patient level into the KPNC Virtual Data Warehouse as previously described and validated.25, 26 Patient vital status was based on comprehensive information from administrative and clinical databases, member proxy reporting, Social Security Administration vital status files, and California state death certificate information.27

In the Ontario cohort, we used the Ontario Drug Benefit database to identify outpatient prescriptions for individuals aged 65 and older, covered through the Ontario Drug Benefit program.28 We used the Ontario Laboratories Information System database to identify laboratory values. We had no information on race and assumed all patients to be non-black for the CKD-EPI equation (less than 5% of the Ontario population is of black race). We used eight additional linked databases held at ICES for information on hospitalizations and emergency department visits, physician billings, treatment for ESKD, and demographics and mortality data. We used Canadian Modified ICD-10 codes to define comorbidities and the Canadian Classification for Health Interventions to define procedures. We also used the Johns Hopkins’ Aggregated Diagnosis Groups and Expanded Diagnosis Clusters to assess baseline comorbidity (Johns Hopkins’ ACG® System Version 10.0).29

Statistical Approach

We conducted all analyses using SAS version 9.4 (Cary, NC). We matched patients in the tested group 1:1 to patients in the untested group using a greedy-matching algorithm based on baseline eGFR ±5 mL/min/1.73 m2, baseline serum potassium ±0.5 mEq/L, age ±2 years, sex, and ±0.2 standard deviations of the logit of a propensity score. In the KPNC cohort, a high-dimensional propensity score (hd-PS) was calculated using an algorithm-generated logistic regression model that included demographic and multi-dimensional patient characteristics. The algorithm selected the 100 best-matched parameters from three data dimensions (i.e., outpatient medications, and prior diagnoses and procedures across care settings) that predicted receipt of follow-up testing.30 In the Ontario cohort, we calculated a propensity score for the probability of receiving routine monitoring using a multivariable logistic regression model that incorporated more than 75 baseline characteristics.31 We matched patients in this cohort using the same criteria as the KPNC cohort, and additionally on residence within areas with linked inpatient laboratory data capture. We compared baseline characteristics for tested and matched untested patients using standardized differences, of which magnitudes of more than 10% were considered potentially meaningful.32

We used multivariable Cox proportional hazards regression stratified on matched pairs to evaluate the association of laboratory testing with outcomes. In the KPNC cohort, we additionally adjusted for pre-specified confounding variables and any variables that were significantly different (standardized difference >0.15) between matched tested and untested patients. Confounding variables included number of days from prescription to index date, class of index prescription (ACEi, ACEi/thiazide diuretic, ARB, ARB/thiazide diuretic), index prescription dose (percent of maximal dose), prior peripheral artery disease, non-steroidal anti-inflammatory drug use, and diuretic use. We also conducted supplementary analyses of all outcomes stratifying on sex.

We created a risk score to predict receipt of an abnormal SCr result (≥30% over baseline) or abnormal potassium (>5.5 mmol/L) after ACEi or ARB initiation among tested patients. To identify predictors of abnormal labs, we performed bootstrapping with replacement to generate 1000 unique samples from our original KPNC cohort and performed bi-directional stepwise logistic regression analyses (with a p<0.05 entry and inclusion criterion) on each sample using all baseline covariates. We flagged significant predictors as variables that were retained in at least 70% of the models and ran a final logistic regression model with these variables to identify the odds ratios and 95% confidence intervals (CI).33 We further selected only predictors with an odds ratio greater than 1.2 or less than 0.8, to isolate the most relevant predictors. We then calculated a weighted integer risk score using rounded model coefficients and ascertained the discrimination and calibration of the risk score in our original cohort. The risk score was developed in the KPNC cohort and externally validated in the Ontario cohort. Finally, we created an online calculator for ease of use.

RESULTS

Study Cohort and Baseline Characteristics

We identified 296,015 eligible adults at KPNC (486,447 older adults in Ontario) initiating ACEi or ARB therapy between 2007 and 2017 (Figure 1 and Supplementary Figure 1), of which 75,823 (25.6%) in KPNC and 153,961 (32%) in Ontario received a routine outpatient serum creatinine and serum potassium test a mean (SD) of 12 (7) days after ACEi or ARB initiation in KPNC and 12 (8.3) days in Ontario. Baseline characteristics for both unmatched cohorts are shown in Supplementary Tables 1 and 2.

The final KPNC cohort included 75,251 eligible adults with follow-up testing matched to 75,251 without testing. KPNC patients were a mean (SD) age of 64 (14) years and 52% were women. After matching, there remained differences in prior valvular disease; prior atrial flutter/fibrillation; use of potassium-sparing diuretics, antiarrhythmic drugs, and anticoagulants; number of primary care visits in the prior year; time from baseline SCr measurement to index date; and time from baseline serum potassium measurement to index date (Table 1). The final Ontario cohort included 84,905 matched pairs; the mean (SD) age was 75 (7) years and 53% were women (Table 1).

Table 1.

Baseline characteristics in a matched cohort of patients initiating ACEi or ARB therapy with and without follow-up SCr and K+ testing within 30 days at KPNC, January 1, 2007- December 31, 2017.

KPNC Cohort
(N=150,502)		 Ontario Cohort
(N=169,810)
Variable Follow-up tests
(N=75,251)		 No Follow-up
Tests
(N=75,251)		 Std.
Difference		 Follow-up tests
(N=84,905)		 No Follow-up
Tests
(N=84,905)		 Std.
Difference
Age, yr, mean (SD) 64.2 (13.6) 64.2 (13.6) 0 75.4 (7.0) 75.4 (7.0) 0
Age Categories, n (%) 0 0
   18-39 3704 (4.9) 3722 (4.9) 0 (0.0%) 0 (0.0%)
   40-65 36,735 (48.8) 36,723 (48.8) 0 (0.0%) 0 (0.0%)
   66-74 17,464 (23.2) 17,462 (23.2) 43,391 (51.1%) 43,397 (51.1%)
   75-84 13,049 (17.3) 13,048 (17.3) 31,202 (36.7%) 31,216 (36.8%)
   >85 4299 (5.7) 4296 (5.7) 10,312 (12.1%) 10,292 (12.1%)
Women, n (%) 39,338 (52.3) 39,338 (52.3) 0 44,273 (52.1%) 45,130 (53.2%) 0.02
Self-reported Race 0.01
   White 42,193 (56.1) 41,826 (55.6) N/A N/A
   Black/African American 5562 (7.4) 5862 (7.8) N/A N/A
   Asian/Pacific Islander 13,120 (17.4) 12,464 (16.6) N/A N/A
   Other/Unknown 14,376 (19.1) 15,099 (20.1) N/A N/A
Hispanic ethnicity, n (%) 10,478 (13.9) 11,232 (14.9) 0.03 N/A N/A
Smoking status, n (%) 0
   Current Smoker 5805 (7.7) 6607 (8.8) N/A N/A
   Former smoker 22,625 (30.1) 21,136 (28.1) N/A N/A
   Nonsmoker 46,821 (62.2) 47,508 (63.1) N/A N/A
Household income <$35,000/year, n (%) 1408 (1.9) 1558 (2.1) 0.06
Income quintile, n (%)* 0
   1 (lowest) N/A N/A 16,711 (19.7%) 16,729 (19.7%)
   2 N/A N/A 18,094 (21.3%) 18,359 (21.6%)
   3 N/A N/A 17,234 (20.3%) 17,318 (20.4%)
   4 N/A N/A 16,559 (19.5%) 16,465 (19.4%)
   5 (highest) N/A N/A 16,307 (19.2%) 16,034 (18.9%)
Time from index date to lab test, days, mean (SD) 11.8 (7.2) 11.8 (7.3) 0 11.5 (8.3) 11.5 (8.3) 0.01
Year of index date, yr 0.05 0.02
   2007 10,549 (14.0) 10,692 (14.2) 52 (0.1%) 42 (0.0%)
   2008 3604 (4.8) 3807 (5.1) 1837 (2.2%) 1660 (2.0%)
   2009 3360 (4.5) 3555 (4.7) 5430 (6.4%) 4837 (5.7%)
   2010 17,561 (23.3) 20,840 (27.7) 7770 (9.2%) 7307 (8.6%)
   2011 7957 (10.6) 7140 (9.5) 8144 (9.6%) 8057 (9.5%)
   2012 6321 (8.4) 5906 (7.8) 9207 (10.8%) 9434 (11.1%)
   2013 6315 (8.4) 5162 (6.9) 9436 (11.1%) 9764 (11.5%)
   2014 5627 (7.5) 4730 (6.3) 10,158 (12.0%) 10,454 (12.3%)
   2015 5920 (7.9) 5272 (7.0) 10,462 (12.3%) 10,661 (12.6%)
   2016 4274 (5.7) 4209 (5.6) 11,086 (13.1%) 11,294 (13.3%)
   2017 3763 (5.0) 3938 (5.2) 11,323 (13.3%) 11,395 (13.4%)
Medical history*, n (%)
   Acute myocardial infarction 1523 (2.0) 1319 (1.8) 0.09 2746 (3.2%) 2697 (3.2%) 0
   Coronary artery bypass graft surgery 1132 (1.5) 962 (1.3) 0.10 1983 (2.3%) 1950 (2.3%) 0
   Percutaneous coronary intervention 7260 (9.6) 7379 (9.8) 0.01 2717 (3.2%) 2695 (3.2%) 0
   Heart failure 3730 (5.0) 3231 (4.3) 0.09 11,260 (13.3%) 10,897 (12.8%) 0.01
   Mitral and/or aortic valvular disease 3315 (4.4) 2607 (3.5) 0.15 1817 (2.1%) 1715 (2.0%) 0.01
   Atrial flutter or fibrillation 5290 (7.0) 3929 (5.2) 0.19 6307 (7.4%) 5972 (7.0%) 0.02
   Ventricular tachycardia or fibrillation 162 (0.2) 143 (0.2) 0.08 366 (0.4%) 365 (0.4%) 0
   Ischemic stroke or transient ischemic attack 1449 (1.9) 1475 (2.0) 0.01 3563 (4.2%) 3564 (4.2%) 0
   Peripheral artery disease 2699 (3.6) 2535 (3.4) 0.04 1365 (1.6%) 1309 (1.5%) 0.01
   Diabetes mellitus 25,266 (33.6) 28,694 (38.1) 0.12 35,585 (41.9%) 35,767 (42.1%) 0
   Hypertension 53,734 (71.4) 55,588 (73.9) 0.08 70,909 (83.5%) 70,641 (83.2%) 0.01
   Dyslipidemia 52,517 (69.8) 53,517 (71.1) 0.04 26,467 (31.2%) 26,387 (31.1%) 0
   Hyperthyroidism 2599 (3.5) 2476 (3.3) 0.03 1646 (1.9%) 1646 (1.9%) 0
   Hypothyroidism 10,470 (13.9) 10,195 (13.5) 0.02 8263 (9.7%) 8383 (9.9%) 0.01
   Chronic liver disease 3058 (4.1) 2656 (3.5) 0.09 865 (1.0%) 832 (1.0%) 0
   Chronic lung disease 18,158 (24.1) 17,654 (23.5) 0.02 18,896 (22.3%) 18,883 (22.2%) 0
   Diagnosed dementia 1478 (2.0) 1687 (2.2) 0.08 8078 (9.5%) 7992 (9.4%) 0
   Diagnosed depression 9698 (12.9) 9990 (13.3) 0.02 1258 (1.5%) 1275 (1.5%) 0
   Hospitalized bleed 1061 (1.4) 995 (1.3) 0.04 9772 (11.5%) 9741 (11.5%) 0
Body mass index category, kg/m2, n (%) 0.07
   <18.5 625 (0.8) 603 (0.8) N/A N/A
   18.5-24.9 14,584 (19.4) 13,436 (17.9) N/A N/A
   25-29.9 24,029 (31.9) 22,936 (30.5) N/A N/A
   30-39.9 24,906 (33.1) 25,527 (33.9) N/A N/A
   ≥40 3672 (4.9) 3871 (5.1) N/A N/A
   Unknown 7435 (9.9) 8878 (11.8) N/A N/A
Systolic blood pressure category, mmHg, n (%) 0.08
   <120 10,439 (13.9) 12,708 (16.9) N/A N/A
   120-129 10,188 (13.5) 11,979 (15.9) N/A N/A
   130-139 16,825 (22.4) 17,879 (23.8) N/A N/A
   140-159 24,108 (32.0) 19,362 (25.7) N/A N/A
   160-180 6485 (8.6) 5205 (6.9) N/A N/A
   ≥180 1430 (1.9) 1164 (1.5) N/A N/A
   Unknown 5776 (7.7) 6954 (9.2) N/A N/A
Index prescription class, n (%) 0.08 0.01
   Angiotensin converting enzyme (ACE) inhibitor 45,053 (59.9) 41,423 (55.0) 50,566 (59.6%) 50,550 (59.5%)
   ACE inhibitor/thiazide combination 10,397 (13.8) 11,241 (14.9) 4662 (5.5%) 5082 (6.0%)
   Angiotensin II receptor blocker (ARB) 19,137 (25.4) 21,814 (29.0) 23,863 (28.1%) 23,599 (27.8%)
   ARB/thiazide combination 664 (0.9) 773 (1.0) 5814 (6.8%) 5674 (6.7%)
Percentage of maximum daily dose, %, mean (SD) 31.1 (23.8) 37.2 (27.7) 0.24
Baseline medication use†, n (%)
   Beta blocker 28,457 (37.8) 25,152 (33.4) 0.12 25,507 (30.0%) 25,220 (29.7%) 0.01
   Calcium channel blocker 12,308 (16.4) 11,542 (15.3) 0.05 25,297 (29.8%) 25,256 (29.7%) 0
   Loop or thiazide diuretic 42,978 (57.1) 40,572 (53.9) 0.08 35,686 (42.0%) 34,720 (40.9%) 0.02
   Potassium-sparing diuretic 937 (1.2) 656 (0.9) 0.22 4119 (4.9%) 4013 (4.7%) 0.01
   Alpha blocker 5510 (7.3) 5124 (6.8) 0.05 2343 (2.8%) 2316 (2.7%) 0.01
   Any anti-hypertensive agent 71,342 (94.8) 70,802 (94.1) 0.08 57,036 (67.2%) 56,592 (66.7%) 0.01
   Nitrates 2166 (2.9) 1834 (2.4) 0.1 4692 (5.5%) 4575 (5.4%) 0
   Antiarrhythmic drug 1012 (1.3) 580 (0.8) 0.34 1209 (1.4%) 1132 (1.3%) 0.01
   Statin 38,767 (51.5) 38,876 (51.7) 0 45,749 (53.9%) 45,779 (53.9%) 0
   Other lipid-lowering agent 3363 (4.5) 3109 (4.1) 0.05 5658 (6.7%) 5684 (6.7%) 0
   Non-aspirin antiplatelet agent 3218 (4.3) 2668 (3.5) 0.12 5420 (6.4%) 5465 (6.4%) 0
   Anticoagulant 4582 (6.1) 2636 (3.5) 0.35 319 (0.4%) 300 (0.4%) 0
   Non-steroidal anti-inflammatory drug 11,296 (15.0) 11,037 (14.7) 0.02 8162 (9.6%) 8237 (9.7%) 0
   Diabetic therapy 11,451 (15.2) 13,171 (17.5) 0.1 22,053 (26.0%) 22,370 (26.3%) 0.01
Baseline laboratory values‡
   CKD-EPI eGFR, mL/min/1.73 m2, mean (SD) 75.2 (20.9) 75.2 (20.9) 0 68.5 (18.4) 68.5 (18.4) 0
   CKD-EPI eGFR category, mL/min/1.73 m2 0 0
     90-150 19,435 (25.8) 19,416 (25.8) 8999 (10.6%) 8977 (10.6%)
     60-89 37,688 (50.1) 37,673 (50.1) 49,157 (57.9%) 49,179 (57.9%)
     45-59 11,675 (15.5) 11,717 (15.6) 16,026 (18.9%) 16,026 (18.9%)
     30-44 5211 (6.9) 5223 (6.9) 8278 (9.7%) 8323 (9.8%)
     15-29 1200 (1.6) 1187 (1.6) 2332 (2.7%) 2300 (2.7%)
     <15 42 (0.1) 35 (0.0) 113 (0.1%) 100 (0.1%)
   Serum potassium, mg/dL, mean (SD) 4.3 (0.4) 4.3 (0.4) 0.03 4.4 (0.4) 4.4 (0.4) 0.02
   Serum potassium category, mg/dL 0 0
     < 5.0 70,581 (93.8) 70,581 (93.8) 77,448 (91.2%) 77,448 (91.2%)
     5.0-5.5 4670 (6.2) 4670 (6.2) 7457 (8.8%) 7457 (8.8%)
   Time from baseline SCr to index date, days 0.01
     Mean (SD) 113.3 (108.3) 95.9 (99.7) 0.17 129.3 (112.1) 128.3 (110.8)
     Median (interquartile range) 80.0 (15.0-192.0) 60.0 (9.0-160.0) 97 (26-217) 98 (26-219)
   Time from baseline K+ to index date, days 0.01
     Mean (SD) 121.0 (111.1) 103.4 (103.4) 0.16 136.4 (114.2) 135.5 (112.4)
     Median (interquartile range) 91.3 (16.5-205.7) 69.3 (11.0-176.0) 108 (28-228) 111 (29-230)
Baseline healthcare utilization§
   Primary care visits 0.13 0
     Mean (SD) 5.2 (6.0) 4.5 (4.8) 8.7 (9.1) 8.7 (9.0)
     Median (IQR) 4 (2-6) 3 (2-6) 6 (4-10) 6 (4-10)
   Hospitalizations 0.10 0
     Mean (SD) 0.2 (0.7) 0.2 (0.6) 0.2 (0.6) 0.2 (0.6)
     Median (IQR) 0 (0-0) 0 (0-0) 0 (0-0) 0 (0-0)
   Emergency department visits 0.06 0.01
     Mean (SD) 0.5 (1.0) 0.4 (1.0) 0.7 (1.3) 0.7 (1.3)
     Median (IQR) 0 (0-1) 0 (0-0) 0 (0-1) 0 (0-1)
   Nephrology visits 0.05 0.02
     Mean (SD) 0.1 (0.5) 0.0 (0.4) 0.2 (0.9) 0.2 (0.8)
     Median (IQR) 0 (0-0) 0 (0-0) 0 (0-0) 0 (0-0)
Open in a new tab

Abbreviations: ACEi, Angiotensin-converting enzyme inhibitor; ARB, angiotensin II receptor blocker; KPNC, Kaiser Permanente Northern California; SCr, serum creatinine; K+, serum potassium; eGFR, estimated glomerular filtration rate.

*

Medical history ascertained within 5 year prior to prescription date.

†

Baseline medication use ascertained within 120 days prior to prescription date

‡

Baseline laboratory values ascertained as the most recent outpatient, non-emergency department value within 1 year prior to prescription date. In Ontario cohort, only assessed for patients in the laboratory catchment area (N=23,821 per group).

§

Baseline healthcare utilization ascertained within 1 year of prescription date.

All-cause Mortality

Overall, 236 patients (0.16%) in the KPNC cohort died within 30 days, 125 (0.17%) in the tested group and 113 (0.15%) in the untested group (Supplementary Table 3). In multivariable analysis, there was no significant association of testing with death at 30 days (adjusted hazard ratio [aHR]: 0.75, 95% CI: 0.54-1.06) or at 60 days (aHR: 1.02, 95% CI: 0.83-1.26) (Figure 2, Supplementary Tables 3-5). Associations were also not statistically significant in sex-stratified analyses, although the direction of the point estimate differed for men (aHR: 1.26, 95% CI: 0.65-1.94) and women (aHR: 0.54, 95% CI: 0.29-1.00) (Supplementary Table 6). Crude rates and associations in the Ontario cohort were in the opposite direction for 30-day all-cause mortality (HR: 1.32, 95% CI: 1.07-1.62) (Figure 2, Supplementary Table 5). In sex-stratified analyses, there was a significant association for women (HR: 1.46, 95% CI: 1.10-1.94) but not men (HR: 1.14, 95% CI: 0.84-1.54) (Supplementary Table 7).

Figure 2. Associations between follow-up testing after ACEi/ARB initiation and outcomes.

Figure 2.

Open in a new tab

All KPNC models adjusted for a priori confounders: time from prescription to follow-up test date, class of index prescription (ACEi, ACEi/thiazide, ARB, ARB/thiazide), percentage of maximum daily dosage of index prescription, prior peripheral artery disease, receipt of NSAIDs, and receipt of diuretics. KPNC models additionally adjusted for covariates differing at baseline: prior valvular disease, prior atrial flutter/fibrillation, receipt of potassium-sparing diuretics, antiarrhythmic drugs, and anticoagulants, number of primary care visits in the prior year, time from baseline SCr measurement to index date and time from baseline potassium measurement to index date. Abbreviations: ACEi, Angiotensin-converting enzyme inhibitor; ARB, angiotensin II receptor blocker; AKI, acute kidney injury; NSAID, non-steroidal anti-inflammatory drug; SCr, serum creatinine

Hospitalization with Acute Kidney Injury and Hyperkalemia

A total of 201 patients (0.13%) at KPNC were hospitalized with acute kidney injury within 30 days. Follow-up testing was significantly associated with a higher risk of hospitalization with acute kidney injury: 133 (0.18%) versus 68 (0.09%) (Supplementary Table 3), which persisted in multivariable analysis (aHR 1.66, 95% CI: 1.10-2.22) (Figure 2). A subset of 14 patients with acute kidney injury-related hospitalizations (0.002%) required acute hemodialysis, of which 11 were in the tested group and 3 were in the untested group. Overall, 61 patients (0.04%) were hospitalized with hyperkalemia within 30 days, and follow-up testing was significantly associated with increased risk of hospitalization with hyperkalemia (aHR: 3.36, 95% CI: 1.08-10.41). (Figure 2). Follow-up testing was not significantly associated with sudden cardiac death or hospital encounter with ventricular arrhythmia but was significantly associated with a higher risk of all-cause hospitalization (Figure 2). The associations between follow-up testing and hospitalizations with acute kidney injury and hyperkalemia were all attenuated in women compared to men (Supplementary Table 6). In addition, follow-up testing was significantly associated with sudden cardiovascular death or a hospital encounter with ventricular arrhythmia in men (aHR: 3.54, 95% CI: 1.25-10.06) (Supplementary Table 6).

In the Ontario cohort, follow-up testing was significantly associated with hospitalization with acute kidney injury (HR: 1.59, 95% CI: 1.24-2.05) but not hyperkalemia (HR 1.45, 95% CI: 0.82-2.56), although the magnitude and direction of the associations were similar to those in the KPNC cohort (Figure 2). Associations with other secondary outcomes were also similar (Figure 2).

Risk Scores for Abnormal Laboratory Results

Among 75,823 patients with follow-up testing in the unmatched KPNC cohort, 2,453 (3.2%) experienced an abnormal increase in SCr and 530 (0.7%) experienced an abnormally high potassium value on testing after ACEi/ARB initiation. These populations were relatively distinct, as only 86 (0.1%) experienced abnormal increases in both SCr and potassium. Significant predictors for an abnormal SCr and/or serum potassium value are listed in Table 2. The risk score for abnormal SCr displayed moderate discrimination, with an AUC of 0.69 in the KPNC derivation cohort and 0.62 in the Ontario validation cohort and was well calibrated to observed absolute risks in the validation cohort (Figure 3A). The risk score for abnormal serum potassium had good discrimination, with an AUC of 0.79 in the KPNC cohort and 0.75 in the Ontario cohort and was also well-calibrated to observed risks in the Ontario cohort (Figure 3B). The risk score for either abnormal SCr or serum potassium displayed very similar performance as the risk score for abnormal SCr (Figure 3C). All calibration plots for the derivation cohort are shown in Supplementary Figure 2. This model is available as an online calculator: https://qxmd.com/calculate/calculator_756.

Table 2.

Predictors and risk scores for abnormal potassium, serum creatinine, and both on follow-up testing after ACEi/ARB initiation among all patients with follow-up SCr and K+ tests within 30 days at KPNC, January 1, 2007- December 31, 2017.

Abnormal Serum Creatinine (≥30% increase) Abnormal Potassium (>5.5 mmol/L) Abnormal Serum Creatinine or Potassium
Predictor Risk Points Predictor Risk Points Predictor Risk Points
Age category Male sex 5 Age category
   18-39 0 Drug class    18-39 0
   40-65 1    ARB 0    40-65 0
   66-74 3    ACEi 5    66-74 3
   75-84 6 Potassium, mmol/L    75-84 5
   >85 8    <5.0 0    >85 5
Female sex 1    5.0-5.5 19 Drug class
Drug class Estimated GFR, mL/min/1.73m2    ARB 0
   ARB 0    90-150 0    ACEi 3
   ACEi 3    60-89 5 Potassium, mmol/L
Estimated GFR, mL/min/1.73m2    45-59 10    <5.0 0
   90-150 9    30-44 14    5.0-5.5 7
   60-89 3    15-29 18 Estimated GFR, mL/min/1.73m2
   45-59 1    <15 16    90-150 6
   30-44 0 Hemoglobin    60-89 0
   15-29 2    ≥13.0 0    45-59 0
   <15 3    12.0-12.9 2    30-44 0
Hemoglobin    11.0-11.9 8    15-29 3
   ≥13.0 0    10.0-10.9 9    <15 4
   12.0-12.9 3    9.0-9.9 8 Hemoglobin
   11.0-11.9 4    <9.0 13    ≥13.0 0
   10.0-10.9 5    Unknown 2    12.0-12.9 3
   9.0-9.9 6 Potassium-sparing diuretic 10    11.0-11.9 4
   <9.0 10 Diabetes mellitus 3    10.0-10.9 6
   Unknown 0    9.0-9.9 6
Diagnosed depression 2    <9.0 11
Heart failure 4    Unknown 0
Hyperthyroidism 4 Heart failure 4
Peripheral artery disease 4 Hyperthyroidism 4
Potassium-sparing diuretic 4 Peripheral artery disease 4
No anti-hypertensive therapy 7 Potassium-sparing diuretic 4
Diabetic therapy 2 No anti-hypertensive therapy 6
Loop or thiazide diuretic 6 Diabetic therapy 3
≥1 hospitalization in prior year 7 Loop or thiazide diuretic 5
≥1 hospitalization in prior year 7
Open in a new tab

Abbreviations: ACEi, Angiotensin-converting enzyme inhibitor; ARB, angiotensin II receptor blocker; SCr, serum creatinine; K+, potassium; GFR, glomerular filtration rate.

Figure 3. Calibration plots of risk scores for abnormal laboratory results after ACEi/ARB initiation, by 5-unit risk score increments, in the Ontario validation cohort.

Figure 3.

Open in a new tab

Abbreviations: ACEi, Angiotensin-converting enzyme inhibitor; ARB, angiotensin II receptor blocker.

DISCUSSION

In two large, population-based cohorts from integrated healthcare delivery systems in North America, we found that routine laboratory monitoring after ACEi or ARB initiation was not associated with lower all-cause mortality within 30 days. However, follow-up testing was associated with increased 30-day hospitalization or emergency department visits with acute kidney injury and hyperkalemia in both cohorts. These increases in hospitalizations with kidney-related outcomes among the tested group may be due to earlier detection of kidney injury and/or severe electrolyte abnormalities, which could be viewed as a positive outcome, especially if earlier detection leads to therapy adjustment or other management strategies that avoid subsequent adverse clinical outcomes. In subgroup analyses by sex, we found differences in the associations of follow-up testing and outcomes between men and women, especially with regard to 30-day all-cause mortality, but results were variable across cohorts and warrant further exploration.

Prior studies examining the association of laboratory monitoring after initiation of antihypertensive agents on outcomes have found mixed results. McDowell et al. found that patients in the UK with serum creatinine monitoring after antihypertensive therapy initiation experienced no significant difference in mortality within 6 months of treatment initiation compared to patients without laboratory monitoring.16 The authors also describe a similar increase in all-cause hospitalization as well as an increase in medication discontinuation among monitored patients. In contrast, Raebel et al. found that in a sample of adults with diabetes, potassium monitoring after ACEi/ARB initiation was associated with lower rates of hyperkalemia-related events including hospitalizations, emergency department visits, and death.15 Our study within larger, more contemporary and ethnically diverse matched populations extend these previous findings by focusing on early laboratory monitoring and the potential clinical value within targeted high-risk subgroups.

To identify subgroups of patients at higher risk for experiencing an abnormal increase in serum creatinine and/or potassium, we created and externally validated risk models using patient characteristics readily available at the time of ACEi/ARB initiation. These included several previously described as risk factors for hyperkalemia and kidney injury after ACEi/ARB initiation, including baseline CKD and potassium levels, heart failure, peripheral artery disease, use of potassium-sparing diuretics, and diabetes mellitus, among others.14, 34 While the models had moderate discrimination (likely affected by the overall low absolute risks of outcomes), they had excellent calibration for accurately predicting absolute risks in both the derivation and external validation cohorts. By using these risk scores to aid clinical judgment, providers and care teams can more efficiently direct resources to the highest-risk patients for laboratory monitoring and avoid low-yield testing in the lowest-risk patients, reducing unnecessary health care utilization and respective costs for patients and health systems. Given that current practices for routine laboratory testing may be ineffective at a population level based on the results of our analyses, eliminating low-yield testing represents an opportunity for health systems to streamline care and reduce costs and unnecessary phlebotomy for patients.

Despite using data from large, population-based cohorts and a rigorous observational methodology, our study had several limitations. We cannot rule out the presence of confounding by indication, as patients may receive laboratory monitoring due to factors that directly increase their risk of adverse outcomes. We attempted to control for this through propensity score matching and further adjustment of remaining baseline differences in multivariable models in the KPNC cohort. We used single imputation to create index dates for the untested group which may be less robust than multiple imputation. The variability in our observed associations between cohorts for the primary outcome of all-cause death may be due to differences in populations, practice patterns, or residual confounding. Variability in the associations within each cohort is most likely due to low numbers of events. It remains unclear whether the increases in hospitalizations for acute kidney injury and hyperkalemia were prompted by laboratory results or occurred independent of monitoring. However, our main findings are strengthened by the parallel analyses in two large, North American population-based cohorts. As both studies were conducted within integrated healthcare delivery systems, our results may not be fully generalizable to other practice settings, geographic areas, or uninsured individuals.

In conclusion, we found that routine laboratory monitoring of serum creatinine and potassium after ACEi/ARB initiation may not be effective in preventing 30-day death and related hospitalizations among all patients initiating ACEi/ARB therapy. However, a validated risk score may help providers stratify patients according to their risk of kidney injury and hyperkalemia to identify subgroups for targeted laboratory monitoring and early intervention.

Supplementary Material

Supplemental Material
Supplemental Material - RECORD

What is Known:

• Initiating angiotensin-converting enzyme inhibitor (ACEi) or angiotensin II receptor blocker (ARB) therapy can increase the risk for kidney dysfunction or hyperkalemia in some patients.

• Multiple practice guidelines recommend routine laboratory monitoring after starting ACEi or ARB therapy; however, little evidence exists that this approach is effective in reducing adverse outcomes.

What the study adds:

• In two large North American cohorts, the rate of routine laboratory monitoring after an initial ACEi/ARB prescription was approximately 30%.

• Laboratory monitoring was not significantly associated with lower mortality or hospitalizations with acute kidney injury and hyperkalemia in either propensity-matched cohorts.

• We created integrated risk scores to identify patients at highest short-term risk for abnormal laboratory results after ACEi/ARB initiation.

Acknowledgments

Sources of Funding

Dr. Go receives support from the National Institute for Diabetes, Digestive and Kidney Diseases (U01 DK060902, U01 DK082223, R01 DK101507, R01 DK098233, R01 DK103612, R01 DK107931, U01 DK108809).

This study was supported by ICES Western site. ICES is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). Core funding for ICES Western is provided by the Academic Medical Organization of Southwestern Ontario (AMOSO), the Schulich School of Medicine and Dentistry (SSMD), Western University, and the Lawson Health Research Institute (LHRI). This research was conducted by members of the ICES Kidney, Dialysis and Transplantation team, at the ICES Western facility, who are supported by a grant from the Canadian Institutes of Health Research (CIHR). Parts of this material are based on data and/or information compiled and provided by CIHI. The opinions, results and conclusions are those of the authors and are independent from the funding and data sources. No endorsement by ICES, AMOSO, SSMD, LHRI, CIHR, CIHI, or the MOHLTC is intended or should be inferred.

Danielle Nash’s training was supported by a Canadian Institutes of Health Research Doctoral Scholarship. Dr. Garg was supported by the Dr. Adam Linton Chair in Kidney Health Analytics and a Clinician Investigator Award from the Canadian Institutes of Health Research.

Dr. Markle-Reid is supported by the Canada Research Chairs Program.

Footnotes

Disclosures

None.

References

  • 1.Xie X, Liu Y, Perkovic V, Li X, Ninomiya T, Hou W, Zhao N, Liu L, Lv J, Zhang H and Wang H. Renin-Angiotensin System Inhibitors and Kidney and Cardiovascular Outcomes in Patients With CKD: A Bayesian Network Meta-analysis of Randomized Clinical Trials. Am J Kidney Dis. 2016;67:728–41. [DOI] [PubMed] [Google Scholar]
  • 2.Raebel MA. Hyperkalemia associated with use of angiotensin-converting enzyme inhibitors and angiotensin receptor blockers. Cardiovasc Ther. 2012;30:e156–66. [DOI] [PubMed] [Google Scholar]
  • 3.Chang AR, Sang Y, Leddy J, Yahya T, Kirchner HL, Inker LA, Matsushita K, Ballew SH, Coresh J and Grams ME. Antihypertensive Medications and the Prevalence of Hyperkalemia in a Large Health System. Hypertension. 2016;67:1181–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Elgendy IY, Huo T, Chik V, Pepine CJ and Bavry AA. Efficacy and safety of angiotensin receptor blockers in older patients: a meta-analysis of randomized trials. Am J Hypertens. 2015;28:576–85. [DOI] [PubMed] [Google Scholar]
  • 5.Yacoub R, Patel N, Lohr JW, Rajagopalan S, Nader N and Arora P. Acute kidney injury and death associated with renin angiotensin system blockade in cardiothoracic surgery: a meta-analysis of observational studies. Am J Kidney Dis. 2013;62:1077–86. [DOI] [PubMed] [Google Scholar]
  • 6.Krogager ML, Torp-Pedersen C, Mortensen RN, Kober L, Gislason G, Sogaard P and Aasbjerg K. Short-term mortality risk of serum potassium levels in hypertension: a retrospective analysis of nationwide registry data. Eur Heart J. 2017;38:104–112. [DOI] [PubMed] [Google Scholar]
  • 7.Schmidt M, Mansfield KE, Bhaskaran K, Nitsch D, Sorensen HT, Smeeth L and Tomlinson LA. Serum creatinine elevation after renin-angiotensin system blockade and long term cardiorenal risks: cohort study. Bmj. 2017;356:j791. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Luo J, Brunelli SM, Jensen DE and Yang A. Association between Serum Potassium and Outcomes in Patients with Reduced Kidney Function. Clin J Am Soc Nephrol. 2016;11:90–100. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Kidney Disease: Improving Global Outcomes (KDIGO) CKD Work Group. KDIGO 2012 Clinical Practice Guideline for the Evaluation and Management of Chronic Kidney Disease. Kidney Int Suppl. 2013;3:1–150. [Google Scholar]
  • 10.National Clinical Guideline Centre (UK). Chronic Kidney Disease (Partial Update): Early Identification and Management of Chronic Kidney Disease in Adults in Primary and Secondary Care. London: National Institute for Health and Care Excellence (UK); 2014. July (NICE Clinical Guidelines, No. 182.) Available from: https://www.ncbi.nlm.nih.gov/books/NBK248058/. [PubMed] [Google Scholar]
  • 11.Kidney Disease Outcomes Quality I. K/DOQI clinical practice guidelines on hypertension and antihypertensive agents in chronic kidney disease. Am J Kidney Dis. 2004;43:S1–290. [PubMed] [Google Scholar]
  • 12.van Blijderveen JC, Straus SM, de Ridder MA, Stricker BH, Sturkenboom MC and Verhamme KM. Adherence to renal function monitoring guidelines in patients starting antihypertensive therapy with diuretics and RAAS inhibitors: a retrospective cohort study. Drug Saf. 2014;37:369–77. [DOI] [PubMed] [Google Scholar]
  • 13.Raebel MA, McClure DL, Simon SR, Chan KA, Feldstein A, Andrade SE, Lafata JE, Roblin D, Davis RL, Gunter MJ and Platt R. Laboratory monitoring of potassium and creatinine in ambulatory patients receiving angiotensin converting enzyme inhibitors and angiotensin receptor blockers. Pharmacoepidemiol Drug Saf. 2007;16:55–64. [DOI] [PubMed] [Google Scholar]
  • 14.Schmidt M, Mansfield KE, Bhaskaran K, Nitsch D, Sorensen HT, Smeeth L and Tomlinson LA. Adherence to guidelines for creatinine and potassium monitoring and discontinuation following renin-angiotensin system blockade: a UK general practice-based cohort study. BMJ Open. 2017;7:e012818. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Raebel MA, Ross C, Xu S, Roblin DW, Cheetham C, Blanchette CM, Saylor G and Smith DH. Diabetes and drug-associated hyperkalemia: effect of potassium monitoring. J Gen Intern Med. 2010;25:326–33. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.McDowell SE, Coleman JJ, Evans SJ, Gill PS and Ferner RE. Laboratory monitoring and adverse patient outcomes with antihypertensive therapy in primary care. Pharmacoepidemiol Drug Saf. 2010;19:482–9. [DOI] [PubMed] [Google Scholar]
  • 17.Gordon NP. Characteristics of Adult Health Plan Members in the Northern California Region Membership, as Estimated from the 2011 Member Health Survey. 2013; Available from: https://divisionofresearch.kaiserpermanente.org/projects/memberhealthsurvey/SiteCollectionDocuments/mhs11reg.pdf.(Accessed 1 May 2017).
  • 18.Ontario Ministry of Finance. 2016 Census Highlights: Factsheet 3. 2017. https://www.fin.gov.on.ca/en/economy/demographics/census/cenhi16-3.html. (Accessed 17 Jun 2018).
  • 19.Ontario Ministry of Health & Long-term Care. Ontario Public Drug Programs. 2016. http://www.health.gov.on.ca/en/public/programs/drugs/. (Accessed 8 Jul 2016).
  • 20.Levey AS and Stevens LA. Estimating GFR using the CKD Epidemiology Collaboration (CKD-EPI) creatinine equation: more accurate GFR estimates, lower CKD prevalence estimates, and better risk predictions. Am J Kidney Dis. 2010;55:622–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Clase CM, Barzilay J, Gao P, Smyth A, Schmieder RE, Tobe S, Teo KK, Yusuf S and Mann JF. Acute change in glomerular filtration rate with inhibition of the renin-angiotensin system does not predict subsequent renal and cardiovascular outcomes. Kidney Int. 2017;91:683–690. [DOI] [PubMed] [Google Scholar]
  • 22.Park IW, Sheen SS, Yoon D, Lee SH, Shin GT, Kim H and Park RW. Onset time of hyperkalaemia after angiotensin receptor blocker initiation: when should we start serum potassium monitoring? J Clin Pharm Ther. 2014;39:61–8. [DOI] [PubMed] [Google Scholar]
  • 23.Bakris GL and Weir MR. Angiotensin-converting enzyme inhibitor-associated elevations in serum creatinine: is this a cause for concern? Arch Intern Med. 2000;160:685–93. [DOI] [PubMed] [Google Scholar]
  • 24.KDIGO AKI Work Group. KDIGO clinical practice guideline for acute kidney injury. Kidney Int Suppl. 2012;2:1–138. [Google Scholar]
  • 25.Go AS, Magid DJ, Wells B, Sung SH, Cassidy-Bushrow AE, Greenlee RT, Langer RD, Lieu TA, Margolis KL, Masoudi FA, McNeal CJ, Murata GH, Newton KM, Novotny R, Reynolds K, Roblin DW, Smith DH, Vupputuri S, White RE, Olson J, Rumsfeld JS and Gurwitz JH. The Cardiovascular Research Network: a new paradigm for cardiovascular quality and outcomes research. Circulation Cardiovascular quality and outcomes. 2008;1:138–47. [DOI] [PubMed] [Google Scholar]
  • 26.Ross TR, Ng D, Brown JS, Pardee R, Hornbrook MC, Hart G and Steiner JF. The HMO Research Network Virtual Data Warehouse: A Public Data Model to Support Collaboration. EGEMS (Wash DC). 2014;2:1049. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Arellano MG, Petersen GR, Petitti DB and Smith RE. The California Automated Mortality Linkage System (CAMLIS). Am J Public Health. 1984;74:1324–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Levy AR, O’Brien BJ, Sellors C, Grootendorst P and Willison D. Coding accuracy of administrative drug claims in the Ontario Drug Benefit database. Can J Clin Pharmacol. 2003;10:67–71. [PubMed] [Google Scholar]
  • 29.The Johns Hopkins University Bloomberg School of Public Health. Health Services Research & Development Center. The Johns Hopkins ACG© Case-Mix System Version 10.0 Release Notes. 2011.
  • 30.Schneeweiss S, Rassen JA, Glynn RJ, Avorn J, Mogun H and Brookhart MA. High-dimensional propensity score adjustment in studies of treatment effects using health care claims data. Epidemiology. 2009;20:512–22. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Austin PC. An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies. Multivariate Behav Res. 2011;46:399–424. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Austin PC. Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Stat Med. 2009;28:3083–107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Austin PC and Tu JV. Bootstrap Methods for Developing Predictive Models. The American Statistician. 2004;58:131–137. [Google Scholar]
  • 34.Bandak G, Sang Y, Gasparini A, Chang AR, Ballew SH, Evans M, Arnlov J, Lund LH, Inker LA, Coresh J, Carrero JJ and Grams ME. Hyperkalemia After Initiating Renin-Angiotensin System Blockade: The Stockholm Creatinine Measurements (SCREAM) Project. J Am Heart Assoc. 2017;6. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Material
NIHMS1610570-supplement-Supplemental_Material.pdf (311.6KB, pdf)
Supplemental Material - RECORD
NIHMS1610570-supplement-Supplemental_Material_-_RECORD.pdf (46.1KB, pdf)

RESOURCES