Measuring Quality in Kidney Care: An Evaluation of Existing Quality Metrics and Approach to Facilitating Improvements in Care Delivery

Significance Statement

Leveraging quality metrics can be a powerful approach to improve patient outcomes. However, the validity of existing kidney-related quality metrics is unknown. To identify whether existing measures can effectively address and guide quality improvement in care of patients with kidney disease, the American Society of Nephrology’s Quality Committee performed a systematic compilation and evaluation of national kidney metrics. They identified 60 metrics, rating only 29 as highly valid and the other 31 metrics as of medium to low validity, on the basis of defined criteria. Almost half of the measures were related to dialysis management, compared with only one metric related to kidney replacement planning and two related to patient-reported outcomes. The authors urge refinement of existing quality metrics and development of new measures that better reflect kidney care delivery.

Abstract

Background

Leveraging quality metrics can be a powerful approach to identify substantial performance gaps in kidney disease care that affect patient outcomes. However, metrics must be meaningful, evidence-based, attributable, and feasible to improve care delivery. As members of the American Society of Nephrology Quality Committee, we evaluated existing kidney quality metrics and provide a framework for quality measurement to guide clinicians and policy makers.

Methods

We compiled a comprehensive list of national kidney quality metrics from multiple established kidney and quality organizations. To assess the measures’ validity, we conducted two rounds of structured metric evaluation, on the basis of the American College of Physicians criteria: importance, appropriate care, clinical evidence base, clarity of measure specifications, and feasibility and applicability.

Results

We included 60 quality metrics, including seven for CKD prevention, two for slowing CKD progression, two for CKD management, one for advanced CKD and kidney replacement planning, 28 for dialysis management, 18 for broad measures, and two patient-reported outcome measures. We determined that on the basis of defined criteria, 29 (49%) of the metrics have high validity, 23 (38%) have medium validity, and eight (13%) have low validity.

Conclusions

We rated less than half of kidney disease quality metrics as highly valid; the others fell short because of unclear attribution, inadequate definitions and risk adjustment, or discordance with recent evidence. Nearly half of the metrics were related to dialysis management, compared with only one metric related to kidney replacement planning and two related to patient-reported outcomes. We advocate refining existing measures and developing new metrics that better reflect the spectrum of kidney care delivery.

Over the past two decades, there has been increased national emphasis on encouraging patient-centered, high-quality, and high-value health care delivery.¹ In kidney care, the Medicare Improvements for Patients and Providers Act of 2008 mandated the first pay-for-performance system in Medicare, which was reinforced in subsequent laws, including the 2010 Affordable Care Act and the 2015 Medicare Access and CHIP Reauthorization Act. Traditionally, health care quality has been monitored by self-regulated credentialing, with oversight by credentialing organizations and, in some instances, market forces.

Health policy experts have argued for a shift from self-regulation, which is marked by variation in care and financial incentives to increase the quantity of care, to accountability and integration, marked by incentives to provide better quality rather than more care. To enable this transition to pay-for-performance, assessment and quantification of measures of quality are critical.² Yet, how to best define and measure quality remains debated and is challenged by a proliferation of quality measures that vary in relevance, validity, and alignment with meaningful patient outcomes.^2,3

CKD represents a significant public health problem, affecting 14% of adults in the United States, with some sociodemographic groups suffering from particularly high disease burden.^4,5 CKD and its associated comorbid conditions are associated with high mortality and frequent complications.^4,5 Kidney failure is associated with higher mortality than most advanced cancers, and incurs >$35 billion dollars in annual Medicare expenditures.⁴ Suboptimal outcomes for individuals with CKD result, in part, from gaps in care.^4,6 Almost 50% of patients who receive dialysis are not seen by a nephrologist before initiation of kidney replacement therapy, and one third receive little or no education regarding dialysis modality choice.^5,7 In the United States in 2016, only 10% of incident kidney failure patients initiated kidney replacement therapy with home dialysis and <3% with a preemptive kidney transplant.⁴ Among incident patients on hemodialysis, only 20% initiated dialysis with an arteriovenous fistula (AVF) or arteriovenous graft, compared with 80% starting with a central venous catheter.⁴ These data demonstrate substantial performance gaps in CKD care. Leveraging quality metrics to improve care delivery for all individuals with kidney disease could be a potentially powerful approach to affect patient outcomes. However, metrics must be meaningful, clearly defined, evidence-based, appropriately attributed and risk-adjusted, and feasible to be utilized by nephrologists to truly improve the care of patients with kidney disease.

The ESRD Quality Incentive Program (QIP), launched in 2012 by the Centers for Medicare and Medicaid Services (CMS), evaluates dialysis facility performance, expanding over the past decade from three metrics to 13 metrics in 2020.⁸ The ESRD QIP highlights many of the strengths and limitations of quality metrics: benefits related to improved performance on clinical outcome metrics, such as vascular access type where a considerable performance gap existed; and risks related to “cherry-picking” and less ability to individualize care. Risks can be amplified when evaluating relatively small populations, as one or two patients can have a dramatic effect on overall dialysis facility performance.

In contrast with the relative homogeneity in structural aspects of dialysis care delivery that facilitates development of facility-level metrics, nephrologists have diverse practices, ranging from office-based clinics for CKD and hypertension, to dialysis clinics, to hospital consultation for AKI, acid-base disorders, and electrolyte derangements, to the care of kidney transplant candidates and recipients. This combination of episodic and longitudinal care makes development of payment models challenging, particularly when determining attribution.

This article, mirroring methods from the American College of Physicians (ACP) for evaluating measures that may be relevant to the Merit-based Incentive Payment System/Quality Payment Program, reports on the independent evaluation of existing quality metrics by the members of the American Society of Nephrology (ASN) Quality Committee. The aim is to identify whether existing measures for physicians can effectively address and guide quality improvement in the care of patients with kidney disease. Notably, many metrics are shaped by CMS, either directly, with CMS acting as a measure steward, or indirectly, with CMS selecting existing measures for inclusion in quality programs and then adapting them for use in these programs.

The ACP recently called for a “time out” with respect to developing and utilizing existing national health care quality metrics.⁹ The Performance Measurement Committee of the ACP found only 37% of national value-based purchasing measures to be valid, whereas 28% were of uncertain validity. Other authors have cautioned about the unintended consequences of implementing quality metrics, namely overtesting, overmedication, inappropriate classification of patients for denominator capture, and distraction from patients’ needs.¹⁰ We apply ACP criteria to evaluate the validity of national kidney disease quality metrics, and assess the scope and unintended consequences of the measures. Although many of the metrics are not in the primary domain of the nephrologist, we provide a framework of how to approach quality measurement in kidney disease moving forward, to guide policy makers and clinicians about how to design, implement, and use measures to guide practice to improve kidney patient outcomes.

Methods

Evaluation of Existing Kidney Disease Quality Metrics

We compiled a comprehensive list of quality measures related to kidney disease from multiple established kidney and quality metric organizations: Renal Physician Association (RPA),¹¹ National Quality Forum (NQF),¹² Healthcare Effectiveness Data and Information Set (HEDIS),¹³ Merit-based Incentive Payment System,¹⁴ and CMS QIP.¹⁵ Quality metrics were included if they met the following criteria: (1) had defined numerator, denominator, and exclusion criteria; (2) were physician-directed measures relevant to the care of kidney disease patients or were classified as a kidney disease metric by the organization; and (3) were published or endorsed by an organization recognized nationally for quality metrics, as opposed to single health system or organization-specific metrics. We did not include clinical practice guidelines without specified numerator and denominator parameters. We organized these metrics on the basis of applicability across the spectrum of kidney disease care delivery: CKD prevention, slowing CKD progression, CKD management, advanced CKD and kidney replacement planning, and dialysis management. We also classified broad measures as applying across the spectrum of kidney disease care and metrics that were patient reported outcome measures (PROMs).

Ratings were completed similar to the approach outlined by the ACP Performance Measurement Committee, utilizing the RAND Corporation/University of California, Los Angeles (UCLA) method of evaluating a medical intervention. As a part of their effort to review existing performance measures, the Performance Measurement Committee of the ACP developed and applied five criteria to evaluate measures included in the Quality Payment Program (Supplemental Table 1):

1. Importance: The metric will lead to measurable and meaningful improvement in clinical outcome or there is an opportunity for improvement.

2. Appropriate care: The metric will stem overuse or underuse of a test or treatment.

3. Clinical evidence base: The metric is on the basis of high-quality and high-quantity evidence, and has consistent data representing clinical knowledge.

4. Measure specifications: The metric has clarity (a clearly defined numerator and denominator), validity, reliability, and appropriate risk adjustment.

5. Feasibility and applicability: The metric is under the influence of the individual or entity being assessed, attribution level is appropriate, data collection is feasible and burden acceptable, and results will help improve care.

ACP criteria as well as consideration of unintended consequences were applied to all measures to evaluate validity.⁹ Validity is defined by this methodology as “the measure is correctly assessing what it is designed to measure, adequately distinguishing good and poor quality.”^9,16 Two rounds of metric evaluation were conducted by 11 members of the ASN Quality Committee (authors M.L.M., S.L.T., K.L.L., K.E., S.Q.L., F.L., E.G., M.S., P.S.G., T.O.N., and S.D.B.). In the first round, members evaluated the measures independently in spring of 2019; the second round of ratings was conducted during an in-person meeting in July 2019, using a formal group process, with a senior committee member (D.E.W.) serving as moderator. After a group discussion of each measure, members provided an individual score from 1 to 9 for each ACP domain, and a separate high/medium/low categorical rating. The overall metric ratings were unchanged from round one to two, and there was strong correlation between domain criteria ratings and overall ratings (see Supplemental Figure 1). Intraclass correlation coefficients (ICCs) showed a moderate correlation among overall ratings (ICC=0.68) and ACP domain ratings (ICC range: 0.59–0.82) (see Supplemental Figure 2 and Supplemental Table 2). Individual comments were collected from the first round of ratings, and group comments were collected from the second round. After the individual metrics were rated for validity, a subset of the committee (M.L.M., S.L.T., D.E.W., and S.D.B.) organized them into subcategories and globally assessed the scope and attribution of the metrics. Please see Supplemental Appendix 1 for additional methods.

Results

Figure 1, Table 1, and Supplemental Tables 3a–g summarizes the 60 metrics included in the following categories: CKD prevention (seven metrics), slowing CKD progression (two metrics), CKD management (two metrics), advanced CKD and kidney replacement planning (one metric), dialysis management (28 metrics), broad measures (18 metrics), and PROMs (two metrics). With respect to validity, 49% of metrics were determined to have high validity, 38% were determined to have medium validity, and 13% were determined to have low validity, on the basis of defined criteria. Three common themes affecting validity emerged. First, there was unclear attribution of many metrics to nephrology (i.e., successful AVF thrombectomies, a procedure not typically performed by noninterventional nephrologists) or unclear delineation from primary care (i.e., comprehensive management of diabetes). Eighteen metrics were determined to not be attributable to nephrologists, of which ten were related to primary care and eight were applicable to other specialties. The second theme was the need for improved metric definition, particularly related to exclusion criteria and risk adjustment. The third theme was nonevidence-based metrics, not reflective of the latest evidence or guidelines. Metrics are discussed in further detail below.

Figure 1.

Measure validity ratings for national kidney disease quality metrics. This figure reports a summary of national kidney disease quality metrics, including the number of measures per category, measure subcategories, and overall measure validity ratings utilizing American College of Physicians-specified criteria. 49% of metrics were determined to have high validity, 38% had medium validity, and 13% had low validity.

Table 1.

Evaluation of existing kidney quality measures

Measure Category (No. of Measures)	Measure Title	RPA/NQF/MIPS/QIP	ACP 1: Importance	ACP 2: Appropriateness	ACP 3: Clinical evidence	ACP 4: Specifications	ACP 5: Feasibility	Overall Measure Validity Rating	Rationale
CKD prevention (7)
Hypertension	Controlling High BP	NQF 0018	+	+	+	+	+	HIGH	BP goals potentially lower <130/80 mm Hg on the basis of the ACC/AHA guideline.
	Preventive Care and Screening: Screening for High BP and Follow-Up Documented	MIPS 317	+	+	+	+	+	HIGH	Well accepted part of high-quality patient care, but may be already universally practiced.
									Not clear what constitutes a recommended follow-up plan, so may add documentation burden without meaningfully improving clinical care.
	HEDIS Controlling High BP		+	±	—	±	+	MED	Masked HTN and white coat HTN are not accounted for. Nonrigorous office BP measurement may not be valid or reliable.
									Does not incorporate home BP or ABPM. Sitting versus standing and measurement technique is important to specify.
Diabetes	Diabetes: Hemoglobin A1c Poor Control	NQF 0059 MIPS 001	+	+	+	±	—	MED	Important for kidney outcomes, but typically the primary responsibility of PCP or endocrinologist.
									Consider adjustment for patient factors.
	Diabetes: LDL-C Control (<100 mg/dl)	NQF 0064	+	±	—	+	—	MED	Newer evidence and guidelines recommend that management should be independent of cholesterol levels.
									Patients on dialysis should be excluded from denominator.
	HEDIS-Comprehensive Diabetes Carea	NQF 0731	+	+	+	±	—	MED	Should be dominant responsibility of PCP or endocrinologist.
									BP goals uncertain after recent ACC/AHA guideline.
									Consider exclusion for dialysis patients, as HbA1c not reliable and BP goal is unclear.
	Diabetes: Medical Attention for Nephropathya	NQF 0062 MIPS 119	+	+	+	±	±	MED	American Diabetes Association guidelines recommend both eGFR and UACR testing.
									ACE-I/ARB use should not count as screening for nephropathy.
Slowing CKD progression (2)
Hypertension	Adult Kidney Disease: BP Management	MIPS 122	+	+	+	+	+	HIGH	Very important metric as a primary factor affecting CVD events and mortality.
									Goal of <140/90 mm Hg not reflective of more recent ACC/AHA guidelines, recommending <130/80 mm Hg.
Hypertension/CKD	Angiotensin Converting Enzyme (ACE) Inhibitor or Angiotensin Receptor Blocker (ARB) Therapy	AKID-2	+	+	+	±	±	HIGH	Strong evidence. Does not specify quantity of proteinuria. May cause hyperkalemia and/or creatinine elevation; requires monitoring.
CKD management (2)
Advance care planning	Advance Directives Completed	RPAQIR18	+	+	+	+	+	HIGH	Important part of high-quality care.
									Unclear need in young patients with mild CKD. Would favor attributing to nephrologist only for CKD stages 4 and 5.
Lipid testing	Adult Kidney Disease: Laboratory Testing (Lipid Profile)	NQF 1668	±	±	±	±	±	MED	Checking lipid panels may not affect management if statins are indicated irrespective of LDL level (2013 KDIGO Lipid Management Guideline recommends statin use in all persons with CKD age ≥50 years). Testing not necessary every 12 mo.
Advanced CKD and kidney replacement planning (1)
Dialysis access	Optimal ESKD Starts	NQF 2594	+	+	+	+	+	HIGH	Current catheter rate at HD start is extremely high, increasing the risk of bloodstream infections. This metric is all-encompassing toward improving quality of initiation of dialysis care.
									Needs appropriate risk adjustment.
									Very applicable to new payment models.
Dialysis management (28)
Dialysis access	Adult Kidney Disease: Catheter Use for ≥90 D	MIPS 330	+	+	+	+	+	HIGH	Important as catheters cause bloodstream infections and are associated with increased mortality, and currently large performance gap.
									Denominator appropriately excludes for patients whom a long-term vascular access is not appropriate (e.g., elderly, imminent transplantation) and patients that decline AVG/AVF.
	Vascular Access Type (VAT) Measure Topic– Catheter >90 D Clinical Measure	NQF 0256 QIP	+	+	+	+	+	HIGH	Denominator does not include exclusions for patients whom a long-term fistula or graft is not appropriate.
									Metric encourages procedures which may not be in line with patient goals.
	Vascular Access—Functional Arteriovenous Fistula (AVF) or AV Graft or Evaluation for Placement	NQF 0251	+	+	+	+	+	HIGH	Most detailed vascular access metric that accounts for different scenarios.
									Annual assessment may not be necessary for long-term catheter use.
	Adult Kidney Disease: Catheter Use at Initiation of Hemodialysis	MIPS 329	+	+	+	+	+	HIGH	Attributable to providers delivering care before dialysis initiation, if seen by a nephrologist.
									Needs appropriate risk adjustment.
	Arteriovenous Fistula Rate	AKID-8	+	+	+	+	+	MED	Important metric and addresses relevant performance gap.
									Needs adequate adjustment for patient factors.
									Should support care individualization and shared decision-making.
	Vascular Access Type (VAT) Measure Topic– Arteriovenous Fistula (AVF) Clinical Measure	NQF 0257 QIP	+	+	+	±	+	MED	Supported by evidence and there is a performance gap, but it may not be the best option for all patients.
									Needs to adequately account for patient-related factors and access to vascular surgery.
									Should support role for patient individualization and shared decision-making.
	Peritoneal Dialysis Catheter Exit Site Infection Rate	RPAQIR17	+	+	+	+	±	HIGH	Measure of PD catheter complication is important for high-quality care, but is not attributable to the nephrologist.
	Peritoneal Dialysis Catheter Success Rateb	RPAQIR16	+	+	±	±	—	LOW	Not appropriately attributed to nephrologist. Definition of successful is not clear, so reporting may be subjective.
	Arterial Complication Rate Following Arteriovenous Access Interventionb	RPAQIR12	+	+	±	—	—	LOW	Unclear how complications will be identified and defined. Not appropriately attributed to nephrologist, unless interventional nephrology. Interventionalists may decrease intervention rate or decrease reporting of complications.
	Arteriovenous Fistulae Thrombectomy Success Rateb	RPAQIR15	+	±	±	±	—	LOW	Not appropriately attributed to nephrologist, unless interventional nephrology. Thrombectomy success may be related to underlying patient factors rather than quality of the interventionalist. May result in more abandoned fistulas.
	Arteriovenous Graft Thrombectomy Success Rateb	RPAQIR14	+	±	±	±	—	LOW	Not appropriately attributed to nephrologist, unless interventional nephrology. Could result in faster referrals of clotted grafts but may result in more abandoned grafts.
Adequacy	Adult Kidney Disease: Hemodialysis Adequacy: Solute	NQF 0323	+	+	+	+	+	HIGH	Certain select situations when care individualized for a given patient may result in Kt/V<1.2. No inclusion of residual kidney function; for example, the consideration of incremental dialysis. Measure likely topped out and so less meaningful.
	Kt/V Dialysis Adequacy Comprehensive Clinical Measure	QIP	+	+	+	+	+	HIGH	More comprehensive measure than other adequacy measures. Certain select situations when care individualized for a given patient may result in Kt/V lower than these thresholds. May disadvantage PD facilities.
	Adult Kidney Disease: Peritoneal Dialysis Adequacy: Solute	NQF 0321	+	+	±	+	+	HIGH	Generally accepted as standard of practice but no clinical trial evidence base.
									Targeting a weekly Kt/V of 1.7 without room for individualization may lead to conversion to hemodialysis.
	Adequacy of Volume Management	AKID-4	+	+	—	—	±	MED	No standardized definition of volume assessment. Lack of strong supporting evidence. Likely already assessed in patients on dialysis.
	Pediatric Kidney Disease: Adequacy of Volume Management	MIPS 327	+	+	—	—	+	MED	Given a lack of strong supporting evidence, may pose documentation and monitoring burden without substantial patient benefit.
Anemia	ESKD Patients Receiving Dialysis: Hemoglobin Level <9 g/dl	AKID-6	+	+	+	+	+	HIGH	Important QOL measure.
									May penalize dialysis facilities with patients who are frequently admitted and miss ESA dosing.
	Anemia Management Reporting Measure	QIP	+	+	+	+	+	HIGH	Important information to track, but not clear that this information will impact or improve patient care.
									Does not account for iron administration.
	Standardized Transfusion Ratio (STrR) Clinical Measure	QIP	+	+	+	±	±	HIGH	Transfusion avoidance important. Should monitor for anemia as a balance measure. Conversely there may be incentive to overtreat with ESAs when not clinically indicated.
	Pediatric Kidney Disease: ESKD Patients Receiving Dialysis Hemoglobin Level <10 g/dl	NQF 1667 MIPS 328	+	+	+	+	+	HIGH	Important for quality of life, and good evidence for this hemoglobin threshold in pediatrics. May not be met by patients who are frequently admitted to hospital.
ESKD-related complications	Mineral Metabolism Reporting Measure	NQF 0255 QIP	+	+	+	+	+	HIGH	Routinely assessed by dialysis facilities, so likely topped out.
	Standardized Readmission Ratio (SRR) Clinical Measure	NQF 2496 QIP	+	+	+	+	+	HIGH	Accepted quality metric and incentivizes to improve care coordination. Readmission high in ESKD population so addresses a large performance gap.
									Readmissions to the hospital in patients with ESKD are often unrelated to the index admission and physicians or facilities may have limited control. Requires adequate risk adjustment for social factors.
	Avoidance of Utilization of High Ultrafiltration Rate (≥13 ml/kg per hour)	NQF 2701	+	+	+	+	+	HIGH	Good observational data evidence base for metric. Difficult to implement in practice because it is easier to increase the ultrafiltration rate rather than extend treatment time (logistically and patient preference).
	Infection Monitoring: National Healthcare Safety Network (NHSN) Bloodstream Infection in Hemodialysis Patients Clinical Measure	NQF 1460 QIP	+	+	+	±	±	MED	Bloodstream infection rate important part of hemodialysis care and modifiable on the basis of dialysis unit practices.
									May encourage underchecking of blood cultures. Self-reported metric markedly limits effectiveness.
									Difficult to differentiate dialysis versus nondialysis-related infections.
Transplant referral, care coordination, advance care planning	Transplant Referral	AKID-13	+	+	+	±	+	HIGH	Important measure for high-quality ESKD care. Some risk of increasing inappropriate referrals. May disadvantage units serving populations with more medical and social risk factors. Requires risk adjustment.
	Adult Kidney Disease: Referral to Hospice	MIPS 403	+	+	±	±	+	MED	Hospice care may not be appropriate for every individual at the end of life. Low number of patients for this measure.
	Rate of Timely Documentation Transmission to Dialysis Unit/Referring Physician	RPAQIR13	+	+	±	+	—	MED	Responsibility of documentation transmission should not be solely on the physician but rather on health care system, EMR vendor, and informatics. Difficult to measure and report.
									May send incomplete documentation to meet 2-d time frame.
	Advance Care Planning (Pediatric Kidney Disease)	PKID-4	+	+	+	+	+	HIGH	Important for patients with organ failure. Documentation may not reflect meaningful discussion.
Broad measures (18)
Preventive Care	Pneumonia Vaccination Status for Older Adults	NQF 0043 MIPS 111	+	+	+	+	+	HIGH	Evidence-based. Shared responsibility with PCP.
									Pneumonia vaccination recommendation was recently updated, so should reflect current recommendations.
	Preventive Care and Screening: Influenza Immunization	NQF 0041 MIPS 110	+	+	+	+	+	HIGH	Important measure with good evidence base. Shared responsibility with PCP.
	Preventive Care and Screening: Tobacco Use: Screening and Cessation Interventiona	NQF 0028	+	+	+	+	+	HIGH	Smoking is a critically important CKD and CVD risk factor.
									Nephrologist should co-own measure along with PCP.
									Unclear that meeting measure is the result of a meaningful patient engagement or just “box checking.”
	One-Time Screening for Hepatitis C Virus (HCV) for Patients at Risk	MIPS 400	+	+	±	+	+	MED	Important measure given antiviral treatments for HCV. Shared responsibility with PCP.
	Diabetes Mellitus: Diabetic Foot and Ankle Care, Peripheral Neuropathy– Neurologic Evaluationa	NQF 0417	+	+	+	±	—	MED	High correlation with falls; easy to perform and document. Should be dominant responsibility of PCP or endocrinologist.
	Diabetes Mellitus: Diabetic Foot and Ankle Care, Ulcer Prevention– Evaluation of Footweara	NQF 0416	+	+	±	±	—	MED	May not be necessary in all patients in the absence of neuropathy. Not appropriate to attribute this measure to providers not trained in the evaluation of footwear.
	Preventive Care and Screening: Body Mass Index (BMI)a	NQF 0421	+	+	±	±	—	MED	Obesity is increasingly recognized as risk factor for CKD, and is also a modifiable risk factor for diabetes.
									May not apply to patients on dialysis.
Medication reconciliation and safety	Medication Reconciliation Post-Discharge	NQF 0097 MIPS 046	+	+	+	+	+	HIGH	Important for high-quality care.
									Difficult to verify accuracy.
	Documentation of Current Medications in the Medical Record	NQF 0419 MIPS 130	+	+	±	+	±	HIGH	Physicians documenting medication list does not necessarily translate into accuracy of medication lists or patients taking medications.
									Challenging because of noninteroperable EHR and pharmacy systems.
	Use of High-Risk Medications in the Elderlya	NQF 0022	±	±	±	±	—	LOW	Difficult for nephrologist to ascertain when and who ordered the medication. Medication list may have important omissions and inappropriate additions.
Advance care planning	Advance Care Plan	NQF 0326 MIPS 047	+	+	+	+	+	HIGH	Reasonable for patients age 65 yr and older, but may not be relevant to care in low acuity patients.
									Should be dominant responsibility of PCP in earlier stage kidney disease.
Falls	Falls: Plan of Carea	NQF 0101	+	+	+	+	—	MED	Important, but not attributable to nephrologist (PCP responsible).
									May be documented by medical assistants or other clinic staff leading to documentation burden without improvement in care.
	Falls: Risk Assessmenta	NQF 0101	+	+	±	+	—	MED	Falls are associated with mortality and other complications.
									Nephrologists lack knowledge to do falls assessments.
	Falls: Screening for Future Fall Riska	NQF 0101	+	+	±	+	—	MED	Not within nephrology practice; should be dominant responsibility of PCP.
Complications/miscellaneous	Prevention of Catheter-Related Bloodstream Infections (CRBSI): Central Venous Catheter (CVC) Insertion Protocolb	NQF 0464	+	+	+	+	—	MED	Not usually relevant to nephrology (except temporary dialysis placement which is often not performed by nephrologist).
	Surgical Site Infection (SSI)b		+	+	+	±	—	LOW	Not appropriately attributed to nephrologist. Surgical centers and hospitals are already required to report SSIs.
	Radiology: Exposure Time Reported for Procedures using Fluoroscopyb	NQF 0510	+	+	±	±	—	LOW	Important to track cumulative ionizing radiation exposure. Not appropriately attributed to nephrologist, unless interventional nephrology.
	Hospitalization Rate Following Procedures Performed under Procedure Sedation Analgesiab	RPAQIR11	±	±	±	±	—	LOW	Not appropriately attributed to nephrologist. Unknown evidence base for established acceptable rate in literature.
PROMs (2)
PROMs	Patient Experience of Care: In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems (ICH CAHPS) Survey Clinical Measure	NQF 0258 QIP	+	±	±	±	±	MED	Important to include patient-reported outcomes. Survey has ≥60 questions and twice-yearly administration, which may lead to survey fatigue. Results may be biased due to low response rate.
	Functional Outcome Assessment	NQF 2624 MIPS 182	+	±	±	±	±	MED	Should be dominant responsibility of PCP. Should target older patients, such as those ≥65 yr, and those who are more vulnerable.

The symbol “+” indicates a median rating of 7–9, “±” indicates a median rating of 4–6, and “—” indicates a median rating of 1–3. Overall rating: HIGH for high, MED for medium, LOW for low. MIPS, Merit-based Incentive Payment System; ACC/AHA, American College of Cardiology/American Heart Association; HTN, hypertension; ABPM, ambulatory blood pressure monitoring; PCP, primary care physician; HbA1C, glycated hemoglobin; UACR, urine albumin-to-creatinine ratio; ACE-I, angiotensin-converting enzyme inhibitor; ARB, angiotensin receptor blocker; CVD, cardiovascular disease; AKID, adult kidney disease; RPAQIR, Renal Physician Association Quality Improvement Registry; AVG, arteriovenous graft; PD, peritoneal dialysis; QOL, quality of life; ESA, erythropoiesis stimulating agent; EMR, electronic medical record; PKID, pediatric kidney disease; HCV, hepatitis C virus; EHR, electronic health record.

^aMetric is PCP-focused.

^bMetric should not be attributable to nephrologists.

CKD Prevention

There were seven metrics in the CKD prevention section: three were related to hypertension and four were related to diabetes. The NQF 0018 controlling high BP metric, defined as patients aged 18–85 years who had a diagnosis of hypertension and whose BP was controlled <140/90 mm Hg during the measurement period, was rated as having high validity; although there was discussion about the need for home BP recordings, lack of applicability to dialysis, and target goal in light of SPRINT findings. The HEDIS controlling high BP metric was rated as having medium validity, primarily because of its recommendation of a BP target of <150/90 mm Hg in patients aged 60–85 years, which does not reflect more recent American College of Cardiology/American Heart Association guidelines. Four metrics were diabetes-centric, involving glycated hemoglobin control, LDL control, comprehensive diabetes care, and the need for “medical attention for nephropathy,” which refers to urine albumin measurement and use of angiotensin-converting enzyme inhibitor or angiotensin receptor blocker therapy. Two of the diabetes metrics were considered to be primary care physician–directed, rather than nephrologist-directed, and were rated as having medium validity. The HEDIS comprehensive diabetes care metric was thought to be the most comprehensive, but delineation of responsibilities was unclear and a specific exclusion for patients with kidney failure is needed, given lack of evidence-based BP targets and potential unreliability of glycated hemoglobin in the population.

Slowing CKD Progression

Only two metrics addressed slowing CKD progression. NQF 1662, which counts adults with a diagnosis of nondialysis CKD and proteinuria who are prescribed angiotensin-converting enzyme inhibitor or angiotensin receptor blocker therapy within a 12-month period, was thought to have high validity, although there were safety concerns regarding hyperkalemia. Similarly, RPA measure 122, which counts adults with CKD stages 3–5 who are not receiving kidney replacement therapy and either have BP <140/90 mm Hg or BP ≥140/90 mm Hg with a documented plan of care, was considered to have high validity, although concerns were raised about nondialysis patients with advanced CKD and appropriate BP targets.

CKD Management

Only two metrics were related to management of a patient with CKD, not specifically to slow progression: (1) an advanced directive completed in adults and (2) lipid profile measurement. The advanced directive measure received a high validity score, highlighting the importance of end-of-life care planning, but concerns were raised about documentation burden and applicability to patients aged 18–65 years without other chronic illnesses. The lipid profile measure received a medium validity score as concerns were raised regarding overtesting in patients already on lipid-lowering therapies and overlap with primary care physician management responsibilities.

Advanced CKD and Kidney Replacement Planning

The NQF 2594 optimal ESKD starts measure was the only metric specifically addressing advanced CKD and kidney replacement planning. The measure captures the percentage of patients with a preemptive kidney transplant, home dialysis initiation, or outpatient in-center hemodialysis via AVF or arteriovenous graft. The measure was rated as having high validity. Committee discussion centered on the importance of having an evidence-based target given potential patient nonadherence, accelerated progression to kidney failure, and lack of timely nephrology referral. Most committee members felt that this metric was a step in the right direction, particularly considering the Advancing American Kidney Health Initiative by Health and Human Services and the Centers for Medicare and Medicaid Innovation kidney care models.¹⁷ These mandatory and optional models emphasize home dialysis, transplantation, and coordinated outpatient dialysis initiation, making the optimal ESKD starts metric particularly relevant.

Dialysis Management

Nearly half of the measures were related to dialysis management, partly reflecting measures incorporated in the ESRD QIP that target dialysis facilities rather than physicians. Metrics focused on AVF and catheter utilization rates were rated high, although the need for robust exclusion criteria were emphasized. Measures of hemodialysis adequacy, defined by solute clearance, and avoidance of high ultrafiltration rates were deemed to have high validity despite being on the basis of limited evidence. Adequacy was cited as a potentially “topped out” measure, and there were concerns raised about the need to individualize care, particularly in peritoneal dialysis, resulting in occasional failure to meet solute clearance targets. Anemia measures with the aim of reducing erythropoiesis stimulating agent overuse were also rated high. The readmission measure was rated as having high validity but attribution concerns were raised; the measure could be improved upon by converting to a rate to benchmark over time, and should better account for patients who are outliers.¹⁸

High-validity measures also included metrics deemed patient-centered, like the transplant referral measure and pediatric advanced care planning, but the committee emphasized the need for robust risk adjustment and exclusion criteria. Metrics rated as having medium validity included erythropoiesis stimulating agent dosage reporting, bloodstream infection rate reporting, and timely documentation from the hospital or provider to dialysis unit, with reporting bias, administrative burden, and attribution raised as potential issues. Measures that were rated as having low validity related to vascular access success or complication rates after interventions. These measures were seen to be attributable to interventional nephrologists or surgeons rather than the broader nephrology field. Specific to the dialysis metrics, topped out status, need for adequate risk adjustment, and overreliance on vascular access measures were highlighted as key concerns.

Broad Measures

Numerous metrics spanned the spectrum of kidney disease care, with variable validity ratings. Measures related to medication reconciliation were rated as high, although documentation burden and workflow challenges were discussed. Similarly, vaccination measures were rated high, although there was recognized overlap with primary care. Low-validity ratings were assigned to hospitalization rate after procedures with sedation, surgical site infections, central venous catheter infection prevention, and radiology exposure time, largely because of issues with unclear relevance to nephrology and unclear attribution to a given specialty.

In addition to the CKD prevention metrics, a number of metrics that spanned the spectrum of CKD were deemed to be primary care physician–directed measures, specifically related to diabetic foot management, body mass index documentation, falls assessment and plan of care, tobacco use screening, and high-risk medication ordering in the elderly. Almost all were rated as having medium validity because of the application to all patients, not those specifically with CKD, and the focus on general primary care. Tobacco use screening was rated as having high validity because of the role of smoking in CKD progression, and high-risk medication ordering as low validity because of the types of medications that are considered to be high risk (and those that are not) in the metric.

PROMs

Only two measures were considered to specifically be a PROM. The In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems (ICH CAHPS) measure was rated as having medium validity because of applicability of the survey to clinician-provided care as opposed to dialysis facilities, given the focus of most questions on facility-related factors. Finally, the functional outcome assessment measure for patients aged ≥18 years was rated as having medium validity because of the inclusion of all adults as opposed to those aged >65 years and the applicability to nephrologists, most of whom are unaware of functional outcome assessment tools.

Global Measure Assessment

Metrics were globally assessed for their scope in measuring quality across the spectrum of kidney disease. The committee noted that almost half of the metrics (46%) were related to dialysis management, whereas only one metric (optimal ESKD starts) related to advanced CKD and kidney replacement therapy planning, and two metrics reflected PROMs. Notably, none of the prevention metrics addressed screening for CKD with GFR biomarkers such as serum creatinine or cystatin C.

Discussion

The ASN Quality Committee’s review of existing kidney care–related metrics determined that less than half of current metrics have high validity. Three frequent concerns were raised, even for measures with high validity: (1) appropriate nephrologist attribution, (2) adequate risk adjustment and denominator exclusions, and (3) nonevidence-based metrics. Lastly, although there are a multitude of dialysis-related metrics, there is a paucity of metrics related to slowing CKD progression, advanced CKD and kidney failure planning, and patient-reported outcomes.

There are significant limitations to current quality measures used to assess kidney care quality. Lack of measure validity may result in potential unintended consequences and increased provider burden, illustrating the need for incorporating the strongest evidence when developing measures. Attribution was a common concern, as many metrics potentially applicable to nephrologists significantly overlap with primary care, contributing to confusion about which provider bears responsibility for the care described in the metric. Notably, we found that many measures related to kidney care should not be attributed to nephrologists, such as CKD prevention measures in the realm of primary care, or vascular access–related measures attributed to interventionalists. Phasing out measures on the basis of the outdated evidence and clinically topped-out measures, such as those for adequacy, is crucial to ensure meaningful measures that improve kidney care.

The scope of existing quality measures highlights opportunities for new measure development and further areas of research. First, there is an abundance of metrics that focus on dialysis care process metrics with varied validity. This is largely driven by the current CMS reimbursement and incentive structure, which focuses on facility-level dialysis care. However, whether these dialysis-centric process measures directly contribute to improvements in patient outcomes remains uncertain.^9,19 Second, there is a paucity of metrics related to CKD prevention and slowing CKD progression, which may, in part, reflect limited evidence-based interventions to prevent or slow the progression of CKD. New therapies such as sodium glucose cotransporter inhibitors may be candidates for new measure development, once appropriate for the indication of slowing CKD progression is established in guidelines and through additional clinical experience. Of note, there is only one measure related to advanced CKD and kidney replacement planning, despite the complex care and high costs in late-stage CKD. Given the upcoming voluntary payment models within the Advancing American Kidney Health Initiative that will include patients with CKD stages 4 and 5, additional measures should be tested and developed for advanced CKD and kidney replacement planning.¹⁷ The absence of metrics reflecting care delivered at various stages in CKD care reflects a significant missed opportunity to drive improvements in these areas. Lastly, a lack of metrics reflecting PROMs as well as limited ability to account for patient choice in many existing measures represent a deficiency in measuring high-quality, patient-centered kidney care. Patient values can conflict with clinical or laboratory metrics of performance, adding a layer of complexity to the assessment of clinician performance. Lack of quality of life measures in the ESRD QIP has been previously noted, leading researchers to propose a patient-focused quality hierarchy pyramid, with quality of life at the apex.^20,21 Existing PROMs, namely the ICH CAHPS Survey, have several limitations noted by the committee, specifically low response rate and survey fatigue. We urge measure developers to incorporate other PROMs related to kidney disease, including PROMs for home dialysis, into new patient-centered quality measures.²²

Other authors have commented on the need to refine existing kidney disease quality measures.^19,23,24 Proposals have included a shared accountability model for patients with ESKD across a defined geographic region¹⁹; incorporation of more patient-centric, goal-directed metrics²³; and greater reliance on defined important outcomes like mortality, hospitalization, and employment.^24,25 The common theme of these proposals is the recognition that quality metrics have the potential to drive care improvements but, in the current state, have not consistently achieved this goal. In Figure 2, we outline a five-step approach to quality metric development and utilization for kidney disease that will support care patient-centric delivery improvements. Critically, this incorporates many tenets of the NQF’s measure endorsement process.

Figure 2.

Framework to refining quality metric development to foster care delivery improvements. Five-step approach outlining ideal steps to refine existing and create new measures that will support patient-centered, high-quality kidney care delivery.

Step 1 involves ensuring that all existing and new quality metrics are valid on the basis of established ACP criteria. Focusing on valid metrics fosters a cohesive discussion with relevant stakeholders regarding the value of establishing measurement in key areas, and avoids overlapping measures of variable validity. Step 2 delineates metrics specific to nephrologists, involving both nephrology and primary care, and those involving other stakeholders. This is an important element of the process of metric definition—specifically, which providers have responsibility for performance. The questions that need to be asked in this step are: (1) whether care should be defined as comanagement by nephrology and primary care, and (2) will care be more specific to one specialty versus another? Step 3 involves expansion of metrics in key areas such as prevention, slowing progression, and planning for kidney failure treatment. These critical aspects of kidney disease care represent established care gaps that often are overlooked. Establishment of meaningful metrics in these areas can facilitate care improvement. For example, a measure that captures rates of patient-centered education and shared decision-making related to conservative management, kidney replacement modality, and pre-emptive transplantation in patients transitioning to kidney failure would be a significant advancement. Step 4 incorporates PROMs, representing a shift to metrics that are patient-centered. Capturing patient symptoms, quality of life, mental health, disability, employment, and values that are most meaningful to patients is necessary in defining quality. Step 5 involves capturing important clinical outcomes: time to kidney failure, utilization, and mortality. These outcomes must be appropriately risk-adjusted, which signifies a clear challenge, but defining quality of care without these paramount outcomes is meaningless.

In conclusion, we reviewed the existing 60 quality metrics related to nephrology care and identified substantial variation in metric validity. Less than half of current metrics were rated with high validity, largely because of unclear attribution, inadequate definitions and risk adjustment, and discordance with the latest evidence. We advocate for refinement of existing measures and development of new, well-defined, well-validated “measures that matter,” characterized as reflecting the spectrum of nephrology practice, capturing clinically relevant outcomes, and shifting focus to tools that will drive meaningful improvements in patient-centered care.

Disclosures

Dr. Bieber is a medical director for Northwest Kidney Centers and serves as faculty for the Home Dialysis University. Dr. Erickson reports personal fees from Acumen LLC, other from American Society of Nephrology, personal fees from Dialysis Clinic, Inc., personal fees from Satellite Healthcare, outside the submitted work. Dr. Lew reports grants from Care First Foundation, personal fees from Reata, outside the submitted work. Dr. O’Neil has a patent null pending and is currently in the very early phases of submitting utility and design patents on a device to reduce the risk of both venous line disconnection and microbial contamination of the bloodline-to-central-hemodialysis-catheter-hub connection points. The effort is entirely self-funded, is not proceeding under the sponsorship or support of any other agency or business entity, and was undertaken more than a year after his retirement to without-compensation emeritus teaching staff status from the James H. Quillen Veterans Affairs Medical Center (VAMC), in Johnson City Tennessee. Although some of the metrics reviewed concerned dialysis safety, the nature and status of the device patents did not in any discernable way alter their assessment of the status of the metrics reviewed as one of the independent review panel. Dr. Mendu reports personal fees from Bayer Pharmaceuticals, outside the submitted work. Dr. Tummalapalli reports personal fees from Bayer Pharmaceuticals, outside the submitted work.

Funding

The authors acknowledge funding for the ASN Quality Committee by the American Society of Nephrology.

Supplemental Material

This article contains the following supplemental material online at http://jasn.asnjournals.org/lookup/suppl/doi:10.1681/ASN.2019090869/-/DCSupplemental.

Supplemental Table 1. ACP Measure Review Criteria.

Supplemental Figure 1. Correlation between ACP domain ratings and overall rating.

Supplemental Figure 2. Box plots showing median, interquartile ranges (IQR), and minimum and maximum values of American College of Physicians (ACP) domain ratings and overall rating across metrics by reviewer (1-11).

Supplemental Table 2. Median and interquartile ranges (IQR) of ACP domain ratings.

Supplemental Appendix 1. Supplemental methods.

Supplemental Table 3a. CKD prevention measures.

Supplemental Table 3b. Slowing CKD progression measures.

Supplemental Table 3c. CKD management measures.

Supplemental Table 3d. Advanced CKD and kidney replacement planning measures.

Supplemental Table 3e. Dialysis management measures.

Supplemental Table 3f. Broad measures.

Supplemental Table 3g. Patient reported outcome measures (PROMs).