Measuring Up

As physicians, we are very used to being graded. Whether it was penmanship and behavior in elementary school, Medical College Admission Tests to get into medical school, or board examinations, our performance has been quantified and ranked since childhood. Although our knowledge is still subject to scrutiny through maintenance of certification, with the increasing conversion from volume-based to value-based payment models, we are increasingly being graded on the quality of our care.

Nephrology has been at the vanguard of this trend. Starting in the mid-1990s with the Core Indicators Project and since 2012 under the ESRD Quality Incentive Program, which was authorized under the Medicare Improvements for Patients and Providers Act of 2008 as the first Medicare pay-for-performance program, metrics of dialysis quality have been under increasing scrutiny. Although these programs focused on dialysis care at the level of the dialysis facility, physician performance is now subject to similar quality-based payments under the Patient Protection and Affordable Care Act of 2010 and the Medicare Access and CHIP Reauthorization Act of 2015, which established the Merit-Based Incentive Payment System (MIPS) and other alternative payment models. The use of clinical performance measures to assess quality of care in nephrology is certain to expand under the Advancing American Kidney Health (AAKH) initiative.

This switch from volume-based to value-based payment begs the questions: what is quality medical care, and how should it be assessed? To paraphrase Justice Potter Stewart, we know it when we see it. But this subjective standard is not sufficient for grading the quality of our care, especially if our grades are to be publicly reported and payment indexed to our grades. The Institute of Medicine defines health care quality as “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.”¹ However, operationalization of this definition for measurement of health care quality is far from straightforward.

In the classic construct, Avedis Donabedian² proposed assessing health care quality using a framework of structural domains, processes of care, and outcomes. Because structural measures generally relate to health systems rather than to individual practitioners, the measures that have been applied to nephrology care can be broadly categorized as process measures, assessing whether parameters (e.g., urine albumin-to-creatinine ratio) are assessed or treatment is implemented (e.g., prescription of an angiotensin-converting enzyme inhibitor or angiotensin receptor blocker in a patient with albuminuria); intermediate outcome measures, assessing change in modifiable causal pathways linked to health outcomes (e.g., achieving a target BP); or definitive outcome measures (e.g., mortality or health-related quality of life).³

In this issue, members of the Quality Committee of the American Society of Nephrology (ASN) report on their assessment of 60 clinical performance measures related to kidney health and nephrology care.⁴ The members of the Quality Committee used five review criteria proposed by the Performance Measurement Committee of the American College of Physicians (ACP) to assess these measures: importance, appropriate care, clinical evidence base, measure specification, and measure feasibility and applicability.⁵ As with assessment of quality itself, these criteria are only semiobjective. Applying these criteria in an iterative two-step Delphi process, 29 (49%) of 60 measures were deemed to have high validity, 23 (38%) were deemed to have medium validity, and 8 (13%) were deemed to have low validity. These ratings should, however, be taken in context. When the ACP evaluated 86 ambulatory general internal medicine measures, only 32 (37%) were rated as valid, 24 (28%) were rated of uncertain validity, and 30 (35%) were rated as not valid.⁵ Several other key observations stand out. Nearly half of the measures identified relate specifically to dialysis management, with a paucity of measures related to early identification of kidney disease and interventions to slow progression. Although these are areas of care that span the spectrum from primary and other specialty care to nephrology, well designed measures related to improving the care of patients with kidney disease will be imperative to meet the goals of the AAKH. Similarly, only one measure addressed advanced CKD management and planning for kidney replacement therapy. With the AAKH goals of increasing utilization of home dialysis and increasing transplantation rates, this is another domain for which additional high-quality measures are needed. Several measures, particularly those focused on interventional management of dialysis accesses, were given low ratings as not relevant to general nephrology care. Although this is certainly true, it should be acknowledged that these measures were developed for use in a Qualified Clinical Data Registry, supplementing measures used in MIPS and specifically addressing the absence of measures relevant to interventional nephrology.⁶

The members of the ASN’s Quality Committee should be lauded for their efforts at reviewing existing quality measures related to kidney health. Their call for refinement of existing measures and development of new high-quality “measures-that-matter” needs to be heeded. However, several caveats need to be borne in mind. Care of patients with kidney disease is rarely a solo enterprise, usually involving a team of providers. Attribution of care to an individual nephrologist is, therefore, often a challenge. The statistical validity of measures, particularly outcome measures, at the level of an individual practitioner may be limited. Measures need to be carefully assessed to determine the number of patients necessary for inclusion in the denominator to be able to differentiate clinically important differences in quality of care from variation due to case mix or random events. Efforts also need to be made to minimize unintended consequences as the result of implementation of specific quality measures.⁷ All aspects of care cannot be assessed; measurement of one facet may divert effort away from other equally important components of care. Further, the development of systems to ensure that specific performance metrics are achieved may dissociateprocesses initially correlated with quality from actual high-quality care. Measures must also be stringently evidence based and must be validated prior to broad implementation. Unfortunately, well intentioned measures can go awry. We should not forget the lesson of a time to antibiotic treatment measure for community-acquired pneumonia that lead to antibiotic overuse and diversion of resources without a demonstrable improvement in patient outcomes.⁸ Finally, there needs to be an increased focus on measures that assess what is important to our patients with greater reliance on patient-reported outcomes.

The ASN’s Quality Committee concludes their analysis by proposing five steps for development of future kidney health quality metrics: ensuring that all existing and proposed metrics have high validity; delineating metrics specific to nephrology or overlapping with primary care and other specialty consultants; expansion of metrics to key areas with current gaps, such as prevention, slowing of progression, and optimal transition to kidney failure treatment; incorporation of patient-reported outcomes; and inclusion of other important clinical outcomes.⁴ These recommendations should be combined with those of the ACP, advocating a time out to assess and revise the current approach to physician performance measurement.⁵ We must move to a new generation of measures that are not limited to easy-to-obtain administrative data but rather, that are fully integrated into care delivery, reliably assess the care that we provide, and help close the performance gaps that are most important to improving the kidney health of our patients.

Disclosures

Dr. Palevsky is the Deputy Editor of JASN; Co-Chair of the Kidney Care Quality Alliance Steering Committee; a member of the Renal Physicians Association Quality, Safety and Accountability Committee (Chair, 2008–2013); and a former Co-Chair of the American Medical Association–Physician Consortium for Performance Improvement Kidney Disease Work Group.