Lessons Learned from Using Global Outcome Measures to Assess Community Pharmacy Performance

Abstract

INTRODUCTION:

As value-based and alternative payment models proliferate, there is growing interest in measuring pharmacy performance. However, little research has explored the development and implementation of systems to measure pharmacy performance. Additionally, systems that currently exist rely on process and surrogate outcome measures that are not always relevant to patients and payers.

PROGRAM DESCRIPTION:

This article describes the process used to design and implement a performance measurement program for a group of enhanced services pharmacies in North Carolina. This program was successful in measuring quality based on medication adherence, hospitalizations, emergency department visits, and total cost of medical care for nearly all North Carolina pharmacies. Measures were scored and combined into a single 11-point composite pharmacy performance score. To demonstrate the measures, we compared performance scores for enhanced services pharmacies (n = 119) to other North Carolina pharmacies (n = 1,616) during the baseline measurement period (March 1, 2015-May 31, 2015). Adherence measure scores for enhanced services pharmacies exceeded those of other pharmacies (P values < 0.0001-0.003), but total scores were not significantly different, with enhanced services pharmacy mean total scores of 6.54 vs. 6.29 for all other pharmacies (P = 0.115).

OBSERVATIONS:

The program described provides an example of a composite performance measurement system that can be used to support alternative pharmacy payment models and shows that case-mix adjustment is possible for broad outcomes such as those used in this program. The measures used for the program depend on timely feeds of medical claims. Payers and pharmacy networks implementing a similar program may need to explore alternative structure or process measures.

IMPLICATIONS:

As pharmacy payment models evolve, there may be value in collaboration between academics, pharmacists, and payers to bring different areas of expertise and perspectives into the performance measurement process. This program demonstrates that global outcome measurement is possible over a broad set of pharmacies and invites additional research to explore the validity of this and other methods to measure pharmacy quality and performance.

What is already known about this subject

• Interest in pharmacy performance measurement is growing.

• Most existing pharmacy performance measure systems rely on process and surrogate outcomes and do not include outcomes-based measures such as total cost of care, inpatient admission rates, or emergency department use.

What this study adds

• Implementation of surrogate and global outcomes measures to support an alternative pharmacy payment model is described.

• Differences in baseline performance is illustrated between pharmacies participating and not participating in an enhanced services network to show the application of surrogate and global outcomes measures.

Adoption of alternative payment models (APMs) is steadily increasing in the U.S. health care system. As an example, the U.S. Department of Health and Human Services recently announced a goal to tie 50% of traditional fee-for-service Medicare payments to quality or value by 2018.¹ APMs commonly use a variety of measures to assess health care quality.² These measures can be defined according to 3 different dimensions: structure, process, and outcome.³ Structural measures assess the fixed attributes of the setting where care is delivered; process measures assess how clinicians deliver health care services; and outcome measures assess the ultimate goals of health care (e.g., reducing morbidity, improving health status, and improving patient satisfaction). Within this framework for quality measures, health care costs are often considered separately from quality either as a measure of value or as a measure of overall performance.^4-6

Alternative Payment Model for Community Pharmacies in North Carolina

APMs are relatively new for community pharmacy practice, and there is no gold standard for design and implementation. The APM for pharmacies described here was implemented as part of a larger Center for Medicare and Medicaid Innovation (CMMI) project that has been described elsewhere.^7,8 Briefly, this project implemented a community pharmacist-delivered medication management service within a statewide integrated Medicaid primary care management program operated by Community Care of North Carolina (CCNC) for the state’s Division of Medical Assistance. Pharmacies participated in the North Carolina community pharmacy enhanced services network (CPESN) and tested an APM for medication management services that was independent from pharmacy prescription reimbursement. Pharmacies that participated in both the CPESN network and the CMMI project are referred here collectively as CPESN-CMMI pharmacies. A full description of the medication management service supported by this program can be found in Smith et al. (2017).⁷

Program Description

The primary objective of this article is to describe the process used to design and implement a performance measurement process for a group of enhanced services pharmacies in North Carolina. The core steps used to accomplish this goal are described in Figure 1.

FIGURE 1.

Steps for Designing a Performance Measure Set

Performance Metric Selection

Pharmacy performance was evaluated using a combination of 3 global outcome measures and 4 adherence measures (Table 1). Hospitalizations, emergency department (ED) visits, and total cost of medical care were selected as global outcomes, since these measures align with measures used by CCNC to evaluate performance of other members of the care team and are closely aligned with patient and payer goals. Adherence, a surrogate outcome, was selected based on well-established research supporting the link between better adherence and improvements in outcomes,^9,10 evidence for the role of pharmacists in improving adherence,^11-13 and use of adherence measurement in many quality rating systems, including Medicare’s Part D star ratings program.¹⁴ Although previous pharmacy APMs have used medication adherence to evaluate pharmacy quality,^5,6 this is the first project to our knowledge to incorporate global outcome measures into an APM for community pharmacies.

TABLE 1.

Surrogate and Global Outcome Metric Mean Score Comparisons

Metric Label	CMMI-CPESN	Non-CPESN	P Value for Comparison
	Score Mean (SD)	Score Mean (SD)
ED score	1.32 (0.73)	1.37 (0.71)	0.465
Hospitalization score	1.45 (0.69)	1.39 (0.70)	0.372
PMPM total cost of medical care score	1.47 (0.86)	1.55 (0.81)	0.266
PDC scores
Oral diabetes medications	0.56 (0.27)	0.49 (0.26)	0.002
Statins	0.57 (0.22)	0.50 (0.25)	0.002
RASAsa	0.57 (0.22)	0.51 (0.24)	0.003
Chronic medications	0.60 (0.25)	0.49 (0.24)	< 0.001
Total score	6.54 (1.59)	6.29 (1.66)	0.115

^aStudent’s t-test used for statistical comparisons.

CMMI = Center for Medicaid and Medicare Innovations; CPESN = community pharmacy enhanced services network; ED = emergency department; PMPM = per member per month; PDC = proportion of days covered; RASA = renin-angiotensin system antagonists; SD = standard deviation.

Performance scores were calculated quarterly. A year-long look-back period was used for the 4 adherence measures, and the 3-month performance quarter was used for the 3 global outcome metrics. The performance quarter was not intended to facilitate observations of program impact; rather it was intended for use by pharmacists to observe their effect on global outcomes as a part of quality improvement. Any patient attributed to a pharmacy for only 1 of 3 months in the performance quarter was excluded, as was any patient with less than 10 months of Medicaid eligibility in the 12-month adherence look-back period.

Data Availability and Lag

Each of the measures used in the CPESN program were constructed from North Carolina Medicaid claims for prescription, outpatient, and inpatient services. To ensure accuracy of data and allow for insurance adjudication, only mature claims were used for performance measurement. Claims were considered “mature” (appropriately adjudicated and collected) following 6 months for inpatient admissions, 3 months for ED visits, and 1 month for all prescriptions. Measures were calculated on a quarterly basis, and all claims within a quarter were required to mature completely before performance measurement.

Performance Metric Definitions

Hospitalizations and ED Visits.

Medical claims data were used to capture rates of all-cause hospitalizations among attributed patients. In addition, all-cause ED visits not resulting in a hospitalization were measured. Rates were calculated as per-member-per-month (PMPM) counts. All-cause rates were chosen to hold pharmacists accountable for the same types of measures that were assessed for other members of the care team.

Total Cost of Medical Care.

Total cost of medical care included payments made by the Medicaid program for all, outpatient, inpatient, and ED services. A number of services were excluded from this measurement, including ancillary care services (e.g., adult care homes and personal care services), dental, and undefined claims. These services were excluded because it was hypothesized that pharmacists could not substantially influence their patients’ use of these services.

Drug costs were eliminated for 2 reasons. First, we wanted to avoid incentives for pharmacists to focus on drug spending without considering overall spending and, second, because of the potential for gross overestimation of drug prices when rebates are unknown.¹⁵ To eliminate outliers, spending was capped at the 99th percentile of average expenditures across all patients. Total cost of medical care was reported as the average PMPM cost for each pharmacy.

Medication Adherence.

Medication adherence was calculated using proportion of days covered (PDC), with the standard 80% cut-off for determination of adherence.¹⁶ Metrics were not adjusted for institutionalized days. The proportion of patients that adhered to renin-angiotensin system antagonists (RASAs), oral diabetes medications, and statin medications was reported for each patient using medication claims from the rolling 12-month period beginning with the first fill of a denominator-eligible medication and ending at the last month of the performance period.

In addition, we also measured PDC for 71 classes of chronic medications, corresponding to 8 primary disease states: asthma, diabetes, hepatitis B, hepatitis C, HIV, hyperlipidemia, hypertension, and mental health. Some of the 71 classes of medications also included contraceptives and treatments for osteoporosis/osteopenia, endocrine disorders, seizures, and other chronic conditions. These classes and conditions were identified from the review of chronic medications defined by Medi-Span and First Data Bank, combined with clinical review by the study team. Patients using medications in 4 or more chronic therapeutic categories were identified as adherent if the PDC for at least 75% of the medications they were using was 80% or higher. This measure was developed by the study team for purposes of performance measurement with this APM. The method for calculating PDC for each class was identical to the method used for RASAs, oral diabetes medications, and statins. The goal of this measure was to assess overall medication adherence in the portion of the population with multiple chronic illnesses. No formal measure testing has been conducted on this measure.

Application of Measurements to Population

Patients were attributed to a pharmacy if they filled at least 1 chronic medication and had 80% or more of their chronic medications filled at a single pharmacy within a 3-month period. Chronic medications were defined by a list of 71 therapeutic classes, the same set used to measure the custom adherence measure. To account for data lag, the last month of the 3-month look-back period was 2 months before the attribution month. For example, if a report on attributed patients was sent to pharmacies in June, prescriptions from February, March, and April were used to run the attribution algorithm.

To protect against potential instability in the performance measurement, pharmacies were required to have a minimum of 20 attributed patients. For global measures (hospitalizations, ED visits, and total cost of medical care) the denominator was equal to the number of attributed patients; therefore, the minimum denominator for these metrics was 20. For each of the medication adherence measures, denominators varied by the number of patients with metric-eligible claims during the measurement period. The minimum denominator was set at 1 for the adherence metrics, and pharmacies that did not have at least 1 patient attributed for each adherence metric were excluded from the analysis.

Case-Mix Adjustment Process

Differences in the age and health status of the populations served by each pharmacy can influence performance. To control for this, we case-mix adjusted each pharmacy using a previously validated tool, Clinical Risk Groups (CRGs).¹⁷ CRGs use diagnosis, inpatient, outpatient, physician, and pharmacy data to assign patients to risk categories that predict use of health care services. Notably, CRGs include data that evaluate the medical complexity of each patient but do not account for other factors influencing health outcomes, such as social determinants. Each CRG group was assigned a severity weight that was used to account for the relative health status of the patients served by each pharmacy to enable accurate comparisons. Severity weights were developed by CCNC using baseline Medicaid claims data from 2012. Patients were stratified into categories based on CRG group, gender, and age category, and separate weights were derived from the count of ED visits, hospitalizations, and total cost of medical care for each grouping. This case-mix adjustment method created the expected event counts/spending levels for each CRG-based severity weight grouping. The ratio of the sum of observed event rates to the sum of what was expected given the pharmacy’s attributed panel’s CRG weights was used to calculate scores for hospitalization and ED measures. An analogous process was used for total cost of medical care. In this way, scores could be equitably compared across all North Carolina pharmacies.

Selection of Scoring Methodology for Each Metric

For each quarterly performance period, each of the 7 performance measures was calculated for all North Carolina pharmacies, regardless of CPESN or CMMI status. For adherence metrics, pharmacies achieving scores 1 standard deviation or more above the average North Carolina pharmacy were deemed high performers and received all possible points (Table 1). For hospitalizations and ED visits, pharmacies achieving scores 1 standard deviation or more below the average North Carolina pharmacy were deemed high performers and received all possible points in that category. For the total cost of medical care measure, pharmacies with scores greater than 2 standard deviations below the mean received 3 points, and pharmacies with 1 to less than 2 standard deviations received 2 points. Pharmacies within 1 standard deviation above or below the mean were deemed average performers and received 1 point for each hospitalization, ED, or total cost of medical care measure or 0.5 points for each of the adherence-based measures. Weights were determined a priori by project team members, with the global outcome measures—the most relevant to patients and payers—weighted more heavily than adherence measures. Scores were summed to create a total performance score ranging from 0 to 11.

Observations Pharmacy Score Comparisons

To better understand the performance scores that we designed and implemented, we analyzed score performance for CPESN-CMMI versus non-CPESN pharmacies during the first period of measurement (March 1, 2015-May 31, 2015). This period did not begin with the calendar year to align with the availability of grant payments. Claims from June 1, 2014, to May 31, 2015, were used to calculate metrics, in accordance with metric-specific lag periods. We compared mean score performance between CPESN-CMMI and non-CPESN pharmacies using simple bivariate statistics. Score means were compared using a t-test to evaluate mean differences in total points. This study was approved by the University of North Carolina at Chapel Hill Institutional Review Board.

Of the 119 CPESN-CMMI pharmacies and 1,616 non-CPESN pharmacies included in the first performance measurement period, 116 (97.4%) and 1,481 (91.6%) were eligible for performance score evaluation, respectively. CPESN-CMMI pharmacies outperformed non-CPESN pharmacies on all adherence metrics (P < 0.001-0.003), with CPESN-CMMI pharmacies scoring 0.07 to 0.11 points greater across the 4 measures. ED score, hospitalization score, and total cost of medical care score were not significantly different between the 2 pharmacy types (P = 0.266-0.465). Accordingly, total score did not differ significantly, with CPESN-CMMI pharmacies scoring an average of 6.54 points and non-CPESN scoring 6.29 (P = 0.115).

The lack of difference between CPESN-CMMI and non-CPESN pharmacies is not unexpected, since these scores were calculated during the first period that the program was implemented, and effect on global outcomes likely takes longer than 3 months. Furthermore, there were 9 months of data before March 2015 included in the adherence measures. Therefore, differences in adherence scores during the first period signify differences in baseline adherence between participating and nonparticipating pharmacies.

Lessons Learned

To our knowledge, this is the first description of the design and implementation of performance measures to support an APM in a network of community pharmacies. Using a mix of surrogate and global outcome measures, this program was successful in measuring pharmacy quality for more than 90% of pharmacies serving attributable North Carolina Medicaid enrollees between March 1, 2015, and May 31, 2015. We do not suggest that this system of measures should be adopted as an industrywide standard; rather, we demonstrate here that global outcome measurement is possible over a broad set of pharmacies and invite additional research to explore the validity of this and other methods to measure pharmacy quality and performance.

A number of key lessons were learned from our design and implementation process. We show that it is possible to differentiate pharmacy performance on the basis of global and surrogate outcomes, not just structure and process. Measuring the ability of pharmacists to positively affect broader health outcomes is an important step toward wider implementation of enhanced clinical services, recognition of the pharmacist role in team-based care models, and reimbursement for the delivery of these services as a part of APMs.

Another lesson learned through this process is the importance of case-mix adjustment of pharmacy populations to ensure equitable quality measurement when using global outcomes. Companion analysis of differences between attributed populations at baseline found that CPESN-CMMI pharmacies served sicker populations with higher health care utilization at base-line.¹⁸ Failure to account for differences in health status across pharmacies would lead to biased measurements of quality.

We acknowledge that not all networks have the ability to implement the same performance measurement process we used. There were several unique circumstances that contributed to the successful implementation of this outcomes-based scoring system, including the availability of medical claims that allowed for a broader measurement of pharmacy performance on relevant health outcomes. Lack of measurement of the effect of pharmacies on total cost of medical care, hospitalizations, and ED visits may dampen payers’ enthusiasm to engage with enhanced services pharmacy networks. Receiving claims data from Medicaid also resulted in a relatively short lag between data availability and performance measurement. Lag times in other programs, such as the star ratings program, which uses previous year data to measure performance, make it difficult to observe the effect of practice changes and reward performance improvements in a timely manner.

In the absence of prescription claims, pharmacy dispensing records may serve as a reasonable proxy for performance measurement. It should be noted, though, that dispensing records only represent prescription-filling histories for the pharmacies that make records available. It is also our experience that the data derived from dispensing systems is often incomplete and not fully adjudicated. In addition, different systems have different data formats and structures, making the process of combining datasets across systems burdensome.

When lacking claims, another alternative for performance measurement is structural and process measures of quality. This alternative requires participating pharmacies to document their actions with a need for audit and patient feedback to verify adherence. We chose not to impose structural or process measure requirements on the network in part because, when these measures were developed, few structural or process measures for pharmacies had a high correlation with better quality care.¹⁹ Structural elements that might be considered for other studies, pending validation of a consistent association with better patient outcomes, include presence of a private consultation room, ability to document care through a dispensing system or other software platform, pharmacy personnel credentialing, and the ability to share quality-related information with other pharmacies and health care providers.^20,21 Process elements could include meaningful use of health information technology and guideline-related measures such as the Statin Use in Persons with Diabetes and High Risk Medications measures endorsed by the Pharmacy Quality Alliance.²²

Implications

Pharmacy has little experience systematically measuring community pharmacy quality. Few examples of scoring systems for pharmacy quality exist, and this is the only study to our knowledge to have used global measures as a part of pharmacy performance measurement. The composite 11-point score and the component measures have not been validated for use in supporting a pharmacy payment model. Formal scientific testing of composite measures such as the measure described here involves measuring reliability, validity, feasibility of measurement, usability of measure output, and comparison with competing composite measures to assess redundancy and relative scientific merit.²³

Our program, however, demonstrates that the composite measure used is feasible to measure and produce results that are usable to payers. Since there are no other publicly available composite measures of pharmacy performance, this measure is not redundant. However, we did not evaluate the consistency of these composite measure scores nor did we evaluate the relationship between composite measure scores and meaningful differences in pharmacy practice. This is an area for further research and should be pursued through collaborations between academics, pharmacists, and payers. Promising measures should proceed through a measure endorsement process supported by an entity such as the Pharmacy Quality Alliance or the National Quality Forum.

There is no single correct system of composite performance measurement. Rather, each system should fit the needs of the network and the payer, and all systems should incorporate processes for continuous quality improvement. This research demonstrates that it is possible to measure pharmacy quality on broad outcomes and that these types of quality measures can be used to support APMs seeking to hold pharmacists accountable for the quality of care they provide.