Evaluation of Eligibility Criteria Used to Identify Patients for Medication Therapy Management Services: A Retrospective Cohort Study in a Medicare Advantage Part D Population

Abstract

BACKGROUND:

General eligibility criteria used by the Centers for Medicare & Medicaid Services (CMS) to identify patients for medication therapy management (MTM) services include having multiple chronic conditions, taking multiple Part D drugs, and being likely to incur annual drug costs that exceed a predetermined threshold. The performance of these criteria in identifying patients in greatest need of MTM services is unknown. Although there are numerous possible versions of MTM identification algorithms that satisfy these criteria, there are limited data that evaluate the performance of MTM services using eligibility thresholds representative of those used by the majority of Part D sponsors.

OBJECTIVES:

To (a) evaluate the performance of the 2013 CMS MTM eligibility criteria thresholds in identifying Medicare Advantage Prescription Drug (MAPD) plan patients with at least 2 drug therapy problems (DTPs) relative to alternative criteria threshold levels and (b) identify additional patient risk factors significantly associated with the number of DTPs for consideration as potential future MTM eligibility criteria.

METHODS:

All patients in the Medicare Advantage Part D population who had pharmacy eligibility as of December 31, 2013, were included in this retrospective cohort study. Study outcomes included 7 different types of DTPs: use of high-risk medications in the elderly, gaps in medication therapy, medication nonadherence, drug-drug interactions, duplicate therapy, drug-disease interactions, and brand-to-generic conversion opportunities. DTPs were identified for each member based on 6 months of most recent pharmacy claims data and 14 months of most recent medical claims data. Risk factors examined in this study included patient demographics and prior health care utilization in the most recent 6 months. Descriptive statistics were used to summarize patient characteristics and to evaluate unadjusted relationships between the average number of DTPs identified per patient and each risk factor. Quartile values identified in the study population for number of diseases, number of drugs, and annual spend were used as potential new criteria thresholds, resulting in 27 new MTM criteria combinations. The performance of each eligibility criterion was evaluated using sensitivity, specificity, positive predictive values (PPVs), and negative predictive values (NPVs). Patients identified with at least 2 DTPs were defined as those who would benefit from MTM services and were used as the gold standard. As part of a sensitivity analysis, patients identified with at least 1 DTP were used as the gold standard. Lastly, a multivariable negative binomial regression model was used to evaluate the relationship between each risk factor and the number of identified DTPs per patient while controlling for the patients’ number of drugs, number of chronic diseases, and annual drug spend.

RESULTS:

A total of 2,578,336 patients were included in the study. The sensitivity, specificity, PPV, and NPV of CMS MTM criteria for the 2013 plan year were 15.3%, 95.6%, 51.3%, and 78.8%, respectively. Sensitivity and PPV improved when the drug count threshold increased from 8 to 10, and when the annual drug cost decreased from $3,144 to $2,239 or less. Results were consistent when at least 1 DTP was used as the gold standard. The adjusted rate of DTPs was significantly greater among patients identified with higher drug and disease counts, annual drug spend, and prior ER or outpatient or hospital visits. Patients with higher median household incomes who were male, younger, or white had significantly lower rates of DTPs.

CONCLUSIONS:

The performance of MTM eligibility criteria can be improved by increasing the threshold values for drug count while decreasing the threshold value for annual drug spend. Furthermore, additional risk factors, such as a recent ER or hospital visit, may be considered as potential MTM eligibility criteria.

What is already known about this subject

• As part of the Medicare Prescription Drug, Improvement, and Modernization Act of 2003, Part D sponsors must offer medication therapy management (MTM) programs to Medicare Part D beneficiaries using specific eligibility criteria updated annually by the Centers for Medicare & Medicaid Services (CMS).

• The majority of MTM programs during the 2013 plan year targeted Medicare beneficiaries with a minimum of 3 chronic diseases, 8 Part D-covered drugs, and $3,144 in annual drug spend.

What this study adds

• This study measured the performance of CMS MTM identification criteria representative of the 2013 plan year and demonstrated that alternative MTM criteria thresholds could be used to improve the identification of patients who might benefit the most from MTM services.

• Findings from this study suggest that additional patient demographic or clinical characteristics readily available from most pharmacy and medical claims data may be helpful in identifying patients who can benefit from MTM services.

• Results from this study are generalizable to other MTM programs that use a similar identification process in a Medicare Advantage Prescription Drug plan population.

As part of the Medicare Prescription Drug, Improvement, and Modernization Act of 2003, Part D sponsors must offer medication therapy management (MTM) programs to Medicare Part D beneficiaries.¹ The goal of MTM programs is to optimize therapeutic outcomes by improving medication use and reducing the risk of adverse events.² Annually, the Centers for Medicare & Medicaid Services (CMS) provide explicit criteria that Part D drug plans must follow when identifying Medicare beneficiaries who are eligible for MTM services. As part of the MTM program, patients are offered a comprehensive medication review (CMR) on an annual basis. A CMR is an interactive consultation provided by a pharmacist or other qualified health care provider who identifies and addresses drug therapy problems (DTPs), such as drug-drug interactions and nonadherence, and provides the patient with an individualized, written summary of the pharmacist’s findings and other follow-up as necessary.

CMS requires Part D sponsors to target Medicare beneficiaries with multiple chronic diseases, multiple Part D chronic medications, and annual spend on Part D-covered drugs likely to exceed a prespecified amount. Each year, CMS may update the eligibility criteria by increasing or decreasing the minimum thresholds for eligibility. During the 2013 plan year, CMS established a ceiling of 3 and a floor of 2 for the minimum number of chronic diseases required for MTM eligibility.³ Most 2013 plans (82.0%) set their criteria threshold at at least 3 chronic diseases.⁴

Furthermore, Part D sponsors are given the choice to target beneficiaries with any chronic diseases or specific chronic diseases. If they target specific chronic diseases, then they must include at least 5 out of 9 core chronic conditions.³ During 2013, 96.3% of MTM programs targeted beneficiaries with specific chronic diseases, with the top 3 targeted diseases being diabetes, chronic heart failure (CHF), and dyslipidemia.⁴ The minimum threshold for the number of Part D-covered drugs was between 2 and 8 during the 2013 plan year.³ Most 2013 MTM programs (58.9%) selected a minimum threshold of 8 drugs.⁴

Lastly, CMS set the minimum annual drug spend (includes ingredient cost, dispensing fee, sales tax, and vaccine administration fee, if applicable) for the 2013 plan year at $3,144.³ For the 2 years following 2013, CMS continued to elect the same minimum number of diseases and medications to define MTM eligibility as in 2013, and it has increased the annual drug spend by 4% every year.⁵

Understanding how well current MTM eligibility criteria perform is imperative when modifying the design of MTM programs for future plan years. There are limited data on the performance of MTM eligibility criteria in identifying Medicare beneficiaries who would benefit the most from MTM services. This study examines current MTM identification criteria and explores how these criteria could be modified, either by changing the thresholds or by including other criteria outside of pharmacy use, to improve its performance in identifying patients who would benefit from MTM programs.

The primary objective of this study was to evaluate the performance of MTM eligibility criteria in a Medicare Advantage Prescription Drug (MAPD) plan population using MTM criteria representative of the 2013 plan year. The secondary study objective was to identify additional risk factors significantly associated with the number of DTPs in a MAPD plan population for potential future use as MTM eligibility criteria.

Methods

Study Design, Setting, and Population

This is a retrospective cohort study using pharmacy and medical claims of a MAPD population managed by a large pharmacy benefits manager (PBM) in the United States. This research was considered exempt by the appointed external institutional review board because the study uses preexisting claims data.

MAPD beneficiaries were included in the study if they had pharmacy benefits as of December 31, 2013. Patients were excluded from the study if they were missing demographic information (Figure 1). The final study population was used to identify patients eligible for MTM in 2013 using various criteria thresholds.

FIGURE 1.

Sample Selection Flow Chart of Patients Identified for the Final Study Population Used to Evaluate MTM Criteria

Identification of Patients Eligible for the 2013 MTM Program

MTM patients were identified from the study population using eligibility criteria representative of those used by the majority of 2013 MTM programs. Specifically, we required a minimum of 3 chronic diseases, 8 Part D-covered drugs, and $3,144 in annual drug spend. Five specific chronic diseases (cardiovascular disease, diabetes, rheumatoid arthritis, dyslipidemia, end-stage renal disease) were used to identify members for MTM and were selected largely based on the prevalence of these diseases in the examined MAPD patient population.

In addition, we identified MTM-eligible patients using 27 alternative versions of MTM eligibility criteria using different combinations for the minimum number of chronic diseases, drug count, and annual drug spend. The minimum number of diseases was varied from 1 to 3. Quartile values identified in the study population for the number of drugs and annual drug spend were used as new threshold cutoffs. In total, 28 different MTM eligibility criteria were tested in the study population.

We used similar identification periods as those used by the MTM program offered at this plan setting that occurred at the end of the 2013 year (Table 1). The patients’ number of chronic diseases were identified using medical claims data based on International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis and procedure codes or J codes (part of the Healthcare Common Procedure Coding System [HCPCS] used to indicate that a drug was administered) between January 1, 2013, and December 31, 2013 (Table 2). In addition, pharmacy claims between October 1, 2013, and December 31, 2013, were used to identify the patients’ number of chronic conditions by proxy based on the medications they filled using generic product identifier (GPI) codes (Table 2). The patients’ number of Part D medications and annual drug spend were calculated using pharmacy claims data between January 1, 2013, and December 31, 2013. In practice, it was possible for members who had been previously identified for MTM during the prior year to be re-identified during the 2013 plan year if they met eligibility criteria. To be consistent with usual practice, members who had been previously identified for MTM in 2012 were included in the identification process.

TABLE 1.

Identification Periods and Claim and Code Types Used to Define Variables for MTM Eligibility Criteria

Variable	IdentificationPeriod	Claim Type Used (Code Types)
Number of chronic diseases	January 1, 2013-December 31, 2013	Medical claims (ICD-9-CM diagnosis, ICD-9-CM procedure, HCPCS,CPT, J codes)
	October 1, 2013-December 31, 2013	Pharmacy claims (GPI codes)
Number of Part D medications	January 1, 2013-December 31, 2013	Pharmacy claims (GPI codes)
Annual drug spend	January 1, 2013-December 31, 2013	Pharmacy claims

CPT = Current Procedural Terminology; GPI = generic product identifier; HCPCS = Healthcare Common Procedure Coding System; ICD-9-CM = International Classification of Diseases, Ninth Revision, Clinical Modification; J code = part of the HCPCS classifications used to indicate that a drug was administered; MTM = medication therapy management.

TABLE 2.

Codes Used to Identify Chronic Conditions for MTM Eligibility Criteria from Medical or Pharmacy Claims Data During the Identification Periods

Condition	ICD-9-CM Diagnosis, ICD-9-CM Procedure, HCPCS, CPT, or J Codes (Using Medical Claims Between January 1, 2013, and December 31, 2013)	GPI Codes (Using Pharmacy Claims Between October 1, 2013, and December 31, 2013)
Cardiovascular disease	ICD-9-CM diagnosis codes: 38.48, 401-405, 410, 412, 427.31, 427.32, 428, 430-435, 443.89, 443.9	GPI2 level: 33, 34, 36; GPI4 level: 3720, 3760; GPI6 level: 379900, 409925; GPI8 level: 33300007
Diabetes	ICD-9-CM diagnosis codes: 250, 357.2, 362.0, 366.41	GPI2 level: 27
Rheumatoid arthritis	ICD-9-CM diagnosis codes: 714.0, 714.1, 714.2, 714.81; J codes: J0129, J0135,J0718,J1438,J1600, J1745,J2910,J3262, J3590, J7500-J7502, J7515-J7518, J8530, J9070, J9080, J9090-J9097,J9250, J9310	GPI4 level: 6620, 6627, 6628, 6629, 6640; GPI8 level: 04000040, 13000020, 21101020, 21300050, 21353060, 52500060, 52505020, 52505040, 66250050, 66260010, 66270015, 66270040, 66500070, 99200030, 99402020, 99403030, 99406010
Dyslipidemia	ICD-9-CM diagnosis codes: 272.0, 272.1, 272.3, 272.4	GPI4 level: 3910, 3920, 3940, 3999; GPI6 level: 409925; GPI8 level: 39450050, 77103010, 77103020; GPI14 level: 39300030000320
End-stage renal disease	ICD-9-CM diagnosis codes: 582, 583, 585.1-585.6, 585.9, 586, 588, v42.0, v45.1, v45.11, v45.12; ICD-9-CM procedure codes: 38.95, 39.27, 39.42, 39.43, 39.53, 39.93-39.95, 54.98, 55.6, 55.61, 55.69; CPT codes: 36147, 36800, 36810, 36815, 36818-36821, 36831-36833, 50300, 50320, 50340, 50360, 50365, 50370, 50380, 90935, 90937, 90940, 90945, 90947, 90957-90962, 90965, 90966, 90969, 90970, 90989, 90993, 90997, 90999, 99512; HCPCS codes: S2065, S9339, G0257

CPT = Current Procedural Terminology; GPI = generic product identifier; HCPCS = Healthcare Common Procedure Coding System; ICD-9-CM = International Classification of Diseases, Ninth Revision, Clinical Modification; J code = part of the HCPCS classifications used to indicate that a drug was administered; MTM = medication therapy management.

Study Outcomes

The primary study outcome was the patients’ total number of identified DTPs as a measure of the patients’ potential value for MTM services. Seven types of DTPs were identified in this study: use of high-risk medications in the elderly, gaps in medication therapy, medication nonadherence, drug-drug interactions, duplicate therapy, drug-disease interactions, and brand-to-generic conversion opportunities. These DTPs were selected based on their high prevalence of DTPs previously identified by this specific MAPD population.

Since patients must receive a CMR offer within 60 days of identification, we assumed that all patients in the study population who were identified for MTM at the end of the 2013 year would have received a CMR by March 1, 2014. DTPs were identified based on a 14-month medical claims measurement period (January 1, 2013, to February 28, 2014) and a 6-month pharmacy claims measurement period (September 1, 2013, to February 28, 2014) occurring before the CMR date. Identification criteria varied by DTP are as described below.

High-Risk Medications in the Elderly.

Use of high-risk medications in the elderly was evaluated using pharmacy claims for each patient filled on or after the beginning of the calendar month when the patient turned 65 years old. High-risk medications were identified during the pharmacy claims measurement period using GPI codes for medications listed by the 2014 Healthcare Effectiveness Data and Information Set (HEDIS) measure (“Use of high-risk medications in the elderly”) and 2012 Pharmacy Quality Alliance/Acumen measure (“High Risk Medication Measures”).^6,7

Gaps in Medication Therapy.

Patients were identified as having medication gap(s) using pharmacy and medical claims during the measurement periods if they were identified (a) with CHF and missing a beta-blocker (BB) or an ACE inhibitor (ACEI) or angiotensin II receptor blocker (ARB); (b) with diabetes and hypertension and missing an ACEI or ARB; (c) with diabetes and missing a statin; (d) as postmyocardial infarction and missing a BB; (e) as a female over the age of 65 years with a history of fracture and no osteoporosis medication; or (f) with rheumatoid arthritis and missing a diseasemodifying antirheumatic drug.

Medication Nonadherence.

Medication nonadherence was evaluated at the medication class level for the following drug classes: antidepressants, antidiabetics, renin-angiotensin system antagonists (includes ACEIs, ARBs, direct renin inhibitors), BBs, calcium channel blockers, and respiratory long-term control medications. Adherence was measured using the proportion of days covered (PDC), which divides the number of days during which the member was covered by at least 1 medication within the medication class by the member’s number of days in the measurement period, which starts on the date of the first fill for a medication class and ends on the last day of the pharmacy claims review period. If prescription fills for the same medication overlapped, the prescription start date was adjusted to be the day after the previous fill ended. A patient was identified with nonadherence as a DTP if he or she had a PDC level of less than 80% for at least 1 of the 6 drug classes.

Drug-Drug Interactions.

Drug-drug interactions were identified if the patient filled 2 interacting medications with at least 1 overlapping days supply any time during the pharmacy claims measurement period. This study included about 2,500 different drug combinations recognized as being of high concern or contraindicated based on Hansten and Horn’s Drug Interactions Analysis and Management,⁸ American Health Formulary Services,⁹ Micromedex,¹⁰ Facts & Comparisons,¹¹ Clinical Pharmacology,¹² and drug-specific prescribing information.

Duplicate Therapy.

Duplicate therapy issues included cases where patients were taking at least 2 medications in the same medication class for at least 90 overlapping days based on pharmacy claims data. This study included about 1,600 potential duplicate therapy issues for DTP identification during the measurement period in the study population.

Drug-Disease Interactions.

Patients were identified with drug-disease interaction(s) based on pharmacy and medical claims if they filled at least 1 medication that interacted with any of the following conditions identified for the patient: asthma, chronic obstructive pulmonary disease, benign prostatic hyperplasia, CHF, closed-angle glaucoma, gallbladder disease, gout, mechanical heart valve implant, Parkinson’s disease, or epilepsy. In addition, patients who were at least 65 years of age on the first day of the medical claims measurement period were identified for drug-disease interactions if they filled at least 1 medication that interacted with their dementia, chronic kidney disease, or having a history of falls. These drug-disease interactions were targeted based on well-established interactions identified from clinical practice, scientific literature, and guidelines set forth by the National Committee for Quality Assurance in HEDIS.¹³

Brand-to-Generic Conversion Opportunities.

We identified brand-to-generic conversion opportunities as a DTP based on the assumption that conversion from brand to generic drugs may indirectly improve clinical outcomes by lowering medication costs and increasing medication adherence levels. Specifically, patients were identified if they filled a brand name medication (which had a generic alternative or equivalent available) belonging to the ARB, bisphosphonate, nasal steroid, proton pump inhibitor, statin, or triptan medication classes.

Patient Risk Factors

Aside from the number of chronic diseases, number of medications, and annual drug spend, we evaluated several patient demographic and clinical characteristics as potential risk factors associated with increased DTPs. These risk factors were chosen based on the likelihood of such data being readily available to other similar PBM organizations and most health plans.

Demographic risk factors included patient age (calculated as of December 31, 2013), gender, race (as collected by the health plan and reported by CMS in the Monthly Member Reports), geographic region, and median household income (geocoded based on patient’s zip code). Patient age was categorically defined as less than 65 years, 65-70 years, or greater than 70 years to examine any differences among those qualifying for Medicare based on age and those with a disability or other condition. Region was determined based on the patient’s residing state and categorized as West, South, Northeast, or Midwest. Household income was also categorically defined as below or above the median value found in the study population. Clinical risk factors included having prior emergency room (ER) or hospital visits between July 1, 2013, and December 31, 2013.

Statistical Analysis

Patient demographics and clinical characteristics of the study population were summarized using descriptive statistics of averages and standard deviations for continuous variables and percentages for categorical variables. The unadjusted relationships between the average number of DTPs identified per member and patient risk factors were evaluated using the t-test if there were 2 categories (such as gender), or analysis of covariance if there were more than 2 categories (such as region). The performance of MTM eligibility criteria in 2013 relative to 27 other criteria threshold combinations was evaluated using sensitivity, specificity, positive predictive values (PPVs), and negative predictive values (NPVs).

Patients identified with at least 2 DTPs were used as the gold standard to identify patients who could benefit from MTM. The choice of gold standard was based on the expected number of DTPs identified for MTM patients extrapolated from the minimum number of chronic diseases and Part D-covered drugs required by CMS using the Resource-based Relative Value Scale (RBRVS) compensation grid. The RBRVS grid has previously been used to infer the number of identified DTPs when chart abstraction is unavailable.¹⁴ As part of a sensitivity analysis, we also used at least 1 DTP as the gold standard. Lastly, a multivariable negative binomial regression model was used to evaluate the relationship between each risk factor and the number of identified DTPs per patient while adjusting for the patient’s number of drugs, number of chronic diseases, and annual drug spend. For all analyses, a two-sided P value of 0.050 was considered as statistically significant and was performed using SAS software package 9.2 (SAS Institute Inc., Cary, NC).

Results

A total of 2,578,511 patients were identified in the MAPD population with pharmacy benefits eligibility as of December 31, 2013. After excluding 175 (0.007%) patients who were missing demographic information, we identified a final study population of 2,578,336 patients (Figure 1). The average standard deviation [SD] age of the study population was 71.2 [10.0] years, and 43.1% were male (Table 3). A greater proportion of patients were located in the South (37.9%) and West (30%) than in other regions. The study population was mostly white (81%), and the median household income was $52,565.

TABLE 3.

Demographic and Clinical Characteristics of Study Population (N = 2,578,336) During Identification Periods^a

Baseline Characteristic	Result
Average age [SD], years	71.2 [10.0]
Age group, % (n)
< 65 years	17.5 (449,881)
65-70 years	32.0 (825,145)
> 70 years	50.6 (1,303,310)
Male, % (n)	43.1 (1,112,125)
Region, % (n)
Midwest	17.1 (441,894)
South	37.9 (977,693)
West	30.0 (773,925)
Northeast	14.9 (384,824)
Race, % (n)
White	81.0 (2,087,914)
African American	10.9 (279,942)
Hispanic	2.2 (55,470)
Asian	2.5 (64,015)
Other	3.3 (85,145)
Native American	0.2 (5,850)
Median (IQR) household income ($)	52,565 (26,025)
Average [SD] number of chronic diseases per patient	1.6 [1.1]
Number of identified chronic diseases, % (n)
0	21.5 (555,339)
1	25.1 (647,230)
2	31.2 (803,897)
3	18.4 (473,883)
4	3.7 (95,170)
5	0.1 (2,817)
Identified with rheumatoid arthritis, % (n)	2.5 (64,337)
Identified with dyslipidemia, % (n)	52.8 (1,361,064)
Identified with diabetes, % (n)	25.1 (648,135)
Identified with end-stage renal disease, % (n)	8.6 (220,358)
Identified with cardiovascular disease, % (n)	6.3 (161,352)
Average [SD] number of Part D medications per patient	7.3 [5.9]
Median (IQR) number of Part D medications per patient	6 (3, 10)
Average [SD] annual drug spend ($)	2,233 [6,146]
Median (IQR) annual drug spend ($)	656 (2,089)
Prior outpatient visit(s), % (n)	87.6 (2,258,888)
Prior emergency room visit(s), all-cause, % (n)	15.7 (403,942)
Prior acute inpatient stay, all-cause, % (n)	5.8 (149,825)

^aSee Table 1 for identification periods used for each variable.

IQR = interquartile range; SD = standard deviation.

During the study identification periods, the average [SD] number of chronic diseases identified per member was 1.6 [1.1]. About 25% of the study population was identified with 1 core chronic disease, 31% with 2 diseases, and 18% with at least 3 diseases. Patients were most commonly identified with hypertension (67.1%), dyslipidemia (52.8%), and diabetes (25.1%). The average [SD] number of drugs identified per member was 7.3 [5.9]. Quartile values for total drug count in this study population were 3, 6, and 10. For annual drug spend, the average [SD] was $2,233 [$6,146] and the quartile values were $150, $656, and $2,239. About 16% of the study population visited the ER and about 6% were hospitalized in the prior 6 months.

Overall Study Population

Among the entire MAPD study population of 2,578,511 patients, about 7% met the 2013 MTM eligibility criteria during the last quarter of 2013, and a total of 2,287,026 DTPs were identified. About 46% of the entire study population was identified with at least 1 DTP, and 23% were identified with at least 2 DTPs. During the 6-month period used to identify DTPs from pharmacy claims, the average [SD] length of pharmacy eligibility per member in the overall study population was 169.1 [24.4] days and the median was 181 days. The majority (80%) of members were continuously eligible during the 6-month period (allowing for gaps in eligibility of fewer than 31 days).

On average, each member was identified with 0.9 [1.3] DTPs. The most commonly identified DTP was medication nonadherence (57.9%), followed by gaps in medication therapy (19.2%), drug-disease interactions (9.6%), high-risk medications in the elderly (6.4%), drug-drug interactions (5.9%), duplicate therapy (0.7%), and brand-to-generic conversions (0.4%; data not shown). Results from the unadjusted analysis revealed that all patient demographic and clinical risk factors were significantly associated with the average number of DTPs identified per member (Table 4).

TABLE 4.

Unadjusted Associations Between Average Number of Identified Drug Therapy Problems per Patient and Additional Patient Risk Factors (N = 2,578,336)

	Average [SD] Number of DTPs per Patient	P value
Age group, years		< 0.001
<65	0.77 [1.15]
65 to 70	0.82 [1.21]
>70	0.97 [1.35]
Gender		< 0.001
Male	0.82 [1.21]
Female	0.94 [1.32]
Region		< 0.001
Midwest	0.82 [1.23]
South	1.06 [1.39]
West	0.75 [1.16]
Northeast	0.81 [1.81]
Race		< 0.001
White	0.85 [1.25]
African American	1.20 [1.45]
Hispanic	1.00 [1.31]
Asian	0.76 [1.14]
Other	0.75 [1.12]
Native American	1.06 [1.40]
Median household income		< 0.001
> $52,565	0.80 [1.20]
≤ $52,565	0.97 [1.34]
Prior outpatient visit(s)		< 0.001
Yes	0.95 [1.31]
No	0.41 [0.90]
Prior emergency room visit(s), all-cause		< 0.001
Yes	1.48 [1.66]
No	0.78 [1.16]
Prior acute inpatient stay, all-cause		< 0.001
Yes	1.77 [1.78]
No	0.83 [1.22]

DTPs = drug therapy problems.

MTM Identification.

Sensitivity, specificity, PPV, and NPV for the CMS MTM eligibility criteria for the 2013 plan year as well as for each of the new 27 MTM eligibility criteria combinations are reported in Table 5. Although sensitivity of the 2013 MTM eligibility criteria was only 15.3%, the specificity, PPV, and NPV were higher at 95.6%, 51.3%, and 78.8%, respectively. In comparison, MTM eligibility criteria requiring at least 3 chronic diseases, 10 drugs, and $2,239 in annual drug cost (combination number 27) achieved higher sensitivity (17.8%), PPV (53.1%), and NPV (79.2%). Specificity only slightly decreased from 95.6% to 95.2%. Furthermore, MTM criteria that required at least 3 chronic diseases, 10 drugs, and either $150 or $656 in annual drug cost (combination numbers 9 and 18) achieved higher sensitivity (26.0%-27.7%), PPV (52.3%), and NPV (80.5%-80.8%), and achieved higher sensitivity (27.7% for combination 9 and 26.0% for combination 18), PPV (52.3% for both combinations 9 and 18), and NPV (80.8% for combination 9 and 80.5% for combination 18). Again, specificity decreased but only slightly (92.3% for combination 9 and 92.8% for combination 18). Results were consistent when using 1 DTP as the gold standard.

TABLE 5.

Performance Metrics for All MTM Eligibility Criteria Tested in Overall Study Population Using 2 DTPs as the Gold Standard (N = 2,578,336)

Criteria Combination	Disease Count	Drug Count	Annual Drug Spend ($)	% of Study Cohort	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
2013 MTM criteria	≥ 3	≥ 8	≥ 3,144	7.0	15.3	95.6	51.3	78.8
1	≥ 1	≥ 3	≥ 150	64.9	88.6	42.3	31.8	92.5
2	≥ 1	≥ 6	≥ 150	51.5	79.3	56.9	35.9	90.0
3	≥ 1	≥ 10	≥ 150	28.4	53.5	79.2	43.8	84.9
4	≥ 2	≥ 3	≥ 150	47.2	75.0	61.2	37.0	88.9
5	≥ 2	≥ 6	≥ 150	39.5	67.8	69.1	40.0	87.6
6	≥ 2	≥ 10	≥ 150	23.0	46.6	84.1	47.2	83.8
7	≥ 3	≥ 3	≥ 150	20.6	40.2	85.4	45.5	82.5
8	≥ 3	≥ 6	≥ 150	18.6	37.5	87.1	47.0	82.1
9	≥ 3	≥ 10	≥ 150	12.3	27.7	92.3	52.3	80.8
10	≥ 1	≥ 3	≥ 656	45.2	69.7	62.3	36.0	87.1
11	≥ 1	≥ 6	≥ 656	40.2	65.8	67.6	38.2	86.7
12	≥ 1	≥ 10	≥ 656	25.8	49.2	81.3	44.5	84.1
13	≥ 2	≥ 3	≥ 656	34.6	59.8	73.1	40.3	85.7
14	≥ 2	≥ 6	≥ 656	31.6	56.8	76.0	41.9	85.3
15	≥ 2	≥ 10	≥ 656	21.1	43.1	85.6	47.6	83.2
16	≥ 3	≥ 3	≥ 656	16.8	33.9	88.5	47.2	81.5
17	≥ 3	≥ 6	≥ 656	15.9	32.7	89.2	47.9	81.4
18	≥ 3	≥ 10	≥ 656	11.6	26.0	92.8	52.3	80.5
19	≥ 1	≥ 3	≥ 2,239	23.1	39.7	82.0	40.1	81.7
20	≥ 1	≥ 6	≥ 2,239	21.5	38.4	83.6	41.6	81.7
21	≥ 1	≥ 10	≥ 2,239	15.8	31.6	89.0	46.5	81.1
22	≥ 2	≥ 3	≥ 2,239	18.4	34.6	86.5	43.8	81.3
23	≥ 2	≥ 6	≥ 2,239	17.5	33.6	87.5	44.9	81.3
24	≥ 2	≥ 10	≥ 2,239	13.2	28.0	91.3	49.3	80.7
25	≥ 3	≥ 3	≥ 2,239	10.1	21.2	93.3	49.1	79.6
26	≥ 3	≥ 6	≥ 2,239	9.8	20.8	93.6	49.6	79.5
27	≥ 3	≥ 10	≥ 2,239	7.8	17.8	95.2	53.1	79.2

DTPs = drug therapy problems; MTM = medication therapy management; NPV = negative predictive value; PPV = positive predictive value.

Multivariable Regression Results.

Results from the multivariable negative binomial regression model revealed that the rate of identified DTPs per member was significantly increased among patients with 8 or more drugs (incident rate ratio [IRR] = 2.06, 95% confidence interval [CI] = 2.05-2.07; P < 0.001); 3 or more chronic diseases (IRR = 1.79, 95% CI = 1.78-1.80; P<0.001); and with at least $3,144 in annual drug spend (IRR = 1.18, 95% CI = 1.18-1.19; P<0.001; Table 6). Older patients had significantly increased rates of DTPs per member by 24%-33% relative to patients less than 65 years of age (P < 0.001). Rates of DTPs per member were 8% higher in females relative to males (P < 0.001). Compared to patients residing in the West, patients from other regions, especially the South, had significantly higher rates of DTPs per member (P < 0.001). Patients with higher median household incomes had significantly lower rates of DTPs per member relative to patients with lower median household incomes (IRR = 0.94, 95% CI = 0.94-0.95; P < 0.001]. All race minorities, with the exception of Asians, had significantly higher rates of DTPs per member relative to whites (P < 0.001). Rates of DTPs per member were significantly higher among patients with prior outpatient visits (IRR = 1.44, 95% CI = 1.43-1.45; P < 0.001), ER visits (IRR = 1.28, 95% CI = 1.27-1.28; P < 0.001), and hospitalizations (IRR = 1.23, 95% CI = 1.22-1.24; P < 0.001).

TABLE 6.

Adjusted Results from Multivariable Regression Model Estimating the Relationship Between Risk Factors and Number of Drug Therapy Problems per Patient Using the Overall Study Population (N = 2,578,336)

Variable	Estimated Parameter	Incident Rate Ratio	95% Confidence Interval	P Value
Drug count
≥ 8	0.72	2.06	2.05-2.07	< 0.001
< 8 (reference group)
Disease count
≥ 3	0.58	1.79	1.78-1.80	< 0.001
< 3 (reference group)
Annual drug spend
≥ $3,144	0.17	1.18	1.18-1.19	< 0.001
< $3,144 (reference group)
Age group, years				< 0.001
> 70	0.22	1.24	1.24-1.25
65-70	0.29	1.33	1.33-1.34
< 65 (reference group)
Gender				< 0.001
Female	0.07	1.08	1.07-1.08
Male (reference group)
Region				< 0.001
Midwest	0.01	1.01	1.00-1.01
Northeast	0.02	1.02	1.02-1.03
South	0.20	1.22	1.21-1.22
West (reference group)
Race				< 0.001
African American	0.21	1.23	1.22-1.24
Asian	-0.02	0.98	0.97-0.99
Hispanic	0.10	1.11	1.09-1.12
Native American	0.12	1.13	1.10-1.17
Other	0.02	1.02	1.01-1.03
White (reference group)
Household income				< 0.001
> $52,565	-0.06	0.94	0.94-0.95
Missing	-0.03	0.97	0.96-0.99
≤ $52,565 (reference group)
Prior outpatient visit(s)	0.36	1.44	1.43-1.45	< 0.001
Prior emergency room visit(s), all-cause	0.25	1.28	1.27-1.28	< 0.001
Prior acute inpatient stay, all-cause	0.21	1.23	1.22-1.24	< 0.001

Deviance value = 2532321; Degrees of Freedom (DF) = 2578316; Value/DF = 0.9822; Akaike Information Criterion = 6149537.

Discussion

This study evaluated the performance of CMS MTM identification criteria during the 2013 plan year and 27 alternative identification criteria in a large MAPD patient population. The sensitivity, specificity, PPV, and NPV of 2013 MTM criteria were 15%, 96%, 51%, and 79%, respectively. Both sensitivity and PPV improved when the drug count threshold increased from 8 to 10, and when the annual drug cost decreased from $3,144 to $2,239 or less.

In addition to identifying the number of chronic conditions, number of Part D-covered drugs, and annual Part D drug spend, this study identified additional risk factors significantly associated with a greater number of DTPs. The associations between the number of DTPs and each risk factor were in the direction we expected. Patient age and recent ER or hospitalization visits could be used during MTM identification to prioritize how patients should be targeted for enrollment. The largest IRR was associated with prior outpatient visits, which may be an indicator of patients who are more difficult to manage. Although most patients in this study had prior outpatient visits, this variable would be easy to identify from medical claims data. Most of the other risk factors examined in this study are also readily available in most health care databases.

Improving sensitivity can help to increase the number of patients identified for MTM among those who have more DTPs. Furthermore, increasing PPV is important from the payer’s perspective by increasing the probability that the patients identified for MTM services will have more DTPs. Although the improvements in PPV found in this study were small in the absolute sense, even a 1% increase in PPV could potentially translate to considerable cost savings for health plans due to the large target population size and substantial CMR costs. This study shows that the existing MTM criteria can be improved to increase sensitivity and PPV in order to identify the patients who can benefit most from MTM services.

By using the number of DTPs for each patient as the gold standard, this study provides benchmark performance measures of potential different MTM identification criteria for others to compare against. Understanding how well MTM eligibility criteria perform and how they could be improved, either by adjusting the thresholds or by including additional criteria, is crucial for policymakers when they design MTM criteria every year. Improving the ability to identify patients who would benefit the most from MTM services would not only increase the benefits of the MTM program but would also reduce wasted resources associated with targeting patients who do not need MTM services.

In the future, policymakers and other stakeholders may also consider directly identifying the number of DTPs to select patients for MTM programs, rather than using drug counts and other patient characteristics that are associated with DTPs. The challenge to this approach may be limited resources available to develop and implement complex algorithms linking both pharmacy and medical claims data to electronically identify DTPs, and to standardize and monitor how DTPs are defined across different organizations.

Study strengths include the use of pharmacy and medical claims to automatically identify DTPs and the use of a large MAPD patient population. Baseline patient characteristics of the study population (Table 3) indicate that the MAPD patients included in the study are similar to the overall Medicare population.¹⁵ External validity of the study is further supported by how patients were identified for MTM, as this process is highly representative of those used by the majority of MTM programs in 2013. Thus, the results from this study are generalizable to MTM programs using similar identification processes in a MAPD population. In addition, the methodology used in this study could be applied when MTM criteria are updated in the future.

Limitations

This study used the number of DTPs identified per patient to measure the patient’s potential value for MTM services. Although several types of DTPs were identified in this study, it was not feasible to electronically capture all types of DTPs that potentially could be identified by the pharmacist. To minimize the effect of this limitation on the number of DTPs identified per patient, we included the most prevalent DTP types identified during the 2013 MTM program. In addition, pharmacists, when speaking directly to the patient, are likely to identify more DTPs due to over-the-counter medications or herbal/dietary supplements that are not captured by claims data.

Continuous pharmacy benefits were not required during the 6-month period used to identify DTPs, which may affect the number of DTPs identified in this population. However, the majority of patients were continuously eligible in terms of pharmacy benefits. Also, we recognize that DTPs are an intermediate measure and assumed that more DTPs would lead to greater clinical and economic impact. This analysis assumed that each DTP was of equal importance. Depending on the health care provider or health plan, certain DTPs may be of more importance clinically and/or economically. Nevertheless, nonadherence was well represented in the study and is widely recognized as an important clinical and economic issue.^16-18

Lastly, the impact of the selection of chronic diseases on criteria performance was outside the scope of this study. Future research may consider which of the 9 core chronic conditions would perform the best.

Conclusions

This study measured the performance of 2013 MTM identification criteria and demonstrated that alternative MTM criteria thresholds could be used to improve the identification of valuable patients for MTM services. In particular, eligibility criteria can be improved by increasing the threshold values for drug count while decreasing the threshold value for annual drug spend. Furthermore, findings from this study suggest that other patient demographic or clinical characteristics readily available from most pharmacy and medical claims data, such as recent ER or hospital visits, may also be helpful in identifying MTM patients.