Disparity Considerations in an Alternative Eligibility Criterion for Medicare Medication Therapy Management Programs with Machine Learning

Abstract

Objectives:

Current eligibility criteria for Medicare Medication Therapy Management (MTM) programs may disadvantage racial and ethnic minorities. As health plans increasingly apply predictive modeling to guide MTM eligibility, these tools risk perpetuating existing disparities. This study examined whether hospitalization and emergency room (ER) costs could serve as an alternative eligibility criterion to improve minority representation in MTM enrollment and whether machine learning models reflect racial and ethnic differences.

Methods:

A 10% random sample of fee-for-service Medicare beneficiaries in 2019 was analyzed. The outcome was inclusion in the top quartile of hospitalization and ER costs. Racial and ethnic differences were assessed using a multivariable logistic regression. Five machine learning algorithms (regularized logistic regression, support vector machines, neural networks, random forest, and gradient boosting trees) were developed, with their predictions combined in a consensus model. A multivariable fractional logistic regression was used to test whether predicted probabilities reproduced observed racial and ethnic differences.

Results:

The analytic sample included 1,848,654 beneficiaries. Black beneficiaries were significantly more likely than non-Hispanic White (White) beneficiaries to be in the top cost quartile (odds ratio [OR]=1.09, 95% confidence interval [CI]=1.07–1.12), while Asian (OR=0.66, 95% CI=0.63–0.69) and Other racial groups (OR=0.87, 95% CI=0.85–0.89) were less likely. No significant Hispanic-White difference was observed. Machine learning models largely reproduced these patterns.

Conclusions:

Hospitalization and ER costs represent a potential alternative MTM eligibility criterion that may enhance inclusion of Black beneficiaries but underrepresent other minority groups. This study highlights the importance of equity-focused design in predictive tools.

Précis:

Eligibility criterion for medication therapy management services (MTM) based on hospitalization and emergency room costs may enhance the inclusion of Black beneficiaries in MTM programs.

Introduction

The Medication Therapy Management (MTM) programs were introduced as a key feature of Medicare Part D to improve medication use, reduce adverse events, and enhance health outcomes among high-risk beneficiaries.¹ To identify eligible individuals, the Centers for Medicare and Medicaid Services (CMS) requires Part D plans to apply criteria based on the number of chronic conditions, the number of covered medications, and an annual threshold for drug spending.¹ Although this structure is intended to direct resources toward beneficiaries at the greatest risk of medication-related problems, evidence suggests these criteria may not equitably capture all racial and ethnic groups. Prior studies have shown that Black beneficiaries are significantly less likely than non-Hispanic White (White) beneficiaries to be enrolled in MTM programs, highlighting disparities in meeting eligibility thresholds established by current criteria.^2,3 These disparities raise concerns that MTM programs may reinforce existing barriers to coordinated care for minority populations.^4,5

Despite growing recognition of disparities in MTM eligibility, the policy framework has remained narrowly scoped. Even when the CMS launched the Enhanced MTM demonstration program in 2017 to allow participating health plans greater flexibility in setting eligibility criteria, plans largely continued to rely on measures tied to medication spending and total healthcare expenditures.⁶ This risks overlooking other dimensions of healthcare utilization that may more directly signal opportunities for intervention. In particular, hospitalizations and emergency room (ER) visits account for a substantial share of Medicare spending.⁷ They are highly sensitive to gaps in medication adherence, care coordination, and chronic disease management.⁷ Prior research has shown that minority populations often experience different patterns of hospitalization and ER use compared with White beneficiaries. Black beneficiaries were more likely to rely on inpatient and emergency services due to delayed access to outpatient care.⁸ Yet these expenditures have not been systematically tested as an alternative basis for MTM eligibility. The current eligibility framework may exclude beneficiaries who would derive benefit from medication-related support, particularly minority groups.

A related and timely consideration is the increasing role of predictive algorithms in health plan operations. Many Part D plans already employ machine learning and predictive modeling tools to identify beneficiaries who are likely to meet MTM eligibility thresholds or who may benefit most from outreach.⁶ These approaches risk perpetuating disparities if not explicitly designed with equity considerations. Algorithms trained on historical data risk perpetuating existing racial and ethnic disparities in service utilization and spending, potentially generating predictions that reinforce inequitable access to care.⁹ Evidence from other healthcare settings has shown that algorithms relying on health outcomes or related proxies for need can understate the care requirements of minority populations,¹⁰ raising concerns that similar disparities could emerge when predictive models are applied to MTM eligibility. The CMS has also acknowledged the potential for disparities in artificial intelligence and machine learning applications, accentuating the importance of evaluating how these tools perform in federal health programs.^11–13 Although machine learning strategies have been widely applied in other areas of healthcare,^14,15 there is limited evidence regarding their role in determining MTM eligibility, particularly when alternative criteria, such as hospitalization and ER costs, are considered. Evaluating whether algorithms replicate or amplify disparities observed in real-world data is essential for informing policy and practice, especially given the growing emphasis on health equity and the expanding role of algorithmic decision-making in access to services.

This study seeks to address the current gaps in the literature by examining racial and ethnic differences in the likelihood of qualifying for hospitalization and ER costs as a potential eligibility criterion for MTM programs. Using both regression-based analyses and machine learning methods, this work assesses whether incorporating hospitalization and ER costs into MTM eligibility could enhance the representation of minority beneficiaries and whether any observed racial and ethnic differences are reflected in predictive models. More broadly, the study findings contribute to ongoing policy discussions about refining MTM program design and ensuring that the MTM eligibility framework aligns with the needs of racial and ethnic populations. This work underscores the importance of considering alternative measures beyond conventional MTM criteria to advance Medicare Part D goals and promote more inclusive access to medication management services.

Methods

Data Source

A cross-sectional, retrospective design was used to analyze Medicare beneficiaries in 2019, the last full year before the COVID-19 pandemic. The selection of 2019 as the study year allowed for an evaluation of healthcare use and system characteristics under structural conditions that predated pandemic-era changes. Demographic characteristics, plan enrollment, eligibility for subsidy programs, and mortality information were drawn from the Medicare Master Beneficiary Summary File.¹⁶ These records were linked to the Medicare administrative claims and the Area Health Resource Files (AHRF).¹⁷ The Medicare claims, maintained by the CMS for oversight and reimbursement, provided details on service use and related expenditures.¹⁶ Inpatient encounters were recorded through Part A, while Part B claims captured outpatient visits and physician services. Both claim types included service dates, diagnosis codes, and reported costs from providers. Measures of local socioeconomic conditions and healthcare resources were extracted from the AHRF and linked to beneficiaries by county of residence.

Study sample

A 10 percent systematic random sample of beneficiaries enrolled in the fee-for-service Medicare was selected to create the study cohort for 2019. Eligibility for study inclusion required beneficiaries to be continuously enrolled in Medicare Parts A, B, and D for the full calendar year and to remain alive through the end of that year. These criteria were used to form a more homogeneous study population and ensure completeness of claims information for analysis. Individuals enrolled in Medicare Advantage plans were excluded due to data unavailability. No sampling weights were applied in the analysis because the study cohort was selected based on the last digit “0” in beneficiary identification numbers.

Racial and ethnic groups

Race and ethnicity were categorized into five distinct groups: White, Black, Hispanic, Asian/Pacific Islander (Asian), and Other racial. This classification was derived from the Research Triangle Institute (RTI) race variable used in Medicare data and aligns with the approach adopted by the CMS for assessing health disparities.¹⁸ The Other group combined beneficiaries coded as American Indian/Alaska Native, those with unspecified race, and individuals labeled as “other.” This aggregation choice was informed by literature showing that the RTI race code does not reliably classify American Indian/Alaska Native beneficiaries.¹⁹

Outcomes

The analysis focused on a binary outcome indicating whether a beneficiary’s combined spending on hospitalizations and ER visits ranked in the top quartile of the cost distribution. Beneficiaries in this top quartile were coded as 1 and classified as potentially eligible for MTM services, while those outside this range were coded as 0. This quartile cutoff corresponds with the CMS policy objective that approximately one-quarter of Part D beneficiaries qualify for MTM programs.²⁰ Hospitalization costs were derived from claims for inpatient hospital stays. ER visit costs were computed from claims containing provider-assigned revenue center codes for professional fees-ER (0981) or ER (0450–0459).²¹ All costs were analyzed from the healthcare sector perspective and represented total expenditures from all reported payment sources related to hospital and ER encounters.

Covariates

Selection of predictor variables for empirical analyses was informed by the Gelberg-Andersen behavioral model for vulnerable populations,²² which is widely used in the literature to examine healthcare utilization patterns. This framework organizes determinants of care utilization into three broad domains: predisposing, enabling, and need factors. Predisposing factors describe demographic and contextual attributes that influence healthcare-seeking behavior. Enabling factors represent the environmental or structural circumstances that affect an individual’s ability to obtain services. Need factors indicate the extent of clinical complexity or health status that drives demand for care.

Predisposing variables at the individual level comprised race/ethnicity, gender, age, status for low-income subsidy assistance, and the original basis for Medicare enrollment. County-level predisposing variables captured local sociodemographic context, including income per capita and the percentages of married-couple families, non-White residents, individuals without health insurance, and individuals with a high school education or greater. Enabling variables reflected geographic context and service availability through indicators for metropolitan statistical area (MSA), Health Professional Shortage Area (HPSA), and U.S. census region.

Need variables were represented using four types of measures: the CMS risk adjustment summary score;²³ hierarchical condition categories (HCCs);²⁴ major diagnostic categories derived from diagnosis-related groups (MDC-DRGs);²⁵ and procedural and surgical classifications.²⁶ Information from the International Classification of Diseases, Tenth Revision (ICD-10) diagnosis and procedure codes, as reported in claims for services rendered in the year prior to the study year, was used to construct these variables. Clinical complexity was assessed using the CMS risk score, a metric developed for risk-based reimbursement in Medicare Advantage plans, where lower scores indicate better health status. The classification systems offered complementary views of morbidity and care needs: HCCs summarized chronic disease burden, MDC-DRGs depicted hospital case mix, and the procedure groupings characterized surgical and treatment patterns.

All previously defined variables were used as features in the machine learning models. The regression analyses assessing racial and ethnic differences in the study outcome used a streamlined specification in which the CMS risk score was the sole need-based variable included to account for health status.

Empirical approach and data analysis

The analytic strategy was implemented in two main steps to assess whether a measure of combined hospitalization and ER costs could serve as an alternative eligibility criterion to improve the representation of racial and ethnic minority groups in MTM programs. The first step examined potential racial and ethnic differences in the likelihood of being ranked in the top quartile of total hospitalization and ER costs. Descriptive comparisons and multivariable logistic regression analysis were conducted on the full study sample. Race and ethnicity were coded as indicator variables using White beneficiaries as the reference group. An odds ratio (OR) greater than 1 for a given minority group indicates that beneficiaries in that group are more likely to qualify under this MTM eligibility criterion than White beneficiaries.

To provide additional context for top-quartile inclusion, the proportions of zero costs and magnitudes of costs among individuals with positive costs were also compared across racial and ethnic groups. A chi-square test and a t-test were used to test for differences in the proportions of zero costs and in the magnitudes of positive costs, respectively.

The second step evaluated whether machine learning models replicated racial and ethnic differences observed in the first step. These models were developed and applied to the study data to emulate how Part D plans use predictive algorithms to determine MTM eligibility. To prevent overfitting, the data were randomly split into a 75% training set and a 25% test set. Stratified splitting was performed within each racial and ethnic subgroup to preserve proportional representation. The resulting subsets were aggregated to create the training and test samples. Five algorithms were used to predict inclusion in the top-cost quartile: regularized logistic regression, support vector machines, multi-layer perceptron neural networks, random forest, and gradient boosting trees. These algorithms were selected to represent a diverse set of modeling paradigms, including linear, non-linear, and tree-based ensemble techniques extensively applied in health services research. A consensus model was created by taking the unweighted average of predicted probabilities across all algorithms to obtain a single, ensemble-based prediction of the study outcome. Hyperparameters for each algorithm were initially set to the default values in the machine learning libraries, with repeated adjustments to ensure successful model training and reasonable run times. Final model performance was evaluated on the held-out test set using the area under the receiver operating characteristic curve (AUC-ROC), with 95% confidence intervals (CIs) for this metric estimated through 1,000 bootstrap replicates drawn with replacement from the test data. Finally, a fractional logistic regression model was fitted, using ensemble-predicted probabilities as the dependent variable, to assess whether predictions replicated the racial and ethnic differences identified in the total study sample.

To assess model performance, fairness metrics for the consensus model were also calculated across racial and ethnic groups. The metrics considered were AUC-ROC, calibration difference (the difference between the observed positive rate and the mean predicted probability), demographic parity (positive prediction rate), true positive rate, equal opportunity difference (differences in true positive rates relative to the reference group), false positive rate, and false negative rate.

To provide a better understanding of the contributions of covariates to model predictions, SHapley Additive exPlanations (SHAP) summary plots of the top 25 features for the machine learning models were also produced. Finally, two sensitivity analyses were conducted to assess the robustness of the study findings. First, a two-part model, consisting of a logistic regression model for the likelihood of incurring non-zero costs and a generalized linear model with a gamma distribution and a log link for positive costs, was used to predict costs. Second, to address differences in model specification between the regression and machine learning models, the regression model was re-estimated using the same covariates included in the machine learning analyses.

All analytic work was completed in the Virtual Research Data Center, a secure research platform maintained by the CMS. Predictive modeling was performed in Databricks, and statistical analyses were executed using SAS Enterprise Guide v7.1. Regression models employed county-level clustering of standard errors to account for potential geographic correlation among beneficiaries. Two-sided hypothesis tests were used throughout, with statistical significance set at an alpha level of 0.05.

Results

The study population consisted of 1,848,654 Medicare beneficiaries, with identical racial and ethnic distributions across the training and test samples: White beneficiaries represented 80.56% of the sample, followed by Black (8.03%), Hispanic (5.49%), Asian (2.88%), and Other racial groups (3.04%). The training and test populations were closely aligned in patient characteristics. Table 1 provides an overview of the characteristics of the training sample by race and ethnicity. Compared with White beneficiaries, Black, Hispanic, and Other racial groups were significantly younger (p < .0001), whereas Asian beneficiaries were older on average (p < .0001). The White population had a higher share of male beneficiaries than Black individuals (p < .0001) and a lower share than those in the Other racial group (p < .0001). The distribution of Medicare entitlement reasons also differed by race and ethnicity, with Black and Hispanic beneficiaries having greater representation in disability or end-stage renal disease categories and lower representation in old-age and survivor’s insurance compared with White beneficiaries (p < .0001).

Table 1.

Characteristics of the training population.

Characteristics	Training sample (N=1,386,489)	Non-Hispanic White (n=1,116,901; 80.56%)	Black (n=111,329; 8.03%)	Hispanic (n=76,119; 5.49%)	Asian/Pacific Islander (n=39,939; 2.88%)	Other racial (n=42,201; 3.04%)
Predisposing factors
Age, mean (SD)	72.23 (11.94)	73.05 (11.21)	66.54* (15.01)	69.44* (14.12)	74.43* (11.15)	68.54* (12.02)
Male, n (%)	585,404 (42.22)	468,222 (41.92)	44,874* (40.31)	32,347 (42.50)	16,598 (41.56)	23,363* (55.36)
Original reason for Medicare enrollment, n (%)
Old age and survivor’s insurance	1,046,501 (75.48)	879,276 (78.72)	53,687* (48.22)	46,705* (61.36)	34,136* (85.47)	32,697* (77.48)
Disability insurance benefits (DIB)	330,163 (23.81)	234,229 (20.97)	53,910 (48.42)	27,545 (36.19)	5,379 (13.47)	9,100 (21.56)
End-stage renal disease (ESRD)	5,072 (0.37)	1,762 (0.16)	1,806 (1.62)	1,026 (1.35)	239 (0.60)	239 (0.57)
Both DIB and ESRD	4,753 (0.34)	1,634 (0.15)	1,926 (1.73)	843 (1.11)	185 (0.46)	165 (0.39)
Low-income subsidy, n (%)	398,642 (28.75)	242,284 (21.69)	69,068* (62.04)	51,798* (68.05)	22,808* (57.11)	12,684* (30.06)
Percentage of non-White population, mean (SD)a	34.54 (20.96)	30.91 (19.30)	47.82* (18.26)	55.69* (21.95)	54.85* (18.78)	38.15* (21.58)
Percentage of married-couple families, mean (SD)a	71.40 (6.63)	72.26 (6.10)	65.37* (7.96)	68.40* (6.29)	70.34* (6.24)	71.25* (7.02)
Percentage with education ≥ high school, mean (SD)a	88.25 (5.04)	88.70 (4.69)	86.95* (4.71)	83.87* (7.38)	87.22 (5.22)	88.69 (4.79)
Per capita income (in $1000), mean (SD)a	55.62 (19.72)	54.94 (18.84)	54.55 (19.62)	57.64 (23.17)	70.42* (26.20)	58.91* (22.56)
Percentage of no health insurance, mean (SD)a	10.50 (4.79)	10.35 (4.65)	11.11* (4.52)	12.61* (6.57)	9.16* (4.20)	10.31 (4.83)
Enabling factors
Metropolitan statistical area, n (%)a	1,084,522 (78.22)	850,421 (76.14)	94,586* (84.96)	67,792* (89.06)	38,406* (96.16)	33,317* (78.95)
Health Professional Shortage Area, n (%)a	1,278,285 (92.20)	1,021,134 (91.43)	106,629* (95.78)	73,537* (96.61)	37,747 (94.51)	39,238* (92.98)
Census regions, n (%)
Northeast	271,671 (19.59)	224,749 (20.12)	17,396* (15.63)	13,203* (17.35)	7,086* (17.74)	9,237* (21.89)
Midwest	326,758 (23.57)	285,721 (25.58)	20,649 (18.55)	7,254 (9.53)	4,074 (10.20)	9,060 (21.47)
South	520,090 (37.51)	410,712 (36.77)	64,807 (58.21)	24,633 (32.36)	7,822 (19.58)	12,116 (28.71)
West	267,970 (19.33)	195,719 (17.52)	8,477 (7.61)	31,029 (40.76)	20,957 (52.47)	11,788 (27.93)
Need factor
Risk adjustment summary score, mean (SD)	1.26 (1.26)	1.22 (1.19)	1.56* (1.69)	1.44* (1.50)	1.26 (1.22)	1.08* (1.20)

Abbreviation: SD = standard deviation.

^aIndicates a county-level characteristic.

^*Indicates that characteristic was statistically different from non-Hispanic White patients by pairwise comparison (p < .05). The significance test for the original reason for Medicare enrollment and census regions is for the overall distribution but not category specific.

Pronounced socioeconomic differences were observed across racial and ethnic groups. Low-income subsidy receipt was more prevalent among all minority groups compared with White beneficiaries, with Hispanic and Black beneficiaries showing the highest proportions (p < .0001). Compared with White beneficiaries, Black and Hispanic beneficiaries lived in counties with higher mean percentages of non-White populations (p < .0001) and of residents lacking health insurance (p < .001), and lower mean percentages of married-couple families (p < .0001) and of residents with educational attainment (p < .0001). Black and Hispanic individuals were also more likely to reside in counties designated as MSAs and HPSAs (p < .0001). In contrast, Asian beneficiaries lived in counties with relatively higher per-capita incomes (p < .0001) and lower uninsured rates (p < .01). Census region distributions showed greater concentrations of Black beneficiaries in the South and Asian beneficiaries in the West. For health status, higher mean risk adjustment summary scores were observed among Black and Hispanic beneficiaries (p < .0001), while lower scores were evident in the Other racial group (p < .0001), relative to White beneficiaries.

Table 2 reports the distribution of total hospitalization and ER costs by race and ethnicity among the total sample. Significant differences were observed between the racial and ethnic majority group and each minority group (p < .0001). Black beneficiaries had the highest top-quartile inclusion at 31.97%, 7.49 percentage points higher than White beneficiaries (24.48%). Hispanic individuals also showed a higher percentage (28.35%), exceeding their White counterparts by 3.87 percentage points. In contrast, Asian beneficiaries had the lowest top-quartile inclusion rate at 18.93%, 5.55 percentage points below that of White beneficiaries. The Other racial group accounted for 20.12%, which was 4.36 percentage points below White beneficiaries.

Table 2.

Distribution of hospitalization and ER costs by race/ethnicity in the total study population.

Measure	Total sample	Non-Hispanic White	Black	Hispanic	Asian/Pacific Islander	Other
Top-quartile cost inclusion, n (%)	462,166 (25.00)	364,532 (24.48)	47,461* (31.97)	28,771* (28.35)	10,081* (18.93)	11,321* (20.12)
Zero cost, n (%)	1,206,105 (65.24)	981,663 (65.92)	82,323* (55.46)	62,034* (61.12)	39,658* (74.47)	40,427* (71.85)
Positive costs,
mean (SD)	$12,132.54 ($24,387.43)	$11,652.64 ($22,615.59)	$14,375.63* ($30,904.24)	$13,616.56* ($29,831.74)	$13,766.84* ($29,246.95)	$13,047.47* ($27,630.27)
median (IQR)	$2,770.19 ($483.06–$14,443.62)	$2,857.74 ($482.59–$14,279.33)	$2,373.40* ($480.58–$15,368.59)	$2,313.94* ($494.60–$14,964.68)	$2,961.46 ($512.15–$15,952.03)	$2,550.52* ($471.87–$15,008.66)

Abbreviations: ER = emergency room; SD = standard deviation; IQR = interquartile range.

^*Indicates that the corresponding measure was significantly different from non-Hispanic White (p < .05).

Among the study sample, 65.24% of beneficiaries incurred no hospitalization or ER costs during the study year. Compared with White beneficiaries (65.92%), Black (55.46%) and Hispanic (61.12%) beneficiaries had lower proportions of zero costs, whereas Asian (74.47%) and Other (71.85%) beneficiaries had higher proportions of zero costs. Among beneficiaries with positive costs, mean costs significantly exceeded median costs within each racial and ethnic group, indicating a highly right-skewed cost distribution. Racial and ethnic minority groups all had higher costs than the White beneficiaries among individuals with positive costs. The patterns of differences across racial and ethnic groups in top-quartile inclusion reflect more their differences in the prevalence of zero costs than the magnitudes of costs among individuals with positive costs.

Table 3 presents estimates from the multivariable logistic regression analysis of top-quartile inclusion for total hospitalization and ER costs in the total study sample. Racial and ethnic differences remained prominent after controlling for patient characteristics, although some findings differed from the descriptive results. Black beneficiaries were more likely than White beneficiaries to be in the top cost quartile (OR = 1.09; 95% CI = 1.07–1.12). In contrast, Asian beneficiaries (OR = 0.66; 95% CI = 0.63–0.69) and Other racial group (OR = 0.87; 95% CI = 0.85–0.89) displayed significantly lower likelihood of top-quartile inclusion than their White counterparts. Unlike the descriptive findings, the adjusted likelihood of top-quartile inclusion for Hispanic beneficiaries was comparable to that of White beneficiaries (OR = 0.99; 95% CI = 0.96–1.01).

Table 3.

Adjusted racial and ethnic differences in the likelihood of top-quartile inclusion for hospitalization and ER costs among the total study population.

Characteristics	Odds ratio	95% confidence interval
Predisposing factors
Race/ethnicity
Black	1.09	1.07–1.12
Hispanic	0.99	0.96–1.01
Asians/Pacific Islander	0.66	0.63–0.69
Other racial	0.87	0.85–0.89
Age	1.017	1.016–1.018
Male	0.91	0.90–0.92
Original reason for Medicare enrollment
Disability insurance benefits (DIB)	1.82	1.79–1.85
End-stage renal disease (ESRD)	2.16	2.04–2.29
Both DIB and ESRD	2.06	1.93–2.20
Low-income subsidy	1.27	1.25–1.29
Percentage of non-White populationa	0.9985	0.9977–0.9994
Percentage of married-couple familiesa	0.998	0.996–1.001
Percentage with education ≥ high schoola	1.004	1.002–1.007
Per capita income (in $1,000)a	0.9999	0.9993–1.0005
Percentage of no health insurancea	1.002	0.999–1.004
Enabling Factors
Metropolitan statistical areaa	0.97	0.96–0.99
Health Professional Shortage Areaa	0.996	0.970–1.022
Census regions
Midwest	1.01	0.97–1.04
South	0.98	0.95–1.02
West	1.04	0.99–1.09
Need Factor
Risk adjustment summary score	1.60	1.59–1.62

Abbreviation: ER = emergency room.

Reference groups: non-Hispanic White, female, old age and survivor’s insurance, non-receipt of low-income subsidy, non-metropolitan statistical area, non-Health Professional Shortage Area, northeast census region.

^aCounty-level characteristic.

Table 4 summarizes the AUC-ROC estimates for all machine learning models based on predicted probabilities of top-quartile hospitalization and ER costs. The models demonstrated consistent, moderate discriminative performance: the AUC-ROC values clustered tightly between 0.71 and 0.72, and the narrow confidence intervals indicated stable estimates. The gradient boosting trees achieved the highest AUC-ROC estimate (0.7206; 95% CI = 0.7189–0.7224), closely followed by the consensus model (0.7199; 95% CI = 0.7182–0.7215) and the multi-layer perceptron neural network (0.7194; 95% CI = 0.7178–0.7210). The regularized logistic regression, the random forest, and the support vector machine produced similar estimates, all exceeding 0.71. These findings highlight minimal variability across modeling approaches, with tree-boosting and deep learning methods offering slightly stronger performance.

Table 4.

Predictive performance of machine learning models for inclusion in the top quartile of cost outcome.

Machine learning models	AUC-ROC point estimate	95% confidence interval
Regularized logistic	0.7158	0.7141–0.7175
Support vector machine	0.7133	0.7116–0.7151
Multi-layer perceptron neural network	0.7194	0.7178–0.7210
Random forest	0.7134	0.7117–0.7150
Gradient boosting trees	0.7206	0.7189–0.7224
Consensus	0.7199	0.7182–0.7215

Abbreviation: AUC-ROC=the area under the receiver operating characteristic curve.

Table 5 presents metrics that measure the consensus model’s performance fairness by race and ethnicity. AUC-ROC estimates were similar across subgroups, ranging from 0.7149 among White beneficiaries to 0.7305 among Black beneficiaries. While calibration differences were relatively small across subgroups, other fairness metrics showed notable differences. Black and Hispanic beneficiaries had higher values for demographic parity, true positive rates, and false positive rates than White beneficiaries. Black and Hispanic beneficiaries had lower false negative rates. Conversely, the comparison with White beneficiaries for Asian and Other groups showed an opposite pattern in demographic parity, true positive rates, false positive rates, and false negative rates.

Table 5.

Performance and fairness metrics of the consensus model by race and ethnicity.

Metric	Non-Hispanic White	Black	Hispanic	Asian/Pacific Islander	Other
	Point estimate (95% CI)	Point estimate (95% CI)	Point estimate (95% CI)	Point estimate (95% CI)	Point estimate (95% CI)
AUC-ROC	0.7149 (0.7130–0.7169)	0.7305 (0.7249–0.7362)	0.7279 (0.7209–0.7349)	0.7216 (0.7107–0.7333)	0.7287 (0.7180–0.7388)
Calibration difference	−0.0001 (−0.0015–0.0012)	−0.0064 (−0.0105– −0.0019)	−0.0022 (−0.0073–0.0028)	0.0154 (0.0088–0.0214)	0.0069 (0.0010–0.0131)
Demographic parity	0.2386 (0.2373–0.2400)	0.3882 (0.3832–0.3929)	0.3166 (0.3105–0.3216)	0.1559 (0.1501–0.1627)	0.1550 (0.1494–0.1610)
True positive rate	0.4525 (0.4494–0.4560)	0.6197 (0.6109–0.6284)	0.5493 (0.5373–0.5606)	0.3668 (0.3478–0.3859)	0.3726 (0.3553–0.3905)
Equal opportunity difference*	Reference	0.1672 (0.1578–0.1760)	0.0967 (0.0846–0.1088)	−0.0858 (−0.1041– −0.0662)	−0.0799 (−0.0981– −0.0619)
False positive rate	0.1691 (0.1677–0.1705)	0.2804 (0.2752–0.2858)	0.2243 (0.2184–0.2301)	0.1056 (0.1000–0.1116)	0.0998 (0.0943–0.1053)
False negative rate	0.5475 (0.5440–0.5506)	0.3803 (0.3716–0.3891)	0.4507 (0.4394–0.4627)	0.6332 (0.6141–0.6522)	0.6274 (0.6095–0.6447)

Abbreviations: AUC-ROC = the area under the receiver operating characteristic curve; CI = confidence interval.

^*Equal opportunity difference was calculated as the true positive rate for each racial and ethnic minority group minus the true positive rate for non-Hispanic White beneficiaries.

The SHAP summary plots for each machine learning model are presented in Appendix 1. The risk adjustment summary score, Medicare enrollment due to disability insurance benefits, age, low-income subsidy receipt, several HCCs, and selected MDC-DRGs were among the most influential predictors of inclusion in the top quartile across all models. Racial and ethnic indicators were frequently ranked lower in importance than these variables, indicating that model predictions were primarily driven by underlying clinical and risk factors rather than by race and ethnicity.

Table 6 presents multivariable regression estimates of predicted probabilities of top-quartile hospitalization and ER costs generated from the consensus model on the test set. The consensus model reproduced most of the racial and ethnic patterns observed in the full sample. Black beneficiaries had a significantly higher predicted probability of being in the top cost quartile than White beneficiaries (OR = 1.070; 95% CI = 1.065–1.075). Asian beneficiaries (OR = 0.75; 95% CI = 0.74–0.76) and Other racial group (OR = 0.909; 95% CI = 0.904–0.915) showed significantly lower predicted probabilities of top-quartile inclusion compared with White individuals. An exception was observed among Hispanic beneficiaries, whose predicted probability of top-quartile inclusion was significantly lower (OR = 0.984; 95% CI = 0.978–0.990), despite no significant difference in observed inclusion compared with White beneficiaries.

Table 6.

Racial and ethnic differences in the predicted probabilities of top-quartile inclusion based on the consensus model.

Characteristics	Odds ratio	95% confidence interval
Predisposing factors
Race/ethnicity
Black	1.070	1.065–1.075
Hispanic	0.984	0.978–0.990
Asians/Pacific Islander	0.75	0.74–0.76
Other racial	0.909	0.904–0.915
Age	1.0144	1.0143–1.0146
Male	0.938	0.937–0.940
Original reason for Medicare enrollment
Disability insurance benefits (DIB)	1.624	1.618–1.630
End-stage renal disease (ESRD)	1.72	1.69–1.75
Both DIB and ESRD	1.62	1.58–1.65
Low-income subsidy	1.225	1.221–1.228
Percentage of non-White populationa	0.9986	0.9985–0.9988
Percentage of married-couple familiesa	0.9970	0.9966–0.9974
Percentage with education ≥ high schoola	0.9998	0.9994–1.0003
Per capita income (in $1,000)a	0.9999	0.9997–1.0000
Percentage of no health insurancea	1.0017	1.0012–1.0022
Enabling Factors
Metropolitan statistical areaa	0.991	0.988–0.995
Health Professional Shortage Areaa	0.998	0.994–1.002
Census regions
Midwest	0.999	0.993–1.005
South	0.987	0.981–0.994
West	1.015	1.007–1.022
Need Factor
Risk adjustment summary score	1.526	1.521–1.532

Reference groups: non-Hispanic White, female, old age and survivor’s insurance, non-receipt of low-income subsidy, non-metropolitan statistical area, non-Health Professional Shortage Area, northeast census region.

^aCounty-level characteristic.

Beyond racial and ethnic differences, several other patient characteristics showed statistically significant and consistent relationships with top-quartile hospitalization and ER costs across both observed likelihood and model-predicted probabilities (Tables 3 and 5). Advancing age, receipt of low-income subsidies, and higher risk adjustment summary scores were each associated with a greater likelihood of inclusion in the top quartile. Similarly, qualification for Medicare based on disability, end-stage renal disease, or a combination of both showed a higher top-quartile inclusion rate than qualification based on old-age and survivor’s insurance. By contrast, male gender, higher county percentages of non-White residents, and residence in MSAs were associated with lower inclusion in the top quartile of costs. Finally, sensitivity analyses using a two-part model and considering the same covariates for the machine learning and regression models produced findings similar to the main analysis.

Discussion

This study examined racial and ethnic differences in hospitalization and ER costs as potential eligibility criteria for MTM in a policy-scenario analysis. While prior work has emphasized existing MTM eligibility criteria, this analysis instead considered hospitalization and ER costs, which may offer an alternative pathway for addressing racial and ethnic disparities. Using a large national sample of fee-for-service beneficiaries, differences were evaluated in both top-quartile inclusion based on observed costs and model-predicted probabilities derived from multiple machine learning approaches. The results highlight key considerations for refining data-driven eligibility criteria to ensure that MTM programs more effectively reach underserved populations. The practical value of these findings lies not primarily in developing an operational prediction tool, but in demonstrating that alternative MTM eligibility approaches may alter beneficiary identification patterns associated with existing predictive methods. Additionally, this paper found that alternative MTM eligibility criteria tested would not fully address racial and ethnic disparities. This consideration is increasingly relevant as predictive analytics become more common in Part D plan operations.

The findings indicate that racial and ethnic groups experience hospitalization and ER costs in systematically different ways. Black beneficiaries had a higher likelihood of top-quartile inclusion in hospitalization and ER costs compared to their White counterparts, even after adjusting for patient and contextual characteristics. This persistence could stem from structural barriers that limit access to timely outpatient care, increasing reliance on costly acute settings rather than preventive or coordinated services.^8,27 In contrast, Asian and Other groups were consistently less likely to be ranked in the highest cost quartile than White beneficiaries. This pattern may reflect lower utilization and health expenditures observed among these populations,²⁸ but it also raises the possibility of underuse of necessary emergency services, as suggested in prior studies.^29,30 Results for Hispanic beneficiaries were inconsistent across analytical approaches. This discrepancy reflects differences in analysis strategies (observed top-quartile inclusion versus predicted probabilities) rather than a meaningful difference in the estimated disparities. Differences in findings for different racial and ethnic minorities suggest that adjusting for socioeconomic and health status factors accounts for differences in some groups but not others, highlighting that variations in service utilization may arise through distinct mechanisms across racial and ethnic groups.

From a policy perspective, incorporating hospitalization and ER costs into MTM eligibility criteria would capture a different dimension of risk by identifying beneficiaries with high acute-care needs who may benefit from enhanced care coordination and medication management. The higher inclusion in the top quartile among Black beneficiaries suggests that using hospitalization and ER costs as the basis for eligibility could help address the underrepresentation of this group in MTM enrollment. This is also consistent with the patterns of utilization of these services across racial and ethnic groups reported in previous research.^2,3 By encompassing individuals with substantial acute care demands more directly, this approach may better align MTM outreach with a population historically facing barriers to participation in health initiatives.⁴ However, reliance on this measure alone could result in the omission of some minority groups from program enrollment.

The findings suggest that the equity implications of alternative eligibility criteria may vary across racial and ethnic groups, highlighting the importance of evaluating their effects across all underserved populations rather than assuming uniform benefits for all. These results illustrate the limitations of one-size-fits-all thresholds and the need for more nuanced eligibility approaches that balance recognition of high-cost service users across various service types with equitable identification across racial and ethnic groups. One potential application of these findings would be to allow Part D plans to consider additional measures that capture distinct dimensions of healthcare need, such as hospitalization and ER costs, as a supplemental pathway for identifying beneficiaries who may benefit from MTM services, in addition to existing eligibility measures. The policy relevance of considering such an approach thus extends beyond identifying high-cost beneficiaries. It includes ensuring that MTM eligibility criteria do not systematically exclude racial and ethnic groups that may benefit from these services.

The machine learning analyses further demonstrated that the consensus model largely reproduced the racial and ethnic disparities observed in regression models, indicating that algorithmic approaches can encode and perpetuate existing inequities. Although the models achieved moderate discriminative performance, future gains in model sophistication, predictive accuracy, and access to granular patient-level data could risk further entrenching subgroup differences if equity considerations are not addressed. Much of the predictive strength is derived from incorporating factors that reflect both real-world cost drivers and embedded structural racial and ethnic differences. This duality underscores a broader trade-off between equity and efficiency: models optimized for predictive performance may do so at the expense of equitable subgroup representation. Given that predictive algorithms are already used by Part D plans to guide MTM eligibility, equity-focused design strategies will be essential to ensure that predictive performance does not come at the cost of equitable representation.^31–33 Simultaneously, policymakers may need to establish market-based incentives to encourage more equitable inclusion of minority populations in MTM programs.

The consistent associations for non-racial covariates across both observed and predicted outcomes provide insight into patient and community factors that influence hospitalization and ER costs, offering important context for structuring MTM eligibility criteria to capture clinical and socioeconomic risks more effectively. Advancing age was associated with a greater likelihood of top-quartile inclusion, consistent with the accumulation of chronic conditions and increased demand for acute care among older beneficiaries.^34–36 Medicare eligibility through disability or end-stage renal disease, relative to old-age and survivor’s insurance, was also linked to higher inclusion, reflecting the more intensive service needs of clinically complex populations.³⁷ Male gender was associated with a lower likelihood of top-quartile inclusion, as supported by prior literature showing fewer inpatient stays and emergency department visits among men compared with women.^38,39 Receipt of low-income subsidies showed a positive association with top-quartile inclusion, potentially reflecting both greater treatment needs and socioeconomic challenges that contribute to hospital and ER use.^40,41 Greater inclusion in the top quartile was observed among beneficiaries with higher CMS risk scores, indicating the importance of underlying health status as a determinant of acute care utilization.^42–44 Residence in counties with higher percentages of non-White populations and in MSAs was associated with a lower likelihood of top-quartile costs. These contextual associations may reflect structural and systemic factors, including differences in healthcare-seeking behavior, availability of provider resources, and patterns of care delivery, which influence individuals’ use of medical services and related expenditures.^45–47

Several limitations should be considered when interpreting these findings. First, the analysis relied on a single year of data, which limits the ability to assess temporal variability or to follow longitudinal cost trajectories. Additionally, because the predictive models were trained and tested within a single-year frame, the predictive relationships identified may not fully extend to subsequent years as utilization patterns and healthcare delivery evolve. Second, the outcome was defined as inclusion in the top quartile of hospitalization and ER costs. While this cutoff aligns with the expected CMS policy threshold, alternative definitions, such as the top decile or modeling costs as a continuous variable, could yield different patterns of racial and ethnic representation. Third, the findings are specific to beneficiaries enrolled in fee-for-service Medicare and may not be generalizable to the overall Medicare population. Specifically, although Medicare Advantage enrollees comprise a large share of Medicare beneficiaries, the study excluded them from the analysis. However, the historical persistent racial and ethnic disparities in hospitalization and ER costs mitigate this limitation. Fourth, the study relied on administrative claims and county-level contextual data. Although adjustments were made for available demographic and clinical characteristics, unmeasured influences in social determinants of health, provider networks, and local access barriers may have introduced bias into the results. Furthermore, costs recorded in claims reflect reimbursement records rather than direct measures of resource intensity, which could result in some degree of measurement error. Lastly, although reasonable default hyperparameter settings were used, internal validation, such as grid search and cross-validation, was not conducted due to limited computational resources at VRDC. Although alternative configurations may have yielded different levels of predictive performance, various analytical strategies, including sensitivity analysis, and the consistency of the findings with the historical racial and ethnic disparities in hospitalization and ER costs helped establish the robustness of the findings.

Conclusions

This study explored the feasibility of using hospitalization and ER costs as an alternative criterion for MTM eligibility. The findings indicated that racial and ethnic groups experienced these costs in systematically different ways, with Black beneficiaries more likely to fall in the top quartile than their White counterparts, while Asian and Other racial beneficiaries were less likely. Machine learning models largely reproduced these cost-based inclusion patterns, suggesting that predictive approaches may reinforce observed group differences. Incorporating hospitalization and ER costs into MTM eligibility could expand outreach to patients with substantial acute care utilization, yet relying on this measure alone may overlook beneficiaries whose costs are disproportionately attributable to other areas of care. Future research should assess longitudinal patterns in hospitalization and ER costs, examine how these outcomes interact with MTM eligibility, and test fairness-aware algorithmic strategies to support more inclusive identification of high-need beneficiaries.

Highlights.

• Current evidence suggests that eligibility criteria for Medicare Medication Therapy Management (MTM) programs may disadvantage racial and ethnic minority groups.

• To the authors’ knowledge, this study represents one of the first evaluations to investigate the intersection of hospitalization and ER costs with race/ethnicity in shaping MTM eligibility, while also assessing whether machine learning models reproduce the observed differences.

• Hospitalization and ER costs represent a potential alternative MTM eligibility criterion that may enhance inclusion of high-need Black beneficiaries. However, relying solely on this criterion may underrepresent other minority groups. Machine learning models reinforced observed differences, highlighting the importance of equity-focused design in predictive tools.