Effects of Real-time Prescription Benefit Recommendations on Patient Out-of-Pocket Costs: A Cluster Randomized Clinical Trial

Sunita M Desai; Alan Z Chen; Jiejie Wang; Wei-Yi Chung; Jay Stadelman; Chris Mahoney; Adam Szerencsy; Lisa Anzisi; Ateev Mehrotra; Leora I Horwitz

doi:10.1001/jamainternmed.2022.3946

. 2022 Sep 12;182(11):1129–1137. doi: 10.1001/jamainternmed.2022.3946

Effects of Real-time Prescription Benefit Recommendations on Patient Out-of-Pocket Costs

A Cluster Randomized Clinical Trial

Sunita M Desai ^1,^✉, Alan Z Chen ², Jiejie Wang ¹, Wei-Yi Chung ¹, Jay Stadelman ¹, Chris Mahoney ¹, Adam Szerencsy ¹, Lisa Anzisi ¹, Ateev Mehrotra ^3,⁴, Leora I Horwitz ¹

¹Department of Population Health, NYU Grossman School of Medicine, New York, New York

²Oscar Health, New York, New York

³Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts

⁴Department of Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts

Accepted for Publication: July 18, 2022.

Published Online: September 12, 2022. doi:10.1001/jamainternmed.2022.3946

^✉

Corresponding Author: Sunita M. Desai, PhD, Department of Population Health, NYU Grossman School of Medicine, 227 E 30th St, New York, NY 10016 (sunita.desai@nyu.edu).

Author Contributions: Dr Desai had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Desai, Mahoney, Szerencsy, Anzisi, Mehrotra, Horwitz.

Acquisition, analysis, or interpretation of data: Desai, Chen, Wang, Chung, Stadelman, Mahoney, Szerencsy, Anzisi, Horwitz.

Drafting of the manuscript: Desai, Chen, Anzisi.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Desai, Chen, Wang, Chung.

Obtained funding: Horwitz.

Administrative, technical, or material support: Desai, Stadelman, Mahoney, Szerencsy, Mehrotra, Horwitz.

Supervision: Desai, Szerencsy, Horwitz.

Other—Pharmacist consultant with meds and plan benefit: Anzisi.

Other—EHR data extraction: Chung.

Conflict of Interest Disclosures: Dr Mehrotra reported support from from Black Opal Ventures Health outside the submitted work. No other disclosures were reported.

Funding/Support: Dr Desai is supported by a grant from the Agency for Healthcare Research and Quality (No. K01HS026980).

Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Data Sharing Statement: See Supplement 3.

^✉

Corresponding author.

PMCID: PMC9468947 PMID: 36094537

Key Points

Question

Do real-time prescription benefit (RTPB) recommendations for clinically appropriate, lower-cost alternatives at the point of prescribing reduce patient medication out-of-pocket costs?

Findings

This cluster randomized clinical trial found that RTPB recommendations led to a 11% reduction in patient out-of-pocket costs for ordered medications. Among high-cost drug classes, the intervention led to a 40% reduction in out-of-pocket costs; however, RTPB recommendations were made for only a small proportion of orders.

Meaning

These findings indicate that medication price transparency solutions that target patient-specific, real-time out-of-pocket cost information to the prescriber and recommend clinically appropriate alternatives can generate savings for patients.

Abstract

Importance

Rising drug costs contribute to medication nonadherence and adverse health outcomes. Real-time prescription benefit (RTPB) systems present prescribers with patient-specific out-of-pocket cost estimates and recommend lower-cost, clinically appropriate alternatives at the point of prescribing.

Objective

To investigate whether RTPB recommendations lead to reduced patient out-of-pocket costs for medications.

Design, Setting, and Participants

In this cluster randomized trial, medical practices in a large, urban academic health system were randomly assigned to RTPB recommendations from January 13 to July 31, 2021. Participants were adult patients receiving outpatient prescriptions during the study period. The analysis was limited to prescriptions for which RTPB could recommend an available alternative. Electronic health record data were used to analyze the intervention’s effects on prescribing. Data analyses were performed from August 20, 2021, to June 8, 2022.

Interventions

When a prescription was initiated in the electronic health record, the RTPB system recommended available lower-cost, clinically appropriate alternatives for a different medication, length of prescription, and/or choice of pharmacy. The prescriber could select either the initiated order or one of the recommended options.

Main Outcomes and Measures

Patient out-of-pocket cost for a prescription. Secondary outcomes were whether a mail-order prescription and a 90-day supply were ordered.

Results

Of 867 757 outpatient prescriptions at randomized practices, 36 419 (4.2%) met the inclusion criteria of having an available alternative. Out-of-pocket costs were $39.90 for a 30-day supply in the intervention group and $67.80 for a 30-day supply in the control group. The intervention led to an adjusted 11.2%; (95% CI, −15.7% to −6.4%) reduction in out-of-pocket costs. Mail-order pharmacy use was 9.6% and 7.6% in the intervention and control groups, respectively (adjusted 1.9 percentage point increase; 95% CI, 0.9 to 3.0). Rates of 90-day supply were not different. In high-cost drug classes, the intervention reduced out-of-pocket costs by 38.9%; 95% CI, −47.6% to −28.7%.

Conclusions and Relevance

This cluster randomized clinical trial showed that RTPB recommendations led to lower patient out-of-pocket costs, with the largest savings occurring for high-cost medications. However, RTPB recommendations were made for only a small percentage of prescriptions.

Trial Registration

ClinicalTrials.gov Identifier: NCT04940988; American Economic Association Registry: AEARCTR-0006909

This cluster randomized trial of medical practices in a large urban academic health system investigates how real-time prescription benefit recommendations affect out-of-pocket costs for medications.

Introduction

Rising drug costs are placing more financial burden on patients.^1,2,3 High out-of-pocket costs contribute to decreased medication adherence, poor health outcomes, and higher overall health care spending.^4,5,6,7,8,9 In many cases, patients could save if they used a mail-order pharmacy, received a longer days’ supply, or were prescribed another clinically appropriate medication.^{10,11,12,13,14} However, patients and prescribers have typically lacked the information necessary at the point of prescribing to identify and prescribe these cost-saving alternatives.^15,16

Real-time prescription benefit (RTPB) tools are integrated into the electronic health record (EHR) order entry system and workflow so that when medication orders are initiated, recommendations are made for clinically appropriate alternatives with lower out-of-pocket costs^17,18 These out-of-pocket cost estimates are based on the specific patient’s benefit design (accounting for formularies, co-pays, coinsurance, and deductibles) and reflect the patient’s contribution, rather than a total or list price. Therefore, the recommendations give prescribers the patient’s actual expected costs.

There has been growing adoption of RTPB tools by clinicians, driven in part by health plan incentives.^17,19 The Centers for Medicare & Medicaid Services implemented a rule requiring Medicare Part D plans to integrate with at least 1 RTPB system.²⁰

Prior interventions incorporating price data in prescriber order entry systems have had mixed effects, although the interventions have differed from RTPB recommendations in key ways.^21,22,23,24 First, previous interventions provided recommendations based on list prices or total costs inclusive of the insurer payment, while RTPB tools base recommendations on patient-specific out-of-pocket costs. Moreover, previous interventions made recommendations for a few select medications or services, while RTPB recommends available alternatives across drug classes. To study the effects of RTPB recommendations, we conducted a cluster randomized clinical trial across outpatient practices within a large, urban, academic health system.

Methods

This cluster randomized clinical trial was part of a quality improvement program at NYU Langone Health, and therefore, was not subject to institutional review board review. The study followed the CONSORT reporting guideline.

Trial Design

This study used a stratified, clustered design with balanced randomization of outpatient practice profiles at NYU Langone Health. Within the health system’s EHR (Epic Systems), every practice is assigned an EHR practice profile. Randomization was done at the practice profile level because this was the lowest grouping at which the RTPB system settings could be customized to intervention (RTPB recommendations) or control (no recommendations). Practitioners in the same profile shared a specialty. Therefore, we stratified randomization by profile specialty because volumes, drug costs, and opportunities for savings vary by specialty. We used the term practice to refer to a given practice profile, recognizing that a given practice location may include multiple clinical locations and departments within a multispecialty practice location can be in different practice profiles. The trial protocol and prespecified analysis plan as well as minor departures from it are available in Supplement 1.

Intervention

The intervention consisted of Surescripts RTPB point-of-prescribing recommendations for clinically appropriate alternatives with daily out-of-pocket costs that were lower than the initiated medication order (eFigure 1 in Supplement 2 for an example recommendation). Recommendations were specific to the patient’s prescription benefit design. Up to 3 alternatives were recommended through pop-up notifications at the point of prescribing.

To limit the frequency of notifications to cases where savings were meaningful, we limited recommendations to orders with (1) patient cost-sharing of at least $5.00 per prescription filled and (2) available alternatives that offered minimum out-of-pocket savings of $0.10 per day (or $3.00 for a 30-day supply). Recommendations were only available if patient and plan information in Epic could be successfully linked to the Surescripts database.

Regarding the prescription initiated, recommended alternatives could be (1) an alternate medication considered by the patient’s prescription benefit manager to be a clinically appropriate substitute, (2) a mail-order prescription that typically incurs lower out-of-pocket costs compared with retail or specialty pharmacy fulfillments; and/or (3) longer days’ supply to benefit from quantity discounts. Prescription benefit managers predetermine alternatives that can be recommended for each medication. Therefore, whether and which recommendations are made for the same medication can differ based on a patient’s drug benefit plan. The NYU Langone’s ordering system automatically reverts to the generic version and RTPB recommendations were relative to the generic unless the prescriber explicitly requested the branded version through a dispense as written request.

Prescribers were informed by email about the introduction of the RTPB tool prior to the trial (eFigure 2 in Supplement 2), but they were not informed about their practice’s assignment to the intervention or control group. The intervention period was from January 13 to July 31, 2021. The planned intervention start date was November 19, 2021, but a technical issue delayed the intervention (eMethods in Supplement 2).

Data Sources

We collected data from the EHR on medication orders at randomized practices and identified orders for which the patient and their health plan information could be linked to the Surescripts database. The study data contained prescription-level information on the medication ordered, patient out-of-pocket cost, drug class, days’ supply, and the pharmacy type (mail order, retail, or specialty). For each ordered medication, we also observed this detailed information (ie, out-of-pocket cost estimate, drug class, days’ supply, and pharmacy type) for all potential alternatives associated with the ordered medication, regardless of whether the alternatives had higher or lower out-of-pocket costs compared with the ordered medication.

An important caveat is that the system did not reliably retain information on medications that were not ordered. If a prescriber initiated an order, saw a RTPB recommendation, and changed the order, we could not observe that a previous order had been initiated. Therefore, we could not determine how prescribers responded to individual alerts; analyses were performed on ordered prescriptions.

Demographic information collected included patient age, sex, zip code of residence, and medical insurance type (Medicare, Medicaid, private, uninsured, or other). We merged zip code-level median income from the US Census based on each patient’s zip code of residence.

Analytic Sample

The analytic sample was adult medication orders that could trigger a recommendation. Specifically, aligned with the system settings, these were medication orders for which the minimum out-of-pocket cost difference between available options for that order was at least $0.10 per day (or $3.00 for a 30-day supply) and the total out-of-pocket cost for the most expensive option was $5.00 or more. To avoid selection bias, the sample was based on out-of-pocket costs for all available options, using data on alternatives associated with each prescription rather than only the prescribed drug.

Power calculations indicated that a sample of 28 221 orders would have 80% statistical power to detect an intervention effect of 10% on out-of-pocket costs at a 5% significance level. Using pilot data, we determined that the intervention should have a duration of 150 or more business days to ensure a sufficient sample size (eMethods in Supplement 2).

Primary and Secondary Outcomes

The primary outcome was the patient-specific out-of-pocket cost for the order. Given the skewed distribution of out-of-pocket cost, and consistent with prior literature, out-of-pocket costs were log-transformed after adding $0.01 to orders without out-of-pocket costs.^25,26 Secondary outcomes were whether a mail-order pharmacy was used and whether a 90-day (instead of 30-day) supply was ordered.

Statistical Analysis

We estimated linear regression models to evaluate the effects of the intervention at the medication order-level. Models included an indicator for whether the order was placed in a practice assigned to the intervention group, patient-level factors (age, sex, and insurance type), medication class fixed-effects to account for cost differences across classes, and specialty fixed-effects to account for differences in the volume of orders across specialties, owing to differing cluster sizes. Standard errors were clustered at the level of randomization (ie, the practice).²⁷ Consistent with the prespecified analysis plan, we reported 95% CIs without P values and did not adjust for multiple comparisons.

We conducted several robustness tests. First, given the large number of orders with no patient out-of-pocket cost, we estimated a 2-part model for the primary outcome, testing the intervention’s effects on (1) the probability of nonzero out-of-pocket cost and (2) out-of-pocket cost among orders with positive out-of-pocket cost. Second, we estimated a logistic model for binary outcomes. Third, we investigated the degree to which the intervention could be causing prescribers to switch medication classes by stratifying orders based on whether any alternatives associated with an order were in a different medication class.

Statistical tests were 2-tailed and P values < .05 were considered statistically significant. Data analyses were performed from August 20, 2021, to June 8, 2022, using R, version 4.1.1 (The R Foundation for Statistical Computing).

Subgroup Analyses

We conducted 3 prespecified subgroup analyses. First, we stratified the sample based on the potential savings from selecting an alternative, which we defined as the cost difference between the highest and lowest out-of-pocket cost option associated with each order (regardless of the out-of-pocket cost for the ordered prescription). Second, to examine the intervention’s effect in high- vs low-cost drug classes, we calculated the average out-of-pocket cost in each drug class represented in the sample (using control group data). We divided drug classes into quartiles and stratified orders based on their corresponding drug class. Third, we stratified analyses based on the median household income in the patient’s zip code of residence. In posthoc analyses, we examined effects separately by insurance type, age category, sex, number of prescriptions for the patient, and specialty (eMethods in Supplement 2).

Results

Of the 867 757 outpatient medication orders placed during the trial period, the analytic sample consisted of 36 419 (4.2%) orders. Prescriptions in the analytic sample accounted for 16% of out-of-pocket cost spending among all orders with out-of-pocket information available and 85% of out-of-pocket costs among orders with alternatives.

Major exclusions from the sample were orders for which patient and/or plan information could not be linked to the RTPB database (88 439 [19.0%] intervention orders; 87 087 [21.6%] control orders) and orders with no alternatives (289 646 [65.3%] intervention orders; 263 488 [65.3%] control orders; Figure 1).

Figure 1. — Data were drawn from the electronic health record database. RTPB refers to real-time prescription benefit.

Of prescriptions in the analytic sample, 21 565 were placed for 15 040 patients by 1180 prescribers at the 97 intervention practices and 14 854 prescriptions were placed for 10 682 patients by 988 prescribers at 99 control practices (eTable 1 in Supplement 2). Among orders in the analytic sample, 57.3% were for Medicare enrollees, 39% for privately insured patients, and 0.2% for Medicaid enrollees (Table 1). As anticipated, there were some imbalances on specialty owing to differences in the size of practices across the health system, underscoring the need to control for specialty and medication class in the analysis. Each order in the sample was assigned to 1 of 257 drug classes; the most common drug classes were statins (7.4%), beta-adrenergic blocking agents (6.8%), thyroid hormones (6.8%), selective serotonin reuptake inhibitors (4.5%), and proton-pump inhibitors (4.2%; eTable 2 in Supplement 2). Most orders (86%) had at least 1 alternative with a mail-order supply, 46% had an alternative with a 90-day supply, and 50% had an alternative for a different medication. Using orders with 1 type of alternative, average potential savings from switching medications was $234.60 per 30-day supply vs $8.70 per 30-day supply from switching to a mail-order pharmacy (eTable 3 in Supplement 2). There was variation in whether alternatives were available across orders placed for the same medication (eTable 4 in Supplement 2); eTable 5 in Supplement 2 compares outpatient prescriptions in the analytic sample to all orders placed at the health system during the study period. Compared with all orders, orders in our analytic sample were less likely to be for Medicaid patients, who face minimal out-of-pocket costs.

Table 1. Baseline Characteristics of Study Participants and Medication Orders^a.

Characteristic	No. (%)
Characteristic	All	Intervention	Control	SMD
Medication orders	36 419 (100)	21 565 (59)	14 854 (41)	NA
Patients	25 113 (100)	15 040 (60)	10 682 (43)	NA
Prescribers	2031 (100)	1180 (58)	988 (49)	NA
Clinical sites	618 (100)	334 (54)	284 (46)	NA
Practice profiles	196 (100)	97 (49)	99 (51)	NA
Patient characteristics
Female	21 282 (58.4)	12 189 (56.5)	9093 (61.2)	0.1
Male	15 137 (41.6)	9376 (43.5)	5761 (38.8)	0.1
Age category, y
18-40	4593 (12.6)	2894 (13.4)	1699 (11.4)	0.2
>40-65	10 204 (28.0)	5887 (27.3)	4317 (29.1)	0
>65	21 622 (59.4)	12 784 (59.3)	8838 (59.5)	0
Insurance type
Private	14 204 (39.0)	8470 (39.3)	5734 (38.6)	0
Medicaid	57 (0.2)	30 (0.1)	27 (0.2)	0
Medicare	20871 (57.3)	1267 (56.9)	8604 (57.9)	0
Other	1287 (3.5)	798 (3.7)	489 (3.3)	0
Zip code-level median household income, mean (SD), $	100 659 (35 794)	99 835 (36 189)	101 861 (35 176)	0.1
Prescriber specialty group
Primary care	12 672 (34.8)	7886 (36.6)	4787 (32.2)	0.1
Cardiology	6763 (18.6)	5547 (25.7)	1216 (8.2)	0.5
Neurology	2659 (7.3)	902 (4.2)	1757 (11.8)	0.3
Surgery	1369 (3.8)	871 (4.0)	498 (3.4)	0
Endocrinology	2324 (6.4)	1537 (7.1)	787 (5.3)	0.1
Other	10 631 (29.1)	4822 (22.4)	5809 (39.1)	0.4

Open in a new tab

Abbreviation: SMD, standard mean difference.

^{^a}

Data on outpatient prescriptions ordered within the NYU Langone Health system from January 13 to July 30, 2021, were drawn from the electronic health record database. Patient and prescriber characteristics were weighted by number of orders. Percentages may not sum to 100 because of rounding and because some patients and prescribers had multiple medication orders with differing exposure to the intervention.

Effects of RTPB Recommendations

Across orders in the analytic sample, out-of-pocket cost adjusted for 30-day supply was $39.90 ($1.33/d) in the intervention and $67.80 ($2.26/d) in the control (Table 2). Controlling for patient factors, medication class, and practice specialty, the intervention led to a 11.2% (95% CI, −15.7% to −6.4%) decrease in out-of-pocket costs (eTable 6 in Supplement 2). In the intervention, 9.6% of orders were mail-order compared with 7.6% in the control (adjusted increase, 1.9 percentage points [pp]; 95% CI, 0.9 to 3.0). Differences in the proportion of orders for 90-day supply were not statistically significant (intervention, 47.8%; control, 40.5%; adjusted difference, 2.0 pp; 95% CI, −2.5 to −6.7). Two-part model estimates indicate that the intervention led to a 1 pp (95% CI, −0.6 pp to −1.5 pp) decrease in the likelihood of positive out-of-pocket cost and a 7.9% (95% CI, −10.0% to −5.7%) decrease in out-of-pocket costs conditional on nonzero out-of-pocket costs (Table 3; eTable 7 in Supplement 2). Logistic model estimates were consistent with the main results (eTable 8 in Supplement 2). There were few orders with alternatives in different medication classes, and the intervention’s effect did not differ substantially based on whether alternatives in different medication classes were available (eTable 9 in Supplement 2).

Table 2. Effects of RTPB Recommendations on Outcomes^a.

Outcome	Mean		Intervention effect estimate, % (95% CI)^b
Outcome	Intervention	Control	Unadjusted^c	Adjusted^d
Out-of-pocket cost, $
30-d Adjusted, $^e	39.90	67.80	NA	NA
Log-transformed^f	−0.76	0.54	−19.8% (−33.7 to −3.1)	−11.2% (−15.7 to −6.4)
Mail-order pharmacy, %	9.6	7.6	2.0 (0.3 to 3.8)	1.9 (0.9 to 3.0)
90-d Supply, %^g	47.8	40.5	7.2 (−4.5 to 20.3)	2.0 (−2.5 to 6.7)

Open in a new tab

Abbreviations: NA, not applicable; RTPB, real-time prescription benefits.

^{^a}

Order-level outcomes of interest were the log-transformed out-of-pocket cost, whether the order was filled by a mail-order pharmacy and whether the order was for a 90-day vs 30-day supply.

^{^b}

Confidence intervals are based on heteroskedasticity-robust standard errors clustered at the level of randomization (the practice profile).

^{^c}

Estimated via linear regression of the outcome on an indicator for the intervention group with no other covariates.

^{^d}

Estimated with linear regression including the covariates: indicators for specialty type, indicators for drug pharmaceutical class, categorical age bins (18-40 y; >40-65 y; >65 y), sex, and insurance type (private/Medicaid/Medicare/other).

^{^e}

Calculated as the daily out-of-pocket cost ×30.

^{^f}

Accounts for the skewed distribution, and the corresponding effect estimate has been transformed to indicate the percent change implied by the coefficient estimate ([exp(coefficient)−1] ×100).

^{^g}

Includes 32 485 orders for a 30- or 90-day supply (89.2% of the full analytic sample).

Table 3. Two-Part Model Regression Estimates for Effects of RTPB Recommendations on Order-Level Out-of-Pocket Costs^a.

Outcome	Mean		Intervention effect estimate, % (95% CI)
Outcome	Intervention	Control	Unadjusted^b	Adjusted^c
Out-of-pocket cost >$0.00, %	97.4	98.6	−1.1 (−0.4 to −1.8)	−1.0 (−0.57 to −1.5)
30-d Adjusted out-of-pocket cost, $^d	40.80	69.00	NA	NA
30-d Adjusted out-of-pocket cost, $^d	−0.66	−0.48	−16.7 (−18.8 to −14.5)	−7.9 (−10.0 to −5.7)

Open in a new tab

Abbreviations: NA, not applicable; RTPB, real-time prescription benefits.

^{^a}

Using logistic regression, the first part estimated the probability of an order having >$0 out-of-pocket cost; using linear regression, the second part estimated the log transformed out-of-pocket cost for orders with nonzero cost. Confidence intervals were based on robust standard errors clustered at the level of randomization.

^{^b}

The unadjusted intervention effect for both parts included only an indicator for the intervention.

^{^c}

The adjusted intervention for both models included the following covariates: indicators for specialty type, drug pharmaceutical class, indicators for age category bins (18-40 y; >40-65 y; >65 y), binary sex, and insurance type.

^{^d}

For 35 652 orders with out-of-pocket cost >$0.

Stratified Results

Stratified results are reported in Figure 2. The intervention lowered out-of-pocket costs regardless of out-of-pocket cost differences between alternatives, although effects were largest for medications with the greatest price difference between alternatives (−16.2%; 95% CI, −20.6% to −11.7%); mail-order prescriptions increased in all quartiles, but were most prominent in the second (1.8 pp; 95% CI, 0.6 to 2.9) and third (4.3 pp; 95% CI, 2.8 to 5.7) quartiles. Effects on 90-day supply were similar across quartiles.

For analyses stratified by medication class cost, the top 10 drug classes by frequency in each quartile are in eTable 10 in Supplement 2. The intervention’s effects on out-of-pocket costs were largest (−38.9%; 95% CI, −47.6% to 28.7%) in the highest out-of-pocket cost drug classes (>$106.50 for a 30-day supply).

The intervention’s effects on out-of-pocket costs were similar across income groups. Increases in mail-order pharmacy use were seen across all drug classes, although precision and magnitude differed. Mail-order pharmacy use increased the most for the highest (2.8 pp; 95% CI, 1.4-4.1) and lowest income groups (2.9 pp; 95% CI, 1.4-4.4). Intervention effects on 90-day supply orders were similar across most subgroups, with 1 notable exception: increases in 90-day supply order among patients in the lowest-income communities (7.4 pp; 95% CI, 4.8-10.0). Results for additional heterogeneity analyses are shown in eFigure 3 in Supplement 2.

Discussion

The RTPB recommendations in this randomized clinical trial led to a 11.2% reduction in medication out-of-pocket costs and a 1.9% increase in mail-order prescriptions. For high-cost drug classes, the out-of-pocket cost reduction was almost 40%. However, fewer than 5% of prescriptions placed during the study period were eligible for RTPB recommendations and analyzed for this study.

Given limitations in the data, we were not able to directly estimate the degree to which different types of changes in prescribing contributed to the observed savings. However, several factors suggest that most of the intervention’s effect on out-of-pocket costs was driven by prescribers switching medications within the drug class vs greater use of generics, switching of drug classes, or switching to mail-order prescriptions. The average potential savings from switching to alternative medications were much larger than the average savings from switching to a mail order prescription. Moreover, RTPB recommendations prompted only a small (2%) increase in mail-order prescriptions. In these data, only a small proportion of orders had any alternatives from different medication classes available. Finally, while we could not directly measure use of generics, we could capture when an order is written as dispense as written indicating that the clinician likely explicitly ordered a branded medication. The rates of dispense as written orders were low in general and not lower in the intervention group (intervention, 8.7%; control, 8.0%).

Although most price transparency tools to date have not lowered health care spending, RTPB may overcome limitations of previous efforts.^{28,29,30,31,32,33,34,35} Instead of making price information available to patients through consumer-facing portals, RTPB recommends lower-cost alternatives to prescribers. Moreover, RTPB systems’ out-of-pocket cost information may be more actionable because it is presented at the point of prescribing. In particular, targeting the information to prescribers at the point of care may enable cost of care conversations in which both cost and clinical factors are balanced, and shared decision-making can occur.^36,37,38 Also, unlike existing price transparency tools that merely display price information, RTPB actively suggests alternatives, potentially addressing inertia and other barriers to selecting a lower-cost alternative.

In addition to reducing financial burden, RTPB may improve health outcomes. Many patients report forgoing necessary medications because of high out-of-pocket costs, and medication nonadherence adversely affects health outcomes across many conditions.⁸ To the extent that medication adherence also reduces avoidable emergency department visits and hospitalizations, RTPB could lower nondrug spending too.⁴ However, patient savings across all orders were limited because recommendations were made for less than 5% of orders.

For many orders, patient information could not be matched with the Surescripts database. Efforts to increase matches between the EHR and RTPB database could increase the frequency of recommendations. Measures to improve the match rates could include more complete and accurate completion of patient and pharmacy benefit manager (PBM) information in the EHR by practices, optimization of the matching algorithms, and more PBMs contributing their enrollees’ benefit design information to the RTPB database. Second, more than 60% of orders did not have alternatives or the savings offered by alternatives were small. Our study data suggest that there is variation in whether a medication is considered to have alternatives, suggesting that some PBMs make recommendations more conservatively than others. Savings could be broader if more alternatives were recommended.

Limitations

This study had several limitations. First, we could not examine the effects of the intervention on total drug spending, inclusive of the insurer’s payments, because most payers did not contribute drug cost information to Surescripts. However, given that out-of-pocket costs and total costs are typically correlated, these results suggest that the intervention likely reduced total spending. Second, we did not investigate whether the intervention led to improved medication adherence and health outcomes. Third, we could not observe patient benefit design, and any imbalances in out-of-pocket cost responsibilities (eg, the proportion of patients in high-deductible health plans) could bias our estimates. However, we do not expect differences in plan design, given insurance type and other patient characteristics were similar for the intervention and control groups. Fourth, we could not validate out-of-pocket cost estimates. However, a previous comparison of claims data from 1 insurer and the Surescripts database suggests a relatively high degree of concordance, although it did not examine out-of-pocket costs specifically.¹ Fifth, our study evaluated the Surescripts RTPB system and was conducted among practices at a single large academic center. Effects could differ in other settings or for other RTPB systems. However, the patient population served by the system is large and diverse, and as the largest e-prescribing network accounting for 80% of e-prescriptions, the Surescripts system may be of broad interest.

Conclusions

This cluster randomized clinical trial indicates that RTPB recommendations can facilitate lower-cost prescribing. Future research should examine whether RTPB improves medication adherence and health.

Supplement 1.

Trial Protocol

Click here for additional data file.^{(234.6KB, pdf)}

Supplement 2.

eMethods

eFigure 1. Example of screenshot of a RTPB recommendation

eFigure 2. Informational email sent to all outpatient physicians regarding RTPB recommendations

eFigure 3. Subgroup analyses by patient factors and specialty: Effects of RTPB recommendations on patient out-of-pocket costs

eTable 1. Number of practices, prescribers, and medication orders using pre-trial volumes in intervention and control practice profiles

eTable 2. Percent of orders by drug class for 15 most common drug classes

eTable 3. Types of alternatives and maximum out-of-pocket cost variation (“potential savings”) by alternative type across orders

eTable 4. Proportion of orders with alternatives by medication (for 25 highest volume medications)

eTable 5. Comparison of all outpatient orders in NYU Langone system versus analytic sample during study period

eTable 6. Model specifications: Sequential inclusion of covariates

eTable 7. Most common medication classes with $0 out-of-pocket cost orders

eTable 8. Logistic regression model estimates for secondary outcomes: Odds ratios and marginal effects

eTable 9. Sensitivity analyses: Stratification by whether an order and its alternatives were in distinct medication classes

eTable 10. Ten most frequent drug classes by quartiles based on average drug class out-of-pocket cost

Click here for additional data file.^{(1.1MB, pdf)}

Supplement 3.

Data Sharing Statement

Click here for additional data file.^{(13.1KB, pdf)}

References

1.Tichy EM, Schumock GT, Hoffman JM, et al. National trends in prescription drug expenditures and projections for 2020. Am J Health Syst Pharm. 2020;77(15):1213-1230. doi: 10.1093/ajhp/zxaa116 [DOI] [PubMed] [Google Scholar]
2.Conti RM, Turner A, Hughes-Cromwick P. Projections of US prescription drug spending and key policy implications. JAMA Health Forum. 2021;2(1):e201613(. R. doi: 10.1001/jamahealthforum.2020.1613 [DOI] [PubMed] [Google Scholar]
3.Rome BN, Feldman WB, Desai RJ, Kesselheim AS. Correlation between changes in brand-name drug prices and patient out-of-pocket costs. JAMA Netw Open. 2021;4(5):e218816. doi: 10.1001/jamanetworkopen.2021.8816 [DOI] [PMC free article] [PubMed] [Google Scholar]
4.Roebuck MC, Liberman JN, Gemmill-Toyama M, Brennan TA. Medication adherence leads to lower health care use and costs despite increased drug spending. Health Aff (Millwood). 2011;30(1):91-99. doi: 10.1377/hlthaff.2009.1087 [DOI] [PubMed] [Google Scholar]
5.Gibson TB, Ozminkowski RJ, Goetzel RZ. The effects of prescription drug cost sharing: a review of the evidence. Am J Manag Care. 2005;11(11):730-740. [PubMed] [Google Scholar]
6.Heaton PC, Tundia NL, Luder HRUS. US emergency departments visits resulting from poor medication adherence: 2005-07. J Am Pharm Assoc (2003). 2013;53(5):513-519. doi: 10.1331/JAPhA.2013.12213 [DOI] [PubMed] [Google Scholar]
7.Eaddy MT, Cook CL, O’Day K, Burch SP, Cantrell CR. How patient cost-sharing trends affect adherence and outcomes: a literature review. P T. 2012;37(1):45-55. [PMC free article] [PubMed] [Google Scholar]
8.Chandra A, Flack E, Obermeyer Z, Kennedy JF, School HK. The Health Costs of Cost-Sharing. National Bureau of Economic Research; 2021. Accessed October 17, 2022. doi: 10.3386/w28439 [DOI]
9.Doshi JA, Li P, Huo H, Pettit AR, Armstrong KA. Association of patient out-of-pocket costs with prescription abandonment and delay in fills of novel oral anticancer agents. J Clin Oncol. 2018;36(5):476-482. doi: 10.1200/JCO.2017.74.5091 [DOI] [PubMed] [Google Scholar]
10.Fernandez EV, McDaniel JA, Carroll NV. Examination of the link between medication adherence and use of mail-order pharmacies in chronic disease states. J Manag Care Spec Pharm. 2016;22(11):1247-1259. [DOI] [PMC free article] [PubMed] [Google Scholar]
11.Carroll NV. A comparison of costs of Medicare Part D prescriptions dispensed at retail and mail order pharmacies. J Manag Care Spec Pharm. 2014;20(9):959-967. doi: 10.18553/jmcp.2014.20.9.959 [DOI] [PMC free article] [PubMed] [Google Scholar]
12.Taitel M, Fensterheim L, Kirkham H, Sekula R, Duncan I. Medication days’ supply, adherence, wastage, and cost among chronic patients in Medicaid. Medicare Medicaid Res Rev. 2012;2(3):1-13. doi: 10.5600/mmrr.002.03.A04 [DOI] [PMC free article] [PubMed] [Google Scholar]
13.King S, Miani C, Exley J, Larkin J, Kirtley A, Payne RA. Impact of issuing longer- versus shorter-duration prescriptions: a systematic review. Br J Gen Pract. 2018;68(669):e286-e292. doi: 10.3399/bjgp18X695501 [DOI] [PMC free article] [PubMed] [Google Scholar]
14.Ma J, Wang L. Characteristics of mail-order pharmacy users: results from the medical expenditures panel survey. J Pharm Pract. 2020;33(3):293-298. doi: 10.1177/0897190018800188 [DOI] [PubMed] [Google Scholar]
15.Mishra SK, Satpathy R. Physicians’ attitudes about prescribing and knowledge of the costs of common medications. Arch Intern Med. 2001;161(10):1352-1353. doi: 10.1001/archinte.161.10.1352 [DOI] [PubMed] [Google Scholar]
16.Allan GM, Lexchin J, Wiebe N. Physician awareness of drug cost: a systematic review. PLoS Med. 2007;4(9):e283. doi: 10.1371/journal.pmed.0040283 [DOI] [PMC free article] [PubMed] [Google Scholar]
17.Everson J, Frisse ME, Dusetzina SB. Real-time benefit tools for drug prices. JAMA. 2019;322(24):2383-2384. doi: 10.1001/jama.2019.16434 [DOI] [PubMed] [Google Scholar]
18.Feldman WB, Rome BN, Avorn J, Kesselheim AS. The future of drug-pricing transparency. N Engl J Med. 2021;384(6):489-491. doi: 10.1056/NEJMp2033734 [DOI] [PubMed] [Google Scholar]
19.Miller BJ, Slota JM, Ehrenfeld JM. Redefining the physician’s role in cost-conscious care: the potential role of the electronic health record. JAMA. 2019;322(8):721-722. doi: 10.1001/jama.2019.9114 [DOI] [PubMed] [Google Scholar]
20.Centers for Medicare & Medicaid Services . Modernizing Part D and Medicare Advantage To Lower Drug Prices and Reduce Out-of-Pocket Expenses. Federal Register. Accessed August 16, 2022. https://www.federalregister.gov/d/2019-10521
21.Monsen CB, Liao JM, Gaster B, Flynn KJ, Payne TH. The effect of medication cost transparency alerts on prescriber behavior. J Am Med Inform Assoc. 2019;26(10):920-927. doi: 10.1093/jamia/ocz025 [DOI] [PMC free article] [PubMed] [Google Scholar]
22.Chien AT, Lehmann LS, Hatfield LA, et al. A Randomized trial of displaying paid price information on imaging study and procedure ordering rates. J Gen Intern Med. 2017;32(4):434-448. doi: 10.1007/s11606-016-3917-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
23.Mummadi SR, Mishra R. Effectiveness of provider price display in computerized physician order entry (CPOE) on healthcare quality: a systematic review. J Am Med Inform Assoc. 2018;25(9):1228-1239. doi: 10.1093/jamia/ocy076 [DOI] [PMC free article] [PubMed] [Google Scholar]
24.Sedrak MS, Myers JS, Small DS, et al. Effect of a price transparency intervention in the electronic health record on clinician ordering of inpatient laboratory tests: The PRICE randomized clinical trial. JAMA Intern Med. 2017;177(7):939-945. doi: 10.1001/jamainternmed.2017.1144 [DOI] [PMC free article] [PubMed] [Google Scholar]
25.Basu A, Manning WG, Mullahy J. Comparing alternative models: log vs Cox proportional hazard? Health Econ. 2004;13(8):749-765. doi: 10.1002/hec.852 [DOI] [PubMed] [Google Scholar]
26.Jones AM. Models for Health Care. Clements MP, Hendry DF, eds. In: The Oxford Handbook of Economic Forecasting. Accessed August 16, 2022. https://global.oup.com/academic/product/the-oxford-handbook-of-economic-forecasting-9780195398649?cc=us&lang=en&
27.Abadie A, Athey S, Imbens GW, Wooldridge J. When should you adjust standard errors for clustering? National Bureau of Economic Research Working Paper 24003. Accessed August 16, 2022. https://www.nber.org/papers/w24003.ack
28.Desai S, Hatfield LA, Hicks AL, Chernew ME, Mehrotra A. Association between availability of a price transparency tool and outpatient spending. JAMA. 2016;315(17):1874-1881. doi: 10.1001/jama.2016.4288 [DOI] [PubMed] [Google Scholar]
29.Desai S, Hatfield LA, Hicks AL, et al. Offering a price transparency tool did not reduce overall spending among California public employees and retirees. Health Aff (Millwood). 2017;36(8):1401-1407. doi: 10.1377/hlthaff.2016.1636 [DOI] [PubMed] [Google Scholar]
30.Sinaiko AD, Rosenthal MB. Increased price transparency in health care--challenges and potential effects. N Engl J Med. 2011;364(10):891-894. doi: 10.1056/NEJMp1100041 [DOI] [PubMed] [Google Scholar]
31.Mehrotra A, Brannen T, Sinaiko AD. Use patterns of a state health care price transparency web site: what do patients shop for? Inquiry. 2014;51(0):2014-2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
32.Mehrotra A, Chernew ME, Sinaiko AD. Promise and reality of price transparency. N Engl J Med. 2018;378(14):1348-1354. doi: 10.1056/NEJMhpr1715229 [DOI] [PubMed] [Google Scholar]
33.Sinaiko AD, Joynt KE, Rosenthal MB. Association between viewing health care price information and choice of health care facility. JAMA Intern Med. 2016;176(12):1868-1870. doi: 10.1001/jamainternmed.2016.6622 [DOI] [PubMed] [Google Scholar]
34.Desai SM, Shambhu S, Mehrotra A. Online advertising increased New Hampshire residents’ use of provider price tool but not use of lower-price providers’. Health Aff (Millwood). 2021;40(3):521-528. doi: 10.1377/hlthaff.2020.01039 [DOI] [PMC free article] [PubMed] [Google Scholar]
35.Brot-Goldberg ZC, Chandra A, Handel BR, Kolstad JT. What Does a Deductible Do? The Impact of Cost-Sharing on Health Care Prices. Quantities, and Spending Dynamics; 2015. [Google Scholar]
36.Alexander GC, Casalino LP, Meltzer DO. Patient-physician communication about out-of-pocket costs. JAMA. 2003;290(7):953-958. doi: 10.1001/jama.290.7.953 [DOI] [PubMed] [Google Scholar]
37.Dine CJ, Masi D, Smith CD. Tools to help overcome barriers to cost-of-care conversations. Ann Intern Med. 2019;170(9_Suppl):S36-S38. doi: 10.7326/M19-0778 [DOI] [PubMed] [Google Scholar]
38.Brick DJ, Scherr KA, Ubel PA. The impact of cost conversations on the patient-physician relationship. Health Commun. 2019;34(1):65-73. doi: 10.1080/10410236.2017.1384428 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials