Abstract
Objective
The purpose of this study was to determine if medication cost transparency alerts provided at time of prescribing led ambulatory prescribers to reduce their use of low-value medications.
Materials and Methods
Provider-level alerts were deployed to ambulatory practices of a single health system from February 2018 through April 2018. Practice sites included 58 primary care and 152 specialty care clinics totaling 1896 attending physicians, residents, and advanced practice nurses throughout western Washington. Prescribers in the randomly assigned intervention arm received a computerized alert whenever they ordered a medication among 4 high-cost medication classes. For each class, a lower cost, equally effective, and safe alternative was available. The primary outcome was the change in prescribing volume for each of the 4 selected medication classes during the 12-week intervention period relative to a prior 24-week baseline.
Results
A total of 15 456 prescriptions for high-cost medications were written during the baseline period including 7223 in the intervention arm and 8233 in the control arm. During the intervention period, a decrease in daily prescribing volume was noted for all high-cost medications including 33% for clobetasol propionate (p < .0001), 59% for doxycycline hyclate (p < .0001), 43% for fluoxetine tablets (p < .0001), and a non-significant 3% decrease for high-cost triptans (p = .65). Prescribing volume for the high-cost medications overall decreased by 32% (p < .0001).
Conclusion
Medication cost transparency alerts in an ambulatory setting lead to more cost-conscious prescribing. Future work is needed to predict which alerts will be most effective.
Keywords: cost transparency, alerts, decision support, behavioral nudges, medications, pragmatic trial
INTRODUCTION
Nearly one-third of all health care in the US system may be wasteful.1 Prescription medication costs represent a sizeable fraction of this low-value care and is among the fastest growing sources of medical spending.2 Compared with other high-income countries, the US has the highest per capita pharmaceutical spending—nearly 2-fold more than the mean of the other 11 advanced world economies.3 In addition to crowding out other health care priorities, such spending can threaten patients’ medication adherence and produce suboptimal clinical outcomes.4
Methods to reduce medication spending, such as formulary restriction and prior authorization, can be successful but remain unpopular and impractical in many situations.5 In contrast, clinical decision support interventions designed to provide information at the point of prescribing may encourage cost-conscious prescribing without restricting choice. Patients and providers both desire using cost information to inform prescribing decisions6,7 but find it difficult to reliably access costs.8,9
Previous efforts to provide medication costs during electronic prescribing in both the inpatient10 and outpatient11,12 settings have furnished mixed results. Challenges have included multiple methods for defining cost,7 wide variation in available costs,13,14 limitations in timing of the information relative to providers’ ordering workflow,11 and methodological limitations that risk confounding and Type I error.15
Our present study aims to build on prior work through a randomly assigned design, large sample size, and careful selection of medications with observable, stable pricing, and implementation of clinical decision support using established design principles.16
MATERIALS AND METHODS
Design
This was a block-randomized study examining the effect of cost transparency alerts on prescribing of high-cost medications. We operationally defined high-cost medications as those for which there are lower cost alternatives of equal safety and effectiveness. The alerts compared standardized cost transparency information about the high-cost medication being ordered to lower cost alternatives. Four medication classes were selected as targets to trigger the alerts on the basis of having relatively stable wholesale acquisition cost and insurance coverage that is consistent across payers.
Randomization was performed at the clinic level because the electronic health record (EHR) system did not reliably accommodate randomization at the level of the clinician. Prescribing volume of selected high-cost medications was heavily skewed across clinics with the top 10 highest volume comprising 50% of the prescriptions and the top 60 comprising 90%. In order to balance prescribing volume and study arm sizes, clinics were rank-ordered by volume of high-cost medication prescribing and then randomly assigned into permuted blocks of 4 with a 1:1 ratio of intervention to control arms.17 The control arm received no alerts. The study protocol was deemed exempt for review by the University of Washington (UW) Medicine Institutional Review Board.
Setting and participants
This study took place in UW Medicine ambulatory practices from February 2018 to May 2018. UW Medicine’s ambulatory practices include over 200 clinics distributed across 5 clinical entities including UW Medical Center, Harborview Medical Center, UW Neighborhood Clinics, Northwest Hospital, and Hall Health. These clinics all use a single commercially available EHR for ambulatory care (EpicCare, Epic Systems Corporation, Verona, WI). Participants included 1896 prescribing clinicians who were a mix of attending physicians, residents, fellows, and advanced practice clinicians. Prescribers represented a variety of specialties including primary care, medical subspecialties, surgical subspecialties, gynecology, behavioral health, and more. There were no exclusions based on site, specialty, or training level. Participants were not informed of the alerts prior to receiving them in the EHR. They were not blinded to arm assignment.
Each clinical department was distributed across several practice locations including some teaching clinics. Included clinics were those that had prescribed any of the targeted medications during the baseline period and were open for the entirety of the study period. We excluded clinicians who had not prescribed any of the targeted medications during the study period, who had retired, or who worked at a refill authorization center. We excluded those working at the refill authorization center because they support many clinics and so could not be isolated to an intervention or control arm.
Intervention
Four high-cost medications were targeted for this study: topical clobetasol propionate, doxycycline hyclate, fluoxetine tabs, and high-cost triptans. These are high-cost generic medications with readily available, lower cost, equally effective, and safe alternatives. Preferred alternatives were chosen based on relatively stable wholesale acquisition cost and consistent coverage across payers. For clobetasol ointment or cream, prescribers in the intervention arm received an alert recommending betamethasone ointment or cream. For fluoxetine tabs, prescribers received an alert recommending fluoxetine caps. For doxycycline hyclate, prescribers received an alert recommending doxycycline monohydrate. Finally, for high-cost triptans which included zolmitriptan, almotriptan, eletriptan, frovatriptan, and naratriptan, prescribers received an alert recommending sumatriptan or rizatriptan.
When prescribing clinicians attempted to place an order for one of these high-cost medications, they received an alert requiring action (Figure 1). The alert consisted of a brief explanation for why the alternative was suggested, a means for ordering an alternative, information about the approximate costs of the originally ordered medication and the suggested alternative, as well as a link to more details about the methodology used to estimate the costs. Prescribers had 3 options for responding to the alert: 1) prescribe no medication, 2) select one of the suggested alternatives, or 3) dismiss the alert and prescribe the original medication.
Previous studies have demonstrated high override rates of active medication alerts, highlighting the importance of well-designed alert content.18,19 Careful consideration was given to ensure that the alerts were concise and provided clear instructions for how to dismiss them.16,20 To keep the intervention as standardized as possible across medications, the description text was written as follows:
[Ordered medication name] is more expensive than [alternative name], costing about $[cost of ordered medication] vs $[cost of alternative medication] per [unit of prescription]. UW Pharmacy and Therapeutics recommends the less expensive [alternative name].
Substitute by selecting one of the options below or “Continue with the Original Order.”
Medication costs
The definition of medication cost varies at each stage of the complex pharmaceutical supply chain. The choice of whether to use wholesale acquisition cost, national average drug acquisition cost, or patient out-of-pocket cost, depends on how one prioritizes the various stakeholders in the process. Providers feel that patient out-of-pocket costs are most relevant to their role as prescribers.21
Communicating patient out-of-pocket costs at the point of care is difficult because it depends on a variety of features including a patient’s insurer, drug benefit plan, and dispensing pharmacy. The patient’s pharmacy benefit manager is responsible for adjudicating real-time prices which individual patients must pay to pharmacies for dispensing a prescription medication. This adjudicated out-of-pocket cost often depends on the negotiated tier structure for each class of medications. It is unavailable at the time of prescribing since adjudication typically does not happen until the point of medication purchase.
Despite this, some medications exhibit predictable pricing, which was necessary for this intervention. The medications in the lowest benefit tiers are usually comprised of generic medications and are generally not subject to restrictions or rebates negotiated among pharmacy benefit managers and drug companies. Medications in these tiers are prescribed at high volumes and have multiple generic manufacturers competing for prescriptions. Thus, these medications usually have the most stable pricing. Therefore, commonly prescribed generic medications known to have multiple formulations of equivalent safety and efficacy were selected. Formularies of the 5 most common payers for the study population were selected and compared against candidate medications. Candidate medications were required to be formulary Tier 1 or Tier 2 for the majority of the examined payers.
The alerts reported wholesale acquisition cost. Though the wholesale acquisition cost is not the same as the average out-of-pocket costs that patients pay for generic medications, it is reliably correlated and was readily obtainable. Wholesale acquisition costs were obtained from UW Medicine’s drug wholesaler (McKesson Corp, San Francisco, CA) in February 2018. Among the 4 intervention alerts, there were several similar medications which could trigger each alert. For example, clobetasol propionate ointment and clobetasol propionate cream would both trigger the betamethasone ordering alert. In this case, the cost per unit was averaged. A list of targeted high-cost medications, their recommended substitutes, and the costs that were used as alert content are available in the Supplementary Table S1.
Study sample and outcomes
The primary study outcome was the change in prescribing volume of high-cost prescriptions within each of the 4 medications during a 12-week intervention period as compared to the prior 24-week baseline period. Expected health system cost savings were calculated as an exploratory outcome using the primary outcome and the average expected unit cost of each prescription. Alert acceptance rate, a measure of the rate at which prescribers chose to accept the alert to substitute a medication, was also calculated as a secondary outcome and balancing metric during the intervention.
Statistical analysis
Analyses were conducted at the prescription level. Chi-squared tests were used to compare the daily prescriptions placed during the baseline and intervention periods for the intervention and control arms. Multivariable Poisson regression, adjusted for prescriber location, level of training, and specialty was used to evaluate the association between alerts and the primary outcome. Statistically significant associations were determined based on the interaction between study arm and study period at an alpha of 0.05. All hypothesis tests were 2-tailed. Analyses were performed in Python (Python version 2.7, Anaconda distribution version 4.5.4).
RESULTS
A total of 210 clinics met the inclusion criteria and were randomly assigned to the intervention and control arms (Figure 2). Six clinics were excluded on the basis of actually being refill authorization centers, and 8 clinics were closed during the baseline period. Clinics remaining included 58 primary care and 152 specialty care clinics.
Sample characteristics
Table 1 summarizes the characteristics of prescriptions placed during the 24-week baseline period. A total of 15 456 prescriptions for high-cost medications were written during the baseline period including 7223 in the intervention arm and 8233 in the control arm. Significant differences were noted between arms in terms of provider practice location, level of training, and specialty.
Table 1.
Control (n = 8233) | Intervention (n = 7223) | |
---|---|---|
Location | ||
Hall Health | 894 (10.86) | – |
Harborview Medical Center | 554 (6.73) | 1866 (25.83) |
Northwest Hospital | 766 (9.3) | 956 (13.24) |
UW Medical Center | 1980 (24.05) | 1439 (19.92) |
UW Neighborhood Clinics | 4039 (49.06) | 2962 (41.01) |
Provider type | ||
Attending | 5512 (66.95) | 4938 (68.36) |
Fellow | 35 (0.43) | 71 (0.98) |
Mid-level | 2202 (26.75) | 1587 (21.97) |
Resident | 484 (5.88) | 637 (8.68) |
Specialty | ||
Behavioral health | 315 (3.83) | 389 (5.39) |
Dermatology | 994 (12.07) | 499 (6.91) |
Family medicine | 3154 (38.31) | 1617 (22.39) |
Gynecology | 183 (2.22) | 401 (5.55) |
Infectious diseases | 16 (0.19) | 348 (4.82) |
Internal medicine | 2793 (33.92) | 2655 (36.76) |
Neurology | 127 (1.54) | 641 (8.87) |
Other medical subspecialty | 298 (3.62) | 280 (3.88) |
Other surgical subspecialty | 266 (3.23) | 77 (1.07) |
Sports medicine and orthopedics | 87 (1.06) | 316 (4.37) |
a p < .05 for all category comparisons.
Change in prescribing volume
Table 2 summarizes the changes in prescribing volume for the intervention and control arms during the baseline and intervention periods. For clobetasol, a 33% reduction in daily prescribing volume was observed among the intervention arm as compared to a 2% reduction in the control arm (p < .0001). For doxycycline hyclate, a 59% reduction was observed among the intervention arm as compared to a 21% increase in the control arm (p < .0001). For fluoxetine tabs, a 43% reduction was observed among the intervention arm as compared to a 16% increase among the control arm (p < .0001). For high-cost triptans, a non-significant 3% reduction was observed as compared to a 4% increase among the control arm. Prescribing volume for the high-cost medications overall decreased by 32% (p < .0001). Table 3 illustrates the results of the Poisson regression analysis demonstrating daily incidence rate ratios for prescriptions of the high-cost medication options with adjustments for prescriber location, level of training, and specialty. The results of this separate methodology were consistent among the 3 models tested and concordant with the chi-squared comparisons.
Table 2.
Control | Intervention | P Value | |
---|---|---|---|
Clobetasol propionate | |||
Baseline period | 6.77 | 5.76 | |
Intervention period | 6.63 | 3.87 | |
Change, % | −2% | −33% | < .0001 |
Doxycycline hyclate | |||
Baseline period | 2.35 | 1.69 | |
Intervention period | 2.85 | 0.68 | |
Change, % | 21% | −59% | < .0001 |
Fluoxetine tablets | |||
Baseline period | 2.14 | 1.48 | |
Intervention period | 2.49 | 0.83 | |
Change, % | 16% | −43% | < .0001 |
High-cost triptans | |||
Baseline period | 1.44 | 2.38 | |
Intervention period | 1.50 | 2.30 | |
Change, % | 4% | −3% | .65 |
All high-cost medications | |||
Baseline period | 12.69 | 11.30 | |
Intervention period | 13.46 | 7.68 | |
Change, % | 6% | −32% | < .0001 |
Table 3.
Model 1 a | P Value | Model 2 b | P Value | Model 3 c | P Value | |
---|---|---|---|---|---|---|
Clobetasol propionate | 0.68 (0.58 to 0.81) | < .001 | 0.81 (0.68 to 0.95) | .013 | 0.75 (0.63 to 0.90) | .001 |
Doxycycline hyclate | 0.33 (0.24 to 0.46) | < .001 | 0.38 (0.27 to 0.53) | < .001 | 0.35 (0.25 to 0.50) | < .001 |
Fluoxetine tablets | 0.48 (0.35 to 0.66) | < .001 | 0.56 (0.40 to 0.78) | .001 | 0.54 (0.39 to 0.77) | .001 |
High-cost triptans | 0.93 (0.71 to 1.22) | .60 | 1.01 (0.75 to 1.35) | .96 | 1.02 (0.76 to 1.38) | .88 |
All high-cost medications | 0.64 (0.57 to 0.72) | < .001 | 0.74 (0.65 to 0.83) | < .001 | 0.70 (0.62 to 0.79) | < .001 |
Model 1: Arm + period.
Model 2: Arm + period + specialty.
Model 3: Arm + period + specialty + location + level of training.
Expected health system cost savings
Based on the observed decreases in prescribing volume of the selected high-cost medications, the annualized reduction in health system drug spending from the alerts if continued in the intervention group and expanded to the control group was calculated to be $127 000 per year for these 4 medications.
Alert acceptance rate
Figure 4 summarizes the actions taken by prescribers in response to the alerts. Prescribers accepted the doxycycline substitution that the alert recommended 37% of the time and rejected the substitution 53% of the time. For clobetasol, prescribers accepted the substitution 26% of the time and rejected the substitution 61% of the time. For fluoxetine tabs, prescribers accepted the substitution 24% of the time and rejected the substitution 60% of the time. For the high-cost triptans, prescribers accepted the substitution only 3% of the time and rejected the substitution 82% of the time.
DISCUSSION
This study describes a pragmatic, health system trial of the effect of medication transparency alerts on prescribing behavior.
Our results demonstrate that EHR cost transparency alerts can reduce low-value prescribing at the point of care in a real-world health care delivery setting. In particular, transparency alerts were deployed system-wide to all primary care and specialty clinics, effectively reducing prescriptions in situations where there was no clear, consistent rationale for prescribing the costlier medication.
Importantly, such efforts support health system efforts to improve the value of care delivery. Without restricting provider choice, the decision support prompted changes which would translate to an estimated $127 000 in annual health system medication spending for these 4 medications. Some of these savings would further accrue to patients under benefit structures where they pay a percentage copay. Since alerts were deployed within a modern EHR, they also represent solutions which could be scalable to other situations or settings. These features—ability to deter low-value care, potential for cost savings, and potential scalability—are particularly important for health systems that are actively engaged in accountable care organizations and other value-based payment arrangements.
The fact that we did not observe decreases in high-cost triptan prescriptions reflects the complexity of prescribing behavior and the need to better understand prescriber rationale for drug choice. In contrast to the others, the triptan alert was targeted at an entire medication class, in which specific agents may have been perceived by patients and/or clinicians to be more effective than others. Moreover, in the context of our health care system, triptans are often prescribed in the setting of a dedicated headache clinic where patients may seek care after already testing other lower cost triptans and finding them to be ineffective. If the higher cost option is not as expensive as expected and there is sufficient perceived value, cost transparency information may even lead to increased prescribing. This effect has been described in a previous study examining cost information for ambulatory imaging and procedures.22 While more work is needed to elucidate factors influencing triptan prescribing, this result underscores the need to design decision support interventions that accommodate the complexity of care delivery.
Providing cost information is a necessary prerequisite for providers and patients to consider cost together with other factors in shared decision-making. The United States has the highest pharmaceutical spending per capita among high-income countries without observable improved performance in many population health outcomes. Despite high rates of generic prescriptions, spending on brand-name pharmaceuticals shows little signs of abatement.3 Cost transparency may assist patients and providers in identifying such opportunities to reduce medication costs without compromising care quality.22
State and federal authorities have called for greater medication cost transparency in health care.23,24 Yet, in practice, frequent changes in negotiated prices and limited order entry integration have made cost information essentially unavailable at the point of prescribing. Thirty yards down the hall at our pharmacy, patients routinely learn exactly what they owe for a prescription within seconds. Our work suggests that closing this information gap with advancements in point of prescribing, real-time pharmacy benefit technologies may be able to drive health system value in a more scalable fashion.
Future work should extend our findings in several ways. Because decision support can lead to unintended consequences, future interventions should be designed to explicitly guard against such effects and to measure them.25 For example, interventions should avoid triggering “alert fatigue” among prescribers18,19 while ensuring that medications are not inappropriately withheld from patients who need them. One approach to achieve this objective is to provide cost information earlier in prescribing workflow, before a drug has been discussed with the patient, selected, and order details entered.26 Additionally, the alerts implemented for this study include 1 type of content used to influence behavior—medication costs—and other behavior-based interventions (eg, social comparisons, active choice) could be explored in future studies. In order to glean insights for future work, interventions should also elucidate reasons that prescribers opt to continue with the costlier option (eg, patient request, perceived differences in clinical efficacy).
Our study has limitations. First, generalizability is limited by implementation in a single health care system and EHR. However, our findings are instructive given UW Medicine’s size, large regional catchment area, and participation in value-based care delivery models that feature medication costs and prescribing behavior. Second, our approach to identifying medication costs is subject to cost fluctuations from supply changes and other drivers of medication prices and costs. However, in the absence of consensus for how to define medication costs, we adopted an approach that attempts to limit these fluctuations and qualified our alerts to provide clinicians with context about the complexities in price and cost determinations. Third, while our study arms were imbalanced with respect to several variables, we employed multivariable analysis to account for these differences. Fourth, our intervention was deployed in a pragmatic fashion with a relatively brief intervention period. Though it allowed us to test our present hypothesis earlier, it does not allow us to determine if the effect of the intervention diminishes over time. Finally, our study was limited to a small subset of prescribed medications for which we had reliable cost data and thus we excluded many other costly medications. Whether for research or operational purposes, extending this intervention to a broad menu of medications will require advancements in real-time pharmacy benefit technology in order to keep pace with the frequent changes in costs and benefit design. Until then, cost transparency decision support will require significant manual effort to maintain or else be highly error prone.
CONCLUSION
We have found encouraging evidence that when accurate cost information is available to providers in circumstances where similarly effective medications have widely diverging costs, costs can be reduced. When broadly available and designed to fit workflow, decision support can be an important tool to increase care value.
FUNDING
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
AUTHOR CONTRIBUTIONS
BG, THP, CBM, and JML conceived of the investigation. CBM and JML designed the methods of analysis. CBM acquired the data and performed computations for the analysis. CBM, JML, BG, and KF contributed to the first draft of the report. All authors reviewed results and contributed critical review in subsequent drafts of the report.
SUPPLEMENTARY MATERIAL
Supplementary material is available at Journal of the American Medical Informatics Association online.
Supplementary Material
ACKNOWLEDGMENTS
The authors would like to acknowledge contributions from Peter Capell, Jane Fellner, Gaylene Pattinson, Julie Wilson, and Lisa Peters, who served on the UW Medicine Medication Cost Transparency Steering Committee.
CONFLICT OF INTEREST STATEMENT
None declared.
REFERENCES
- 1. Institute of Medicine. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. National Washington, DC:Academies Press; 2013. doi: 10.17226/13444. [PubMed] [Google Scholar]
- 2. NHE Fact Sheet. NHE Fact Sheet (2018) https://www.cms.gov/research-statistics-data-and-systems/statistics-trends-and-reports/nationalhealthexpenddata/nhe-fact-sheet.html. Accessed April 7, 2018.
- 3. Papanicolas I, Woskie LR, Jha AK.. Health care spending in the United States and other high-income countries. JAMA 2018; 31910: 1024.. [DOI] [PubMed] [Google Scholar]
- 4. Choudhry NK, Avorn J, Glynn RJ, et al. Full coverage for preventive medications after myocardial infarction. N Engl J Med 2011; 36522: 2088–97. [DOI] [PubMed] [Google Scholar]
- 5. Flynn KE, Smith MA, Davis MK.. From physician to consumer: the effectiveness of strategies to manage health care utilization. Med Care Res Rev 2002; 594: 455–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Collado M, Ducas A.. Patients, physicians, and price transparency: if you build it, will they come? Health Affairs Blog 2016. https://www.healthaffairs.org/do/10.1377/hblog20160831.056288/full/. Accessed March 13, 2018. [Google Scholar]
- 7. Schiavoni KH, Lehmann LS, Guan W, et al. How primary care physicians integrate price information into clinical decision-making. J Gen Intern Med 2017; 321: 81–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Mehrotra A, Dean KM, Sinaiko AD, Sood N.. Americans support price shopping for health care, but few actually seek out price information. Health Aff (Millwood) 2017; 368: 1392–400. [DOI] [PubMed] [Google Scholar]
- 9. Sinaiko AD, Rosenthal MB.. Increased price transparency in health care—challenges and potential effects. N Engl J Med 2011; 36410: 891–4. [DOI] [PubMed] [Google Scholar]
- 10. Conway S, Brotman D, Pinto B, et al. Impact of displaying inpatient pharmaceutical costs at the time of order entry: lessons from a tertiary care center. J Hosp Med 2017; 128: 639–45. [DOI] [PubMed] [Google Scholar]
- 11. Ornstein SM, MacFarlane LL, Jenkins RG, Pan Q, Wager KA.. Medication cost information in a computer-based patient record system. Impact on prescribing in a family medicine clinical practice. Arch Fam Med 1999; 82: 118–21. [DOI] [PubMed] [Google Scholar]
- 12. Vedsted P, Nielsen JN, Olesen F.. Does a computerized price comparison module reduce prescribing costs in general practice? Fam Pract 1997; 143: 199–203. [DOI] [PubMed] [Google Scholar]
- 13. Binder L. Why health-care price transparency isn’t enough for consumers. Wall Str J 2015. https://blogs.wsj.com/experts/2015/03/26/why-health-care-price-transparency-isnt-enough-for-consumers/. Accessed March 13, 2018. [Google Scholar]
- 14. Gellad WF, Choudhry NK, Friedberg MW, et al. Variation in drug prices at pharmacies: are prices higher in poorer areas? Health Serv Res 2009; 44 (2 Pt 1): 606–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Mummadi SR, Mishra R.. Effectiveness of provider price display in computerized physician order entry (CPOE) on healthcare quality: a systematic review. J Am Med Inform Assoc 2018; 259: 1228–39. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Horsky J, Phansalkar S, Desai A, Bell D, Middleton B.. Design of decision support interventions for medication prescribing. Int J Med Inf 2013; 826: 492–503. [DOI] [PubMed] [Google Scholar]
- 17.Ivers NM, Halperin IJ, Barnsley J, Grimshaw JM, Shah BR, Tu K, Upshur R, Zwarenstein M. Allocation techniques for balance at baseline in cluster randomized trials: a methodological review. Trials 2012; 13 https://trialsjournal.biomedcentral.com/articles/10.1186/1745-6215-13-120. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Nanji KC, Slight SP, Seger DL, et al. Overrides of medication-related clinical decision support alerts in outpatients. J Am Med Inform Assoc 2014; 213: 487–91. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Isaac T, Weissman JS, Davis RB, et al. Overrides of medication alerts in ambulatory care. Arch Intern Med 2009; 1693: 305. [DOI] [PubMed] [Google Scholar]
- 20. Shah NR, Seger AC, Seger DL, et al. Improving acceptance of computerized prescribing alerts in ambulatory care. J Am Med Inform Assoc 2006; 131: 5–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Shrank WH, Joseph GJ, Choudhry NK, et al. Physicians’ perceptions of relevant prescription drug costs: do costs to the individual patient or to the population matter most? Am J Manag Care 2006; 129: 545–51. [PubMed] [Google Scholar]
- 22. Chien AT, Ganeshan S, Schuster MA, et al. The effect of price information on the ordering of images and procedures. Pediatrics 2017; 1392: e20161507. [DOI] [PubMed] [Google Scholar]
- 23. Azar A. Remarks on Drug Pricing Blueprint. Washington, DC. May 14, 2018. Speech. Accessed Sept 3, 2018 from https://www.hhs.gov/about/leadership/secretary/speeches/2018-speeches/remarks-on-drug-pricing-blueprint.html. [Google Scholar]
- 24. Weintraub A. The call for drug-price transparency is growing louder—but will it matter? Forbes 2018. https://www.forbes.com/sites/arleneweintraub/2018/03/30/the-call-for-drug-price-transparency-is-growing-louder-but-will-it-matter/#76eaa47b3267. Accessed September 4, 2018. [Google Scholar]
- 25. Patel MS, Volpp KG, Asch DA.. Nudge units to improve the delivery of health care. N Engl J Med 2018; 3783: 214–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Hayward J, Thomson F, Milne H, et al. Too much, too late’: mixed methods multi-channel video recording study of computerized decision support systems and GP prescribing. J Am Med Inform Assoc 2013; 20 (e1): e76–84. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.