Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Aug 1.
Published in final edited form as: Drug Alcohol Depend. 2019 Jun 25;201:260–265. doi: 10.1016/j.drugalcdep.2019.03.034

Advanced visualizations to interpret prescription drug monitoring program information*

Scott G Weiner 1, Karen M Sherritt 2, Zoe Tseng 3, Jaya Tripathi 4
PMCID: PMC6737934  NIHMSID: NIHMS1534130  PMID: 31276986

Abstract

Objectives:

Prescription drug monitoring programs (PDMPs) are state-based databases that contain information about controlled substance prescriptions dispensed by pharmacies. Many states now mandate PDMP use by prescribers, despite unclear effectiveness. We hypothesize that it is possible to improve the interpretability, and hence effectiveness, of PDMPs by enhancing them.

Methods:

This was a real-time simulation of an enhanced PDMP. Fifty practicing physicians (25 primary care, 25 emergency medicine) were randomized to see three cases with a standard profile or an enhanced profile that included graphical representation of prescriptions and identified risky prescribing patterns. After a two-month washout period, participants were placed in the opposite group.

Results:

Physicians presented with the enhanced profile were more likely to correctly identify patients with multiple providers (97.0% vs. 85.8%, p=0.002), overlapping opioid and benzodiazepine prescriptions (94.7% vs. 87.5%, p=0.03), overlapping opioid prescriptions (89.5% vs. 70.8%, p<0.01), high daily dosages of opioids (99.2% vs. 25.0%, p=0.02), and traveling to distant pharmacies (79.7% vs. 2.5%, p<0.01). There was no difference in interpretation time for the three cases (standard profile 657.3 seconds vs. enhanced profile 686.3 seconds, p=0.31).

Conclusions:

A simulated PDMP with graphical displays and interpretation of findings was, for this cohort of emergency physicians and primary care physicians, associated with an increased ability to determine high-risk features on PDMP profiles.

Keywords: Prescription Drug Monitoring Programs, Opioids, Prescribing, Polypharmacy, Informatics, Decision-Making Tools, Emergency Medicine, Primary Care

1. Introduction

1.1. Background

The United States continues to face an opioid overdose and abuse crisis, with an average of 130 people dying each day due to an opioid-related overdose in 2017 (CDC, 2019). Although the number of prescribed opioids is finally decreasing after more than a decade of increases – a 22% decrease between 2013 and 2017 – more than 40% of all U.S. opioid overdose deaths in 2016 still involved a prescription opioid (IQVIA, 2019; Seth et al., 2018).

Prescription drug monitoring programs (PDMPs) are state-based databases that contain information on controlled substance prescriptions dispensed by pharmacies and prescribers. They can be used by prescribers to monitor controlled substance use by patients and by states to monitor prescribing practices and population-level medication use trends. There has been rapid uptake of PDMPs around the country, and as of February 2018, 50 states, the District of Columbia, Guam and Puerto Rico had some form of a PDMP (Sacco et al., 2018). PDMPs can be expensive, with startup costs that can range as high as $1.5 million and annual operating costs ranging from $125,000 to nearly $1.0 million (Sacco et al., 2018).

1.2. Importance

Despite the widespread adoption and expenditure, the efficacy of PDMPs has been questioned. A large systematic review found insufficient evidence that PDMPs either increased or decreased fatal and nonfatal overdoses (Fink et al., 2018). Overall use of PDMPs is variable, both between specialties and states (Irvine et al., 2014; Deyo et al., 2015; Sun et al., 2018). To address this dilemma, multiple states have instituted mandated use of PDMPs, either by mandating that prescribers register or mandating access prior to every opioid prescription written (Shev et al., 2018; Williams et al., 2018; Grecu et al., 2019). These mandates carry significant penalties in certain states if the PDMP is not accessed and do appear to be associated with decreased opioid prescribing in several settings (Haffajee et al., 2015; Rasubala et al., 2015; Winstanley et al., 2018; Suffoletto et al., 2018). Furthermore, states with more robust PDMP programs do appear to have decreased opioid prescriptions compared with states which do not (Haffajee et al., 2018).

Although there are PDMPs in all 50 states, and prescribers are mandated to use the PDMP in at least 41 of those states, it is surprising that there have been few studies that evaluate how the systems can be improved and become more useful for clinicians. Past work has determined that the transitional workflow of accessing the PDMP through a website outside of a provider’s electronic medical record is a time-consuming barrier (Poon et al., 2016). Additionally, research has demonstrated that providers seeing the exact same PDMP report may interpret it in different ways, with variations in rates of opioid prescribing despite seeing the same clinical scenario and same PDMP report (Hoppe and Weiner, 2015). PDMP profiles can be complicated to quickly synthesize and interpret for a provider, especially if there are multiple prescriptions present (Rutkow et al., 2015). Enhancing the PDMP in a way that simplifies use and interpretation, quickly identifies “red flags” of concern, and highlights data not easily determined by the clinician – such as total daily morphine milligram equivalents (MMEs) or distance the patient traveled to get to a pharmacy – could increase utility and also improve the quality of the prescriber’s interpretation leading to increased patient safety.

1.3. Goals of This Investigation

We performed a real-time simulation of an enhanced PDMP. A simulated website was created and current opioid prescribers viewed either a standard PDMP or an enhanced version. The primary aim was to determine the ability of clinicians to detect high-risk opioid features from prescription drug monitoring program (PDMP) profiles, comparing standard with enhanced displays. We hypothesized that significant risk factors, such as overlapping opioid prescriptions, high daily doses of opioids, concurrent opioid and benzodiazepine prescriptions, traveling far distances to have a prescription filled, or having a large number of self-pay prescriptions (avoiding use of insurance) would be missed when clinicians evaluated standard profiles. The secondary aims were to determine time spent reviewing the standard vs. enhanced profiles, and to ascertain clinician impressions about the enhanced profile.

2. Materials and Methods

2.1. Study Design and Setting

This was a prospective cross-over study of a cohort of attending physicians affiliated with a health system in an urban area. The health system includes an academic Level 1 trauma center with 777 licensed beds, an affiliated community hospital with 162 licensed beds, and 15 affiliated primary care practices. We recruited a convenience sample of 25 emergency physicians and 25 primary care physicians from across the organization. Prescribers were required, by self-report, to be actively practicing, had written at least 6 opioid prescriptions in the preceding 6 months, and had accessed the state’s PDMP at least 6 times in the preceding 6 months. The study was approved by the hospital’s institutional review board.

2.2. Selection of Participants

Physicians were enrolled by responding to an email sent out by an investigator, and then they were randomly assigned to the enhanced profile group (25 participants) or the standard profile group (25 participants). As an exploratory study without preexisting data, no sample size calculation was performed. In Phase 1, which was open from December 22, 2017 to January 9, 2018, all participants were provided with written instructions and a brief online video to orient them to the study. Participants logged onto a website at a place and time of their choosing, reviewed three clinical scenarios with accompanying PDMP profiles (either enhanced or standard) and then answered questions about the cases and the profile on the website. After a two-month washout period, Phase 2 was launched, open from March 10, 2018 to March 23, 2018, in which the participants were assigned the opposite PDMP profile (enhanced instead of control and vice versa) and were presented with the same three cases and asked the same questions. Participants were compensated with a $50 gift card each of the two times they completed the task ($100 total). In both phases, subjects were contacted by email up to three times until they completed the study.

2.3. Development of The Enhanced Profile

As part of an ongoing research project on prescription drug fraud and abuse, members of our team conducted a survey of existing decision-making tools that assist physicians when prescribing opioids. The respondents provided a wish list of features, which included three categories: 1) clear indication of medications with MME calculations; 2) visualizations to help interpret data; 3) concise and easily interpretable metrics. Using this as guidance, a web application called the Medication Decision Support Suite (MeDSS) was created (Tripathi and Weiner, 2017).

MeDSS contains two components, a patient timeline graph and a risk assessment summary. Usability and interpretability were tested at the Indiana Board of Pharmacy and content was improved based on user feedback. The patient timeline graph contains information on multiple provider episodes, polypharmacy, distance traveled to obtain a prescription, dosage escalations, MMEs and payment type. Payment type was included because of the potential for riskier prescribing if the patient self-pays for a prescription outside of their insurance (Becker et al., 2017). The risk assessment summary displays predictors of risk that were determined by applying advanced analytics on a large longitudinal PDMP dataset (12 million records from INSPECT, Indiana’s PDMP program). The CRISP-DM data mining model was employed with features that were derived from the raw data attributes, to feed into machine learning algorithms (Sharer, 2000). The enhanced profile is based on the MeDSS tool. The three vignettes, which varied for emergency medicine and primary care, are demonstrated in Appendix A 1The standard PDMP profile and enhanced PDMP profile are demonstrated in Figure 1.

Figure 1.

Figure 1

Figure 1

Open in a new tab

A sample of the standard prescription drug monitoring report (panel A) and enhanced report (panel B). The information presented was unique to each of the three patient vignettes. Pharmacy names were chosen randomly from local pharmacies, but all prescription and provider data are simulated.

2.4. Methods of Measurement

Participants were first asked basic demographic questions including specialty and years in practice. They then viewed each case vignette presented in either the enhanced profile or standard profile view. Next, the participants were then asked to respond to questions about the patient’s timeline chart, to identify the presence of high-risk features and to decide whether or not they would prescribe an opioid medication to the simulated patient. After completing the three vignettes and associated questions, the participants answered questions about the tool itself, such as the format and value of an enhanced profile.

2.5. Primary Data Analysis

After study completion, data was extracted from the website to an Excel spreadsheet and double-checked for accuracy. Data were imported into JMP v12 (SAS Corporation, Cary, NC) and were analyzed with descriptive analytics.

3. Results

There were 50 total participants in the cohort. Half (25/50) were specialized in primary care and half (25/50) in emergency medicine. For the entire cohort, the mean number of years in practice was 16.2 (SD 10.2) years, range 5–41 years. Twenty-nine providers (58%) were male. Participants estimated that the percentage of patients they prescribed an opioid to in their practice was an average of 5.7% (SD 4.0), range 0–20%. The number of times that the PDMP was checked prior to writing an opioid prescription was 78% (SD 32.0), range 0–100%.

Table 1 demonstrates the participants’ answers regarding the presence of high-risk factors on the patients’ profiles. Physicians presented with the enhanced profile were more able to correctly identify patients with multiple providers, overlapping prescriptions, high dosages, and other risk factors such as paying in cash and traveling to distant pharmacies. The only category where the enhanced profile group responded with lower accuracy than the standard group was in identifying patients who had visited many pharmacies (more than 4) in the last year.

Table 1:

Rate of correct responses to questions about identifying risk factors in 3 hypothetical patients.

Standard Profile Group (n=120 observations) Enhanced Profile Group (n=133 observations) Comparison
Question Percent Correct Percent Incorrect/Unsure Percent Correct Percent Incorrect/Unsure P-value
1. Identify the number of prescriptions in the patient’s profile 95.0% 5% 96.2% 3.8% 0.63
2. Identify if there were 4 or more providers for controlled substances used in the past year 85.8% 14.2% 97.0% 3% <0.01
3. Identify if there were any overlapping opioid and benzodiazepine prescriptions 87.5% 12.5% 94.7% 5.3% 0.03
4. Identify if there were any overlapping opioid prescriptions 70.8% 29.2% 89.5% 10.5% <0.01
5. Identify if there were 4 or more pharmacies use for controlled substances in the past year 81.7% 18.3% 43.6% 56.4% <0.01
6. Identify if there were times when the patient was taking more than 100 morphine milligram equivalents (MME) per day 25.0% 75% 99.2% 0.8% 0.02
7. Identify if there were any self-pay (cash) prescriptions filled 89.2% 10.8% 93.2% 6.8% 0.25
8. Identify if there were any times that the patient traveled more than 50 miles from their home address to have a prescription filled 2.5% 97.5% 79.7% 20.3% <0.01
9. Identify if the patient had any buprenorphine prescriptions 89.2% 10.8% 94.7% 5.3% 0.16
Open in a new tab

P values <0.05 are considered statistically significant and are in bold text.

Table 2 describes provider responses on whether they would prescribe an opioid to the patient after seeing either the standard or enhanced profile. There was a significant difference for Patient A (19.5% standard profile vs. 4.3% enhanced profile prescribing an opioid), and no significant difference for the other two patients or for all three patients combined.

Table 2:

Determination if the provider would provide an opioid prescription after reviewing the vignette and profile.

Standard Profile Group Enhanced Profile Group P-value
Overall 18.3% 12.0% 0.22
Patient A (dental pain) 19.5% 4.3% 0.04
Patient B (rib fracture) 33.3% 27.3% 0.72
Patient C (back pain) 2.3% 4.7% 1.00
Open in a new tab

P values <0.05 are considered statistically significant and are in bold text.

We also evaluated the time it took to complete the activity. On average the standard profile group (n=40 observations) spent a mean of 657.3 (95% CI 561.9–752.6) seconds, or 10 minutes, 57 seconds reviewing the three profiles and answering questions, and the enhanced profile group (n=43 observations) spent a mean of 686.3 (95% CI 618.2–754.5) seconds, or 11 minutes, 26 seconds reviewing the three profiles and answering questions, p=0.31. As Figure 2 demonstrates, the time it took to complete each profile decreased with each subsequent profile.

Figure 2.

Figure 2

Open in a new tab

Mean time in seconds required to evaluate the three cases for both the standard and enhanced profiles. Error bars represent 95% confidence intervals.

Table 3 demonstrates the usefulness of various features of the enhanced profile on a scale from 1–5 (5 being most useful). Participants ranked the graphical representation of prescriptions, color-coding of medication types, identification of early refills and overlapping of same-type prescriptions, and identification of polypharmacy as being the most useful features on the enhanced profile.

Table 3:

Physician evaluation of the usefulness of various features on the enhanced profile, on a scale of 1 to 5 (5 being “most useful”).

Information presented Median Rating
Daily morphine milligram equivalent graph 4.5
Distance from patient’s home to prescribers/pharmacies 4
Specialty of prescribers 4
Payment methods used 4
Graphical representation of prescriptions 5
Color-coding of medication types 5
Identification of early refills and overlapping of same-type prescriptions 5
Identification of polypharmacy 5
Open in a new tab

4. Discussion

The increased adoption of PDMPs has seen a rapid progress in their use of technology. Only a few years ago, there was limited interstate data sharing, delays between the time the prescription was filled and when it appeared in the state’s database, partial reporting of controlled substance prescriptions (such as only reporting of schedule II medications) and a tabular format presentation that made it challenging to quickly get the picture of a patient’s PDMP history (Manasco et al., 2016). Certain states have undertaken their own enhancements to improve the system. Wisconsin, for example, uses a proprietary system that includes graphical representations of patient, prescriber and pharmacy locations to quickly and visually identify doctor- and pharmacy-shopping behavior (Wisconsin, 2019). The site also highlights “red flag” concerning features such high MME.

The majority of states (currently 42 states and territories) use a PDMP created by Appriss, Inc. called PMP AWARxE (Appriss, 2019). The basic system is continually undergoing updates, including improved patient data matching algorithms, identification of daily MME and easy identification of cash payments. Appriss also offers a proprietary system that condenses findings from the PDMP profile into three-digit scores and also has a graphical representation. A retrospective study associating higher numeric scores with higher overdose death risk using Ohio’s PDMP data can be found on their website (Huizenga et al., 2017). However, we are unaware of any studies comparing this system to the standard PDMP profile, both in terms of provider satisfaction, detection of concerning profile features or its influence on medical decision-making. Likewise, it is possible that patients with higher numeric scores have higher overdose death risk because they are more complex patients that need to use higher doses of opioids or have concurrent prescriptions for opioids and benzodiazepines (e.g. those with cancer pain or severe trauma). Higher scores might also not mean that an opioid is contraindicated for that patient; each patient’s care must be personalized. Other recent work has highlighted that physicians are concerned that overdose risk scores may overly determine prescribing decision and expose the prescriber to liability if an opioid is prescribed despite a high-risk score, even if deemed clinically necessary (Leichtling et al., 2019).

In our study, the indicators for risky behavior were the results algorithms which learned from training on historical data where the output (or target variable) was risk of overdose. It is important to understand that these indicators should not be viewed independently but in conjunction with each other. Indicators such as self-pay or driving far distances to fill a prescription may unintentionally discriminate against vulnerable patient populations that have to travel far distances for care or self-pay because of lack of insurance coverage, and are better indicators when viewed as co-mingled payments (Medicaid and cash) or simultaneous visits to other prescribers who are located at closer proximity to the patient.

We detected that the enhanced profile was associated with several benefits, including more accurate detection of overlapping opioid and benzodiazepine prescriptions, overlapping opioids between prescribers and high MME per day. The pharmacy indicators, including use of multiple pharmacies and determining the distance traveled to fill a prescription were also improved with the enhanced profile. In one out of the three patients the probability of prescribing an opioid was significantly lower (19.5% vs 4.3%) in the enhanced profile groups, but there was no significant difference in opioid prescription rates for the other two patients. This was a simulated patient with dental pain, and it is likely that the nuanced PDMP data affected prescribing behavior. It is also possible that for the other cases (a confirmed rib fracture and musculoskeletal back pain) the providers had already decided their prescribing decision regardless of the PDMP information.

We tested several features of an enhanced PDMP profile, but it is also important to recognize that other enhancements may be of value. For example, the aforementioned Wisconsin system also includes important information that is not routinely reported in PDMP data, including if a patient has suffered an overdose. Given that most providers continue to write opioid prescriptions after their patients have suffered an overdose, this functionality could markedly improve PDMPs (Larochelle et al., 2016). Other ideas would be to provide a prompt to the provider to ask about their concurrent benzodiazepine and opioid prescribing and provide resources to explain the risk of these combined medication or encourage a provider to prescribe naloxone for higher risk patients. Regardless of what enhancements are made, it is essential that evaluation occurs to ensure protection of individual patient and overall population health and avoid unintended consequences like under-treatment of pain. Going forward, for maximum effectiveness we would recommend a dashboard that contains additional components including alerts, geospatial maps, and incorporation of clinical data not currently reported in PDMPs such as medical diagnoses and non-controlled substance prescriptions.

We propose that the enhanced profile does not alter prescribing behavior much for obvious cases of aberrant drug-related behavior, such as a long list of different providers for opioid prescriptions, or for cases when there are few or no prescriptions on the PDMP. Likewise, it may not change decision-making when the provider was not going to prescribe an opioid in any case (e.g. musculoskeletal back pain) or was going to prescribe regardless (e.g. for a fracture). Rather, it is most helpful for the more nuanced cases where the provider is undecided about prescribing or for identifying patient safety concerns such as overlapping opioid and benzodiazepine prescriptions or high daily MMEs. While PDMPs may alter provider prescribing behavior, it has yet to be seen what additional benefits they may provide to improving the health of individuals and at the population level.

There was no significant difference in time needed to review the two different profiles. We hypothesize that part of the reason for the insignificant time difference was lack of familiarity with the enhanced tool, and the fact that the enhanced view offers more information with two components in two tabs within the web application. However, we found that progressively less time was spent with each successive patient review. We predict that time spent will vastly improve if used in routine clinical practice.

Satisfaction with the tool overall was high, with each of the features we enquired about having a median of at least 4 out of 5 for usefulness. The tools with a usefulness median of 5 out of 5 were easy ways to identify polypharmacy, early refills, color coding of the types of prescriptions and graphical representations of prescriptions. Some of these features have recently been incorporated in some states’ PDMPs; the other features could be considered for further expansion.

4.1. Limitations

Our study has several limitations that must be considered. This was a single site study of mainly academic physicians at one health system, in a state where PDMP use is mandatory prior to every opioid prescription. Results from other sites and states may vary. The cases the participants saw were simulated based on a small case vignette and patient histories selected from a PDMP database, that were then anonymized; in an actual patient encounter PDMP data would be presented synchronously alongside the patient’s electronic health records in real-time. The selected PDMP profiles were ‘extreme’ to capture complexities, with multiple provider episodes, multiple prescriptions and features that are purposefully difficult to ascertain by just evaluating a standard profile. Not all of the participants completed every vignette, as some had technical problems related to browser issues with our prototype website. This was a crossover study, and although several weeks passed between phases, it is possible that participants remembered certain case features in Phase 2. Although participants were shown an orientation video about the enhanced profile, they had otherwise never used it. A wealth of information was displayed on the timeline graph but even more could be gleaned by right-clicking on each color-coded rectangular bar that represents a prescription. Some participants may not have been aware of this feature which displays information about the pharmacy, its location, the specific drug and dosage. We imagine that with more use and familiarity the tool would become even easier to use, exemplified by the time data which showed a decreasing amount of time needed to review each subsequent case.

4.2. Conclusions

A simulated PDMP with advanced graphical displays and interpretation of findings was, for this cohort of emergency physicians and primary care physicians, associated with an increased ability to determine high-risk features on PDMP profiles. The tool did not reduce the time needed to review cases and interpret the PDMP. However, providers in the enhanced profile were able to identify several important nuances compared with the standard view. The providers rated several features of the enhanced profile highly, and these should be considered for incorporation in further iterations of states’ PDMP systems to enhance accuracy of interpretation, improve prescribing habits, increase patient safety, and encourage provider use of PDMPs overall.

Supplementary Material

1

Highlights.

  • We created a simulated enhanced prescription drug monitoring program (PDMP) system.

  • The enhanced PDMP was associated with improved comprehension of high-risk factors.

  • The enhanced PDMP did not require less time to use than the standard version.

  • Opioid prescribing decision-making was changed for one of three cases.

Acknowledgements

We gratefully acknowledge the sponsorship of MITRE Corporation, who funded the study. Dr. Weiner is also funded by NIH grant 1RO1DA044167.

Role of Funding Source

This work was funded by MITRE Corporation, a not-for-profit company. Ms. Tripathi is an employee of MITRE. MITRE did not have control over the conduct of the study, but it did assist in the data analysis.

Ms. Tripathi is an employee of MITRE, the not-for-profit company that funded the study.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Conflict of Interest

No other authors declare any relevant conflict of interest.

1

Supplementary material can be found by accessing the online version of this paper at http://dx.doi.org and by entering doi:…

*

Supplementary material can be found by accessing the online version of this paper at http://dx.doi.org and by entering doi:…

References

  1. Appriss Health, 2019. PMP AWARxE Available at: https://apprisshealth.com/solutions/pmp-awarxe. Accessed 03.23.19.
  2. Becker WC, Fenton BT, Brandt CA, Doyle EL, Francis J, Goulet JL, Moore BA, Torrise V, Kerns RD, Kreiner PW, 2017. Multiple sources of prescription payment and risky opioid therapy among veterans. Med. Care 55, S33–S36. [DOI] [PubMed] [Google Scholar]
  3. Centers for Disease Control and Prevention (CDC), 2019. Drug overdose deaths Available at: https://www.cdc.gov/drugoverdose/data/statedeaths.html. Accessed 03.23.19.
  4. Deyo RA, Irvine JM, Hallvik SE, Hildebran C, Beran T, Millet LM, Marino M, 2014. Leading a horse to water: Facilitating registration and use of a prescription drug monitoring program. Clin. J. Pain 31, 782–787. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Fink DS, Schleimer JP, Sarvet A, Grover KK, Delcher C, Castillo-Carniglia A, Kim JH, Rivera-Aguirre AE, Henry SG, Martins SS, Cerdá M, 2018. Association between prescription drug monitoring programs and nonfatal and fatal drug overdoses: A systematic review. Ann. Intern. Med 168, 783–790. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Freeman PR, Curran GM, Drummond KL, Martin BC, Teeter BS, Bradley K, Schoenberg N, Edlund MJ, 2018. Utilization of prescription drug monitoring programs for prescribing and dispensing decisions: Results from a multi-site qualitative study. Res. Social Adm. Pharm doi: 10.1016/j.sapharm.2018.09.007. [Epub ahead of print]. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Grecu AM, Dave DM, Saffer H, 2019. Mandatory access prescription drug monitoring programs and prescription drug abuse. J. Policy Anal. Manage 38, 181–209. [PubMed] [Google Scholar]
  8. Haffajee RL, Jena AB, Weiner SG, 2015. Mandatory use of prescription drug monitoring programs. JAMA 313, 891–892. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Haffajee RL, Mello MM, Zhang F, Zaslavsky AM, Larochelle MR, Wharam JF, 2018. Four states with robust prescription drug monitoring programs reduced opioid dosages. Health Aff. (Millwood) 37, 964–974. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Hoppe JA, Weiner SG, 2015. Emergency physician interpretation of prescription drug monitoring program profiles. Acad. Emerg. Med 22, S16–S17. [Google Scholar]
  11. Huizenga JE, Breneman BC, Patel VR, Raz A, Speights DB, 2017. NARxCHECK score as a predictor of unintentional overdose death, available at: https://apprisshealth.com/wp-content/uploads/sites/2/2017/02/NARxCHECK-Score-as-a-Predictor.pdf. Accessed 03.23.19.
  12. Irvine JM, Hallvik SE, Hildebran C, Marino M, Beran T, Deyo RA, 2014. Who uses a prescription drug monitoring program and how? Insights from a statewide survey of Oregon clinicians. J. Pain 15, 747–55. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. IQVIA Institute for Human Data Science, 2018. Medicine use and spending in the U.S. A review of 2017 and Outlook to 2022 Available at: https://www.iqvia.com/institute/reports/medicine-use-and-spending-in-the-us-review-of-2017-outlook-to-2022. Accessed 03.23.19.
  14. Larochelle MR, Liebschutz JM, Zhang F, Ross-Degnan D, Wharam JF, 2016. Opioid prescribing after nonfatal overdose and association with repeated overdose: A cohort study. Ann. Intern. Med 164, 1–9. [DOI] [PubMed] [Google Scholar]
  15. Leichtling G, Hildebran C, Novak K, Alley L, Doyle S, Reilly C, Weiner SG, 2019. Physician responses to enhanced prescription drug monitoring program profiles. Pain Med 10.1093/pm/pny291 [Epub ahead of print]. [DOI] [PubMed]
  16. Manasco AT, Griggs C, Leeds R, Langlois BK, Breaud AH, Mitchell PM, Weiner SG, 2016. Characteristics of state prescription drug monitoring programs: A state-by-state survey. Pharmacoepidemiol. Drug Saf 25, 847–51. [DOI] [PubMed] [Google Scholar]
  17. Poon SJ, Greenwood-Ericksen MB, Gish RE, Neri PM, Takhar SS, Weiner SG, Schuur JD, Landman AB, 2016. Usability of the Massachusetts prescription drug monitoring program in the emergency department: A mixed-methods study. Acad. Emerg. Med 23, 406–14. [DOI] [PubMed] [Google Scholar]
  18. Rasubala L, Pernapati L, Velasquez X, Burk J, Ren YF, 2015. Impact of a mandatory prescription drug monitoring program on prescription of opioid analgesics by dentists. PLoS One 10, e0135957. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Rutkow L, Turner L, Lucas E, Hwang C, Alexander GC, 2015. Most primary care physicians are aware of prescription drug monitoring programs, but many find the data difficult to access. Health Aff. (Millwood) 34, 484–92. [DOI] [PubMed] [Google Scholar]
  20. Sacco LN, Duff JH, Sarata AK, 2018. Prescription drug monitoring programs Available at: https://fas.org/sgp/crs/misc/R42593.pdf. Accessed 03.23.19.
  21. Seth P, Scholl L, Rudd RA, Bacon S, 2018. Overdose deaths involving opioids, cocaine, and psychostimulants - United States, 2015–2016. MMWR Morb. Mortal. Wkly. Rep 30, 349–358. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Shearer C, 2000. The CRISP-DM model: The new blueprint for data mining. J. Data Warehouse 5, 13–22. [Google Scholar]
  23. Shev AB, Wintemute GJ, Cerdá M, Crawford A, Stewart SL, Henry SG, 2018. Prescription drug monitoring program: Registration and use by prescribers and pharmacists before and after legal mandatory registration, California, 2010–2017. Am. J. Public Health 108, 1669–1674. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Suffoletto B, Lynch M, Pacella CB, Yealy DM, Callaway CW, 2018. The effect of a statewide mandatory prescription drug monitoring program on opioid prescribing by emergency medicine providers across 15 hospitals in a single health system. J. Pain 19, 430–438. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Sun BC, Lupulescu-Mann N, Charlesworth CJ, Kim H, Hartung DM, Deyo RA, John McConnell K, 2018. Variations in prescription drug monitoring program use by prescriber specialty. J. Subst. Abuse Treat 94, 35–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Tripathi J, Weiner SG, 2017. MeDSS: A data-driven tool for pain management, available at: http://opioidconference.org/uploads/1_7_Tripathi_Weiner_Presentation_ICOO2017.pdf. Accessed 03.23.19.
  27. Williams KS, Magalotti S, Schrouder K, Knox M, Feldman L, Ujwal D, Lynch D, 2018. Prescription drug monitoring programs: Relationships among program awareness, use, and state mandates. J. Pain Palliat. Care Pharmacother 10, 1–5. [DOI] [PubMed] [Google Scholar]
  28. Winstanley EL, Zhang Y, Mashni R, Schnee S, Penm J, Boone J, McNamee C, MacKinnon NJ, 2018. Mandatory review of a prescription drug monitoring program and impact on opioid and benzodiazepine dispensing. Drug Alcohol Depend 188, 169–174. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Wisconsin.gov, 2019. Wisconsin enhanced prescription drug monitoring program (ePDMP) Available at: https://pdmp.wi.gov. Accessed 03.23.19.

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS1534130-supplement-1.docx (344.7KB, docx)

RESOURCES